Continuous Integration avoids unpleasant surprises

The Continuous Integration (CI) project’s goal is to reduce the amount of human effort needed to verify each code release, and thus to reduce the frequency of wasted computing and human resources. The CI system is a set of tools, applications and machines that allows users to execute their validation tests with minimal effort. It is based on the open source Jenkins toolkit, which offers a powerful tool for complex software, and associated database, testing interfaces, and web facilities. More information (including links to detailed instructions on how to run CI tests) can be found at

Experiments/collaborations already on-boarded are: uBooNE, DUNE (35T), LArIAT, ArgoNeut (from the existing LAr-CI), NOvA and art. Other experiments have expressed interest and will soon be on-boarded.

As Alex Himmel, who is monitoring the CI build for NOvA, said, ”The CI tests picked up a problem introduced in a commit which has now been fixed, and it would be great to have another set of CI tests that show that things are now fixed.”

The CI project aims to provide rapid feedback to help identify defects introduced by the software framework changes as soon as possible, before the code goes in production. Issues detected early in development are typically smaller, less complex and easier to resolve. CI is planned to be used by other frameworks, such as the art system. As Marc Paterno said, “The art team would like to use the CI system in order to help reduce the amount of human effort needed to verify each art release candidate, and thus to reduce to zero the frequency of incidents in which a newly-minted release is discovered to fail on a lesser-used operating system.”

Real-time test status, progress and results are viewable via a Web GUI running on a database node called the CI results display page. The colors indicate the status of the phase. CI_status

Objectives for the CI project include:

  1. Monitor the revision control system for changes and automatically run the build process to verify that introduced changes integrate correctly.
  2.  Provide reports on current and past builds, along with detailed information on pre- and post-install tests of the software.
  3. Automatic notification including log information.
  4. Many more.

By using CI, experiments can avoid the situation where the code and infrastructure is ready, and everything seems to be in place, but a large submission of jobs causes a failure. The goal is to ensure that the data needed for that important conference is available on time.

— Anna Mazzacane and Katherine Lato