The Continuous Integration system continues to be improved and new features added to fulfill user needs in terms of code testing. There are different way for users to keep testing their code to make sure that changes integrate into the existing code without breaking anything.
The most common practice is to test the whole software suite of the experiment for each commit to verify that the code builds properly and can run a suite of quick CI tests to provide quick feedback on the status of the code. This configuration is really useful when there are many developers contributing to their experiment code daily. This is how NOvA, LArSoft and LArSoft-based experiments (i.e. ArgoNeuT, DUNE/protoDUNE, LArIAT, SBND and uBooNE) use CI.
In the case of LArSoft-based code, in order to guarantee faster feedback on the code status, the CI introduced new features. Those features include the possibility to concatenate CI workflows and to reuse previously built code. This allows the LArSoft-based experiment code to be tested on different build nodes and, if it is not changed, to use the LArSoft code already built. Other CI users, such as MINERvA, GENIE, g-2, GlideinWMS and Art, test their code nightly. The CI team is working with mu2e to setup a CI configuration that meets their needs.
CI users appreciate the ability to constantly monitor the status of their code and they collaborate with the CI team to further improve the CI system. James Stapleton, the g-2 software manager, commented:
“The nightly build system caught its first true build error last night. It gave me plenty of information, and I could find which commit it was within a couple of minutes (without running a single command). I just sent an email to the developer responsible.”
In the last few months there has been a lot of work to extend and improve the CI validation phase. This CI phase is devoted to CPU-intensive validation tests that can be run as grid jobs to split the load. The main use of the CI validation phase is to monitor the code output in terms of Physics results providing easy access to plots and reports through the CI web application. For example, DUNE is using the CI validation phase to monitor the statuses of the ‘DUNE Tracking Efficiency’ and the ‘DUNE Shower Reco Efficiency’ for each code release. uBooNE is also working to set up their CI validation workflow that also includes the comparison of data results with MC results. A useful feature of the CI validation phase is to provide plots that collect statistics about grid jobs’ usage resources. This feature has been extensively used by the protoDUNE production team to estimate the grid job resources required by their workflow.
If your experiment is not yet onboarded to the CI service and you think CI would be beneficial, or there is a feature that CI is missing for your experiment, please contact the CI team and open a request to Continuous Integration under Scientific Production Processing.
–Vito Di Benedetto