HEPCloud doubles the size of CMS computing

High-energy physics experiments have an ever-growing need for computing, but all the experiments don’t need all the cycles all the time. The need is driven by machine performance, experiment and conference schedules, and even new physics ideas. Computing facilities are purchased with the intention to meet peak workload rather than the average, which impacts the overall computing cost for the facility and the experiment.

bholzman_googlecloud_20161201The HEPCloud program enables facility owners to seamlessly expand their resources from Fermilab to other grid and cloud resources as transparently to the user as possible. At the heart is a decision engine that chooses when to provision off-premises resources based on cost, efficiency, and the requirements of the scientific workflow. It is currently in R&D and is scheduled to go into production in 2018.

For the Supercomputing 2016 (SC16) conference, we added Google Cloud Platform as a resource provider for HEPCloud. As a demonstration, we started with an ambitious goal – to double the reach of computing for the CMS experiment during conference hours (12 hours a day, for 4 days). The demo was a big success! Thanks to the Fermilab HEPCloud team, we were able to demonstrate the capability and scale HEPCloud during the conference, utilizing over 150,000 compute cores to the CMS experiment for physics.




Here’s a plot of running job cores during the four days of the conference.

–Burt Holzman