This presentation covers the improvements we made to our HPC setup and workflow, implementing best practices from the DevOps toolchain. The age of bash scripting, for system setup and configuration, has seen its dawn with the rise of configuration management systems like Puppet, Chef, and Ansible.
After we switched from shell scripts to Puppet, we felt the need to take another step in automating our workflow. We implemented an automated validation and build setup using Git for version control, Jenkins for automation, and Docker containers for reproducible builds of our HPC master package.
Deploying our code more often and in a more controlled way, gives us a more consistent code base to deploy our HPC environment. It improved our code quality and diminished bugs and mistakes that went into our production package.
The presentation will focus on how we used tools like Jenkins to improve the reliability of our Puppet code base, and how some simple improvements (git precommit hooks, syntax validation, test deployments,…) can help in the daily management of a Linux HPC environment.
