Slurm Training '15

We were pleased to host the Slurm Training'15 organized by HPCNow! and IQTC-UB. This was a training course aimed to share expertise and strategies of the most popular Batch Queue System in High Performance Computing. It was held at the University of Barcelona, from 5th to 6th February, 2015 in conjunction with HPC Knowledge Meeting'15 and BeeGFS Training'15, also organized by HPCNow!, IQTC-UB and ThinkParQ.

Sponsors Adamantium Sponsors Platinum Sponsors Gold Sponsors Silver

   
     

 

Abstract :

Current SLURM versions are featured with a variety of job completion logging plugins such as mysql, filetxt or script. The purpose of this presentation is to raise awareness about a new jobcomp-family related plugin that manages the job completion information so that it is stored into a user configured elasticsearch1 cluster. An elasticsearch cluster structure is composed by nodes, indexes, types and documents. A document is a basic unit of information that can be indexed, and it is expressed in JSON2 . Elasticsearch operations are performed through HTTP requests against a configured URL in the JobCompLoc parameter inside the SLURM configuration file. The main functionalities of this new plugin include checking whether the configured elasticsearch server is accessible and operational, preprocess the job completion data so that it is encoded in a proper JSON format and saving state of the data that could not be indexed (for instance because the elasticsearch index is temporarily in read-only mode) in the StateSaveLocation so that it can be restored for further indexing retries. Plugin implementation has been held in a context that is part of a bigger project for a Final Master Thesis in the Barcelona School of Informatics3 and in collaboration with the Barcelona Supercomputing Center4 .

____________________________________________

 

  1. Elasticsearch is a highly scalable open-source full-text search and analytics engine. It allows you to store, search, and analyze big volumes of data quickly and in near real time: http://www.elasticsearch.org/.
  2. JavaScript Object Notation, which is an ubiquitous internet data interchange format.
  3. Facultat d’Informàtica de Barcelona: http://www.fib.upc.edu/en.html
  4. Barcelona Supercomputing Center: http://www.bsc.es/

 

 

Title: New jobcomp plugin for elasticsearch

 

Speaker : Alejandro Sanchez Graells (BSC)

 

Date : 6th February 2015

 

Location : Faculty of Chemistry (UB), C/ Martí i Franqués 1, 08028 Barcelona, Spain.

 

Slides : PENDING

Schedule Slurm Training'15

Beginner to Intermediate
Registration and Coffee Meet Up
9:00h
9:30h Jordi Blasco (HPCNow!)
10:00h Jordi Blasco (HPCNow!)
11:15h Jordi Blasco (HPCNow!)
Lunch Break
12:00h
13:30h Jordi Blasco (HPCNow!)
QoS
14:00h Jordi Blasco (HPCNow!)
Coffee Break
15:00h
15:30h Jordi Blasco (HPCNow!)
16:30h Jordi Blasco (HPCNow!)
6 February 2015
Intermediate to Advanced
Coffee Meet Up
9:00h
9:30 Jordi Blasco (HPCNow!)
10:30h Jordi Blasco (HPCNow!)
11:00h Carles Fenoy (BSC)
11:30h Jordi Blasco (HPCNow!)
Lunch
12:00h
13:30h Sergio Iserte (UJI)
14:30h Michel Ringenbach (HPC Center @ University of Strasbourg)
15:30h Alejandro Sanchez Graells (BSC)
Wrap-Up
16:30h