Toolkit

VECMA M42 final release version of all components:

Overview of the VECMA Toolkit

The VECMA Toolkit (VECMAtk) enables automated VVUQ for multiscale applications that can be deployed on emerging exascale platforms. VECMAtk provide support for software applications regardless of the domain of interest. The month 42 release is made up of the following tools:

• The full VECMA toolkit (VECMAtk)
• FabSim3
• EasySurrogate
• EasyVVUQ
• QCG Pilot Job
• QCG-Now
• QCG-Client
• EasyVVUQ-QCGPilotJob
• MUSCLE3

In Figure 1 we show how the different tools are combined when using each of the four application tutorials, after which we provide a short introduction to each component.

Figure 1: Tube Map showing which VECMAtk components are used in which tutorials. VECMAtk components are given in boxes, and the tutorials are indicated using colored lines. Note that in particular EasyVVUQ can be used both on the local desktop for ease of use, or on a remote HPC resource for improved performance.

Overview of the VECMAtk toolkit and its components

The full VECMA toolkit (VECMAtk) establishes a platform for verification, validation and uncertainty quantification (VVUQ). The goal is to provide tools that can be combined to capture complex scenarios, applied to applications in disparate domains, and used to run multiscale simulations on any desktop, cluster or supercomputing platform.

In addition to installing each component of VECMA VVUQ Toolkit individually, the full VECMAtk Toolkit is available for download and installation; this contains all the software components in the form of a Python VirtualEnv:

• Tutorials: https://github.com/vecma-project/VECMAtk/blob/master/VECMAtk_static_tutorials/Tutorial_Links.md.
• Installation: https://github.com/vecma-project/VECMAtk/blob/master/README.md#installation.
• VECMAtk_setup script: https://github.com/vecma-project/VECMAtk/blob/master/VECMAtk_setup.sh
• Download the VECMAtk setup script for installation.

FabSim3 is an automation toolkit for complex simulation tasks. FabSim3 helps users to perform complex remote tasks from a local command-line, and to automatically organise their data and environment variables when they perform these tasks. FabSim3 supports the execution of single jobs, ensembles of multiple jobs, and dynamic workflows through schedulers such as Slurm, PBSPro, LoadLeveller and QCG. Further information can be found here:

• Documentation: https://fabsim3.readthedocs.io/en/latest/
• Code: http://www.github.com/djgroen/FabSim3
• Download version v3.6.

EasySurrogate is a toolkit designed to facilitate the creation of surrogate models for multiscale simulations. The beta version of the EasySurrogate toolkit we use is for replacing the expensive (small-scale) components of multiscale simulations by a surrogate model. The overall design is similar to EasyVVUQ, our forward uncertainty propagation toolkit. 

• Tutorial: https://github.com/wedeling/EasySurrogate/tree/master/tutorials
• Code: https://github.com/wedeling/EasySurrogate
• Download version v0.18

EasyVVUQ is a Python library designed to facilitate verification, validation and uncertainty quantification (VVUQ) for a wide variety of simulations. It was concieved and developed within the EU funded VECMA (Verified Exascale Computing for Multiscale Applications) project. Further information can be found here:

• Documentation: https://easyvvuq.readthedocs.io/en/latest/
• Code: http://www.github.com/UCL-CCS/EasyVVUQ
• Download version v1.1.1

QCG Pilot Job is a lightweight implementation of the Pilot Job mechanism. It can be easily incorporated into scientific workflows to provide efficient and reliable execution of large number of computational jobs.

• Code: https://github.com/vecma-project/QCG-PilotJob
• Documentations can be found here
• Download version v0.13.0

QCG-Now is a portable desktop program that allows to prepare and run computational jobs on the HPC machines. With an intuitive interface and in a flexible and secure manner it automatises a process of job submission, data management and simulation progress monitoring. The sources are currently stored on our servers, the access to them could be requested. The released packages are available from the downloads webpage. 

QCG-Client a command-line interface to the QCG middleware. Its interface is similar to the well known interfaces of queuing systems, such as Slurm or PBS. QCG-Client provides support for a variety of computing jobs, from simple ones to complex distributed workflows.

• Documentation and Code: http://www.qoscosgrid.org/trac/qcg-broker#QCG-BrokerClient

EasyVVUQ-QCGPilotJob is a lightweight integration code that simplifies usage of EasyVVUQ with a QCG Pilot Job execution engine. The tool could be used for efficient, parallel execution of demanding EasyVVUQ scenarios on HPC resources.

• Documentation here
• Code: https://github.com/vecma-project/easyvvuq-qcgpj
• Download version v0.5.

MUSCLE 3 is the third incarnation of the Multiscale Coupling Library and Environment. Its purpose is to make creating coupled multiscale simulations easy, and to then enable efficient Uncertainty Quantification of such models using advanced semi-intrusive algorithms. MUSCLE 3 uses the Multiscale Modelling and Simulation Language (MMSL) to describe the structure of a multiscale model. MMSL can be expressed in the form of a diagram (gMMSL; not yet implemented) or as a YAML file (yMMSL; this is convenient both for people and for software). The MMSL lets one describe which compute elements (submodels, scale bridges, data converters, UQ components, etc.) a multiscale model consist of, how many instances of each we need, and how they are wired together. MUSCLE 3 is intended to scale from your laptop to the exascale, and we intend to test and apply the toolkit to progressively larger resources with each new release.