Tutorials

Application tutorials and documentations in M42 final release

The VECMAtk tutorials are divided into two formats:

  • Static tutorials provide informative read-only information for the VECMAtk components with internal and external application instances.
  • Interactive tutorials are easy-to-use as installation of the VECMAtk components are not required and they are versatile because users are able to append with additional remarks, which are hosted in the VECMA JupyterHub environment.

Here, we provide the links to the documentation of VECMAtk components and tutorials for applications from a diverse range of scientific domains:

1. FabSim3 is an automation toolkit written in Python 3 featuring an integrated test infrastructure and a flexible plugin system. There are several plugins available from a diverse range of scientific domains, such as

    • FabUQCampaign for coupling EasyVVUQ with FabSim3;
    • FabMD for a molecular dynamics modelling using Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS);
    • FabFlee for a migration modelling;
    • FabMogp for an earthquake modelling;
    • FabCovid19 for a Covid-19 modelling.
    • FabCovidsim for CovidsSim epidemiological code.

2. EasyVVUQ is a Python library designed to facilitate verification, validation and uncertainty quantification (VVUQ) for a wide variety of simulations. It accounts for uncertainty quantification (UQ) and validation patterns in application to various domains.

We provide tutorials for the basic concepts of EasyVVUQ workflows using Jupyter Notebooks:

    • Encoder and Decoder to set-up an input file for a simulation code and to parse the output of the simulation;
    • Campaign to operate and run simulations;
    • Analysis to analyse the simulation output.

2.1 UQ Sampling techniques and tutorials

EasyVVUQ provides four sampling methods for analysis (see Wright et al. (2020) for explanation of these sampling techniques):

We provide the static tutorials for these samplers below:

Interactive tutorials for EasyVVUQ samplers with examples are available here.

2.2 Verification and Validation (V&V) patterns and tutorials

There are four prominent V&V patterns in VECMAtk, namely:

They are most suitable for multiscale computing applications with source codes available here.

V&V examples:

3. Quality in Cloud and Grid (QCG) is an integrated system offering advanced job and resource management capabilities to deliver to end-users supercomputer-like performance and structure. QCG consists of the following components:

    • QCG-PilotJob is a Pilot Job system that allows to execute many subordinate jobs in a single scheduling system allocation. We demonstrate how EasyVVUQ workflows can be adapted to enable their executions with QCG-PilotJob on HPC machines here.
    • QCG-Client is a command line client for execution of computing jobs on the clusters offered by QCG middleware.
    • QCG-Now is a desktop, GUI client for easy execution of computing jobs on the clusters offered by QCG middleware.

4. EasySurrogate is a toolkit designed to facilitate the creation of surrogate models for multiscale simulations.

5. MUSCLE3 is the third incarnation of the Multiscale Coupling Library and Environment.

    • ISR3D is the 3D in-stent restenosis application aiming to simulate smooth muscle cells (SMC) proliferation and restenosis process as a complication after coronary stenting procedure. It is a fully coupled 3D multiscale model, which includes several single-scale submodels as well as utility modules which facilitate communication between the submodels.

We also present the application of these VECMAtk components to five domains, which are forced migration, fusion energy, climate, biomedicine and urban air pollution, in the publication by Suleimenova et al.(2021) to perform uncertainty quantification analysis, use surrogate models, couple multiscale models and execute sensitivity analysis on HPC.

 

Tutorials and documentations in previous releases

Application tutorials and documentations up to M33 release

We have updated detailed documentations and tutorials in this M33 release which updates and merges with the previous releases. For each application tutorial, based on users feedback we obtained for this release and previous ones, we have made an improvement on both learning side and development side of the documentation. We divided VECMAtk tutorials into two formats, namely static and interactive. The static tutorials are accompanied with the previous M24 VECMAtk release while the interactive tutorials continued to be available in this release. Both static tutorial interactive tutorial are constantly updated in the latest release. The publicly available deliverable D3.5 published on the main VECMA website provides further information.

VECMAtk tutorials

The comprehensive all-in-one VECMAtk tutorials containing all following components which establishes a platform for verification, validation and uncertainty quantification (VVUQ).

Interactive tutorials

The interactive tutorials with Jupyter Notebook provide a portable training environment without requiring the installation of the VECMAtk components. Specifically, these tutorials focus on trying out FabSim3, EasyVVUQ, QCG-PilotJob and EasyVVUQ-QCGPJ using example applications. These interactive tutorials offer unique opportunities to teach and learn independently on how to perform VVUQ analysis.

The links to the detailed tutorials for the VECMAtk components and applications are provided below:

FabSim3 is an automation toolkit written in Python 3  featuring an integrated test infrastructure and a flexible plugin system. There are several plugins available from a diverse range of scientific domains, such as

  • FabUQCampaign for a climate modelling;
  • FabMD for a molecular dynamics modelling using Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS);
  • FabFlee for a migration modelling;
  • FabMogp for an earthquake modelling;
  • FabCovid19 for a Covid-19 modelling.

EasyVVUQ is a Python library designed to facilitate verification, validation and uncertainty quantification (VVUQ) for a wide variety of simulations. It accounts for uncertainty quantification (UQ) and validation patterns in application to  earlier described domains.

  • UrbanAir for modelling air quality at street level in the cities.

 

UQ Sampling techniques and tutorials

  • Stochastic Collocation (SC) sampler examples:
  • Polynomial Chaos Expansion (PCE) sampler example:

 

Validation pattern tutorials:

 

EasySurrogate is a toolkit designed to facilitate the creation of surrogate models for multiscale simulations.

  • L96_QSN: Quantized Softmax Network (QSN) surrogates for atmospheric model equations. In this tutorial, the subgrid-scale term of the Lorenz96 equations is replaced by a QSN surrogate.
  • L96_ANN: Artificial Neural Network (ANN) surrogates for atmospheric model equations. In this tutorial, the subgrid-scale term of the Lorenz96 equations is replaced by an ANN surrogate.
  • gray_scott: reduced surrogates for a reaction diffusion equation. This applies the data compression technique to the two-dimensional gray-scott equations.
  • EasyVVUQ_coupling: a tutorial where an ANN surrogate model is trained with a EasyVVUQ data frame.

QCG-Pilot Job is a Pilot Job system that allows to execute many subordinate jobs in a single scheduling system allocation.

EasyVVUQ-QCGPJ is a lightweight plugin for parallelization of EasyVVUQ with the QCG-PilotJob mechanism. To learn more, please see the tutorial example here.

QCG-Client is a command line client for execution of computing jobs on the clusters offered by QCG middleware.

QCG-Now is a desktop, GUI client for easy execution of computing jobs on the clusters offered by QCG middleware.

MUSCLE3 is the third incarnation of the Multiscale Coupling Library and Environment.

                                          

xxxxxxx

 

Application tutorials provided in M12 release 

We provide four application tutorials to show that VECMAtk can be applied to solve some real world problems. These are:

  • FabFlee (migration modelling)
  • FabUQCampaign (basic CFD models)
  • Fusion
  • FabMD (molecular dynamics)

Each tutorial highlights different components in VECMAtk, as indicated by the corresponding figures.

FabFlee

https://github.com/djgroen/FabFlee/blob/master/doc/Tutorial.md

In this tutorial we will explain how you can combine a simple stochastic conflict evolution model (Flare) with an agent-based migration model (Flee), perform a set of runs based on different conflict evolutions, and visualize the migrant arrivals with confidence intervals.

 

FabUQCampaign

https://github.com/wedeling/FabUQCampaign/blob/master/Tutorial_ADE.md

This tutorial describes how to create a Stochastic Collocation EasyVVUQ campaign.

Figure 2: Overview of VECMAtk components used in the FabUQCampaign tutorial.

HPC execution of EasyVVUQ with QCG
Pilot Job

 

This tutorial allows users to submit an entire integrated EasyVVUQ and QCG Pilot Job Manager workflow as a single job into the HPC cluster.

Figure 3: Overview of VECMAtk components used in the Fusion tutorial.

FabMD

https://github.com/UCL-CCS/FabMD/blob/master/doc/EasyVVUQ_FabMD_example.md

This example shows how to create an ensemble of LAMMPS simulations using EasyVVUQ, execute the jobs through FabMD, then analyse them within the EasyVVUQ architecture.

Figure 4: Overview of VECMAtk components used in the FabMD tutorial.