Documentation for JAWS

_images/JAWS5.tiny.png

The Joint Genome Institute has developed JAWS (JGI Analysis Workflow Service) as a framework to run computational workflows. Its purpose is to improve the re-usability and robustness of workflows in a high performance computing (HPC), cluster, and cloud environments.

In Short…

  • JAWS should simplify your workflow submissions because it takes care of resource configuration and scheduling
  • Before using JAWS, workflows need to be wrapped in an easy-to-understand Workflow Definition Language (WDL)
  • JAWS presents the user with many commands to manage your jobs.

Some Definitions

WDL
The Workflow Description Language is essentially a
wrapper around the commands in your pipeline code.
Cromwell
A workflow engine which takes a WDL and converts it
to bash commands that can be run on a “backend”.
Backend
Any compute resource like a private labtop, or a public
compute cluster like Cori. In our case we use JTM as a backend.
JTM
A customized backend that uses different compute sites (i.e. Cori, and JGI)
and is responsible for reserving workers on the cluster.
Workers
Workers are “processes” that run the Cromwell commands.
They can be part of a “worker pool” to process parallel tasks.
Tasks
Each WDL or run is composed of multiple tasks.