Subworkflows

A subworkflow is just a WDL that can be imported into a main WDL and then called like a task.

Here is a more detailed description of subworkflows on Cromwell’s docs

Why use subworkflows

  • re-use code

  • use code that has been expertly vetted

Example of Subworkflow

Main WDL: main.wdl

../_images/main.png

Subworkflow: sub.wdl

../_images/sub.png

How to Run Subworkflows in JAWS

You can try running this example:

# activate the environment you set up above
# on DORI
module load jaws

# clone the example code
git clone https://code.jgi.doe.gov/official-jgi-workflows/wdl-specific-repositories/jaws-tutorial-examples.git

cd jaws-tutorial-examples/subworkflow

# run jaws submit <workflow> <inputs> <site>
jaws submit main.wdl inputs.json dori

How to supply subworkflows

Subworkflows may be imported using relative paths. The jaws-client app will validate your WDL, find the subworklow files, and prepare a ZIP file automatically. Alternatively, if you have a ZIP file of your subworkflows, you can supply it using the --sub option (jaws submit --sub <zip>).

import ./FastqToUBam.wdl as FastqToUBam

Also, subworkflows may be imported using HTTP URLs, so long as no login is required, such as a public git repository (use the “Open raw” button in gitlab to get the URL to use).

import "https://raw.githubusercontent.com/HumanCellAtlas/skylab/optimus_v2.0.0/library/tasks/FastqToUBam.wdl" as FastqToUBam

JGI Workflows

A few of JGI’s workflows have been ported over to WDL. Here you can find a list of them.

JGI WDL Pipelines.

See the tabs:

  1. Subgroups and Projects (projects created under this repository).

  2. Shared Projects (projects created under some other repository).

How to Share Workflows

You can share JGI workflows with the scientific community by adding them to the JGI maintained gitlab group JGI WDL Pipelines. A gitlab group is just a collection of repositories that share permissions.

There are two options:

  1. you can just create a link in this group to your existing repository

  2. you can create the repository under this group.

Directions are under the readme How to Add Projects to this Group

The Group’s Page Looks Like

../_images/jgi-wdl-pipelines.png

JGI Subworkflows and Tasks

These are sub-workflows and tasks that are maintained by the JAWS team that can be used when developing your own WDLs.

JGI Subworkflows and Tasks

See the README for that repo to see how to use the subworkflows in your own WDL.