Cromwell

Installing and Running Cromwell locally

Summary

Here we will go over getting Cromwell setup on your local machine so you can test & develop WDLs.

Required

Important!

You need to have a java –version that is compatible with Cromwell or Cromwell will complain. If you get an error that says some component of Cromwell’s was compiled with a newer version of java than you have, then update java: java downloads.

Install Cromwell

The below steps are copied from Cromwell’s official github repository:

  1. Download the latest cromwell and womtool jar file from the above repository.

  2. Copy the files to some place like ~/cromwell/cromwell-<VERSION>.jar and ~/cromwell/womtool-<VERSION>.jar.

Test Run Cromwell

To make sure cromwell is working, copy the WDL file example below to a file called my.wdl and then run:

java -jar ~/cromwell/cromwell-<VERSION>.jar run my.wdl
my.wdl Example
workflow myTest {
    call TaskOne
}

task TaskOne {
    command {
        echo "hello jackson" > output.txt
    }
    output {
        String out = "output.txt"
    }
}

The cromwell Config File

You can configure cromwell settings in 3 ways. The default settings are in a file called cromwell.conf file which is packaged with every cromwell.jar. However, you don’t set the variables directly in this file. Instead, when you run the cromwell.jar command, you can:

  1. Point to your own cromwell.conf file.

  2. Include an options.json file that has the variables set.

  3. Specify the variables from the command line itself.

Note

Check out the tutorial on How to Configure Cromwell for more information.

1. Overriding Variables with a cromwell.conf File

The cromwell.conf file that is packaged with the jar is much larger than this example. This example only includes variables you are likely to alter.

An example cromwell.conf file
title:

jgi-purple text-white font-weight-bold box

body:

bg-light

animate:

fade-in

include required(classpath("application"))

webservice
{
  port = 50011
}

workflow-options
{
  workflow-log-dir: "cromwell-workflow-logs"
  workflow-log-temporary: false
}

call-caching
{
  enabled = true
  invalidate-bad-cache-result = true
}
docker {
  hash-lookup {
      enabled = true
  }
}


backend
{
  default = "Local"

  providers
  {

    Local
    {
      actor-factory = "cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory"

      config
      {

        concurrent-job-limit = 5
        run-in-background = true
        #temporary-directory = "`mktemp -d \"/global/cscratch1/sd/jaws/cromwell-tmp\"/tmp.XXXXXX`"

        # The list of possible runtime custom attributes.
        runtime-attributes = """
        String? docker
        String? docker_user
        """

        # Submit string when there is no "docker" runtime attribute.
        submit = "/usr/bin/env bash ${script}"

        # Submit string when there is a "docker" runtime attribute.
        submit-docker = "docker run --volume=${cwd}:${docker_cwd} ${docker} /usr/bin/env bash ${script}"

        root = "/cromwell-executions"
        dockerRoot = "/cromwell-executions"

        filesystems
        {
          local
          {
            localization: [ "soft-link", "hard-link", "copy" ]
            http {}

            caching {
              duplication-strategy: [ "soft-link" ]
              hashing-strategy: "path"
            }
          }
        }

        default-runtime-attributes
        {
          failOnStderr: false
          continueOnReturnCode: 0
        }
      }
    }
  }
}

database
{
  profile = "slick.jdbc.MySQLProfile$"
  db
  {
    driver = "com.mysql.cj.jdbc.Driver"
  url = "jdbc:mysql://localhost:3306/cromwell?rewriteBatchedStatements=true&useSSL=false&autoReconnect=true&useUnicode=true&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC&allowPublicKeyRetrieval=true"
    user = "userName"
    password = "somePassword"
    connectionTimeout = 5000
  }
  insert-batch-size = 2000
}

Note

The example config has different sections or “stanzas” defined by {} and can be nested.

One of the larger stanzas is called backend. It has only one backend defined, default = "Local", but could have more like slurm, and custom backends.

The official cromwell configuration documentation is the best place to learn about the configs, which includes:

  1. a tutorial on creating your own config.

  2. description of the different parameter settings.

  3. examples using different backends beside Local.

  4. how to set variables using the command line arguments (also described below).

  5. how to setup the mysql database (we have a tutorial under Configuring MySQL Database for Cromwell)

2. Overriding Variables with a options.json File

See the official documentation from https://cromwell.readthedocs.io/ concerning options.json.

3. Overriding Variables from the Command Line

Setting variables on the command line is done with the -D flag, with no spaces. The variable name must include the full path as it exists in the cromwell_docker.config `file, for example, the variable :bash:`dockerRoot exists under:

backend {
    providers {
        Local {
            config {
                dockerRoot = "/cromwell-execution"

Following that logic for other variables.

java -Dconfig.file=cromwell_docker.conf \
     -Dbackend.providers.Local.config.dockerRoot=$(pwd)/cromwell-executions \
     -Dbackend.providers.Local.config.root=$(pwd)/cromwell-executions \
     -Dbackend.default=Local \
     -jar ~/cromwell/cromwell.jar run my.wdl

where

-Dconfig.file points to a cromwell conf file that is used to overwrite the default configurations

-Dbackend.providers.Local.config.dockerRoot this overwrites a variable ‘dockerRoot’ that is in cromwell_docker.conf and tells cromwell to mount this path within the docker container.

-Dbackend.providers.Local.config.root this overwrites a variable ‘root’ that is in cromwell_docker.conf and tells cromwell to use this path for its output outside of the docker container.

-Dbackend.default=[Local|Slurm|AWS|etc.] this will allow you to choose between the Local and other backends. No other backends are included in the cromwell_docker.conf example.

Configure Backends

One thing you can customize in a Cromwell configuration file is the backend which, among other things, specifies where your jobs will run. You can, for example, choose to use SLURM or your local machine.

Note

Running in JAWS

when running in JAWS, the HTCondor backend is set for you by default and can’t be changed.

This page describes how you can change backend parameters in the config file for testing your WDL. When running JAWS you cannot change the configuration settings at all.

How Cromwell uses the Configuration file

Cromwell is installed with a complete default config file. However, for testing, you can overwrite parts of it by specifying another config file that has, for example, docker backend specifications. By including the -Dconfig.file=<your.config> option to your Cromwell command you can overwrite the default config.

Example configs

Backend Examples

Cromwell has two modes of operation: server and non-server (client). During development, the client mode is the simplest way to test WDLs.

You can find examples of backends for our various sites here.

Example of Running a WDL with Slurm Config

You can see a working example of how to run a task on slurm. working example using slurm

Follow the README.md in the above repository to run the WDL (test.wdl) and inspect the WDL and cromwell_dori.conf file.

Essentially, you specify that you want to use slurm in the runtime section of your WDL. If you don’t specify anything for the backend: then the default will run jobs locally, as set in the config (see below).

runtime {
        backend: "SLURM"
}

And in the config file you would define SLURM to be one of the backend providers.

backend {
        default = "Local"
        providers {
                SLURM {
                        submit = "sbatch <command>"
                }
}