Configuring the JAWS Client: A Step-by-Step Guide

Summary

The JAWS Client is a command-line interface (CLI) that allows you to interact with the JAWS service. This guide will help you set up the JAWS Client on one or more locations where the JAWS Client is installed.

Prerequisites

• JAWS Token: You must have a JAWS token to access the JAWS Client.

• Location of your input data:

  • DORI: If your data is stored on Dori, you must have a Dori account.

  • NERSC: If your data is stored on the NERSC file system, you must have a NERSC account and be a member of the genome group (or nmdc for the NMDC project).

If you have questions about how to get an account, please get in touch with our team.

Steps to Set Up JAWS Client

These are the steps you’ll be required to complete to start running a pipeline in JAWS:

1. Obtain a JAWS Token;

2. Create a configuration file to store your token;

3. choose how you’d like to access the JAWS Client:

   • You have three options for accessing the JAWS Client:

     • Option 1: activate the JAWS virtual environment using module load jaws;

     • Option 2: use jaws client container;

     • Option 3: install the jaws-client into your own virtual environment using pip.

Let’s get started!

JAWS TOKEN

JAWS TOKEN

Before completing the JAWS setup below, you need to get a JAWS token from the JAWS admin.

Please fill this out form. The JAWS Team will contact you soon!

Setting up JAWS Client

1. Create a config file that has the JAWS token in it:

Select the cluster where your data is stored and to which you will be submitting to JAWS. Next, copy the jaws.conf file to your home directory ($HOME). Ensure that the file permissions are set so that only you can access it.

DORI

cp /clusterfs/jgi/groups/dsi/homes/svc-jaws/dori-prod/jaws.conf ~/jaws.conf
chmod 600 ~/jaws.conf

NERSC - Perlmutter

cp /global/cfs/cdirs/jaws/perlmutter-prod/jaws.conf ~/jaws.conf
chmod 600 ~/jaws.conf

NERSC - NMDC

cp /global/cfs/cdirs/m3408/jaws-install/jaws-client/nmdc-prod/jaws.conf ~/jaws.conf
chmod 600 ~/jaws.conf

TAHOMA - NMDC

cp /tahoma/mscnmdc/jaws-install/jaws-client/nmdc_tahoma-prod/jaws.conf ~/jaws.conf
chmod 600 ~/jaws.conf

2. Add your token and default_team to ~/jaws.conf:

[USER]
token = <your token>
default_team = <team ID>

Find here information about JAWS Teams.

Use JAWS CLI

To use the JAWS CLI, you can choose from three options tailored to different needs and setups.

1. You could activate the JAWS environment with the command module load jaws for easy integration.

2. Alternatively, using the jaws client container offers a more isolated setup.

3. If you prefer direct integration with your existing Python projects, installing jaws-client package into your virtual environment is also an option.

1. module load jaws

You can use Environment Modules to load JAWS in the environment.

General Usage:

module use <PATH/jaws/modulefiles/>
module load jaws

Note

The purpose of module use is to point to our custom module files. You can add one or both of these lines to your ~/.bashrc file.

DORI

module use /clusterfs/jgi/groups/dsi/homes/svc-jaws/modulefiles
module load jaws

NERSC - Perlmutter

module use /global/cfs/projectdirs/jaws/modulefiles
module load jaws

NERSC - NMDC

If you are part of NMDC project, please use:

module use /global/cfs/cdirs/m3408/jaws-install/jaws-client/modulefiles
module load jaws

TAHOMA - NMDC

If you are part of NMDC project on Tahoma, please follow:

ssh twf2
module use /tahoma/mscnmdc/jaws-install/jaws-client/modulefiles
module load jaws

NERSC - KBase

If you are part of KBase project on NERSC, please follow:

module use /global/cfs/projectdirs/kbase/jaws/modulefiles
module load jaws

• To deactivate the environment, use the following:

  module unload jaws
  

Useful commands 🔗

module list        ## List actyive modules in the environment
module avail jaws  ## List available JAWS modules

2. JAWS Client Container

You can use the JAWS Client Container.

DORI

JAWS_USER_CONFIG=~/jaws.conf JAWS_CLIENT_CONFIG=/clusterfs/jgi/groups/dsi/homes/svc-jaws/dori-prod/jaws-prod.conf apptainer run docker://doejgi/jaws-client:latest jaws --help

• Append to the end of your ~/.bashrc on DORI:

jaws() {
JAWS_USER_CONFIG=~/jaws.conf JAWS_CLIENT_CONFIG=/clusterfs/jgi/groups/dsi/homes/svc-jaws/dori-prod/jaws-prod.conf apptainer run docker://doejgi/jaws-client:latest jaws "$@"
}

NERSC - Perlmutter

JAWS_USER_CONFIG=~/jaws.conf JAWS_CLIENT_CONFIG=/global/cfs/projectdirs/jaws/perlmutter-prod/jaws-prod.conf shifter --image=doejgi/jaws-client:latest jaws --help

• Append to the end of your ~/.bashrc:

jaws() {
JAWS_USER_CONFIG=~/jaws.conf JAWS_CLIENT_CONFIG=/global/cfs/projectdirs/jaws/perlmutter-prod/jaws-prod.conf shifter --image=doejgi/jaws-client:latest jaws "$@"
}

NERSC - NMDC

If you are part of NMDC project, please use:

JAWS_USER_CONFIG=~/jaws.conf JAWS_CLIENT_CONFIG=/global/cfs/cdirs/m3408/jaws-install/jaws-client/nmdc-prod/jaws-prod.conf shifter --image=doejgi/jaws-client:latest jaws --help

• Append to the end of your ~/.bashrc:

jaws() {
JAWS_USER_CONFIG=~/jaws.conf JAWS_CLIENT_CONFIG=/global/cfs/cdirs/m3408/jaws-install/jaws-client/nmdc-prod/jaws-prod.conf shifter --image=doejgi/jaws-client:latest jaws "$@"
}

TAHOMA - NMDC

If you are part of NMDC project on Tahoma, please follow:

JAWS_USER_CONFIG=~/jaws.conf JAWS_CLIENT_CONFIG=/tahoma/mscnmdc/jaws-install/jaws-client/nmdc_tahoma-prod/jaws-prod.conf apptainer run docker://doejgi/jaws-client:latest jaws --help

• Append to the end of your ~/.bashrc:

jaws() {
JAWS_USER_CONFIG=~/jaws.conf JAWS_CLIENT_CONFIG=/tahoma/mscnmdc/jaws-install/jaws-client/nmdc_tahoma-prod/jaws-prod.conf  apptainer run docker://doejgi/jaws-client:latest jaws "$@"
}

NERSC - KBase

If you are part of KBase project, please use:

JAWS_USER_CONFIG=~/jaws.conf JAWS_CLIENT_CONFIG=/global/cfs/projectdirs/kbase/jaws/jaws-client/kbase-prod/jaws-prod.conf shifter --image=doejgi/jaws-client:latest jaws --help

• Append to the end of your ~/.bashrc:

jaws() {
JAWS_USER_CONFIG=~/jaws.conf JAWS_CLIENT_CONFIG=/global/cfs/projectdirs/kbase/jaws/jaws-client/kbase-prod/jaws-prod.conf shifter --image=doejgi/jaws-client:latest jaws "$@"
}

3. Install jaws-client Package into Your Virtual Environment

• Back on your terminal (i.e. on Dori):

  • create a venv and activate it (or active your existing venv):

  python3 -m venv <some-env-name>
  source <some-env-name>/bin/activate
  

  • Install jaws-client package:

  pip install jaws-client==2.3.0 --index-url https://code.jgi.doe.gov/api/v4/projects/312/packages/pypi/simple
  

Note: jaws-client requires Python version between 3.10 and 3.12.

• Run the JAWS Client Python Library:

  We have a dedicated session on how to use the JAWS Client Python Library. Please refer to JAWS API for more information.