biomodal CLI on Biowulf

The biomodal pipeline processes FASTQ files generated using the biomodal library preparation kits. The pipeline utilises Nextflow as the orchestration tool.

Documentation
Important Notes

Interactive job
Interactive jobs should be used for debugging, graphics, or applications that cannot be run as batch jobs.

Allocate an interactive session and run the program.
Sample session (user input in bold):

[user@biowulf]$ sinteractive
salloc.exe: Pending job allocation 46116226
salloc.exe: job 46116226 queued and waiting for resources
salloc.exe: job 46116226 has been allocated resources
salloc.exe: Granted job allocation 46116226
salloc.exe: Waiting for resource configuration
salloc.exe: Nodes cn3144 are ready for job

[user@cn3144 ~]$ module load biomodal

[user@cn3144 ~]$ cd /data/$USER/project_dir 

[user@cn3144 user]$ biomodal auth # run once
Enter your email: 
Enter your password:

[user@cn3144 user]$ biomodal init # run per project

[user@cn3144 user]$ cat $BIOMODAL_NF_OVERRIDE >> nextflow_override.conf # add biowulf specific nextflow options

[user@cn3144 user]$ biomodal run duet \
    --input-path /path/to/input \
    --output-path /path/to/output \
    --mode 5bp \
    --tag mytag

[user@cn3144 user]$ exit
salloc.exe: Relinquishing job allocation 46116226
[user@biowulf ~]$

Batch job
Most jobs should be run as batch jobs.

Create a batch input file (e.g. biomodal.sh). For example:

#!/bin/bash
set -e
module load biomodal
biomodal run duet \
    --input-path /path/to/input \
    --output-path /path/to/output \
    --mode 5bp \
    --tag mytag

Make sure to update the paths according to your data location. Submit this job using the Slurm sbatch command. You do not need to provide large resources to this job since it is the orchestrating job; just provide enough wall time. For example:

sbatch --cpus-per-task=4 --mem=8G --time=2-00:00:00 biomodal.sh
Biomodal Project Configuration

When you run biomodal init for a specific biomodal project you will see the following prompts. We have highlighted the recommended settings.

Your computing platform is currently undefined.
Select your computing platform:
  - aws
  - gcp
  - lsf
  - pbs
  - sge
  - slurm
  - local
Enter choice: slurm

Select your container engine:
  - apptainer
  - docker
  - singularity
Enter choice: singularity

Define where to store reference files. These are large files that can be shared across all installs.
Example: '/path/to/dir' or './subdir'
Enter your reference files location: /data/$USER/biomodal/2.0.0

Define the location where to store container images for the analysis modules.
Example: '/path/to/dir' or './subdir'
Enter where to store container images: /data/$USER/biomodal/sing/2.0.0

Your Nextflow work directory is currently undefined. It is where temporary data are saved to disk during a pipeline run. Please define it.
Example: '/path/to/dir' or './subdir'
Enter the new work directory location: ./work

Your biomodal duet version is currently undefined.
Do you want to use the default version (1.5.0)? [Y/n]: Y

The default error strategy for biomodal pipelines is to fail-fast. (i.e., terminate the task immediately after the first failure.)
Do you want to change it to retry? (i.e., retry a task up to 10 times.) [y/N]: N

# Telemetry:
If you enable optional telemetry sharing, you are helping biomodal improve the CLI through non-sensitive usage metrics and faster support, while keeping your data files and personal information private.
For more information, please see: https://software-docs.biomodal.com/projects/biomodal-cli/en/latest/#sharing-event-metric-data-with-biomodal

Would you like to share anonymous usage events with biomodal? This includes actions like downloading and running the CLI. [y/N]: N

Would you like to share anonymous metrics with biomodal? This includes metadata about the results at the end of your run. (No sequencing data will be shared with biomodal.) [y/N]: N

Your current CLI config is defined as follows':

cli:
    max_concurrent_transfers: 6
    max_retries: 3
computing_platform:
    container_engine: singularity
    error_strategy: fail_fast
    images_registry_location: /data/$USER/biomodal/sing/2.0.0
    nextflow_work_directory_location: work
    reference_files_location: /data/$USER/biomodal/2.0.0
    type: slurm
pipelines:
    duet:
        version: 1.5.0
telemetry:
    share_events: false
    share_metrics: false