cibrrig.run_sasquatch

Need: 1) Local run path 2) Sasquatch directory 3) active ALF target 4) Archive ALF target

Steps: 1) Compress local 2) Backup to Baker 3) Copy to Sasquatch 4) Remove local 5) Submit slurm job 5) Move to helens

Attributes

SASQUATCH_WORKING_DIR

BAKER_DEST

BAKER_DEST_RSS

HELENS_DEST

job_params

Functions

gen_sbatch_script(→ str)

Generate SLURM batch script as a string.

gen_tarball_script(→ str)

Generate SLURM batch script as a string.

remove_empty_folders(root_dir)

main()

from_NPX(local_run_path[, sasquatch_working_dir, ...])

Run the pipeline on sasquatch from the acquisition computer using SSH commands

from_baker(run_src[, sasquatch_working_dir, ...])

Run the pipeline on sasquatch from data already on baker.

Module Contents

cibrrig.run_sasquatch.SASQUATCH_WORKING_DIR[source]
cibrrig.run_sasquatch.BAKER_DEST[source]
cibrrig.run_sasquatch.BAKER_DEST_RSS[source]
cibrrig.run_sasquatch.HELENS_DEST[source]
cibrrig.run_sasquatch.job_params[source]
cibrrig.run_sasquatch.gen_sbatch_script(job_params: dict, run_folder: pathlib.Path, helens_dest: pathlib.Path | None = None, baker_dest_rss: pathlib.Path | None = None, opto_flag=True, QC_flag=True) → str[source]

Generate SLURM batch script as a string.

Parameters:

  • job_params (dict) – Dictionary containing job parameters.

  • run_folder (Path) – Path to the run folder on the remote server.

  • helens_dest (Path) – Path to the destination folder on Helen’s server.

  • baker_dest_rss (Path) – Path to the destination folder on Baker’s RSS server.

Returns:

SLURM batch script as a string.

Return type:

str

cibrrig.run_sasquatch.gen_tarball_script(run_folder: pathlib.Path) → str[source]

Generate SLURM batch script as a string.

Parameters:

run_folder (Path) – Path to the run folder on the remote server.

Returns:

SLURM batch script as a string.

Return type:

str

cibrrig.run_sasquatch.remove_empty_folders(root_dir)[source]
cibrrig.run_sasquatch.main()[source]
cibrrig.run_sasquatch.from_NPX(local_run_path, sasquatch_working_dir=SASQUATCH_WORKING_DIR, helens_dest=HELENS_DEST, baker_dest_rss=BAKER_DEST_RSS, opto_flag=True, QC_flag=True)[source]

Run the pipeline on sasquatch from the acquisition computer using SSH commands

Parameters:

  • local_run_path (Path) – Path to the local run directory.

  • sasquatch_working_dir (Path, optional) – Path to sasquatch working directory. Defaults to SASQUATCH_WORKING_DIR.

  • helens_dest (Path, optional) – Path to Helen’s destination directory. Defaults to HELENS_DEST.

  • baker_dest_rss (Path, optional) – Path to Baker’s RSS destination directory. Defaults to BAKER_DEST_RSS.

  • opto_flag (bool, optional) – Whether to remove opto artifact. Defaults to True.

  • QC_flag (bool, optional) – Whether to run ephys QC. Defaults to True.

Returns:

None

Raises:

FileNotFoundError – If the local run path does not exist.

cibrrig.run_sasquatch.from_baker(run_src, sasquatch_working_dir=SASQUATCH_WORKING_DIR, helens_dest=HELENS_DEST, opto=True, qc=True)[source]

Run the pipeline on sasquatch from data already on baker. Must be run from the login node.

This script is accessed by the entry point pipeline_hpc after installing cibrrig in a conda environment. e.g.:

pipeline_hpc /data/rss/baker/ramirez_j/ramirezlab/alf_data_repo/ramirez/Subjects/m2025-01

Best practice is to create a tmux session on login node since you will move large files, activate iblenv, then run this

Parameters:

  • run_src (str) – Source run path, e.g. ‘/data/rss/baker/ramirez_j/ramirezlab/alf_data_repo/ramirez/Subjects/m2025-01’

  • sasquatch_working_dir (Path, optional) – Path to sasquatch working directory. Defaults to SASQUATCH_WORKING_DIR.

  • helens_dest (Path, optional) – Path to Helen’s destination directory. Defaults to HELENS_DEST.

  • opto_flag (bool, optional) – Whether to remove opto artifact. Defaults to True.

  • QC_flag (bool, optional) – Whether to run ephys QC. Defaults to True.

Raises:

FileNotFoundError – If the baker path does not exist.