Skip to content
Snippets Groups Projects

feat: let gitlab parallel drive multi-node snakemake

Open Wouter Deconinck requested to merge snakemake-batch into master
2 unresolved threads
1 file
+ 2
1
Compare changes
  • Side-by-side
  • Inline
@@ -11,11 +11,12 @@ bench:ecal_gaps:
stage: benchmarks
needs:
- ["sim:ecal_gaps"]
parallel: 8
Please register or sign in to reply
script:
- ln -s $LOCAL_DATA_PATH/input input
- export PYTHONUSERBASE=$LOCAL_DATA_PATH/deps
- pip install -r benchmarks/ecal_gaps/requirements.txt
- snakemake $SNAKEMAKE_FLAGS --cores 8 results/epic_inner_detector/ecal_gaps
- snakemake $SNAKEMAKE_FLAGS --cores 1 --batch results/epic_inner_detector/ecal_gaps=$CI_NODE_INDEX/$CI_NODE_TOTAL
    • Suggested change
      19 - snakemake $SNAKEMAKE_FLAGS --cores 1 --batch results/epic_inner_detector/ecal_gaps=$CI_NODE_INDEX/$CI_NODE_TOTAL
      19 - snakemake $SNAKEMAKE_FLAGS --cores 5 --batch results/epic_inner_detector/ecal_gaps=$CI_NODE_INDEX/$CI_NODE_TOTAL
Please register or sign in to reply
collect_results:ecal_gaps:
extends: .det_benchmark
Loading