Skip to content

Pipeline overview

What You Get

Category What It Finds Steps
Variant Calling SNPs, indels, structural variants, copy number variants 3, 4, 4b, 18, 19
Clinical Screening Pathogenic variants, carrier status, cancer predisposition (CPSR panels), SMN1/SMN2 copy number and the alpha-globin gene count (opt-in), a couple's shared carrier risk (opt-in), the variants that could explain a list of symptoms (HPO terms, opt-in) 6, 17, 35, 35b, 39, 48
Pharmacogenomics Drug-gene interactions (23+ genes, CYP2C19, CYP2D6 SV, DPYD, etc.), with HLA-A/B from T1K (and, with HLA_QC=true, only when OptiType agrees) and CYP2D6 only when two callers agree 7, 21, 27, 32, 36
Structural Variants Deletions, duplications, inversions, translocations (4 callers + consensus) 4, 4b, 5, 15, 18, 19, 22
Functional Annotation Impact prediction for every variant (VEP + CADD, SpliceAI, REVEL, AlphaMissense) 13, 30
Variant Prioritization Rare deleterious variants, compound hets, gene constraint filtering 31
Repeat Expansions Huntington's, Fragile X, ALS and the other disorders at STRchive's 82 disease loci 9, 9b
Blood Groups ABO, RhD, RhCE, Kell, Duffy, Kidd, MNS, Lutheran, Diego, Dombrock, Colton, the platelet antigens HPA-1 to -6 and -15 and the neutrophil antigens HNA-1, -3, -4 and -5, predicted from the genome (opt-in) 41
MUC1 VNTR (ADTKD-MUC1) Frameshifts inside the MUC1 coding repeat, which no standard caller sees, read by VNtyper with its sensitivity at the sample's depth (opt-in; needs the BAM) 52
Lp(a) A genetically high Lp(a): the LPA KIV-2 repeat count from read depth and the risk variants rs10455872 and rs3798220 (opt-in; needs the BAM) 46
Ancestry & Haplogroups Mitochondrial haplogroup (with an mtDNA contamination check), Y-chromosome haplogroup and Y-STR repeat counts in FamilyTreeDNA's order (opt-in), consanguinity check, projection onto a 1000 Genomes reference panel, admixture fractions and a chromosome painting against an 80-population panel (opt-in), the nearest ancient populations on a PCA of the AADR (opt-in) 11, 12, 26, 26b, 37, 44, 45
Telomere Length Relative telomere content estimation from WGS reads 10
Mitochondrial Heteroplasmy detection, mitochondrial disease variants 12, 20
Polygenic Risk Scores for 13 common conditions and traits (CAD, T2D, cancers, blood pressure, lipids, etc.) with pgsc_calc; a percentile among the most similar ancestry group with the panel installed 25
Quality Control Adapter trimming, coverage statistics, aggregated QC report, sex check, sample identity and contamination, SV filtering 1b, 15, 16, 16b, 28, 33
Storage Alignments kept as a checked CRAM, about half the size of the BAM 34
Clonal Haematopoiesis Acquired low-VAF mutations of the blood in DNMT3A, TET2, ASXL1, JAK2 and 9 other CHIP genes, with the detection limit at the sample's depth (opt-in) 49
Traits Lactase persistence, caffeine, alcohol flush, bitter taste, earwax, pigmentation, APOE and other replicated variants, each row citing its primary paper 42
Archaic Ancestry The Neanderthal and Denisovan shares and the segments carried, with IBDmix against the Altai and Vindija Neanderthals and the Altai Denisovan (opt-in) 43
Viral DNA Inherited integrated HHV-6 and the DNA of 23 human viruses (herpesviruses, HPV, HBV, HIV-1, polyomaviruses and others) in the reads that do not map to the human genome, with phiX as the lab control (opt-in) 47
Mobile Elements Alu, L1 and SVA insertions the reference lacks, each genotyped as one or two copies, and three SVA insertions known to cause disease (FKTN, TAF1, BBS1) read as seen, not seen or not callable (opt-in; needs the BAM) 50
Mosaic Alterations Large gains, losses and copy-neutral loss of heterozygosity on chr1 to chr22 in a share of the blood cells, with the cell fraction (opt-in; mosaic loss of Y and X are step 33's) 51

Pipeline Overview

graph LR
    FASTQ["FASTQ"] --> fastp["fastp<br/><small>QC + trim</small>"]
    fastp --> align["minimap2<br/><small>Alignment</small>"]
    align --> BAM["Sorted BAM"]
    BAM --> DV["DeepVariant<br/><small>SNPs + indels</small>"]
    DV --> VCF["VCF"]

    %% VCF-based analyses
    VCF --> clinvar["ClinVar Screen"]
    VCF --> pharmcat["PharmCAT<br/><small>PGx</small>"]
    pharmcat --> cpic["CPIC<br/><small>Drug recs</small>"]
    VCF --> vep["VEP<br/><small>Annotation</small>"]
    vep --> vcfanno["vcfanno<br/><small>CADD / SpliceAI<br/>REVEL / AlphaMissense</small>"]
    vcfanno --> slivar["slivar<br/><small>Prioritization</small>"]
    vcfanno --> clinical["Clinical Filter"]
    VCF --> cpsr["CPSR<br/><small>Cancer predisposition</small>"]
    VCF --> roh["ROH Analysis"]
    VCF --> prs["PRS<br/><small>Polygenic risk</small>"]
    VCF --> ancestry["Ancestry<br/><small>panel projection</small>"]

    %% BAM-based analyses
    BAM --> manta["Manta<br/><small>SVs</small>"]
    BAM --> delly["Delly<br/><small>SVs</small>"]
    BAM --> cnvpytor["CNVpytor<br/><small>CNVs</small>"]
    manta --> duphold["duphold"]
    duphold --> annotsv["AnnotSV"]
    delly --> annotsv
    cnvpytor --> annotsv
    manta --> consensus["SV Consensus"]
    delly --> consensus
    cnvpytor --> consensus

    BAM --> eh["ExpansionHunter<br/><small>STRs</small>"]
    BAM --> pypgx["pypgx<br/><small>23-gene PGx<br/>+ CYP2D6 SV</small>"]
    %% pypgx reads the VCF too, so it waits for DeepVariant
    VCF --> pypgx
    BAM --> cyrius["Cyrius<br/><small>CYP2D6, once installed</small>"]
    BAM --> hla["T1K<br/><small>HLA (+ KIR, opt-in)</small>"]
    BAM --> paralogs["Parascopy<br/><small>SMN1/SMN2, opt-in</small>"]
    hla --> consensus36["PGx consensus<br/><small>outside calls</small>"]
    pypgx --> consensus36
    cyrius --> consensus36
    consensus36 --> pharmcat
    BAM --> telomere["TelomereHunter"]
    BAM --> coverage["mosdepth<br/>+ indexcov"]
    BAM --> mito["Mutect2<br/><small>Mitochondrial</small>"]
    BAM --> haplo["Haplogrep3<br/><small>mtDNA haplogroup</small>"]
    BAM --> sampleqc["somalier + VerifyBamID2<br/><small>Identity + contamination</small>"]
    BAM --> cram["CRAM archive"]

    %% Reporting
    clinical --> report["HTML Report<br/>+ MultiQC"]
    slivar --> report
    clinvar --> report
    pharmcat --> report
    cpsr --> report
    sampleqc --> report

    %% Styling
    classDef input fill:#0ea5e9,stroke:#0284c7,color:#fff
    classDef core fill:#8b5cf6,stroke:#7c3aed,color:#fff
    classDef analysis fill:#10b981,stroke:#059669,color:#fff
    classDef sv fill:#f59e0b,stroke:#d97706,color:#fff
    classDef annotation fill:#ec4899,stroke:#db2777,color:#fff
    classDef report fill:#ef4444,stroke:#dc2626,color:#fff

    class FASTQ,BAM,VCF input
    class fastp,align,DV core
    class clinvar,pharmcat,cpic,cpsr,eh,roh,prs,ancestry,pypgx,cyrius,hla,paralogs,consensus36,telomere,coverage,mito,haplo,sampleqc,cram analysis
    class manta,delly,cnvpytor,consensus,duphold,annotsv sv
    class vep,vcfanno,slivar,clinical annotation
    class report report

All Steps

# Step Tool Image variable Required?
1 ORA to FASTQ orad orad binary Only for Illumina ORA files
1b QC & Trimming fastp FASTP_IMAGE Recommended
2 Alignment minimap2 + samtools MINIMAP2_IMAGE + SAMTOOLS_IMAGE Yes (if starting from FASTQ)
3 Variant Calling DeepVariant DEEPVARIANT_IMAGE Yes
4 Structural Variants Manta MANTA_IMAGE Recommended
5 SV Annotation AnnotSV ANNOTSV_IMAGE If step 4 run
6 ClinVar Screen bcftools isec, then the cDNA read check (pysam) BCFTOOLS_IMAGE + PYPGX_IMAGE Yes
7 Pharmacogenomics PharmCAT PHARMCAT_IMAGE Yes
8 HLA Typing T1K (+ OptiType with HLA_QC=true) T1K_IMAGE (+ OPTITYPE_IMAGE + SAMTOOLS_IMAGE) Optional
9 STR Expansions ExpansionHunter EXPANSIONHUNTER_IMAGE Recommended
9b STR Annotation Stranger (+ the RFC1 motif count) STRANGER_IMAGE (+ PYPGX_IMAGE when Stranger flags RFC1) If step 9 run
10 Telomere Length TelomereHunter TELOMEREHUNTER_IMAGE Optional
11 ROH Analysis bcftools roh BCFTOOLS_IMAGE Recommended
12 Mito Haplogroup haplogrep3 + haplocheck HAPLOGREP3_IMAGE + HAPLOCHECK_IMAGE Optional (reads step 20's calls when they exist)
13 VEP Annotation VEP VEP_IMAGE Recommended
14 Imputation Prep bcftools BCFTOOLS_IMAGE Optional, opt-in (IMPUTATION=true)
15 SV Quality duphold DUPHOLD_IMAGE If step 4 run
16 Coverage QC indexcov GOLEFT_IMAGE Recommended
16b Coverage Stats mosdepth MOSDEPTH_IMAGE Recommended
17 Cancer Predisposition CPSR PCGR_IMAGE Recommended
18 CNV Calling CNVpytor CNVPYTOR_IMAGE Optional
19 SV Calling (Delly) Delly DELLY_IMAGE Optional
20 Mitochondrial GATK Mutect2 GATK_IMAGE Optional

Post-Processing Steps

These run after the core pipeline completes and combine outputs from earlier steps.

# Step Tool Image variable Required?
21 CYP2D6 Star Alleles Cyrius PYTHON_IMAGE Runs by default once setup.sh --cyrius has installed it (non-commercial licence); skipped otherwise
22 SV Consensus Merge SURVIVOR SURVIVOR_IMAGE If two SV callers ran
23 Clinical Filter bcftools +split-vep BCFTOOLS_IMAGE If step 13 run
24 HTML Report bash + bcftools BCFTOOLS_IMAGE Recommended
25 Polygenic Risk Scores pgsc_calc PGSC_UTILS_IMAGE, PLINK2_IMAGE Exploratory; percentiles need the ancestry panel
26 Ancestry pgsc_calc PGSC_FRAPOSA_IMAGE, PGSC_UTILS_IMAGE After setup.sh --ancestry-panel (runs inside step 25)
26b Fine-Grained Ancestry ADMIXTURE, Beagle, RFMix ADMIXTURE_IMAGE, BEAGLE_IMAGE, RFMIX_IMAGE Opt-in, after setup.sh --ancestry-fine-panel (ADMIXTURE: academic use; RFMix: academic research only)
27 CPIC Recommendations Python + CPIC PYTHON_IMAGE If step 7 run
28 MultiQC Report MultiQC MULTIQC_IMAGE Recommended
29 Somatic Variants GATK Mutect2 GATK_IMAGE Experimental, opt-in (SOMATIC=true)
30 Annotation Enrichment vcfanno VCFANNO_IMAGE If step 13 run
31 Variant Prioritization slivar SLIVAR_IMAGE If step 13 run
32 pypgx Pharmacogenomics pypgx PYPGX_IMAGE Recommended
33 Sample Identity and Contamination somalier + VerifyBamID2 SOMALIER_IMAGE + VERIFYBAMID2_IMAGE Recommended
34 CRAM Archive samtools SAMTOOLS_IMAGE Optional, when the analysis is done
35 Paralog Genes: SMN1/SMN2, HBA1/HBA2 Parascopy (+ HBA1/HBA2 from depth) PARASCOPY_IMAGE Opt-in (TOOLS=...,parascopy)
35b Paralog Genes from PacBio HiFi Reads Paraphase + Python PARAPHASE_IMAGE + PYTHON_IMAGE Opt-in (TOOLS=...,paraphase; PacBio HiFi BAMs only)
36 PGx Consensus Python PYTHON_IMAGE Runs with step 7 when step 8, 21 or 32 ran
37 Y-Chromosome Haplogroup Yleaf YLEAF_IMAGE Opt-in (TOOLS=...,y_haplogroup, after setup.sh --yleaf-data; male samples)
38 Family Analysis GLnexus + somalier + WhatsHap; shared segments: Beagle + hap-ibd GLNEXUS_IMAGE + SOMALIER_IMAGE + WHATSHAP_IMAGE (with BCFTOOLS_IMAGE + PYTHON_IMAGE); BEAGLE_IMAGE + HAP_IBD_IMAGE Opt-in (a pedigree naming two or more samples, each with a gVCF; after setup.sh --sample-qc-data; the shared segments after setup.sh --ancestry-fine-panel)
39 Couple Carrier Screen Python PYTHON_IMAGE Opt-in (a pedigree marking two samples as partners; after step 6 for both)
40 Raw-Data Export Python PYTHON_IMAGE Opt-in (TOOLS=...,consumer_export, after setup.sh --consumer-sites; needs the gVCF)
41 Blood Groups (predicted) samtools + Python SAMTOOLS_IMAGE + PYTHON_IMAGE Opt-in (TOOLS=...,bloodtype; RhD needs the BAM)
42 Traits Panel bcftools + Python + WhatsHap BCFTOOLS_IMAGE + PYTHON_IMAGE + WHATSHAP_IMAGE Recommended (reads the gVCF of step 3; WhatsHap phases from the BAM when a rule needs it)
43 Archaic Ancestry IBDmix + Python IBDMIX_IMAGE + PYTHON_IMAGE Opt-in (runs once setup.sh --archaic installed its table)
44 Ancient Populations smartpca (EIGENSOFT) + bcftools + Python EIGENSOFT_IMAGE + BCFTOOLS_IMAGE + PYTHON_IMAGE Opt-in (TOOLS=...,ancient, after setup.sh --ancient-panel; needs the gVCF)
45 Y-STR Markers HipSTR HIPSTR_IMAGE Opt-in (TOOLS=...,y_str; male samples)
46 Lp(a): LPA KIV-2 Repeat Count samtools + Python SAMTOOLS_IMAGE + PYTHON_IMAGE Opt-in (TOOLS=...,lpa; needs the BAM)
47 Viral DNA in the Unmapped Reads samtools + minimap2 + Python SAMTOOLS_IMAGE + MINIMAP2_IMAGE + PYTHON_IMAGE Opt-in (TOOLS=...,viral, after setup.sh --viral-data; needs the BAM)
48 Symptom-Driven Variant Ranking Exomiser + Python EXOMISER_IMAGE + PYTHON_IMAGE Opt-in (HPO=..., the symptoms as HPO terms; after setup.sh --exomiser-data)
49 Clonal Haematopoiesis (CHIP) Mutect2 + bcftools csq + Python GATK_IMAGE + BCFTOOLS_IMAGE + PYTHON_IMAGE Opt-in (TOOLS=...,chip, after setup.sh --chip-data; needs the BAM)
50 Mobile Element Insertions (Alu, L1, SVA) MEGAnE + samtools + Python MEGANE_IMAGE + SAMTOOLS_IMAGE + PYTHON_IMAGE Opt-in (TOOLS=...,mei, after setup.sh --megane-kmers; needs the BAM)
51 Mosaic Chromosomal Alterations MoChA + Beagle + Python MOCHA_IMAGE + BEAGLE_IMAGE + PYTHON_IMAGE Opt-in (TOOLS=...,mocha, after setup.sh --mocha-data)
52 MUC1 VNTR Frameshifts (VNtyper) VNtyper VNTYPER_IMAGE Opt-in (TOOLS=...,vntyper; needs the BAM)

What a default run covers

A default ./scripts/run-all.sh <sample> <sex> runs 37 numbered steps: 1b and 2 (only when there is no BAM yet), 3 (only when there is no VCF yet), 4, 5, 6, 7, 8, 9, 9b, 10, 11, 12, 13, 15, 16, 16b, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 26b, 27, 28, 30, 31, 32, 33, 36, 42 and 43. Steps 5, 8, 13, 17, 18, 32 and 33 are reported as skipped when their data is not installed (for 33, somalier's sites and VerifyBamID2's panel, which setup.sh installs), 21 when Cyrius is not installed (setup.sh --cyrius; non-commercial licence), 25 when no score file is, 26 (inside the PRS run of step 25) when the ancestry panel is not, 26b when its panel (setup.sh --ancestry-fine-panel) is not, 43 when its table (setup.sh --archaic) is not, 23, 30 and 31 when step 13 did not run, 36 (inside the PharmCAT stage) runs when step 8 or 32 did, and 42 is reported as skipped for a sample that starts from a VCF, unless it has a BAM too and a gVCF beside the VCF (the gVCF run-all.sh passes on). It ends with the summary report (generate-report.sh).

Off unless you ask for them: 35 (TOOLS=...,parascopy, after setup.sh --parascopy-data), 35b (TOOLS=...,paraphase, on a PacBio HiFi BAM), KIR typing in step 8 (KIR=true, after setup.sh --kir-data), OptiType beside T1K in step 8 (HLA_QC=true), 4b (GRIDSS=true), 14 (IMPUTATION=true), 29 (SOMATIC=true), 37 (TOOLS=...,y_haplogroup, after setup.sh --yleaf-data, male samples), 40 (TOOLS=...,consumer_export, after setup.sh --consumer-sites), 41 (TOOLS=...,bloodtype), 44 (TOOLS=...,ancient, after setup.sh --ancient-panel), 45 (TOOLS=...,y_str, male samples), 46 (TOOLS=...,lpa), 47 (TOOLS=...,viral, after setup.sh --viral-data), 48 (HPO=... with the symptoms as HPO terms, after setup.sh --exomiser-data), 49 (TOOLS=...,chip, after setup.sh --chip-data), 50 (TOOLS=...,mei, after setup.sh --megane-kmers), 51 (TOOLS=...,mocha, after setup.sh --mocha-data), 52 (TOOLS=...,vntyper), the alternative callers 3a to 3d (EXTRA_CALLERS=gatk,freebayes,strelka2,octopus) and the caller comparison (BENCHMARK=true). Step 1 (ORA input) and the other alternative scripts (2a, 2b, 3e, 4a, 4c) run only by hand. Step 33 stops the run when somalier's sex from the reads differs from the declared one, as step 16 does (--sex_check warn logs it and goes on). Step 34 runs by hand, or in the Nextflow pipeline with cram_archive in --tools. Steps 38 and 39 work on a family, not one sample, so run-all.sh never runs them: run 38-family.sh and 39-couple.sh by hand with a pedigree, or give the Nextflow pipeline --pedigree (step 39 then runs for two samples the pedigree marks as partners, when clinvar is in --tools).

The Nextflow pipeline runs the same chain from a samplesheet, from FASTQ (steps 1b, 2, 16 and 3) to the report, with the steps above that have a module.

The minimum useful run is steps 2, 3, 6 and 7 (alignment, variant calling, ClinVar, PharmCAT). Runtimes per step and for a whole run are on Hardware and storage requirements.

Alternative Tools (Benchmarking)

No single variant caller is universally best. The pipeline includes alternative tools that output to separate directories so you can compare results without overwriting the defaults.

Script Tool Alternative To Output Directory
02a BWA-MEM2 minimap2 (step 2) aligned_bwamem2/
03a GATK HaplotypeCaller DeepVariant (step 3) vcf_gatk/
03b FreeBayes DeepVariant (step 3) vcf_freebayes/
04a TIDDIT Manta (step 4) sv_tiddit/
03c Strelka2 DeepVariant (step 3) vcf_strelka2/
03d Octopus DeepVariant (step 3) vcf_octopus/
04b GRIDSS Manta (step 4) sv_gridss/
benchmark bcftools isec / hap.py — benchmark/

See docs/benchmarking.md for how to run and interpret results, and docs/tool-rationale.md for why each default was chosen.