Sovereign 8s.is mirror of standardgalactic/denovo_calling
  • Python 99.7%
  • Shell 0.3%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
michelle noyes 0153592514
Update README.md
Add DOI badge!
2026-02-22 18:27:48 -08:00
callable_genome Initial commit 2025-07-25 15:02:44 -07:00
indel_calling Initial commit 2025-07-25 15:02:44 -07:00
manifests Initial commit 2025-07-25 15:02:44 -07:00
mnm_validation Initial commit 2025-07-25 15:02:44 -07:00
phasing Initial commit 2025-07-25 15:02:44 -07:00
sex_chromosomes Initial commit 2025-07-25 15:02:44 -07:00
snv_calling Initial commit 2025-07-25 15:02:44 -07:00
LICENSE Create LICENSE 2025-07-25 15:06:00 -07:00
README.md Update README.md 2026-02-22 18:27:48 -08:00

Long-read sequencing of trios reveals increased germline and postzygotic mutation rates in repetitive DNA

DOI badge

This repository contains six Snakemake pipelines to call, phase, and validate de novo variants and to calculate callable genome size on the T2T-CHM13 v2.0 reference. Pipelines cover autosomes and sex chromosomes and are intended for use with multi-platform sequencing (HiFi, ONT, Illumina).

For full methodological details, see our arXiv submission: link to arXiv manuscript.

Notes:

  • Each pipeline has its own config.json specifying reference paths and manifests. Update these paths for your environment.
  • Example manifests are provided under manifests/.

Pipelines

Pipeline-specific expected outputs and typical runtimes (assuming resources encoded in each Snakefile):

  • callable_genome/

    • Expected output: BED files of accessible regions and per-sample callable genome size TSVs under results/.
    • Typical runtime: 2–3 weeks.
  • snv_calling/

    • Expected output: validated de novo SNVs under results/ (for example, results/{family}_{sample}.snv_calls.tsv).
    • Typical runtime: 4–8 hours.
  • indel_calling/

    • Expected output: validated de novo indels under validated_calls/.
    • Typical runtime: 4–8 hours.
  • phasing/

    • Expected output: per-sample haplotype assignments and origin (postzygotic/germline) integration under results/.
    • Typical runtime: 2–4 hours.
  • mnm_validation/

    • Expected output: validated de novo multinucleotide mutations under validated_mnms/.
    • Typical runtime: 1–2 days.
  • sex_chromosomes/

    • Expected output: validated sex-chromosome DNMs for males and females (SNVs and indels) under each subpipeline’s outputs.
    • Typical runtime: 2–4 hours.

Notes and tips:

  • Some pipelines consume outputs of others. For example, phasing/ expects SNV results from snv_calling/ (see phasing/Snakefile), and indel_calling/ expects prior indel candidates generated by snv_calling/; ensure upstream outputs exist or paths are adjusted accordingly.

System requirements

  • Operating system: Pipelines were written and tested on Ubuntu 22.04. Other operating systems have not been tested.
  • Required software:
    • Miniconda v4.12.0 (to enable per-rule conda environments).
    • Snakemake 4.12.0 (other versions have not been tested with these workflows).
    • The pipelines declare software dependencies in their envs/env.yaml files and will create/activate those environments automatically when run with --use-conda.
  • Hardware: No non-standard hardware is required. Jobs can be run locally or on a cluster. Example commands below include an SGE/DRMAA submission alias; cluster use is optional.

Installation

The steps below install Miniconda and Snakemake on Ubuntu 22.04 using the Bioconda channels.

# Install Miniconda 4.12.0 (Linux x86_64)
wget https://repo.anaconda.com/miniconda/Miniconda3-py39_4.12.0-Linux-x86_64.sh -O miniconda.sh
bash miniconda.sh -b -p "$HOME/miniconda3"
eval "$($HOME/miniconda3/bin/conda shell.bash hook)"
conda init bash  # or 'zsh' if you use zsh

# Configure Bioconda channels
conda config --add channels defaults    
conda config --add channels bioconda
conda config --add channels conda-forge

# Create an environment with Snakemake 4.12.0
conda create -y -n snakemake-4.12.0 snakemake=4.12.0
conda activate snakemake-4.12.0

# Optional (only if submitting via DRMAA to an SGE grid):
# conda install -y python-drmaa
  • Environment provisioning: When you run any pipeline with --use-conda, Snakemake will automatically create and manage per-rule environments from the envs/env.yaml files.
  • Typical install time: ~10–20 minutes on a standard desktop with broadband internet.

Demo Instructions

All commands below are executed from a pipeline directory (for example, snv_calling/ or callable_genome/). Before running, edit that pipeline’s config.json to point to your reference, manifests, and other inputs. Example manifests are in manifests/.

You can run locally or submit to an SGE cluster. To submit via DRMAA on SGE, define this alias (from the repo root or any pipeline directory):

alias snakesub='mkdir -p log; snakemake --ri --jobname "{rulename}.{jobid}" --drmaa " -V -cwd -j y -o ./log -e ./log -l h_rt={resources.hrs}:00:00 -l mfree={resources.mem}G -pe serial {threads} -w n -S /bin/bash" -w 60'

Then submit with:

snakesub -j [number of jobs] --use-conda
  • Cluster integration beyond SGE/DRMAA is not provided here. If running on a different scheduler, adapt the submission command accordingly.

Although we cannot provide demo data from our samples, you can test these pipelines using trio data from Genome in a Bottle (GIAB) by following these instructions for the GIAB Ashkenazi trio (HG002—son, HG003—father, HG004—mother), all aligned to GRCh38.

  1. Download the BAMs

You will need aligned HiFi, ONT, and Illumina BAMs for each memeber of the trio. Run the following commands to fetch the data into your directory:


# HG002 (son)
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG002_NA24385_son/NIST_NA24385_HG002_HiSeq300x/HG002_HiSeq300x_GRCh38.bam
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG002_NA24385_son/NIST_NA24385_HG002_HiSeq300x/HG002_HiSeq300x_GRCh38.bam.bai

wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG002_NA24385_son/PacBio_CCS_15kb/HG002_PacBio_CCS_15kb_GRCh38.bam
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG002_NA24385_son/PacBio_CCS_15kb/HG002_PacBio_CCS_15kb_GRCh38.bam.bai

wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG002_NA24385_son/ONT_UL/HG002_ONT_UL_GRCh38.bam
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG002_NA24385_son/ONT_UL/HG002_ONT_UL_GRCh38.bam.bai

# HG003 (father)
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG003_NA24149_father/Illumina300XWGS/HG003_Illumina_300x_GRCh38.bam
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG003_NA24149_father/Illumina300XWGS/HG003_Illumina_300x_GRCh38.bam.bai

wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG003_NA24149_father/PacBio_CCS_15kb/HG003_PacBio_CCS_15kb_GRCh38.bam
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG003_NA24149_father/PacBio_CCS_15kb/HG003_PacBio_CCS_15kb_GRCh38.bam.bai

wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG003_NA24149_father/ONT_UL/HG003_ONT_UL_GRCh38.bam
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG003_NA24149_father/ONT_UL/HG003_ONT_UL_GRCh38.bam.bai

# HG004 (mother)
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG004_NA24143_mother/Illumina300XWGS/HG004_Illumina_300x_GRCh38.bam
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG004_NA24143_mother/Illumina300XWGS/HG004_Illumina_300x_GRCh38.bam.bai

wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG004_NA24143_mother/PacBio_CCS_15kb/HG004_PacBio_CCS_15kb_GRCh38.bam
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG004_NA24143_mother/PacBio_CCS_15kb/HG004_PacBio_CCS_15kb_GRCh38.bam.bai

wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG004_NA24143_mother/ONT_UL/HG004_ONT_UL_GRCh38.bam
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/data/AshkenazimTrio/HG004_NA24143_mother/ONT_UL/HG004_ONT_UL_GRCh38.bam.bai
  1. Download the VCFs
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/release/AshkenazimTrio/HG002_GRCh38/latest/HG002_GRCh38_1_22_v4.2.1_benchmark.vcf.gz
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/release/AshkenazimTrio/HG002_GRCh38/latest/HG002_GRCh38_1_22_v4.2.1_benchmark.vcf.gz.tbi

wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/release/AshkenazimTrio/HG003_GRCh38/latest/HG003_GRCh38_1_22_v4.2.1_benchmark.vcf.gz
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/release/AshkenazimTrio/HG003_GRCh38/latest/HG003_GRCh38_1_22_v4.2.1_benchmark.vcf.gz.tbi

wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/release/AshkenazimTrio/HG004_GRCh38/latest/HG004_GRCh38_1_22_v4.2.1_benchmark.vcf.gz
wget ftp://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/release/AshkenazimTrio/HG004_GRCh38/latest/HG004_GRCh38_1_22_v4.2.1_benchmark.vcf.gz.tbi
  1. Combine VCFs into one for the trio

Although GIAB does not provide separate GATK and DeepVariant files like these pipelines expect, you can use just one VCF by listing the same file twice in vcf_manifest.tab. Generate this file by merging variant data from the three VCFs.

bcftools merge -m all -Oz \
   -o GIAB_trio_GRCh38_merged.vcf.gz \
   HG002.vcf.gz HG003.vcf.gz HG004.vcf.gz

tabix -p vcf GIAB_trio_GRCh38_merged.vcf.gz
  1. Reference Genome (GRCh38)

Make sure to use GRCh38 as the reference - you will need to update this in your config.json files. You can download the same version used by GIAB:

wget ftp://ftp-trace.ncbi.nlm.nih.gov/giab/release/references/GRCh38_no_alt_analysis_set.fna.gz
gunzip GRCh38_no_alt_analysis_set.fna.gz
samtools faidx GRCh38_no_alt_analysis_set.fna

Once you have obtained these data, create manifests that match those in manifests/, update your config.json files, and you should be good to go! Happy DNM hunting!

Reproduction Instructions

  • The original data used for analysis are available in the SFARI Base under accession SFARI_DS714840 and in the NIH NDA Collection ID 3780. Access may require data use agreements through those portals.
  • Update each pipeline’s config.json to point to your local copies of the reference (T2T-CHM13v2.fasta), manifests (for example, manifests/bam_manifest.tab, manifests/vcf_manifest.tab, manifests/hifi_sources.tsv), and any other required inputs (for example, TRF/RepeatMasker BEDs).

Contact

Please open an issue or pull request with questions or improvements.