nf-core/callingcards

Edit

Introduction

This document describes the output produced by the pipeline. Most of the plots are taken from the MultiQC report, which summarises results at the end of the pipeline.

In the output directory of a given run of this pipeline, there will be a subdirectory for each sample in the samplesheet where the directory name is the sample name (first field in the samplesheet).

For example, if you are running mammals data and your samplesheet has two samples, like so:

sample,fastq_1,fastq_2,barcode_details
AAV9_cortex,data/AAV9_1-1_cortex_L004_R1.fastq.gz,,barcode_details.json
AAV9_hindbrain,data/AAV9_1-1_hindbrain_L004_R1.fastq.gz,,barcode_details.json

Then the output directory would have the following structure:

example_results/
├── AAV9_cortex
│   ├── hops
│   └── sequence
├── AAV9_hindbrain
│   ├── hops
│   └── sequence
├── multiqc
└── pipeline_info
 

If you are running yeast data, then the samplesheet would look the same, except fastq_2 must be present. The output directory will be structured similarly, except that within each sample subdirectory, there will be multiple qbeds/qc files for each of the TFs included in the multiplexed library. Mammals libraries, on the other hand, are not multiplexed and contain data for only 1 TF.

For either datatype, you can include the parameters --save_genome_intermediate, --save_sequence_intermediate and/or --save_alignment_intermeidate to save the intermediate files generated at each processing stage.

The directories listed below will be created in the results directory after the pipeline has finished. All paths are relative to the top-level results directory.

Pipeline overview

The pipeline is built using Nextflow and processes data using the following steps:

1. Prepare the Genome
   • Mask Fasta
   • Concat Fasta
   • GTF2BED
   • Aligner Index
   • Genome Index
2. Prepare the Reads
   • FastQC
   • Divide Fastq
   • UMITools Extract (mammals only)
   • Demultiplex (yeast only)
   • Concat Fastqs (yeast only)
   • Concat QC (for mammals, this occurs in the Align step)
   • Trimmomatic
3. Align
   • Align
4. Count Hops
   • Count Hops
   • Alignment QC and Metrics
5. Present QC data
6. Pipeline information

Mask Genome

This step does not output, except in the work-dir. However, if a bed file is provided to the regions_mask parameter, then it will be used to mask the genome with bedtools maskfasta. The masked genome will be used for the rest of the pipeline.

Concatenate Additional Sequences

This step does not output, except in the work-dir. However, if a fasta file is provided to the the additional_sequence parameter, it will be appended to the (possibly masked) genome and used for all subsequent steps.

GTF2BED

Translation of the gtf file to a bed file. This is only output if save_genome_intermediate is true

Aligner Index

The index produced by the aligner of your choice. This will only be saved if save_genome_intermediate is true

Genome Index

Samtools index of the fasta file (after appending any additional sequences)

FastQC

FastQC gives general quality metrics about your sequenced reads. It provides information about the quality score distribution across your reads, per base sequence content (%A/T/G/C), adapter contamination and overrepresented sequences. For further reading and documentation see the FastQC help pages.

NB: The yeast workflow runs FastQC both before and after processing (including demultiplexing) the reads. The mammalian workflow runs FastQC only on un-processed reads

Divide Fastq

seqkit split2 is used to divide each of the samples’ fastq files into parts for parallel processing. For the yeast pipeline, these parts are concatenated back together by the respective TFs while for mammals, the divided parts are aligned and processed in parallel

UMItools Extract

UMItools extract is used to extract barcode and other CallingCards specific sequences from single or paired-end reads.

Demultiplex

This step is only run for the yeast pipeline. It calls the callingCardsTools utility parse fastq and uses the barcod_details json file to demultiplex the reads according to the TF barcodes. This step will typically be performed on split fastq files (for parallel processing) and not output unless save_sequence_itermediate is set.

Concat Fastq

This step is only run for the yeast pipeline. It concatenates the splits of each demultiplexed TF fastq file so that there is a single fastq file for each TF for each sample.

Concat QC

This is performed in both the yeast and mammals workflows, but at different points. For yeast, this is performed in the Prepare Reads subworkflow while in mammals it is performed after alignment in the Process Alignment subworkflow.

Trimmomatic

Trimmomatic is used to trim reads after extracting known non-genomic Calling Cards specific sequence

Align

The user may select any one of the following:

• bwa aln
• bwa mem2
• bowtie
• bowtie2

If save_intermediate is set, and the selected Aligner generates an index, the index will be saved at the main level of the outdir in a directory called genome. In each sample subdirectory, there will also be a directory named by the name of the selected aligner which will store the aligner’s output.

Count Hops

Alignment Metrics

Picard, RSeQC and Samtools provide alignment level quality control metrics. Qualifying alignments are counted as ‘hops’ of a given transcription factor and those hops are quantified in a qBed file. Hop level QC metrics are also generated.

Picard

Picard CollectMultipleMetrics gathers multiple QC metrics from alignments files. These are performed on the post-processed alignments files, after they have been partitioned into passing and failing alignments.

SAMtools

The original BAM files generated by the selected alignment algorithm are further processed with SAMtools to sort them by coordinate, for indexing, as well as to generate read mapping statistics.

RSeQC

RSeQC is a package of scripts designed to evaluate the quality of RNA-seq data. This pipeline runs several, but not all RSeQC scripts. You can tweak the supported scripts you would like to run by adjusting the --rseqc_modules parameter which by default will run all of the following: bam_stat.py, inner_distance.py, infer_experiment.py, junction_annotation.py, junction_saturation.py,read_distribution.py and read_duplication.py.

The majority of RSeQC scripts generate output files which can be plotted and summarised in the MultiQC report.

Infer experiment

A number of RSeQC modules are available. Only the read distribution

RSeQC documentation: infer_experiment.py

Read distribution

This tool calculates how mapped reads are distributed over genomic features. A good result for a standard RNA-seq experiments is generally to have as many exonic reads as possible (CDS_Exons). A large amount of intronic reads could be indicative of DNA contamination in your sample but may be expected for a total RNA preparation.

RSeQC documentation: read_distribution.py

BAM stat

This script gives numerous statistics about the aligned BAM files.

MultiQC plots each of these statistics in a dot plot. Each sample in the project is a dot - hover to see the sample highlighted across all fields.

RSeQC documentation: bam_stat.py

MultiQC

MultiQC is a visualization tool that generates a single HTML report summarising all samples in your project. Most of the pipeline QC results are visualised in the report and further statistics are available in the report data directory.

Results generated by MultiQC collate pipeline QC from supported tools e.g. FastQC. The pipeline has special steps which also allow the software versions to be reported in the MultiQC output for future traceability. For more information about how to use MultiQC reports, see http://multiqc.info.

Pipeline information

Nextflow provides excellent functionality for generating various reports relevant to the running and execution of the pipeline. This will allow you to troubleshoot errors with the running of the pipeline, and also provide you with other information such as launch commands, run times and resource usage.