Mapping reads to Schistocerca RefSeq genome

workflowr

• Summary
• Checks
• Past versions

Last updated: 2024-01-29

Checks: 7 0

Knit directory: locust-comparative-genomics/

This reproducible R Markdown analysis was created with workflowr (version 1.7.1). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.

---

R Markdown file: up-to-date

Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.

Environment: empty

Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.

Seed: set.seed(20221025)

The command set.seed(20221025) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.

Session information: recorded

Great job! Recording the operating system, R version, and package versions is critical for reproducibility.

Cache: none

Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.

File paths: relative

Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.

Repository version: 7f93ae2

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.

The results in this page were generated with repository version 7f93ae2. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:

Ignored files:
    Ignored:    .DS_Store
    Ignored:    .RData
    Ignored:    .Rhistory
    Ignored:    .Rproj.user/
    Ignored:    analysis/.DS_Store
    Ignored:    analysis/.Rhistory
    Ignored:    data/.DS_Store
    Ignored:    data/.Rhistory
    Ignored:    data/metadata/.DS_Store
    Ignored:    figures/

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.

---

These are the previous versions of the repository in which changes were made to the R Markdown (analysis/3_rna-mapping.Rmd) and HTML (docs/3_rna-mapping.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.

File	Version	Author	Date	Message
Rmd	7f93ae2	Maeva A. TECHER	2024-01-29	wflow_publish("analysis/3_rna-mapping.Rmd")
html	c11436d	Maeva A. TECHER	2024-01-24	Build site.
html	1b09cbe	Maeva A. TECHER	2024-01-24	remove
html	71b33ec	Maeva A. TECHER	2023-12-18	Build site.
Rmd	53877fa	Maeva A. TECHER	2023-12-18	add pages

---

NEED TO EDIT AND CHANGE with map to own genome and gregaria

Load R libraries (install first from CRAN or Bioconductor)

library("knitr")
library("rmdformats")
library("tidyverse")
library("DT")  # for making interactive search table
library("plotly") # for interactive plots
library("ggthemes") # for theme_calc
library("reshape2")

## Global options
options(max.print="10000")
knitr::opts_chunk$set(
    echo = TRUE,
    message = FALSE,
    warning = FALSE,
    cache = FALSE,
    comment = FALSE,
    prompt = FALSE,
    tidy = TRUE
)
opts_knit$set(width=75)

Short Reads mapping and Gene Quantification

STAR only

We used STAR for mapping reads to either 1) their own species reference genome or 2) an alternate sister reference genome. The pipeline is the same, except that the code will change index.

We index each genome using the following bash script:

#!/bin/bash

##NECESSARY JOB SPECIFICATIONS
#SBATCH --job-name=STARindex        #Set the job name to "JobExample4"
#SBATCH --time=06:00:00         #Set the wall clock limit to 1hr and 30min
#SBATCH --ntasks=2              #Request 1 task
#SBATCH --cpus-per-task=4       #Request 1 task
#SBATCH --mem=30G              #Request 100GB per node

module purge
module load GCC/11.2.0 STAR/2.7.9a

REFDIR="/scratch/user/maeva-techer/refgenomes/locusts_complete/index_rRNA_gregaria/STAR"
GENOME="/scratch/user/maeva-techer/refgenomes/locusts_complete/rRNA_gregaria.fasta"
ANNOTATION="/scratch/user/maeva-techer/refgenomes/locusts_complete/rRNA_gregaria.gtf"

STAR --runMode genomeGenerate --runThreadN 8 --genomeDir $REFDIR --genomeFastaFiles $GENOME --genomeSAindexNbases 10 --sjdbGTFfile $ANNOTATION --sjdbGTFfeatureExon exon --sjdbOverhang 149

The parameters were chosen for the following reasons:
--runMode genomeGenerate indicates we are in the mode to build genome index
--genomeSAindexNbases 22 should be min(18,log2[max(GenomeLength/NumberOfReferences,ReadLength)]) according to the manual. We have a genome of ~9000000000 and with around 1475 contigs so our value is around 22.
--sjdbGTFfile $ANNOTATION --sjdbGTFfeatureExon exon we use these parameters to indicates that we want the annotation file to be already accounted for.
--sjdbOverhang 149 this should be ideally (mate_length -1), and our reads are PE150.

########################################
# Snakefile rule
########################################

#Ahead of the alignment I will build independently the index for STAR, HiSat2 and Segemehl

rule STAR_align:
    input:
        index = REFdir + "/locusts_complete/index_{genome}/STAR",
        annotation = REFdir + "/locusts_complete/{genome}.gtf",
        read1 = OUTdir + "/trimming/{locust}_trim1P_1.fastq.gz",
        read2 = OUTdir + "/trimming/{locust}_trim2P_2.fastq.gz"
    params:
        prefix = OUTdir + "/alignment/STAR/{genome}/{locust}_"
    output:
        OUTdir + "/alignment/STAR/{genome}/{locust}_Aligned.sortedByCoord.out.bam"
    shell:
        """
        module load GCC/11.2.0 STAR/2.7.9a
        STAR --runThreadN 8 --genomeDir {input.index} --readFilesIn {input.read1} {input.read2} --outFileNamePrefix {params.prefix} --readFilesCommand zcat --genomeLoad NoSharedMemory --twopassMode Basic --sjdbGTFfile {input.annotation} --sjdbScore 2 --sjdbOverhang 149 --outFilterMultimapNmax 20 --alignSJoverhangMin 8 --alignSJDBoverhangMin 1 --outFilterMatchNmin 50 --outFilterMismatchNmax 999 --outFilterMismatchNoverReadLmax 0.04 --alignIntronMin 20 --alignIntronMax 1000000 --alignMatesGapMax 1000000 --outSAMunmapped Within --outFilterType BySJout --outSAMattributes NH HI AS NM MD --outSAMtype BAM SortedByCoordinate --quantMode TranscriptomeSAM GeneCounts --quantTranscriptomeBan IndelSoftclipSingleend --limitBAMsortRAM 32000000000 --seedSearchStartLmax 30 --outFilterScoreMinOverLread 0 --outFilterMatchNminOverLread 0
        """

        
########################################
# Parameters in the cluster.json file
########################################

    "STAR_align":
    {
        "cpus-per-task" : 10,
        "partition" : "medium",
        "ntasks": 1,
        "mem" : "100G",
        "time": "0-08:00:00"
    }

The parameters were chosen as follow:
--runThreadN 8 indicates that we run the mapping process using 8 threads.
--genomeDir {input.index} indicates where the genome index is located.
--readFilesIn {input.read1} {input.read2} path to the PE reads.
--outFileNamePrefix {params.prefix} is the prefix of our output file.
--readFilesCommand zcat signifies that we our reads are compressed and need to be read as fastq.gz files.
--genomeLoad NoSharedMemory
--twopassMode Basic indicates that we wish to use a two-passes mapping mode that will first extract the junctions and insert them into the genome index and re-map everything during a 2nd pass. The option basic allows us to perform this on multiple files in parallel.
--sjdbGTFfile {input.annotation} indicates the path of our annotation file.
--sjdbScore 2
--sjdbOverhang 149 is the same parameter used for building our index.
--outFilterMultimapNmax 20
--alignSJoverhangMin 8
--alignSJDBoverhangMin 1
--outFilterMatchNmin 50
--outFilterMismatchNmax 999
--outFilterMismatchNoverReadLmax 0.04
--alignIntronMin 20
--alignIntronMax 1000000
--alignMatesGapMax 1000000
--outSAMunmapped Within
--outFilterType BySJout
--outSAMattributes NH HI AS NM MD indicates that the temporary .sam alignment file should contain headers. --outSAMtype BAM SortedByCoordinate indicates that the output should be in a .bam format and sorted by coordines.
--quantMode TranscriptomeSAM GeneCounts indicates that we wish to have two outputs, one with the Read Count for each gene and one with the gene aligned to the transcriptome only.
--quantTranscriptomeBan IndelSoftclipSingleend
--limitBAMsortRAM 32000000000
--seedSearchStartLmax 30
--outFilterScoreMinOverLread 0
--outFilterMatchNminOverLread 0

After mapping, we obtained alignment statistics from the *_Log.final.out file and filled out the metadata table with it.

grep 'Number of input reads' *_Log.final.out
grep 'Average input read length' *_Log.final.out
grep 'Uniquely mapped reads number' *_Log.final.out
grep 'Number of reads mapped to multiple loci' *_Log.final.out
grep 'Number of reads mapped to too many loci' *_Log.final.out
grep 'Number of reads unmapped: too many mismatches' *Log.final.out
grep 'Number of reads unmapped: too short' *Log.final.out
grep 'Number of reads unmapped: other' *Log.final.out

STAR only

Quantification using GeneCount

We can note that the option --quantMode GeneCounts from STAR produces the same output as the htseq-count tool if we used the –-sjdbGTFfile option.

In the output file {locust}_ReadsPerGene.out.tab we can obtain the read counts per gene depending if our data is unstranded (column 2) or stranded (columns 3 and 4).

column 1: gene ID
column 2: counts for unstranded RNA-seq.
column 3: counts for the 1st read strand aligned with RNA
column 4: counts for the 2nd read strand aligned with RNA (the most common protocol nowadays)

For our pilot S. gregaria project, we know we used Illumina stranded kit but to check we can with the following code:

grep -v "N_" {locust}_ReadsPerGene.out.tab | awk '{unst+=$2;forw+=$3;rev+=$4}END{print unst,forw,rev}'

#or as a loop
for i in *_ReadsPerGene.out.tab; do echo $i; grep -v "N_" $i | awk '{unst+=$2;forw+=$3;rev+=$4}END{print unst,forw,rev}'; done

In a stranded library preparation protocol, there should be a strong imbalance between number of reads mapped to known genes in forward versus reverse strands. This is what we observe for example on S. cancellata libraries here.

PREFERRED OPTION: We need to extract in our case the 1st and 4th columns for each file.

########################################
# Snakefile rule
########################################

#either ran the following rule
rule reads_count:
        input:
                readtable = OUTdir + "/alignment/STAR2/{locust}_ReadsPerGene.out.tab",
        output:
                OUTdir + "/DESeq2/counts_4thcol/{locust}_counts.txt"
        shell:
                """
                cut -f1,4 {input.readtable} | grep -v "_" > {output}  
                """

#or simply this loop for less core usage
# for i in $SCRATCH/locust_phase/data/alignment/STAR/*ReadsPerGene.out.tab; do echo $i; cut -f1,4 $i | grep -v "_" > $SCRATCH/locust_phase/data/DESeq2/counts_4thcol/`basename $i ReadsPerGene.out.tab`counts.txt; done

ALTERNATIVE OPTION: We can also build a single matrix of expression with all individuals targeted. Below is the example for S. piceifrons:

paste SPICE_*_ReadsPerGene.out.tab | grep -v "_" | awk '{printf "%s\t", $1}{for (i=4;i<=NF;i+=4) printf "%s\t", $i; printf "\n" }' > tmp
sed -e "1igene_name\t$(ls SPICE_*ReadsPerGene.out.tab | tr '\n' '\t' | sed 's/_ReadsPerGene.out.tab//g')" tmp > raw_counts_piceifrons_matrix.txt

paste *ReadsPerGene.out.tab | grep -v “” | awk ‘{printf “%s, $1}{for (i=4;i<=NF;i+=4) printf”%s, $i; printf “” }’ > tmp

ls *.tab | awk ‘BEGIN{ORS=““;print”gene name}{print $0”}END{print “”}’| sed ’s/_ReadsPerGene.out.tab//g’ > raw_count_ALB.txt ; cat tmp >> raw_count_ALB.txt

This document was written in R Markdown, and translated into html using the R package knitr. Press the buttons labelled Code to show or hide the R code used to produce each table, plot or statistical result. You can also select Show all code at the top of the page.

Short Reads mapping and Gene Quantification

STAR mapping to own species

# Load our SRA metadata table
metaseq <- read_table("data/metadata/RNAseq_modified_METADATA2022.txt", col_names  = TRUE,  guess_max = 5000)

## Create an interactive search table
metaseq %>%
    datatable(extensions = "Buttons", options = list(dom = "Blfrtip", buttons = c("copy",
        "csv", "excel"), lengthMenu = list(c(10, 20, 50, 100, 200, -1), c(10, 20,
        50, 100, 200, "All"))))

mycol_species <- c("green", "deeppink", "orange", "orange2", "blue2", "red2", "yellow2")

## READS AVERAGE colored by STATUS
eren <- ggplot(metaseq, aes(x=Map_SUM, y = Inputtrim_reads, color = Species, label = SampleID))
eren <- eren + geom_point(size=2, alpha =0.7)
eren <- eren + scale_color_manual(values = mycol_species)
eren <- eren + theme_calc()
eren <- eren + geom_hline(yintercept=30000000, linetype="dotted", color = "green3")
eren <- eren + geom_hline(yintercept=50000000, linetype="dotted", color = "green3")
eren <- eren + geom_vline(xintercept=80, linetype="dotted", color = "blue2")
eren <- eren +  xlim(0,100)
options(scipen=20) #to remove the scientific annotation of the axis

## make an interactive version of the scatter plot
attacktitan <- ggplotly(eren)
attacktitan

Figure XX: Interactive plot of the mapping rate success of each sample against their respective reference genome.

We added the green thresholds to indicate how many reads are recommended by Illumina (lower end and optimal). The blue line demonstrates where the mapping ratio could be considered not contaminated.

STAR mapping to alternate species

To gregaria

## READS AVERAGE colored by STATUS
mikasa <- ggplot(metaseq, aes(x=Map_SUM, y = Inputtrim_reads, color = Species, label = SampleID))
mikasa <- mikasa + geom_point(size=2, alpha =0.7)
mikasa <- mikasa + scale_color_manual(values = mycol_species)
mikasa <- mikasa + theme_calc()
mikasa <- mikasa + geom_hline(yintercept=30000000, linetype="dotted", color = "green3")
mikasa <- mikasa + geom_hline(yintercept=50000000, linetype="dotted", color = "green3")
mikasa <- mikasa + geom_vline(xintercept=80, linetype="dotted", color = "blue2")
mikasa <- mikasa +  xlim(0,100)
options(scipen=20) #to remove the scientific annotation of the axis

## make an interactive version of the scatter plot
attacktitan <- ggplotly(eren)
attacktitan

Figure XX: Interactive plot of the mapping rate success of each sample against gregaria genome.

We added the green thresholds to indicate how many reads are recommended by Illumina (lower end and optimal). The blue line demonstrates where the mapping ratio could be considered not contaminated.

To piceifrons

To cancellata

To americana

To cubense

To nitens

Session information

sessionInfo()

FALSE R version 4.3.1 (2023-06-16)
FALSE Platform: x86_64-apple-darwin20 (64-bit)
FALSE Running under: macOS Sonoma 14.1.2
FALSE 
FALSE Matrix products: default
FALSE BLAS:   /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libRblas.0.dylib 
FALSE LAPACK: /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0
FALSE 
FALSE locale:
FALSE [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
FALSE 
FALSE time zone: America/Chicago
FALSE tzcode source: internal
FALSE 
FALSE attached base packages:
FALSE [1] stats     graphics  grDevices utils     datasets  methods   base     
FALSE 
FALSE other attached packages:
FALSE  [1] reshape2_1.4.4   ggthemes_5.0.0   plotly_4.10.4    DT_0.31         
FALSE  [5] lubridate_1.9.3  forcats_1.0.0    stringr_1.5.1    dplyr_1.1.4     
FALSE  [9] purrr_1.0.2      readr_2.1.5      tidyr_1.3.0      tibble_3.2.1    
FALSE [13] ggplot2_3.4.4    tidyverse_2.0.0  rmdformats_1.0.4 knitr_1.45      
FALSE [17] workflowr_1.7.1 
FALSE 
FALSE loaded via a namespace (and not attached):
FALSE  [1] gtable_0.3.4       xfun_0.41          bslib_0.6.1        htmlwidgets_1.6.4 
FALSE  [5] processx_3.8.3     callr_3.7.3        tzdb_0.4.0         vctrs_0.6.5       
FALSE  [9] tools_4.3.1        ps_1.7.5           generics_0.1.3     fansi_1.0.6       
FALSE [13] pkgconfig_2.0.3    data.table_1.14.10 lifecycle_1.0.4    compiler_4.3.1    
FALSE [17] git2r_0.33.0       munsell_0.5.0      getPass_0.2-4      httpuv_1.6.13     
FALSE [21] htmltools_0.5.7    sass_0.4.8         yaml_2.3.8         lazyeval_0.2.2    
FALSE [25] later_1.3.2        pillar_1.9.0       jquerylib_0.1.4    whisker_0.4.1     
FALSE [29] cachem_1.0.8       tidyselect_1.2.0   digest_0.6.34      stringi_1.8.3     
FALSE [33] bookdown_0.37      rprojroot_2.0.4    fastmap_1.1.1      grid_4.3.1        
FALSE [37] colorspace_2.1-0   cli_3.6.2          magrittr_2.0.3     utf8_1.2.4        
FALSE [41] withr_2.5.2        scales_1.3.0       promises_1.2.1     timechange_0.2.0  
FALSE [45] rmarkdown_2.25     httr_1.4.7         hms_1.1.3          evaluate_0.23     
FALSE [49] viridisLite_0.4.2  rlang_1.1.3        Rcpp_1.0.12        glue_1.7.0        
FALSE [53] formatR_1.14       rstudioapi_0.15.0  jsonlite_1.8.8     R6_2.5.1          
FALSE [57] plyr_1.8.9         fs_1.6.3