Tuesday, April 11, 2017

two-speed genome analysis using R and perl

I have discussed about my speed genome analysis on my previous blog, now am writing the steps how to do that

1. calculate the intergenic distance of ur organism "x" from gtf file using the perl script , I have used the augustus predicted gtf file , the file needs to be modified according to the perl script so that it looks for the particular feature and pattern such as it looks for gene/exon/mRNA feature

sample of the augustus gtf file
 (make sure the last column is mentioned in the gene row)

scaffold_1904   AUGUSTUS        gene    1       775     0.09    +       .       transcript_id "g18705"; gene_id "temp";
scaffold_1      AUGUSTUS        transcript      1       4141    0.05    +       .       g1.t1
scaffold_1      AUGUSTUS        intron  1       180     0.55    +       .       transcript_id "g1.t1"; gene_id "g1";
scaffold_1      AUGUSTUS        intron  2022    3511    0.21    +       .       transcript_id "g1.t1"; gene_id "g1";
scaffold_1      AUGUSTUS        CDS     181     2021    0.09    +       2       transcript_id "g1.t1"; gene_id "g1";
scaffold_1      AUGUSTUS        CDS     3512    4141    0.49    +       0       transcript_id "g1.t1"; gene_id "g1";
scaffold_1      AUGUSTUS        stop_codon      4139    4141    .       +       0       transcript_id "g1.t1"; gene_id "g1";

use the perl script using the gtf file to calculate the intergenic distance between the features, already complimentbed is there but that is not the one which we need, that is for difference purpose when I posted in a github forum to author I came to know the difference, this is the link 

the intergenic distance between the gene file looks like this

2. make a gtf file of Avh's with the location and calculate the intergenic distance for Avh's , intergenic 5 prime and 3 prime distance needs to be calculated the intergenic distance of effector file should look like this

3.  then just Run the R script which is pasted below just by changing the names of the file, if the points or plots are going out the cut off please change the bin size and num of bins before its creating the heatmap don't change after the heatmap is made

if ((max(whole_intergene$fiveprime, na.rm=TRUE)>max(whole_intergene$threeprime, na.rm=TRUE)) == TRUE) { whole_intergene2Bin=whole_intergene$fiveprime} else { whole_intergene2Bin=whole_intergene$threeprime}
BinSteps=round(length(whole_intergene2Bin)/ (NumBins-20) , digits=10)
#### The [2*BinSteps] has been changed to [1*Binsteps it was producing an error after googling the error has been fixed]
TempBinLimits[length(TempBinLimits)+1]=max(whole_intergene2Bin, na.rm=TRUE)
fit<-nls(log(TempBinLimits) ~ a*x + b, start= c(a=0, b=0),algorithm='port',weights=((x-1.0* NumBins)^2))
pred=predict(fit, x)
BinLimits=c(1, round(exp(pred),0), max(whole_intergene2Bin))
xbin=cut(whole_intergene$fiveprime, breaks=c(BinLimits))
ybin=cut(whole_intergene$threeprime, breaks=c(BinLimits))
whole_intergene=cbind(whole_intergene, xbin, ybin, genevalue=rep(1, length (whole_intergene$fiveprime)))
GenValMatrix<-with(whole_intergene, tapply (genevalue, list(xbin, ybin), sum))
zlim = range(as.numeric (unlist(GenValMatrix)) , finite=TRUE)
mypalette<-colorRampPalette(c( "white","darkblue", "forestgreen", "goldenrod1","orangered", "red3", "darkred"), space="rgb")
mycol=mypalette(2*max(GenValMatrix, na.rm=TRUE))
mylabels<-paste(BinLimits[1:length(BinLimits)-1], BinLimits[2:length(BinLimits)], sep="- ", collapse=NULL)
filled.contour(x, y, z=GenValMatrix,plot.title = title(main ="Phytophthora ramorum Pr102 genome",xlab = "five prime intergenic regions", ylab= "three prime intergenic regions", cex.main=0.8, cex.lab=0.8),key.title = title(main ="Number ofgenes", cex.main=0.5,line=1),col=mycol,levels = pretty(zlim, 1*max(GenValMatrix,na.rm=TRUE)),plot.axes={axis(1,at=x, labels=mylabels, las=2,cex.axis=0.5);axis(2,at=y, labels=mylabels,cex.axis=0.5)})
image_name<-paste(as.character(format(Sys.time(),"%Y%m%d%H%M%S")), "_graph", sep="")
png(filename = paste(image_name, ".png", sep=""))
filled.contour(x, y, z=GenValMatrix,col=mycol,levels = pretty(zlim, 2*max(GenValMatrix,na.rm=TRUE)),frame.plot = FALSE,axes = FALSE)
img <- readPNG(paste(image_name, ".png", sep=""))
g <- rasterGrob(img, interpolate=TRUE)'rxrl_whole_intergenic.csv',header=TRUE))
ggplot(data=rxlrData,aes(x=rxlrData$fiveprime,y=rxlrData$threeprime,geom="blank"))+annotation_custom(g,xmin=-Inf,xmax=Inf,ymin=-Inf,ymax=Inf)+coord_fixed(ratio=1)+geom_point(shape=21,fill="red",colour="black",size=2,alpha=0.7,na.rm=FALSE)+scale_y_log10(breaks=BinLimits[2:length(BinLimits)],limits=c(BinLimits[2],BinLimits[NumBins +1]))+scale_x_log10(breaks= BinLimits[2:length(BinLimits)],limits= c(BinLimits[2] ,BinLimits[NumBins +1]))+theme(axis.text.y=element_text(size = 10,vjust=0.5))+theme(axis.text.x=element_text(size=10,vjust=0.5,angle=90))+theme(axis.title.x = element_text(face="bold",size=12))+xlab("five prime intergenic region")+theme(axis.title.y = element_text(face="bold",size=12)) + ylab("threeprime intergenic region")

Below I have put the screen shot for the Phytophthora sojae genome from a sample dataset
 minor edits can be done to make it good !!
This code and concept has been acquired from 

Sunday, March 26, 2017

analysing the alleles from haplotypes on pacbio data

I had been working in pacbio data and when am trying to identify the alleles from haplotypes from diploid assembly, in the very early step itself i got many errors, because i had been following the illumina dataset method like for pacbio data, but the developed tools behaves strange with the data and I got stuck for 3-4 days i googled the maximum and tried various approaches, Finally i posted in the forums and interacted with the GATK developers, they suggested me a simple solution for solving my errors, so those who are working in long reads and want to identify the haplotypes here is my commandline and verified one
[ Any aligners can be used even BLASR initially i was thinking there was a problem with my aligner, but really not] and no need to mark duplicates in case of long reads only for illumina reads its been recommended by the developer, i had reached till the step of HaplotypeCaller so far no error its running smooth, If i change commands or face any problems, will be updated, once the output is ready maybe i can paste some of my output

bwa index 2017_V6_Pr102_assembly.fa
bwa mem -x pacbio 2017_V6_Pr102_assembly.fa /data/results1/STLab/Takao_data/Raw_data/ND886/all_ND886.fastq > aln.sam
samtools view -b -S aln.sam -o aln.bam
samtools sort aln.bam > aln_sorted.bam
samtools index aln_sorted.bam
samtools mpileup -uf 2017_V2_ND886_assembly.fa aln_sorted.bam | /share/apps/bcftools-1.2/bcftools call  -cv - > out.vcf [use bcftools1.2 otherwise its not producing the genotype information]

Use any of your favourite haplotype phaser (whatshap/ hapcut) along with the above produced bam and vcf file

Now u get the phased  alleles from haplotypes u can compare them and these can be used to downstream analysis

Friday, March 24, 2017

Fancy genomics “Iam taking you all to the world of two-Speed genomes concept"

My Phd problem includes the various approaches for solving genome assembly problems. When I was working on oomycetes project, I was attracted by the effector proteins, Evolution, pathogenicity, synteny, transposon, Repeat regions, suddenly the fancy thing which came in the mind after reading an interesting paper from biorxiv that is verticullum genome, a group from Netherlands have sequenced and studied the 2-speed genome concepts among the strains. I was impressed by the work, then I showed the work to my PI even she was impressed by the speed genomes. I work in a collaborative program where exactly my collaborator also was fascinated by the  speed genome work.
Let me explain what is 2 speed genomes?
It was already known that fungi and the plant pathogen genomes comprises of Effector proteins. Which plays an important role in causing pathogenicity to the host. These Effector genes are not randomly distributed across the genomes, tend to be associated with the compartments enriched with repeat sequences and transposons. This led to the ‘two-speed genome’ model in which filamentous pathogen genomes have a bipartite architecture with gene sparse, repeat rich compartments for adaptive evolution.  The unusual genome architecture and occurrence of effector genes in specific genome compartments is a feature that has evolved repeatedly in independent phylogenetic lineages of filamentous pathogens. Genome analyses of P. infestans and three of its sister species revealed uneven evolutionary rates across genomes with genes in repeat-rich regions showing higher rates of structural polymorphisms and positive selection.  Two-speed genome architecture with the effector genes populating the more rapidly evolving sections of the genomes.  Lineages that acquired two-speed genomes have increased survivability — they are less probabe to go extinct compared to lineages with less adaptable genomes, which are more probabe to be purged out of the biota as their hosts develop full resistance or become extinct. In this ‘jump or die’ model, pathogen lineages that have an increased likelihood to produce virulent genotypes on resistant hosts and non-hosts benefit from a macroevolutionary advantage and end up dominating the biota. Several filamentous plant pathogens have evolved by shifting or jumping from one host plant to another.
The information has been shared from this paper a great detailed review by Sophien and Raffaele et al its available here .
For who don’t have access to science direct the same paper is available at biorxiv repository please find the link

Wednesday, October 26, 2016

Structural variation in the genomes

Structural variation:Structural variation is a change or variation which leads to change in the structure of organisms's chromosome. structural variants can be of Insertions, duplication, Inversion and translocation. According to the human genome or people work in genome say that if there is a variant more than of 50 base pairs changes in the human genome of 1%. Its believed that some of the genetic diseases are caused due to the structural variations.whats the difference between the SNP's and structural variation?SNP's are single nucleotide base mutations  which have been validated to be present in more than 1% of the population when a single base differes between the 2 genomes.  These are any mutations which cause a change in the organism's chromosome structure, such as Insertions, deletions, copy number variations, duplications, inversions and translocation.  SNPs and INDELs are about low-level genomic variation. The structural variants which affect the genome at larger scales. Events like gene duplications, tandem repeats, transposon insertions, inversions, and other chromosomal rearrangements. The long read sequencing technology paves the way to understand the structural variants using the split read alignment.[Information from literature Structural variation in two human genomes mapped at single-nucleotide resolution by whole genome de novo assembly  Yingrui Li, et al]  structural variations from short sequencing reads are hampered by one or more of the following limitations: (i) the methods may favor a particular length range of structural variations; (ii) they may favor discovery of particular types of structural variations; (iii) they may be unable to resolve the exact structural variation genotypes and/or breakpoints at single nucleotide resolution; and (iv) because of difficulties mapping reads to the genome, they may not be able to accurately identify complex rearrangements. Paired-end mapping, for example, can only predict insertion breakpoints within a few base pairs of the exact breakpoint position, and it can only detect insertions when the entire sequence is contained within the DNA fragment whose ends are being sequenced; thus, the maximum size of an insertion that can be detected by paired-end mapping is limited by the largest insert size present in a library. Split-read methods, on the other hand, can precisely define a breakpoint and genotype of an insertion, but only when it is shorter than the read length. Thus, studies carried out so far have been of limited completeness, accuracy and/or resolution.
BWA-MEM or BLASR this is a very nice blog discusses about the alignment methods useful of the pacbio long reads.  forum discusses about the split read alignments.
Tips for structural variant analysis:
1. The maximum number of Reads should be mapped in the breakpoints of the chromosome and the coverage should be high.
2. How many Individual reads are supporting the translocation versus supporting assembly for identifying the translocations.
[ I spoke with some of the developers asking about the structural variants of draft pacbio assembly plant pathogen human said completely I can use the tools for predicting , am trying to do for one of the plant pathogen genome]
one of the paper in 2014 talks about all approaches

Tuesday, September 27, 2016

Posters from ECCB2016

I found some interesting poster and thought it will help my friends who are working on the same area , and same type of work going in my lab those are here

Sunday, September 25, 2016

ECCB 2016 Den Hague, Netherlands computational Biologist and Bioinformaticians gatherings at a sweet Dutch country !

I have been to several conferences within India, while ECCB 2016 which happened in Den Hague, from September 3 2016 - September 7 2016. It was the first time for me travel outside India, had butterflies on my stomach the day before I travel.The trip really went well. It was a gathering of computational Biologist and Bioinformaticians over the world. Well I should thank Department of Science and Technology, Government of India for providing me the travel award. The Meeting started with the workshop on discussing Pacbio and Nanopore data. Expertise from the field of nanopore and Pacbio were discussing the problems with the long reads. People were complaining about the "error rates" of these reads, and difficulties in genome assembly of these reads. Had a great opportunity to discuss with the experts. The Nanopore experts were suggesting that Canu assembler can do better when handling the problematic regions in the genome. The Miniasm and Racon assembler also be tried . There were sessions about Irys to create a genome map and align the created  map back to the genome assembly to get a better genome assembly. The structural variants and the comparative genomics are also studied from the graph. Next topic was using Isoseq from pacific Bio-systems to produce a full length transcripts without assembly, followed by promethion and squiggle sequencing system from nanopore technology"Read Until " approach it enables selection of individual DNA molecules for sequencing from a pool of DNA molecules. Then there was a session of Minotour where the base calling of the nanopore reads where done without performing the cloud base calling since there is a dependency of high speed internet. The developers of the tools and technologies were very friendly and gave suggestions on working with the long reads. after the workshop the conference scientific sessions started and many interesting talks where there, I was more interested towards the error correction algorithm development, genome assembly tools, new ortholog prediction tools. Most of the sessions and posters where about the cancer ( a devil), and ENCODE. I can say that 60% of people presented towards cancer transcriptomics and genomics, and 30 % of work in ENCODE, rest where like plant, bacteria, database development, Docking and simulations. The talks and discussions can be retrieved from twitter via #ECCB2016.  I liked the theme of the conference here not only PhD students and Scientists were presenting the work, even people working in companies were also showcased the ongoing work.Some people were very happy and showed interest towards the  poster of my PhD work, since its a plant pathogen. I am more interested towards studying the environmental organisms, pathogens of human, discovering various new species from the environment. About the food it was good, had a varieties of cheese. I had time to visit Amsterdam its a very nice place with a polite people. Visited churches, Museum, had a good canal Boat riding.I had few friends from conference and joined with them and rented a boat and rode over the Amsterdam city. The future ECCB2017 conference will be held in Prague.

Tuesday, September 20, 2016

Analyzing Differential expression analysis data using the tuxedo suite (cummeRbund)

Tuxedo suite comprises of bowtie, tophat, cufflink, cummeRBund and many more accessory tools.

First get your genome fasta file (final genome assembly file).
1. Map your RNAseq fastq files using tophat (if all is well your run will be seamless)
2. Run cufflink over your tophat output file (cufflinks accepted_hits.bam). This run will take a while since cufflink will actually merge the reads into transcripts, isoforms, genes and so on. If your files are large then in a good enough server expect it to run for 8-12 hours.
3. Run cuffmerge: cuffmerge list.txt -> where list.txt carries the names of the files of *_transcripts.gtf files. This will run very fast and will merge all the gene_ids that will be same across all your samples. The output of this file is a merged.gtf file.
4. For running differential expression analysis run the following:
/cuffdiff merged.gtf tophat_HTI1-vs-HTI4/accepted_hits.bam tophat_HTI2-vs-HTI4/accepted_hits.bam tophat_HTI3-vs-HTI4/accepted_hits.bam

This will create a plethora of files, but the following files are the ones you will be proceeding with for cummeRbund for result visualization and generating publication quality images.

For running cummeRbund, get all these files to your working directory

The best option will be to put all of these 11 files into a separate directory inside your working directory: say 'diff_exp'
You can run Rstudio if you like in your windows machine or run R in your server. For running CummeRbund you will need the following packages that you can go ahead and download upfront:

  • RSQLite
  • ggplot2 v0.9.2
  • reshape2
  • plyr
  • fastcluster
  • rtracklayer
  • Gviz
  • BiocGenerics (>=0.3.2)
  • Hmisc
In case you have forgotten how to install R packages go this way: source('') biocLite('cummeRbund') And follow this same protocol for installing other R packages. Once done you can start with setting your working directory using setwd() command.
 For example: setwd("C:/Users/Sucheta/Documents/MyLabIICB/AllCollaborations/NahidAliCollaboration/companion")
Then load the library:


Now read your 11 files using this command
data <-readCufflinks("diff_exp")

This will take a while to read but will create a db file in your source directory. This is your database file.

Now you can plot gene density using the following command:


Or can do a volcano plot of differentially expressed genes using:


As you can see from this file, the different conditions have least difference among themselves.

This will continue in next blog...