error running presplit_map.py
4
0
Entering edit mode
Eichan • 0
@eichan-14174
Last seen 6.5 years ago
Japan/Tokyo

Hi,

May I ask a naive question on what could be wrong when I try to run the chimeric alignment using presplit_map.py from diffHic package. It seems an error from samtools but I couldn't figure out what causes it. I updated my samtools version from 1.31 to 1.6 and I still having the same problem. I thank you in advance for any help.

Command

python /pathto/presplit_map.py \
    -G /pathto/bowtie2_idx/mm10 \
    -1 SRR_1.fastq \
    -2 SRR_2.fastq \
    --sig AAGCTAGCTT \
    -o SRR.bam

 

Error

Traceback (most recent call last):
  File "/home/R/x86_64-pc-linux-gnu-library/3.4/diffHic/python/presplit_map.py", line 197, in <module>
    pysam.sort("-n", outbam, bsorted)
  File "/home/local/share/bcbio/anaconda/lib/python2.7/site-packages/pysam/utils.py", line 75, in __call__
    stderr))
pysam.utils.SamtoolsError: 'samtools returned with error 1: stdout=, stderr=[bam_sort] Use -T PREFIX / -o FILE to specify temporary and final output files\nUsage: samtools sort [options...] [in.bam]\nOptions:\n  -l INT     Set compression level, from 0 (uncompressed) to 9 (best)\n  -m INT     Set maximum memory per thread; suffix K/M/G recognized [768M]\n  -n         Sort by read name\n  -o FILE    Write final output to FILE rather than standard output\n  -T PREFIX  Write temporary files to PREFIX.nnnn.bam\n  -@, --threads INT\n             Set number of sorting and compression threads [1]\n      --input-fmt-option OPT[=VAL]\n               Specify a single input file format option in the form\n               of OPTION or OPTION=VALUE\n  -O, --output-fmt FORMAT[,OPT[=VAL]]...\n               Specify output format (SAM, BAM, CRAM)\n      --output-fmt-option OPT[=VAL]\n               Specify a single output file format option in the form\n               of OPTION or OPTION=VALUE\n      --reference FILE\n               Reference sequence FASTA FILE [null]\n'

diffHic • 1.9k views
ADD COMMENT
0
Entering edit mode
Aaron Lun ★ 28k
@alun
Last seen 20 hours ago
The city by the bay

This is probably because the samtools interface changed between software versions. The presplit_map.py script itself doesn't call samtools, but does it through the pysam module. My best guess is to make sure your pysam version in your Python installation is up to date, e.g., via pip.

ADD COMMENT
0
Entering edit mode
Eichan • 0
@eichan-14174
Last seen 6.5 years ago
Japan/Tokyo

May I ask you which version of Python I should use for it? 

ADD COMMENT
0
Entering edit mode
Eichan • 0
@eichan-14174
Last seen 6.5 years ago
Japan/Tokyo

Dear Aaron,

 

I followed your recommendation and after trying several times running without success I uninstalled pysam, created a Python2 environment with conda, installed again the pysam 0.12.0.1 and then ran it again.

Unfortunately, I got the same error as previously. I really don't know what's the problem here.

 

Bellow is the conda info here, can you see something that might be wrong?

Current conda install:

platform : linux-64
          conda version : 4.3.29
       conda is private : False
      conda-env version : 4.3.29
    conda-build version : not installed
         python version : 2.7.14.final.0
       requests version : 2.18.1
       root environment : /home/nagailae/local/share/bcbio/anaconda  (writable)
    default environment : /home/nagailae/local/share/bcbio/anaconda/envs/py27
       envs directories : /home/nagailae/local/share/bcbio/anaconda/envs
                          /home/nagailae/.conda/envs
          package cache : /home/nagailae/local/share/bcbio/anaconda/pkgs
                          /home/nagailae/.conda/pkgs
           channel URLs : https://conda.anaconda.org/bioconda/linux-64
                          https://conda.anaconda.org/bioconda/noarch
                          https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.continuum.io/pkgs/main/linux-64
                          https://repo.continuum.io/pkgs/main/noarch
                          https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/linux-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
                          https://conda.anaconda.org/r/linux-64
                          https://conda.anaconda.org/r/noarch
            config file : /home/nagailae/.condarc
             netrc file : None
           offline mode : False
             user-agent : conda/4.3.29 requests/2.18.1 CPython/2.7.14 Linux/2.6.32-696.10.1.el6.x86_64 Red Hat Enterprise Linux Server/6.9 glibc/2.12
                UID:GID : 14809:10693

ADD COMMENT
0
Entering edit mode

Hm... I suspect this may be something to do with my sort call in pysam. I will test this myself, but you can try this:

  1. Copy presplit_map.py to your local directory (to avoid changing the installed copy).
  2. Change the sort call (on line 197) to pysam.sort("-o", outbam, "-n", bsorted).
  3. Change line 198 to os.rename(bsorted, outbam).

This should reflect the updates to the SAMtools API. (Make sure you don't mess with the indenting when you edit the code!)

I'll check this myself when I can, so don't worry if you can't get it to work.

P.S. Use the "add comment" button to respond to answers - unless you're answering your own question!

ADD REPLY
0
Entering edit mode

Thank you for the advice, I'm very new in this kind of forum, sorry for any mistake.

So, I've done the suggested changes, but got another errors:

 python presplit_map.py     -G /home/nagailae/database/mm10/bowtie2_idx/mm10     -1 SRR_1.fastq     -2 SRR_2.fastq     --sig AAGCTAGCTT     -o SRR.bam
[E::hts_open_format] Failed to open file ./tmpkuRGt0/sorted
Traceback (most recent call last):
  File "presplit_map.py", line 197, in <module>
    pysam.sort("-o", outbam, "-n", bsorted)
  File "/home/nagailae/local/share/bcbio/anaconda/envs/py27/lib/python2.7/site-packages/pysam/utils.py", line 75, in __call__
    stderr))
pysam.utils.SamtoolsError: 'samtools returned with error 1: stdout=, stderr=samtools sort: can\'t open "./tmpkuRGt0/sorted": No such file or directory\n[bam_sort] Note the <out.prefix> argument has been replaced by -T/-o options\n'

ADD REPLY
0
Entering edit mode

Ugh. Looks like the updated samtools needs an additional field on line 197:

pysam.sort("-o", bsorted, "-n", outbam)

Notice that I've flipped bsorted and outbam around, I made a mistake in my previous comment.

Edit: I realized that you shouldn't need the -T at all, the above corrected code should be sufficient.

ADD REPLY
0
Entering edit mode

Thank you very much Aaron, it worked perfectly now!!

P.S. there's an extra parenthesis in the end ;)

ADD REPLY
0
Entering edit mode

Ops,  another error...

 

Traceback (most recent call last):
  File "presplit_map.py", line 140, in <module>
    raise SystemError, "bowtie2 failed for presplit alignment"

 

My bowtie2 version is 2.3.3.1. 

ADD REPLY
0
Entering edit mode

Sorry, this error was caused because an interruption of the job here. I ran using qsub and seems like it is working well so far. As soon as I get results from alignment I report here again.

ADD REPLY
0
Entering edit mode
Eichan • 0
@eichan-14174
Last seen 6.5 years ago
Japan/Tokyo

Hi!

I tested and it ran perfectly with all data. It took a little longer than I was expecting but very nice! Alignment was higher than BWA mem.

presplit_map.py is working well after changes suggested by Aaron Lun :

 

  1. Copy presplit_map.py to your local directory (to avoid changing the installed copy).

  2. Change the sort call (on line 197) to pysam.sort("-o", bsorted, "-n", outbam).

  3. Change line 198 to os.rename(bsorted, outbam).​

 

Thank you very much Aaron!

ADD COMMENT

Login before adding your answer.

Traffic: 513 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6