Search
Question: error running presplit_map.py
0
gravatar for Eichan
4 weeks ago by
Eichan0
Japan/Tokyo
Eichan0 wrote:

Hi,

May I ask a naive question on what could be wrong when I try to run the chimeric alignment using presplit_map.py from diffHic package. It seems an error from samtools but I couldn't figure out what causes it. I updated my samtools version from 1.31 to 1.6 and I still having the same problem. I thank you in advance for any help.

Command

python /pathto/presplit_map.py \
    -G /pathto/bowtie2_idx/mm10 \
    -1 SRR_1.fastq \
    -2 SRR_2.fastq \
    --sig AAGCTAGCTT \
    -o SRR.bam

 

Error

Traceback (most recent call last):
  File "/home/R/x86_64-pc-linux-gnu-library/3.4/diffHic/python/presplit_map.py", line 197, in <module>
    pysam.sort("-n", outbam, bsorted)
  File "/home/local/share/bcbio/anaconda/lib/python2.7/site-packages/pysam/utils.py", line 75, in __call__
    stderr))
pysam.utils.SamtoolsError: 'samtools returned with error 1: stdout=, stderr=[bam_sort] Use -T PREFIX / -o FILE to specify temporary and final output files\nUsage: samtools sort [options...] [in.bam]\nOptions:\n  -l INT     Set compression level, from 0 (uncompressed) to 9 (best)\n  -m INT     Set maximum memory per thread; suffix K/M/G recognized [768M]\n  -n         Sort by read name\n  -o FILE    Write final output to FILE rather than standard output\n  -T PREFIX  Write temporary files to PREFIX.nnnn.bam\n  -@, --threads INT\n             Set number of sorting and compression threads [1]\n      --input-fmt-option OPT[=VAL]\n               Specify a single input file format option in the form\n               of OPTION or OPTION=VALUE\n  -O, --output-fmt FORMAT[,OPT[=VAL]]...\n               Specify output format (SAM, BAM, CRAM)\n      --output-fmt-option OPT[=VAL]\n               Specify a single output file format option in the form\n               of OPTION or OPTION=VALUE\n      --reference FILE\n               Reference sequence FASTA FILE [null]\n'

ADD COMMENTlink modified 26 days ago • written 4 weeks ago by Eichan0
0
gravatar for Aaron Lun
4 weeks ago by
Aaron Lun17k
Cambridge, United Kingdom
Aaron Lun17k wrote:

This is probably because the samtools interface changed between software versions. The presplit_map.py script itself doesn't call samtools, but does it through the pysam module. My best guess is to make sure your pysam version in your Python installation is up to date, e.g., via pip.

ADD COMMENTlink modified 4 weeks ago • written 4 weeks ago by Aaron Lun17k
0
gravatar for Eichan
4 weeks ago by
Eichan0
Japan/Tokyo
Eichan0 wrote:

May I ask you which version of Python I should use for it? 

ADD COMMENTlink written 4 weeks ago by Eichan0
0
gravatar for Eichan
4 weeks ago by
Eichan0
Japan/Tokyo
Eichan0 wrote:

Dear Aaron,

 

I followed your recommendation and after trying several times running without success I uninstalled pysam, created a Python2 environment with conda, installed again the pysam 0.12.0.1 and then ran it again.

Unfortunately, I got the same error as previously. I really don't know what's the problem here.

 

Bellow is the conda info here, can you see something that might be wrong?

Current conda install:

platform : linux-64
          conda version : 4.3.29
       conda is private : False
      conda-env version : 4.3.29
    conda-build version : not installed
         python version : 2.7.14.final.0
       requests version : 2.18.1
       root environment : /home/nagailae/local/share/bcbio/anaconda  (writable)
    default environment : /home/nagailae/local/share/bcbio/anaconda/envs/py27
       envs directories : /home/nagailae/local/share/bcbio/anaconda/envs
                          /home/nagailae/.conda/envs
          package cache : /home/nagailae/local/share/bcbio/anaconda/pkgs
                          /home/nagailae/.conda/pkgs
           channel URLs : https://conda.anaconda.org/bioconda/linux-64
                          https://conda.anaconda.org/bioconda/noarch
                          https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.continuum.io/pkgs/main/linux-64
                          https://repo.continuum.io/pkgs/main/noarch
                          https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/linux-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
                          https://conda.anaconda.org/r/linux-64
                          https://conda.anaconda.org/r/noarch
            config file : /home/nagailae/.condarc
             netrc file : None
           offline mode : False
             user-agent : conda/4.3.29 requests/2.18.1 CPython/2.7.14 Linux/2.6.32-696.10.1.el6.x86_64 Red Hat Enterprise Linux Server/6.9 glibc/2.12
                UID:GID : 14809:10693

ADD COMMENTlink written 4 weeks ago by Eichan0

Hm... I suspect this may be something to do with my sort call in pysam. I will test this myself, but you can try this:

  1. Copy presplit_map.py to your local directory (to avoid changing the installed copy).
  2. Change the sort call (on line 197) to pysam.sort("-o", outbam, "-n", bsorted).
  3. Change line 198 to os.rename(bsorted, outbam).

This should reflect the updates to the SAMtools API. (Make sure you don't mess with the indenting when you edit the code!)

I'll check this myself when I can, so don't worry if you can't get it to work.

P.S. Use the "add comment" button to respond to answers - unless you're answering your own question!

ADD REPLYlink modified 4 weeks ago • written 4 weeks ago by Aaron Lun17k

Thank you for the advice, I'm very new in this kind of forum, sorry for any mistake.

So, I've done the suggested changes, but got another errors:

 python presplit_map.py     -G /home/nagailae/database/mm10/bowtie2_idx/mm10     -1 SRR_1.fastq     -2 SRR_2.fastq     --sig AAGCTAGCTT     -o SRR.bam
[E::hts_open_format] Failed to open file ./tmpkuRGt0/sorted
Traceback (most recent call last):
  File "presplit_map.py", line 197, in <module>
    pysam.sort("-o", outbam, "-n", bsorted)
  File "/home/nagailae/local/share/bcbio/anaconda/envs/py27/lib/python2.7/site-packages/pysam/utils.py", line 75, in __call__
    stderr))
pysam.utils.SamtoolsError: 'samtools returned with error 1: stdout=, stderr=samtools sort: can\'t open "./tmpkuRGt0/sorted": No such file or directory\n[bam_sort] Note the <out.prefix> argument has been replaced by -T/-o options\n'

ADD REPLYlink written 4 weeks ago by Eichan0

Ugh. Looks like the updated samtools needs an additional field on line 197:

pysam.sort("-o", bsorted, "-n", outbam)

Notice that I've flipped bsorted and outbam around, I made a mistake in my previous comment.

Edit: I realized that you shouldn't need the -T at all, the above corrected code should be sufficient.

ADD REPLYlink modified 4 weeks ago • written 4 weeks ago by Aaron Lun17k

Thank you very much Aaron, it worked perfectly now!!

P.S. there's an extra parenthesis in the end ;)

ADD REPLYlink written 4 weeks ago by Eichan0

Ops,  another error...

 

Traceback (most recent call last):
  File "presplit_map.py", line 140, in <module>
    raise SystemError, "bowtie2 failed for presplit alignment"

 

My bowtie2 version is 2.3.3.1. 

ADD REPLYlink written 4 weeks ago by Eichan0

Sorry, this error was caused because an interruption of the job here. I ran using qsub and seems like it is working well so far. As soon as I get results from alignment I report here again.

ADD REPLYlink modified 4 weeks ago • written 4 weeks ago by Eichan0
0
gravatar for Eichan
26 days ago by
Eichan0
Japan/Tokyo
Eichan0 wrote:

Hi!

I tested and it ran perfectly with all data. It took a little longer than I was expecting but very nice! Alignment was higher than BWA mem.

presplit_map.py is working well after changes suggested by Aaron Lun :

 

  1. Copy presplit_map.py to your local directory (to avoid changing the installed copy).

  2. Change the sort call (on line 197) to pysam.sort("-o", bsorted, "-n", outbam).

  3. Change line 198 to os.rename(bsorted, outbam).​

 

Thank you very much Aaron!

ADD COMMENTlink written 26 days ago by Eichan0
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.2.0
Traffic: 150 users visited in the last hour