Question: Clomial parallel jobs
0
gravatar for jmfernandesalves
2.9 years ago by
jmfernandesalves0 wrote:

Dear all,

I am interested in running Clomial R package for a fairly large dataset (23 regional samples with 247 SNVs) and, while reading the documentation, I’ve found that one can run this tool in parallel in a Unix SGE cluster.

Since we have one available at our lab, I am trying to implement Clomial to work in parallel but I keep getting error messages such as:

Unable to run job: denied: host is no submit host.

I was therefore wondering whether you have any example on how to use Clomial with SGE. 

Here's the Clomial command line that I'm using:

1. sh script

#!/bin/sh
module load R/3.2.2_1
R CMD BATCH /home/jmf.alves/Clomial.c3.R

2. R script (Clomial.c3.R)

#!/usr/bin/env Rscript
library(Clomial)
data.Dc <- as.matrix(read.table("Dc.txt", head=T, sep="\t", row.names=1, as.is=T))
data.Dt <- as.matrix(read.table("Dt.txt", head=T,  sep="\t", row.names=1, as.is=T))

Clomial.c3 <- Clomial(Dc=data.Dc, Dt=data.Dt, maxIt=20, C=3, doParal=T, binomTryNum=100, outPrefix="/home/jmf.alves/", jobNamePrefix="C3", fitBinomJobFile="/home/jmf.alves/Clomial.C3", doTalk=T)

 

 

 

sge clomial • 519 views
ADD COMMENTlink modified 2.9 years ago by Habil Zare170 • written 2.9 years ago by jmfernandesalves0
Answer: Clomial parallel jobs
1
gravatar for Habil Zare
2.9 years ago by
Habil Zare170
United States/Austin Area
Habil Zare170 wrote:

rsub() is an internal Clomial function. You got this error because the qsub command in your cluster does not have -l mem_free= option. Options can vary cluster to cluster. 

You can check with your IT support teams and let me know what options are available, and which are required, to submit a job on your cluster. Then, I will generalize the Clomial code accordingly.  

The current code (Clomial 1.8.0) assumes a qsub command is something like below:

qsub -js <jobShare> -l mem_free=<memory> -l h_vmem=<memory>  -l mem_requested=<memory> -l longjob=TRUE -N <jobName> -e <ste> -o <sto>

ADD COMMENTlink written 2.9 years ago by Habil Zare170

Hi @Habil,

I managed to fix it by modifying the Rsub function and the fitBinomJobFile template
Now everything seems to be working properly.
Thank you again for your help,

J

ADD REPLYlink written 2.9 years ago by jmfernandesalves0

You're welcome!

ADD REPLYlink written 2.9 years ago by Habil Zare170
Answer: Clomial parallel jobs
0
gravatar for Habil Zare
2.9 years ago by
Habil Zare170
United States/Austin Area
Habil Zare170 wrote:

It is possible to run Clomial in parallel.  Basically, the idea is to run thousands of jobs on the cluster using qsub. Clomial submits each job by calling the rsub() function, which needs a template similar to this one but everything else is done automatically. Clomial just needs a proper template and you should tell it where the template is using its fitBinomJobFile argument. The template file linked above might need some modification with the current version of Clomial and also according to your cluster settings. Then, you can login to your cluster and run Clomial in parallel using:

Clomial.c3 <- Clomial(Dc=data.Dc, Dt=data.Dt, maxIt=20, C=3, doParal=T, binomTryNum=100, outPrefix="/home/jmf.alves/", jobNamePrefix="C3", fitBinomJobFile="fit.binom.job_v1.R", doTalk=T)

Please let me know if you need more help.

ADD COMMENTlink written 2.9 years ago by Habil Zare170

Thanks for the kind reply @Habil. 

I've tried your example (running directly from the front-end) but I got the following:
Unable to run job: unknown resource "mem_requested"

Is this related with the rsub() function? Btw, is this function part of a different package that I need to load? 
Thanks.

J

 

ADD REPLYlink written 2.9 years ago by jmfernandesalves0
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 299 users visited in the last hour