mcmapply with edgeR?
1
0
Entering edit mode
Aaron Mackey ▴ 200
@aaron-mackey-3833
Last seen 9.9 years ago
I have mcmapply (and related parallel library functions) working fine with example code, but when I try to run my own function (which makes calls to estimateGLMCommonDisp, etc) in parallel, only one core remains engaged, no forking happens. The only thing I can imagine is that the parallel library has detected my use of a dynamically loaded .so, and has reverted to serial behavior? Or is edgeR preventing it's functions from being fork'ed somehow? Thanks in advance, -Aaron [[alternative HTML version deleted]]
edgeR edgeR • 1.0k views
ADD COMMENT
0
Entering edit mode
@martin-morgan-1513
Last seen 4 months ago
United States
On 08/13/2012 10:07 AM, Aaron Mackey wrote: > I have mcmapply (and related parallel library functions) working fine with > example code, but when I try to run my own function (which makes calls to > estimateGLMCommonDisp, etc) in parallel, only one core remains engaged, no > forking happens. The only thing I can imagine is that the parallel library > has detected my use of a dynamically loaded .so, and has reverted to serial > behavior? Or is edgeR preventing it's functions from being fork'ed somehow? using a shared object shouldn't be a problem with mclapply (rather than mcmapply, which is not from the parallel library?); maybe you can create a short script that has the essentials for someone else to evaluate, maybe using small simulated data, perhaps from one of the help pages of edgeR? Martin > > Thanks in advance, > -Aaron > > [[alternative HTML version deleted]] > > _______________________________________________ > Bioconductor mailing list > Bioconductor at r-project.org > https://stat.ethz.ch/mailman/listinfo/bioconductor > Search the archives: http://news.gmane.org/gmane.science.biology.informatics.conductor > -- Computational Biology / Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109 Location: Arnold Building M1 B861 Phone: (206) 667-2793
ADD COMMENT
0
Entering edit mode
Turns out it is a bug in the mcmapply() code -- when the number of jobs is less than the number of cores, then the function routes it to standard mapply() rather than letting the jobs run in parallel. Sorry for the red herring. -Aaron On Mon, Aug 13, 2012 at 1:14 PM, Martin Morgan <mtmorgan@fhcrc.org> wrote: > On 08/13/2012 10:07 AM, Aaron Mackey wrote: > >> I have mcmapply (and related parallel library functions) working fine with >> example code, but when I try to run my own function (which makes calls to >> estimateGLMCommonDisp, etc) in parallel, only one core remains engaged, no >> forking happens. The only thing I can imagine is that the parallel >> library >> has detected my use of a dynamically loaded .so, and has reverted to >> serial >> behavior? Or is edgeR preventing it's functions from being fork'ed >> somehow? >> > > using a shared object shouldn't be a problem with mclapply (rather than > mcmapply, which is not from the parallel library?); maybe you can create a > short script that has the essentials for someone else to evaluate, maybe > using small simulated data, perhaps from one of the help pages of edgeR? > > Martin > > >> Thanks in advance, >> -Aaron >> >> [[alternative HTML version deleted]] >> >> ______________________________**_________________ >> Bioconductor mailing list >> Bioconductor@r-project.org >> https://stat.ethz.ch/mailman/**listinfo/bioconductor<https: stat.e="" thz.ch="" mailman="" listinfo="" bioconductor=""> >> Search the archives: http://news.gmane.org/gmane.** >> science.biology.informatics.**conductor<http: news.gmane.org="" gmane="" .science.biology.informatics.conductor=""> >> >> > > -- > Computational Biology / Fred Hutchinson Cancer Research Center > 1100 Fairview Ave. N. > PO Box 19024 Seattle, WA 98109 > > Location: Arnold Building M1 B861 > Phone: (206) 667-2793 > [[alternative HTML version deleted]]
ADD REPLY

Login before adding your answer.

Traffic: 666 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6