Dear Bioc, in my lab we are evaluating the possibility to aquire a linux farm based on openMosix. In order to do this, we have to evaluate if the application we usually run, are going to improve their performance with this solution. Therefore I want to ask you if R, and in particular BioC packages can fork in this openMosix enviroment, and if so how to do it. Thanks -- Claudio Isella

Hi Claudio, You'll benefit with openMosix because your R session can migrate to less-used nodes in the cluster. Extra benefit might come if you use or develop some of the parallel processing tools available through R. For instance, snow, Rmpi, and rpvm can be used to 'spawn' and communicate with new processes, including new R processes. I think (no direct experience, though) that openMosix will allow these processes to migrate to under-used nodes, resulting in efficient computation. R processes are relatively large, and computations likely to benefit from parallelization may often involve very large data sets. For these situations (large processes, large data) migration between nodes is likely to be 'costly'. Whether costs outweigh benefits likely depends on the details of the problem being solved. It would be great to hear from people with direct experience using openMosix. Martin Claudio Isella <claudio.isella at="" ircc.it=""> writes: > Dear Bioc, > > > in my lab we are evaluating the possibility to aquire a linux farm based > on openMosix. In order to do this, we have to evaluate if the > application we usually run, are going to improve their performance with > this solution. > > Therefore I want to ask you if R, and in particular BioC packages can > fork in this openMosix enviroment, and if so how to do it. > > Thanks > > -- > Claudio Isella > > _______________________________________________ > Bioconductor mailing list > Bioconductor at stat.math.ethz.ch > https://stat.ethz.ch/mailman/listinfo/bioconductor

> Claudio Isella <claudio.isella at="" ircc.it=""> writes: >> in my lab we are evaluating the possibility to aquire a linux farm >> based on openMosix. In order to do this, we have to evaluate if the >> application we usually run, are going to improve their performance >> with this solution. It really depends on your application. OpenMosix provides automatic load balancing of separate individual processes among a set of compute nodes. This could benefit you in two ways: 1. You have many people sharing a compute resource, all launching jobs. Instead of trying to organize who should use which server, all users can launch jobs on the openMosix head node and the jobs will migrate to less busy nodes. 2. You have an application that can take advantage of some parallell processing and you have written some support scripts to launch many processes (which will be load balanced for you). IMO, openMosix isn't providing all that much help in this case. >> Therefore I want to ask you if R, and in particular BioC packages >> can fork in this openMosix enviroment, and if so how to do it. I have some limited experience mixing R and openMosix. At that time, I found the migration hit or miss. Some R codes would migrate fine, others would segfault upon migration attempt. I ended up using the snow package along with Rpvm and ran pvm on top of the openMosix cluster. This worked very well. In summary: - I think you will get the most bang for your buck by taking advantage of pvm or mpi via snow (it is easier than it sounds). Whether or not you have openMosix is less important, but having a number of reasonably powerful Linux servers is important :-) - openMosix (1 year ago) was something of a maintence headache and I'm not sure the advantages it brought were worth it. For use-case #1 above, many jobs just didn't migrate as desdired. For use-case ##2, using snow on top of mpi is a better bet, IMO. HTH, + seth