Entering edit mode
pierrick.roger • 0@pierrickroger-21304
Last seen 12 months ago
I've switched recently from custom cache system to BiocFileCache inside my biodb package.
However, now, in one of the comming extension packages of biodb (biodbHmdb) I need to move more than 100,000 files at once into BiocFileCache.
For this I call
bfcadd() on each file inside a loop, and it takes forever to complete.
Is there a faster way of achieving this or would it be possible to improve BiocFileCache for such a purpose?
Do you really need to move all 100,000 files at the same time? It sounds like you're pre-loading the cache. To my mind a cache is useful for speeding up multiple similar operations, but not necessarily the first one. I don't know the context for biodbHmdb, but I'm wondering if it might be more effective to cache the result of a query after it has been executed.
This might also get more traction on the developers mailing list (https://stat.ethz.ch/mailman/listinfo/bioc-devel)