Best way of creating a large RleMatrix
1
0
Entering edit mode
maltethodberg ▴ 180
@maltethodberg-9690
Last seen 2 hours ago
Denmark

What's the preferred way of creating a large RleMatrix?

The documentation page ?RleMatrix only covers how make an RleMatrix from a single Rle (which doesn't work for very large RleMatrix objects)

First creating a normal matrix and then coercing to RleMatrix, requires keeping the original matrix in memory which is not possible.

I'm currently using the approach of first creating a DataFrame of Rle's, and then coercing that to an RleMatrix, but that seems a bit slow. Is there a smarter way? 

DelayedArray RleMatrix • 1.3k views
ADD COMMENT
3
Entering edit mode
@herve-pages-1542
Last seen 3 hours ago
Seattle, WA, United States

Hi,

Sorry for the late answer. I've been doing some work on the RleArray() constructor to improve support for long RleArray/RleMatrix objects. One of them is that calling the RleArray() constructor directly on a DataFrame with Rle columns now is supported and fast. I've also added plenty of examples to the man page so make sure to check them (see ?RleArray, in particular the D. CONSTRUCTING A LARGE RleArray OBJECT section).

These improvements are in the devel version of DelayedArray (v 0.9.5), which should become available to BioC 3.9 users via BiocManager::install() in the next 24h. Let me know if you run into problems with this.

Happy new year,

H.

ADD COMMENT
0
Entering edit mode

Thank you! Does the new constructor affect how the chunks are structured? If you construct an RleMatrix from a DataFrame of Rle's, is the RleMatrix then chunked by column?

ADD REPLY
1
Entering edit mode

The change to the constructor is that it now accepts a list-like object of Rle objects (e.g. RleList object or DataFrame of Rle's). Each list element in this list-like object becomes a chunk. So yes, if the input to the constructor is a DataFrame of Rle's then each column of the DataFrame becomes a chunk.

H.

ADD REPLY

Login before adding your answer.

Traffic: 630 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6