Best practice for storing large amounts of data with J2ME


I am developing a J2ME application that has a large amount of data to store on the device (in the region of 1MB but variable). I can't rely on the file system so I'm stuck the Record Management System (RMS), which allows multiple record stores but each have a limited size. My initial target platform, Blackberry, limits each to 64KB.

I'm wondering if anyone else has had to tackle the problem of storing a large amount of data in the RMS and how they managed it? I'm thinking of having to calculate record sizes and split one data set accross multiple stores if its too large, but that adds a lot of complexity to keep it intact.

There is lots of different types of data being stored but only one set in particular will exceed the 64KB limit.

9/18/2008 8:21:16 PM

Accepted Answer

For anything past a few kilobytes you need to use either JSR 75 or a remote server. RMS records are extremely limited in size and speed, even in some higher end handsets. If you need to juggle 1MB of data in J2ME the only reliable, portable way is to store it on the network. The HttpConnection class and the GET and POST methods are always supported.

On the handsets that support JSR 75 FileConnection it may be valid alternative but without code signing it is an user experience nightmare. Almost every single API call will invoke a security prompt with no blanket permission choice. Companies that deploy apps with JSR 75 usually need half a dozen binaries for every port just to cover a small part of the possible certificates. And this is just for the manufacturer certificates; some handsets only have carrier-locked certificates.

9/15/2008 1:25:10 PM

I think the most flexible approach would be to implement your own file system on top of the RMS. You can handle the RMS records in a similar way as blocks on a hard drive and use a inode structure or similar to spread logical files over multiple blocks. I would recommend implementing a byte or stream-oriented interface on top of the blocks, and then possibly making another API layer on top of that for writing special data structures (or simply make your objects serializable to the data stream).

Tanenbaum's classical book on operating systems covers how to implement a simple file system, but I am sure you can find other resources online if you don't like paper.


Under Blackberry OS 4.6 the RMS store size limit has been increased to 512Kb but this isn't much help as many devices will likely not have support for 4.6. The other option on Blackberry is the Persistent Store which has a record size limit of 64kb but no limit on the size of the store (other than the physical limits of the device).

I think Carlos and izb are right.


It is quite simple, use JSR75 (FileConnection) and remember to sign your midlet with a valid (trusted) certificate.


For read only I'm arriving at acceptable times (within 10s), by indexing a resource file. I've got two ~800KB CSV price list exports. Program classes and both those files compress to a 300KB JAR.

On searching I display a List and run a two Threads in the background to fill it, so the first results come pretty quickly and are viewable immediately. I first implemented a simple linear search, but that was too slow (~2min).

Then I indexed the file (which is alphabetically sorted) to find the beginnings of each letter. Now before parsing line by line, I first InputStreamReader.skip() to the desired position, based on first letter. I suspect the delay comes mostly from decompressing the resource, so splitting resources would speed it up further. I don't want to do that, not to loose the advantage of easy upgrade. CSV are exported without any preprocessing.


I'm just starting to code for JavaME, but have experience with old versions of PalmOS, where all data chunks are limited in size, requiring the design of data structures using record indexes and offsets.