Handling very large sets of records

The getRecords() and getAllRecords() methods load all data records into the server memory before your code can do anything with these records. If the data set contains hundreds of thousands of records, it will consume a very large amount of server memory, potentially slowing down the server operation speed and affecting all interactive users.

If you need to handle very large sets of records, you can do so by scrolling the result set and handling one record at a time. Using this technique, only a handful of records (a buffer) is loaded into the server memory at once.

To scroll though the result set, use the DataSource.queryRecords() method and provide your own callback object, as shown in the example below. The DataSource will open the database cursor, and will call your handleRecord() callback method for each returned record.

ds.queryRecords(new RecordHandler() {

public boolean handleRecord(DataRecord record) {
// the code that handles each record goes here

return true; // to continue scrolling through the result set
}
});

You can stop the scrolling at any time by returning false from the callback method.