Sometimes in Android you connect to a third party API and you need to download gobs of JSON data. One of my current clients has an API endpoint that is used for syncing data to the device. One API call returns hundreds of thousands of lines of JSON resulting in a JSON file (when saved) of around 8MB or more. Processing the entire file by loading it into memory can be painful and can crash a lot memory constrained devices. However, using the Jackson JSON parsing library you can stream the http-request as a Reader into Jacksons Streaming API which will allow you to process each JSON object as soon as it comes down from the pipe. This allows you to process the data as it comes down the pipe instead of loading the entire JSON file.
I use the ObjectMapper from Jackson to deserialize the JSON into a POJO. This allows me to work with domain objects throughout my app, therefore, I don’t have to worry about parsing JSON objects all over the place (or cursors if you move the data into a DB locally on the device). If you were to load the entire JSON file into the object mapper you’d create a humongous object graph which may ballon your memory out of the bounds of what’s available and you might get a OutOfMemoryException. The end goal is to process EACH of the JSON objects as they come down, but also have the benefit of using a POJO to do so. Here’s how you can do it.
Below we have a simple customer object that we need to download. Imagine having a large accounting app that has millions of customers. Each customer would be represented as a POJO as defined below.
Connecting to the API / Consuming the Stream
This actually quite easy. Simply connect to your API endpoint with your http library of choice (In this case I’m using http-request, but you can easily use OkHttp, etc). Using the reader() method on the HttpRequest object I can then pass this reader into the JsonParser (shown below) and then let the loop build each Customer and then I handle each customer accordingly. Comments are inline:
How to Use Jackson
Simply add it as a library or if you’re using Maven add the Maven dependency in your POM and you’re off to the races. 🙂 Now you can download HUGE JSON streams without the worry of exploding your memory footprint. 🙂