Java RestFUL API Pagination

Hello Guys,

This issue is been perplexing me for a while and I can't figure the issue. I have to admit I normally program in php and not Java, so I would really appreciate your help.

So my java code below makes a REST GET request, retrieves the xml and outputs it into a file. It also paginates through if the REST service has more than 100 records. The code works fine if we don’t paginate. But when I do paginate it does not return the correct result.

So lets say there are 100 records in the REST service and result per page is 10. My java code is supposed to output 10 xml files, but instead only outputs 4 files.

The rest service says:

Parameters
● currentPage - the page of results to get. If not specified assume the first page.
● resultsPerPage - how many results are returned per page.

Metadata
The following meta data will be returned
● totalResults - the total number of results
● resultsPerPage - the number of results per page
● currentPage - the current page number of results
● lastPage - the page number of the last page of result

Sample xml:

<?xml version="1.0" encoding="UTF-8" ?>
<results />
<metaData>
<totalResults>100</totalResults>
<resultsPerPage>10</resultsPerPage>
<currentPage>1</currentPage>
<lastPage>10</lastPage>
</metaData>

my code

Code:

---

int currentPage; //THIS SHOULD BE USED FOR PAGINATION
int totalPages; //THIS WILL BE THE TOTAL NUMBER OF PAGES AS RETURNED IN THE METADATA IN THE FIRST ITERATION
String outputFolder = "/xxxx/xxxx/xxxx/";

 

public String getData(String brandSiteCode, String fromDate, String toDate, int cp) {

        currentPage = cp;

        String s="";

        String response ="";

        //need to rename filename to get rid of colons.

        String filename = "Test2" + brandSiteCode + "_" + fromDate.replace(":","-") + "_" + toDate.replace(":","-") + "_" + currentPage + ".xml";

        String metaData = "";

  try

  {

    URL url;

    HttpURLConnection urlConn;

    InputStream dis;
     
 String myURL = "http://xxxxxx.stg.xxxxx/v1/user-records/?brandSiteCode="+brandSiteCode+"&fromDate=" + fromDate + "&currentPage=" + currentPage;

 
//logInfo("getting data for this date range: " + myURL);

    url = new URL(myURL);

 
    urlConn = (HttpURLConnection)url.openConnection();

    urlConn.setDoInput(true);

    urlConn.setRequestMethod("GET");

    urlConn.setUseCaches(false);

    urlConn.setRequestProperty("Accept","application/xml");

    //dis = new DataInputStream(urlConn.getInputStream());
 

    int status = urlConn.getResponseCode();

    System.out.println("Status code:" + status);

    //if http code 200 is not returned, there's an error.

    if (status!=200) {

        dis = new DataInputStream(urlConn.getErrorStream());

      }else {

        dis= new DataInputStream(urlConn.getInputStream());

      }

    InputStreamReader inReader = new InputStreamReader(dis, "utf-8");

    StringBuffer responseBuffer = new StringBuffer();


if (status!=200) { 

    failSession("HTTP status code wasn't 200, so must be something wrong - FAILED");

    }


    char[] buffer = new char[1024];

    int bytes;

    while ((bytes = inReader.read(buffer)) != -1) {

        responseBuffer.append(buffer, 0, bytes);

    }

        response = responseBuffer.toString();

    int metaDataIndex = response.indexOf("<metaData");

    metaData = response.substring(metaDataIndex,response.length());

    response = response.substring(0,metaDataIndex);


    metaDataIndex = metaData.indexOf("<lastPage>")+10;

    metaData = metaData.substring(metaDataIndex,metaData.length()); //the number of pages will now be at the start of metaData

    metaDataIndex= metaData.indexOf("<");

    totalPages=Integer.parseInt(metaData.substring(0,metaDataIndex));

    FileOutputStream out = new FileOutputStream(outputFolder + filename);

    PrintStream p = new PrintStream(out);

    p.println(response);
 
    System.out.println("metadata:" + metaData);

    p.close();

    dis.close();

 
// PAGINATION

    currentPage++;

    while (currentPage<=totalPages) {

        getData(brandSiteCode,fromDate, toDate, currentPage);
               
}
        //currentPage++;

    }

        catch (Exception e) {e.printStackTrace();}

 

      return response;

  }

---

Just looking at your code I see two problems. Post incrementing currentPage instead of pre and using the while loop for recursion. In your while loop, you don't increase the control variable so presumably, it won't end and even if it did, you would be executing the code way more times than you want because it's going to want to get from the starting page count to the end for each time it goes back into itself. The first iteration would count 1 to 10, the second 2 to 10, the third 3 to 10, etc. Change that to a simple if(currentPage<=totalPages) and it should unwind the way you want.

Just tried that, It did not make a difference. But you are right, its my pagination logic that is wrong.

So it looks like the my java code is skiping result per page. so if there are 10 results per page, rather than retrieving records 1,2,3,4,5,6,7,8,9,10 its retrieving records per file like this:
file 1: 1,4,5,6,11,14,15,20,22,24
file 2: 25,26,29,36,38,40,43,46,50,54
file 3: 58,59,62,64,75,77,83,85,86,89
file 4: 93,94,95,98,99

Any Ideas?

That code is really hard to follow with the seemingly random indentation.

Thanks Tolls, I have resolved the issue now. it was not coding related.