Pagination

The Hurdlr API endpoints utilize pagination to efficiently provide data to the apps and services developed by FinTechs, banks and other platforms. By following the best practices explained below, your usage of the Hurdlr API will reliably scale along with your user base.

1. How it works

Pagination is a process that divides a large dataset into smaller chunks a.k.a. pages. All Hurdlr API endpoints that return a list support pagination. For example, the GET /expenses endpoint supports pagination, but the GET /userTaxSetup endpoint does not.

As an example, suppose your GET /expenses result set contains 202 expenses and your page size is set to 100. The response to your first request will contain 100 expenses along with a cursor that you can use in your subsequent request. Your second request will return the next 100 expenses along with a new cursor that you can use for your third request. The third request will return the remaining 2 expenses and no cursor, which indicates that there are no remaining pages of expenses to request.

2. Specifying cursor and page size

All Hurdlr API endpoints that support pagination allow you to specify a limit, which indicates the maximum number of items to include in the response. If you are unsure what limit to use, the Hurdlr API team recommends starting with 100, as shown in the sample request for pending expenses below:

curl \
  --request GET \
  --url https://sandbox.hurdlr.com/rest/v5/expenses/expenses?type=PENDING \
  --header 'Authorization: Bearer ${access_token}' \
  --header 'Content-Type: application/json' \
  --header 'limit: 100'

As explained above, when fetching a dataset with more items than the limit specified, the response will contain a cursor field, as shown below in the sample response:

{
  "data": [
    {
      "id": 1118496,
      "type": "PENDING",
      "date": "2020-08-27 00:00:00.000",
      "amount": 500.39,
      "categoryId": 13325612,
      "personalCategoryId": 233391,
      ...,
    }, {
      "id": 1118497,
      "type": "PENDING",
      "date": "2020-08-28 00:00:00.000",
      "amount": 5423.12,
      "categoryId": 13325613,
      "personalCategoryId": 233392,
      ...,
    },
    ...,
  ],
  "lastUpdatedDate": "2021-03-28T19:49:23.288Z",
  "cursor": "eyJkYXRlQ3Vyc29yIjo5MjIzMzcyMDM2ODU0Nzc1ODA3LCJpZEN1cnNvciI6OTIyMzM3MjAzNjg1NDc3NTgwNywicGVuZGluZ0RhdGVDdXJzb3IiOjE2NDM2NzM2MDAwMDAsInBlbmRpbmdJZEN1cnNvciI6OTIzNDJ9"
}

The subsequent request to fetch the next page of data would look like the request below:

curl \
  --request GET \
  --url https://sandbox.hurdlr.com/rest/v5/expenses/expenses?type=PENDING \
  --header 'Authorization: Bearer ${access_token}' \
  --header 'Content-Type: application/json' \
  --header 'cursor: eyJkYXRlQ3Vyc29yIjo5MjIzMzcyMDM2ODU0Nzc1ODA3LCJpZEN1cnNvciI6OTIyMzM3MjAzNjg1NDc3NTgwNywicGVuZGluZ0RhdGVDdXJzb3IiOjE2NDM2NzM2MDAwMDAsInBlbmRpbmdJZEN1cnNvciI6OTIzNDJ9' \
  --header 'limit: 100'

3. Only fetching updated data

In order to reduce unnecessary computation cycles and bandwidth consumption on your servers, all Hurdlr API endpoints that support pagination allow you to specify a lastUpdatedDate, which requests only the data that has been updated since a given date. The first time you request a given dataset, you can leave the lastUpdatedDate empty, which will return the entire dataset, as shown in the examples above.

The response will contain a lastUpdatedDate, which you should store in your database. The next time you request the same dataset, in order to only fetch the data that was updated since your last request, you should provide that lastUpdatedDate in your request, as shown in the sample request below:

curl \
  --request GET \
  --url https://sandbox.hurdlr.com/rest/v5/expenses/expenses?type=PENDING \
  --header 'Authorization: Bearer ${access_token}' \
  --header 'Content-Type: application/json' \
  --header 'limit: 100' \
  --header 'lastUpdatedDate: 2021-03-28T19:49:23.288Z'

A sample response for this request is shown below:

{
  "data": [
		{
      "id": 1118497,
      "type": "PENDING",
      "date": "2020-08-28 00:00:00.000",
      "amount": 5423.12,
      "categoryId": 13325614,
      "personalCategoryId": 233392,
      ...,
    }
  ],
  "lastUpdatedDate": "2021-04-01T19:49:23.288Z",
  "deletedIds": [1118496]
}

In the above response, there is a deletedIds attribute, which indicates that an item was removed from the request's dataset since the lastUpdatedDate of 2021-03-28T19:49:23.288Z. Specifically, the expense with id: 1118496 is no longer pending and therefore will no longer be returned in requests for /expenses?type=PENDING. Additionally, the expense with id: 1118497 was updated to have a new categoryId.

Those are the only changes since 2021-03-28T19:49:23.288Z, so there was only one page of data, and therefore no cursor was returned. A lastUpdatedDate of 2021-04-01T19:49:23.288Z was returned for you to store for your next request to /expenses?type=PENDING.

👍

High performance

Utilizing lastUpdatedDate in your requests can exponentially reduce the amount of data transferred over time, saving you and your users' significant time and money.

In the above example, if lastUpdatedDate was not specified, then there would have been 202 items fetched using three GET requests; instead, by utilizing lastUpdatedDate, there was only 1 item (and 1 removal) requiring only one GET request.