Pagination

In this example we take you through how you can paginate through large GraphQL result sets.

5 minute read

What you’ll learn

In this article you’ll learn about pagination, specifically:

Result set size
Cursors (moving forward in the result set)

What is pagination?

Pagination (or paging) is the process of dividing a document, (in our case a GraphQL JSON result-set), into a sequence of pages that are then “numbered”.

In the context of a GraphQL result set this means that we can deal more effectively with large sets of data, i.e. we don’t need to have the entire result set returned in 1 request, instead splitting it up into pages and then moving through, (retrieving), those pages as required.

Using pagination we can move forwards through a result set.

Note

We don’t currently support backwards pagination

Using Pagination in your queries

We make use of the Relay Specification, which is specifically designed to allow us to use pagination best practices. In this article we’ll go through the basics of how to query using pagination, but for a full discussion please refer to the Relay Specification documentation.

Specifying the “page size”

Probably the first thing to consider is what “size” you want your result set to be, and by that we mean the max number of objects returned per request. You can specify this using the first parameter.

Defaults

Marketplacer employ the following page limits across the GraphQL api:

Default Page Size (if none specified): 20,000
Maximum Page Size: 20,000 unless otherwise specified

So requesting the “first” 3 Adverts using advertsWhere, would look something like this:

query productPagination {
  advertsWhere(first: 3) {
    nodes {
      id
      legacyId
      title
    }
  }
}

Executing that query will return up to 3 Advert objects,

{
  "data": {
    "advertsWhere": {
      "nodes": [
        {
          "id": "QWR2ZXJ0LTEwMDAwNTMzMQ==",
          "legacyId": 100005331,
          "title": "Mens T-Shirt"
        },
        {
          "id": "QWR2ZXJ0LTEwMDAwNTMzMg==",
          "legacyId": 100005332,
          "title": "Plastic Brush"
        },
        {
          "id": "QWR2ZXJ0LTEwMDAwNTM0MA==",
          "legacyId": 100005340,
          "title": "Boxer Shorts"
        }
      ]
    }
  }
}

That’s great and everything, but how do we get to see the next page, (if there is one)?

Do we have more pages?

In order to retrieve the next page of data, we first need to determine if indeed there is a next page, (we have more than 3 Adverts), or if in fact there isn’t a next page (we have 3 or fewer Adverts).

To do this we introduce the use of pageInfo.

Updating our query to the following details the use of pageInfo:

query productPagination {
  advertsWhere(first: 3) {
    pageInfo{
      hasNextPage
    }
    nodes {
      id
      legacyId
      title
    }
  }
}

Running that query again we see:

{
  "data": {
    "advertsWhere": {
      "pageInfo": {
        "hasNextPage": true
      },
      "nodes": [
        {
          "id": "QWR2ZXJ0LTEwMDAwNTMzMQ==",
          "legacyId": 100005331,
          "title": "Mens T-Shirt"
        },
        {
          "id": "QWR2ZXJ0LTEwMDAwNTMzMg==",
          "legacyId": 100005332,
          "title": "Plastic Brush"
        },
        {
          "id": "QWR2ZXJ0LTEwMDAwNTM0MA==",
          "legacyId": 100005340,
          "title": "Boxer Shorts"
        }
      ]
    }
  }
}

In this case pageInfo and specifically hasNextPage tells us that we do indeed have more pages…

Introducing our cursor

To retrieve the next page of data we need to know how to get to the next page, we do this by specifying endCursor in pageInfo as follows:

query productPagination {
  advertsWhere(first: 3) {
    pageInfo{
      hasNextPage
      endCursor
    }
    nodes {
      id
      legacyId
      title
    }
  }
}

This will yield the following result:

{
  "data": {
    "advertsWhere": {
      "pageInfo": {
        "hasNextPage": true,
        "endCursor": "Mw"
      },
      "nodes": [
        {
          "id": "QWR2ZXJ0LTEwMDAwNTMzMQ==",
          "legacyId": 100005331,
          "title": "Mens T-Shirt"
        },
        {
          "id": "QWR2ZXJ0LTEwMDAwNTMzMg==",
          "legacyId": 100005332,
          "title": "Plastic Brush"
        },
        {
          "id": "QWR2ZXJ0LTEwMDAwNTM0MA==",
          "legacyId": 100005340,
          "title": "Boxer Shorts"
        }
      ]
    }
  }
}

We can then “request” this page of data the next time we make a request, we do this by updating our query as follows:

query productPagination {
  advertsWhere(first: 3, after:"Mw") {
    pageInfo{
      hasNextPage
      endCursor
    }
    nodes {
      id
      legacyId
      title
    }
  }
}

Noting that we have used the value of endCursor from our previous query as the value of the after parameter.

This will result in the next page of 3 Adverts, observing that we now have a different value for endCursor that will be used to retrieve the next page, and so it goes on…

{
  "data": {
    "advertsWhere": {
      "pageInfo": {
        "hasNextPage": true,
        "endCursor": "Ng"
      },
      "nodes": [
        {
          "id": "QWR2ZXJ0LTEwMDAwNTM0MQ==",
          "legacyId": 100005341,
          "title": "iPhone"
        },
        {
          "id": "QWR2ZXJ0LTEwMDAwNTM0Mg==",
          "legacyId": 100005342,
          "title": "Blue Training Potty"
        },
        {
          "id": "QWR2ZXJ0LTEwMDAwNTM0Mw==",
          "legacyId": 100005343,
          "title": "Green Training Potty"
        }
      ]
    }
  }
}

Total Count

A final useful piece of information that you can query when using pagination, is the totalCount of records, this is added outside of the pageInfo object as follows:

query productPagination {
  advertsWhere(first: 3, after:"Mw") {
    totalCount
    pageInfo{
      hasNextPage
      endCursor
    }
    nodes {
      id
      legacyId
      title
    }
  }
}

The result set would look like this:

{
  "data": {
    "advertsWhere": {
      "totalCount": 24,
      "pageInfo": {
        "hasNextPage": true,
        "endCursor": "Ng"
      },
      "nodes": [
        {
          "id": "QWR2ZXJ0LTEwMDAwNTM0MQ==",
          "legacyId": 100005341,
          "title": "iPhone"
        },
        {
          "id": "QWR2ZXJ0LTEwMDAwNTM0Mg==",
          "legacyId": 100005342,
          "title": "Blue Training Potty"
        },
        {
          "id": "QWR2ZXJ0LTEwMDAwNTM0Mw==",
          "legacyId": 100005343,
          "title": "Green Training Potty"
        }
      ]
    }
  }
}

Summary

The table below summarizes the main paging concepts, and specifically how they relate to GraphQL querying:

	Moving Forwards
Specify Page Size	first
Determine Page Availability	hasNextPage
Get Cursor	endCursor
Use Cursor	after
Total number of records for query	totalCount