How to implement Pagination and Mutation in GraphQL

This article is in continuation of the previous one on Building a GraphQL Server using NodeJs and Express. In the previous article, we started on building a GraphQL server for food ordering system and we were able to fetch details of the restaurants, customers and the existing orders. In this article, we will take it forward by adding pagination for the queries and placing orders using GraphQL mutations.

Quick heads-up on what we have built so far:

Use-case:

Customers can place orders in any restaurants of their choice.

Technical Dependencies:

We're using the node modules express and express-graphql as dependencies for building the GraphQL server. The root level query is of type RootQueryType and it has the following fields:

restaurants: for fetching the list of all the restaurants

restaurant: takes an input parameter called id and returns the details of a particular restaurant

customers: for fetching the list of all the customers

customer: takes an input parameter called id and returns the details of a particular customer

orders: for fetching the list of all the orders

Here's an overview of how the restaurants field fetches the list of restaurants:

Image loading...

You can always refer to this GitHub repository if you get lost anywhere in the code. With that said, let's continue building our food ordering system.

Adding Pagination in the GraphQL queries

It is a good practice to implement pagination in data-heavy queries. For example, fetching all of the restaurants in one-go might put a load on the server. We can request restaurants in batches using pagination. Let's see how this would work:

Fetching the first 2 restaurants:

We'll have to modify the restaurants' query to fetch only the first 2 restaurants as below:

javascript
{
    restaurants (first:2) {
        id
        name
        email
        location
    }
}

If you try to run this query in the graphiql, you will get the following error:

"Unknown argument "first" on field "restaurants" of type "RootQueryType"."

The error occurs because we have not yet added first as an argument in the restaurants field. Let's do that right away!

javascript
restaurants: {
    type: GraphQLList(Restaurant),
    args: {
        first: { type: GraphQLInt }
    },
    resolve (parentValue, args) {
        return axios.get(`http://localhost:4200/restaurants?_end=${args.first}`)
            .then(res => res.data)
    }
}

The restaurants field now accepts an argument first of type GraphQLInt. Notice how we have changed the URL in the resolve function. The _start and _end query parameters are used to slice the original array, with _start specifying the start index and _end specifying the length of the array. Please note this implementation of the resolve function is specific to the json-server. We'll see how to implement this in SQL and MongoDB in the later articles.

If we have to keep iterating through the list, there should be a means to specify the value for offset to the server. The offset property would tell the server to send in the records after this particular one. Let's say if we specify first as 2 and offset as 1, the server would send the first two records that are placed after the offset record.

Here's how our restaurants would look after adding the offset property:

javascript
restaurants: {
    type: GraphQLList(Restaurant),
    args: {
        first: { type: GraphQLInt },
        offset: { type: GraphQLInt }
    },
    resolve (parentValue, args) {
        return axios.get(`http://localhost:4200/restaurants?_start=${args.offset}&_end=${args.first+args.offset}`)
            .then(res => res.data)
    }
}

The argument offset is of type GraphQLInt and _end param now accepts the value as args.first + args.offset to return the list of restaurants after the offset. Let's check this out in graphiql:

Image loading...

The above query returns two restaurants that are placed after the offset.

This approach of paginating records using first and offset is ideal for static data. However, it is not recommended for the data that changes over time. The GraphQL specification recommends Cursor-based pagination.

Before getting into the details of Cursor-based pagination, let's first try to understand the problems with the offset approach:

Problems with the Offset-based Pagination Technique

Image loading...

As it can be seen from the above image, we're fetching the records in batches of 5. When offset is 0, we fetch the topmost 5 records. While we were fetching the next 5 records, two new records got added in the system. We'll get the next 5 records from the 7th one as the value of the offset is 5. Something went wrong here! We fetched the records 7 and 6 twice.

Cursor-based Pagination

There should be a way to tell the server about the last fetched record on the client-side instead of relying on the offset value. We can do so by using cursors. A cursor is basically a unique string that identifies a particular row. The server can send the row data along with its respective cursor. The client can then use this cursor for fetching records and that would solve our problem! Please note: we're not relying on the offset anymore; rows are being identified by their respective cursors here! This approach is being used by Facebook, Twitter, and GitHub for fetching the paginated records.

Let's see how we would implement this in our case:

How to add the cursor information in nodes?

The types Restaurant, Customer, etc specifically define the properties of the respective fields and cursor is a connection related information. We cannot add cursor in these types directly. We can do something like this:

javascript
{
    restaurants {
        edges {
            node
            cursor
        }
    }
}

The edges field now includes information about the node as well as the cursor. We can also include the total count of nodes along with the page information in the above query as:

javascript
{
    restaurants {
        totalCount
        edges {
            node
            cursor
        }
        pageInfo {
            startCursor,
            endCursor,
            hasNextPage
        }
    }
}

The above fields are self-explanatory. We're all good with the query! Let's modify the resolve function of the restaurants field to accommodate the above changes.

Adding new types - Edge, PageInfo and Page

I'm creating a generic function for these types so that they can also be used with other fields like Customer and Order. Let's create a new file called pagination.js and add the following code:

javascript
const graphql = require('graphql')

const {
    GraphQLString,
    GraphQLInt,
    GraphQLBoolean,
    GraphQLObjectType,
    GraphQLList
} = graphql

const Edge = (itemType) => {
    return new GraphQLObjectType({
        name: 'EdgeType',
        fields: () => ({
            node: { type: itemType },
            cursor: { type: GraphQLString }
        })
    })
}

const PageInfo = new GraphQLObjectType({
    name: 'PageInfoType',
    fields: () => ({
        startCursor: { type: GraphQLString },
        endCursor: { type: GraphQLString },
        hasNextPage: { type: GraphQLBoolean }
    })
})

const Page = (itemType) => {
    return new GraphQLObjectType({
        name: 'PageType',
        fields: () => ({
            totalCount: { type: GraphQLInt },
            edges: { type: new GraphQLList(Edge(itemType)) },
            pageInfo: { type: PageInfo }
        })
    })
}

module.exports = {
    Page
}

The above code defines types for Edge, PageInfo and Page. We'll import the Page function in our main schema.js file to use it with the restaurants field. This all looks good but how do we generate value for the cursor?

Helper functions for cursor based Pagination

Cursors are usually generated using base64 encoding. You can read more on the Base64 encoding and decoding here.

Following are the two functions in JavaScript used for encoding and decoding strings:

btoa: creates a base64 encoded ASCII string from a string of binary data

atob: decodes a string of data that has been encoded using base64 encoding

Let's write a few helper functions in the pagination.js file for handling the cursor value using base64 encoding decoding:

javascript
const convertNodeIdToCursor = (node) => {
    return new Buffer(node.id, 'binary').toString('base64')
}

const convertCursorToNodeId = (cursor) => {
    return new Buffer(cursor, 'base64').toString('binary')
}

Your final pagination.js file should look like this.

There's just one thing left in our cursor based implementation -- modifying the restaurants field to make room for cursor based pagination. Let's do that right away!

Modifying the type and args keys of the restaurants field

javascript
restaurants: {
    type: Page(Restaurant),
    args: {
        first: { type: GraphQLInt },
        afterCursor: { type: GraphQLString }
    }
}

Notice the value of the type as Page(Restaurant).

Modifying the resolve function

We'll first extract the id of the node using the value of the argument afterCursor and then we'll find its index in the main restaurants array.

javascript
resolve (parentValue, args) {

    let { first, afterCursor } = args
    let afterIndex = 0

    return axios.get(`http://localhost:4200/restaurants`)
        .then(res => {
            let data = res.data

            if (typeof afterCursor === 'string') {
                /* Extracting nodeId from afterCursor */
                let nodeId = convertCursorToNodeId(afterCursor)
                /* Finding the index of nodeId */
                let nodeIndex = data.findIndex(datum => datum.id === nodeId)
                if (nodeIndex >= 0) {
                    afterIndex = nodeIndex + 1 // 1 is added to exclude the afterIndex node and include items after it
                }
            }

            const slicedData = data.slice(afterIndex, afterIndex + first)
    })
}

In the above code, we're first fetching the entire list of restaurants and then slicing it as per the requirement. This is done only to demonstrate how cursor-based pagination works. In real-world scenarios where you will work with databases like MongoDB or MySQL, you would be querying on the indexed column instead of fetching the entire list. We'll see this in action in the GraphQL Advanced series.

Creating the edges object to include the node and its respective cursor

javascript
const edges = slicedData.map (node => ({
    node,
    cursor: convertNodeToCursor(node)
}))

Notice how we are using the helper function convertNodeToCursor from the pagination file to encode the nodeId

Finding the values for startCursor, endCursor and hasNextPage

javascript
let startCursor, endCursor = null
if (edges.length > 0) {
    startCursor = convertNodeToCursor(edges[0].node)
    endCursor = convertNodeToCursor(edges[edges.length - 1].node)
}
let hasNextPage = data.length > afterIndex + first

Returning the Page type from the restaurants field

javascript
return {
    totalCount: data.length,
    edges,
    pageInfo: {
        startCursor,
        endCursor,
        hasNextPage
    }
}

Wrapping-up the resolve function

Finally, this is how your resolve function should look after doing all the above changes

javascript
resolve (parentValue, args) {

    let { first, afterCursor } = args
    let afterIndex = 0

    return axios.get(`http://localhost:4200/restaurants`)
        .then(res => {
            let data = res.data

            if (typeof afterCursor === 'string') {
                /* Extracting nodeId from afterCursor */
                let nodeId = convertCursorToNodeId(afterCursor)
                /* Finding the index of nodeId */
                let nodeIndex = data.findIndex(datum => datum.id === nodeId)
                if (nodeIndex >= 0) {
                    afterIndex = nodeIndex + 1 // 1 is added to exclude the afterIndex node and include items after it
                }
            }

            const slicedData = data.slice(afterIndex, afterIndex + first)
            const edges = slicedData.map (node => ({
                node,
                cursor: convertNodeToCursor(node)
            }))

            let startCursor, endCursor = null
            if (edges.length > 0) {
                startCursor = convertNodeToCursor(edges[0].node)
                endCursor = convertNodeToCursor(edges[edges.length - 1].node)
            }
            let hasNextPage = data.length > afterIndex + first

            return {
                totalCount: data.length,
                edges,
                pageInfo: {
                    startCursor,
                    endCursor,
                    hasNextPage
                }
            }
    })
}

Fetching results using Cursor based pagination

Let's check the below query for fetching the first two records in graphiql:

javascript
{
    restaurants (first: 2) {
        totalCount
        pageInfo {
            startCursor
            endCursor
            hasNextPage
        }
        edges {
            node {
                id
                name
                location
                email
            }
            cursor
        }
    }
}

Here's the result from the graphiql:

Image loading...

The structure of the response is as expected! The value of hasNextPage is true because there is one more restaurant. Please note the value of cursor for each of the rows. Let's fetch the third row using the cursor of the second row:

javascript
{
    restaurants (first: 1, afterCursor: "cmVzXzI="){
        totalCount
        pageInfo {
            startCursor
            endCursor
            hasNextPage
        }
        edges {
            node {
                id
                name
                location
                email
            }
            cursor
        }
    }
}

Here's the result of the above query:

Image loading...

The value of the afterCursor argument is cmVzXzI=, which is the cursor of the second row. Notice the value of hasNextPage as false because there are no more restaurants left. And with this, we have completed the implementation for cursor based pagination! Let's get to the another interesting topic in GraphQL called Mutations.

What are GraphQL Mutations

It is a common practice in REST to use POST, PUT or DELETE requests to make changes or side-effects on the server. GraphQL follows the same ideology by using another ObjectType called mutation for causing side-effects on the server.

Let's create an ObjectType for mutation in the schema.js file as below:

javascript
const mutation = new GraphQLObjectType({
    name: 'Mutation',
    fields: () => ({
        addOrder: {
            type: Order,
            args: {
                customerId: { type: new GraphQLNonNull(GraphQLString) },
                restaurantId: { type: new GraphQLNonNull(GraphQLString) },
                order: { type: new GraphQLNonNull(GraphQLList(GraphQLString)) }
            },
            resolve (parentValue, args) {
                let { customerId, restaurantId, order } = args
                return axios.post(`http://localhost:4200/orders`, {
                    customerId,
                    restaurantId,
                    order
                }).then(res => res.data)
            }
        }
    })
})

It has one field addOrder. The addOrder field returns an object of type Order. The addOrder field takes in customerId, restaurantId and order as the input. The type of these arguments is wrapped in GraphQLNonNull type to make sure these arguments are present while performing the addOrder mutation. The resolve function simply makes a POST request to the json-server to save the newly created order. Let's create our very first order!

javascript
mutation {
    addOrder (customerId: "ct_3", restaurantId: "res_2", order: ["mitem_1"]) {
        id
        order
    }
}

The customer ct_3 places an order for mitem_1 from the restaurant res_2. Don't worry about the weird ids for customer, restaurants and menu items as of now. They would make sense once we build the front-end for this project.

The above query returns the following result:

javascript
{
    "data": {
        "addOrder": {
            "id": "KPOPXmF",
            "order": [
                "mitem_1"
            ]
        }
    }
}

The value of id is generated automatically by the json-server. Let's query the orders field to check if our order was placed successfully in the system:

Image loading...

There you go! Our system is working as expected. Let's write mutations for updating and deleting orders.

javascript
updateOrder: {
    type: Order,
    args: {
        id: { type: new GraphQLNonNull(GraphQLString) },
        order: { type: new GraphQLNonNull(GraphQLList(GraphQLString)) }
    },
    resolve (parentValue, args) {
        let { id, order } = args
        return axios.patch(`http://localhost:4200/orders/${id}`, {
            order
        }).then(res => res.data)
    }
}

The updateOrder takes in the updated order as an argument and patches it to the existing order. Let's see this in action in graphiql:

The customer ct_2 had initially ordered mitem_3 from the restaurant res_3. He has now changed his mind and would like to order mitem_2 from the same restaurant.

Image loading...

Notice the updated value of the order returned after executing the mutation.

Here's the code for deleting a particular order:

javascript
deleteOrder: {
    type: Order,
    args: {
        id: { type: new GraphQLNonNull(GraphQLString) }
    },
    resolve (parentValue, args) {
        let { id, order } = args
        return axios.delete(`http://localhost:4200/orders/${id}`)
            .then(res => res.data)
    }
}

The deleteOrder field accepts only the id of the order and deletes that order from the system. Let's check this out in graphiql:

Image loading...

The value of the id is null as that order no longer exists in the system. You can find the entire code for building a GraphQL server in this GitHub repository

Conclusion

In this tutorial, we learned how to implement pagination using offset and cursor-based approaches. We also learned how to implement mutations to create, update and delete records from the database.

Next in series: Introducing the Apollo GraphQL Platform for implementing the GraphQL Specification