Pagination / Filter

API resources need to be paginated when the number of records growths. Pagination concept does not come alone. To obtain subset of a resource, data can be filtered then ordered and paginated.

πŸ‘‰ Operation order is: Filter β‡’ Sort β‡’ Paginate

We use document paths for filtering and sorting. In this case the document is the response payload. Path should be a top level or a nested property of the document. We can restrict this usage to a subsets of document path.

1. Filter

To filter data, you should use the filter keyword in your query string.

You can use a predefined subset of document paths for filtering.

Query can be composed with many filters.

Filters use a function like syntax eq(path,value) .

1.1 Comparison operators

You can use operators to filter data with logical comparison.

Filter path can be used to build object filters. In this case you can use a reducer in your server side code to rebuild the object from filter parameters.

# date should be greater than or equal 2020-01-01
curl https://api.tld/resource?filter=gte(date,2020-01-01)

Comparison filter is like a function that is taking two parameters.
The first parameter is the comparison path and the second parameter is the comparison value.

curl https://api.tld/resource?filter=eq(path,value)

The in and nin operators can take more than on values.

Operator first parameter is the path. Rest of parameters are the values.

# name should be 'alice' or 'bob'
curl https://api.tld/resource?filter=in(name,alice,bob)

# name should be different than 'alice' or 'john'
curl https://api.tld/resource?filter=nin(name,alice,john)
eqMatches values that are equal to a specified value.1
neqMatches all values that are not equal to a specified value.1
gtMatches values that are greater than a specified value.1
gteMatches values that are greater than or equal to a specified value.1
ltMatches values that are less than a specified value.1
lteMatches values that are less than or equal to a specified value.1
inMatches any of the values specified in an array.1..n
ninMatches none of the values specified in an array.1..n
likePattern matching in strings. SQL Format.1
nlikeNegated version of the like operator. SQL Format.1
regexIn last instance we can use regex filter. JavaScript Format.1

For example, if you want to filter advances to get those that has a LATE status, to know which of your client is also late to repay you :

curl https://api.tld/advances?filter=eq(status,LATE)

1.2 Logical operators

When the query contains more than one filter, you can define by what type of logical relationship they are linked.

Logical filter is not mandatory. Implicitly the default logical relationship is a logical and.

curl https://api.tld/resource?filter=and(eq(owner.type,company),eq(,123))
andAll filter constraints should be satisfied.default
orOne of the filter constraint should be satisfied.

To add logical comparison, you need to wrap the value with an operator using parenthesis.

1.3 Formats


Dates values should follow the ISO 8601 spec.

2. Sort

You can sort data using the sort keyword in the query string.

2.1 Sort order

By default the sort is ascendant.

For strings values we use an alphabetical order sort.

If you need to specify the sort order, you can use an operator. Sort operators use the same syntax as the filter operators.

ascStarting from the lowest value to the highestdefault operator
descStarting from the highest value to the lowest
# sort data by date from older to newer
curl https://api.tld/resource?sort=date

# sort data by date from newer to older
curl https://api.tld/resource?sort=desc(date)

2.2 Sort chaining

You can also chain sorts operation.

The first sort create an ordered list of records.

Sometimes more than one records have identical values in the field used to perform the sort which we will call subgroups. Then, you can use an other field to sort each primary subgroup with an other property.

# sort ba date then by age
curl https://api.tld/resource?sort=date&sort=desc(age)

2.3 Limitations

You can not define more than one operator per field.

2.4 Sortable keys

If paths in the sort input aren't valid, a 400 BAD REQUEST is returned.

$ cat resource.ts
export class Resource {
	static sortableKeys = ['createdAt', 'updatedAt']

$ curl https://api.tld/resource?sort=foo
	"message": "Cannot sort on foo"
	"code": "BAD_REQUEST"
	"status": 400

3. Paginate

Pagination can be useful if you don't need the entire data set.

Pagination uses two concepts: slicing and sizing. Those concepts will be used to query a specific part of the entire data set.

Key wordTypeDescriptionConcept
afterstringCreate subset from cursor to the last record.slicing
beforestringCreate subset from first record to the cursor.slicing
firstunsigned integerTake the n first records of the subset.sizing
lastunsigned integerTake the n last records of the subset.sizing

3.1 Sizing

Sizing defines the number of records that the request will return.

# get 10 first records
curl https://api.tld/resource?first=10

# get 10 records after cursor
curl https://api.tld/resource?after=eyJsYXs&first=10

3.2 Slicing

If you does not provide slicing parameter, the first item of the result will be the first item of the data set.

Slicing define the first element of the subset. Slicing use opaques cursors. When you paginate data, a cursor is returned. You can use the cursor to define the first record of tour next request.

# get records after cursor
curl https://api.tld/resource?after=eyJsYXs

# get records before cursor
curl https://api.tld/resource?before=YXN0X2l

3.3 Pagination info

The query output contains some details about pagination.

Those details are returned as http headers on the response.

X-Start-CursorstringSubset start cursor
X-End-CursorstringSubset end cursor
X-Total-Countunsigned intTotal record count
X-Has-Next-PagebooleanSubset is not the first
X-Has-Prev-PagebooleanSubset is not the last