> For the complete documentation index, see [llms.txt](https://doc.duaer.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://doc.duaer.com/guides/rest-api-process-source.md).

# A REST API as a process source: pagination, filters and incremental reads

A system without a ready-made connector can be a process source as long as it has an API that answers with JSON: set it up in the wizard, without code.
## What do you need?

A GET endpoint that answers with JSON holding a list of records, each with at least a case id, a step (or status) and a time. For an API that needs sign-in, create a Header Auth, Query Auth, Basic Auth or Bearer Auth credential in Duaer; a public API needs none.

## How do you set it up?

1. As an owner or admin, open Data sources, select Add source and choose REST API under API.
2. Enter the endpoint (without a query) and the records path, such as data.items; leave it empty when the answer is the list.
3. Add filters when needed: each goes as a query parameter with every request, such as status = open.
4. Choose the pagination: None, Page number, Offset, Cursor (the path of the next cursor in the answer), Next page address in the answer, or Link header, and enter the parameter names the API uses.
5. When the API can filter by a field such as the last change, enter the incremental parameter (such as updated_since). After Read fields, choose the incremental field (such as updated_at): each read fills the parameter with the largest value read so far and reads only what changed since.
6. Select Read fields; once you see the columns and a few records, select Next, choose the case id, step and time as for a spreadsheet, and confirm. Read the fields again after any change.

## How often does it read, and how much?

Each run reads at most 20 pages of at most 10 MB each; a long list finishes over several runs, and the next round starts from the largest incremental value read. Without an incremental parameter each round starts at the first page again; a step is never written twice.

## Is it safe?

- Direct calls use https only and never reach private, local or link-local addresses, checked again at every redirect.
- A system inside your network is read only through an on-premises data gateway that lists its address; http is allowed there.
- Credentials go only to the API’s own host: a redirect or a next page on another host stops the read and says why.
- Reading fields in the wizard uses only a credential you may use yourself.

## What are the limits?

- GET and JSON only.
- Nested objects become a.b.c columns, three levels deep; lists are kept as JSON text; 200 columns at most.
## Questions

### How is this different from the push API?

With the push API the system sends events to Duaer; with a REST API Duaer reads the system on a schedule. Use push when the system can send, REST when it can only be read.

## Related

- [Add a process source in Duaer](https://doc.duaer.com/build/process-sources.md)
- [See the process in your data and generate workflows in Duaer](https://doc.duaer.com/build/process.md)
- [Process findings and SOP conformance in Duaer](https://doc.duaer.com/build/process-findings.md)
- [ServiceNow ticket process: see where each incident and requested item waits](https://doc.duaer.com/guides/servicenow-ticket-process.md)
- [Jira issue process: see which status each issue waits in longest](https://doc.duaer.com/guides/jira-issue-process.md)