Data Synchronization

In order to keep your user data as up-to-date as possible, we recommend a two-step approach using both Notifications and Batch Queries

Primary Mechanism: Notifications

Notifications are sent whenever a user adds a new data source or has new data available. For this reason, notifications should be the primary mechanism for receiving the latest user data from Human API.

In order to receive notifications, you must implement a listener on your server that will receive notifications and subsequently pull the updated data from the appropriate endpoint.

📘

To implement Notifications, see the guide here

Secondary Mechanism: Batch Queries

A Batch Query allows you to pull data from a specific endpoint for all of your users at once for a specified time period. We recommend that Batch Queries be implemented at regular intervals to ensure that your user data is up to date in the event that a notification fails to deliver. A good practice is to rely on notifications for the ongoing data synchronization and do a few catch-up synchronizations each 24 hour period.

You need to keep track of the last query time for your batch queries so that you only query for the data that was updated since the last time you ran the query.

🚧

Please do not perform regular batch queries without the updated_since query parameter. By supplying updated_since with your last query time, you can ensure that your queries are quick and efficient. You should schedule your queries frequently enough so that you regularly receive a small number of data pages per query.

Making Efficient Queries

For both the individual and batch endpoints, you can utilize the updated_since query parameter to retrieve only data that has been updated since a certain date, often the 'last updated' timestamp for you local app. Doing so can make the synchronization efficient and reliable, and will ensure that you get data that is recently added or changed even it if is older than the most recent data retrieved.

To do this you should use the updated_since query parameter and a date in the format YYYYMMDDTHHmmssZ. This is an example query for an individual endpoint:

curl https://api.humanapi.co/v1/human?...&updated_since=20140110T000000Z

📘

If the queries return more than the default 50 items per page, you should make sure you use the Link Header to retrieve all the pages of data. This can be done by walking through links until there are no more "next" links available.

See Patterns and Conventions section for more details on using the updated_since query parameter and how to process paginated results.