Data Availability & Design

This page exists as a guideline on how to design the full authentication flow for your users based on when you can expect newly authenticated data to be available for each data type.

Wellness Data

After a user connects a source, we will immediately start the process of querying their historical data from the source to make it available to your application. However, some consideration must be made for the time it takes to do so when designing the user flow in your system.

Best Practices: Asynchronous Update Process

If you would like to update your UI to reflect a successful connection, the best thing to do is to query the /sources endpoint. The source and supportedDataTypes properties here will be available as soon as you have an accessToken for the user.

You can then rely on Notifications to know when data is available to sync and use in your application. For the Wellness API, these should begin to appear within seconds of a successful authentication.

🚧

When checking the /sources endpoint immediately after Connect finishes, the devices array may not yet be updated. It is updated on the first successful sync with the source.

How long will it take for all data to be available?

We start synchronizing user data as soon as a user connects a source and selects "Done" in Connect. Specifically, we start pulling the most recent data from the source and work backward with a full historical data sync from there. As new data comes in, we will send you notifications to then sync that data for your users.

As a result, you can expect to receive recent data very soon after a user connects, but it is often best to communicate to your users that historical data will continue to come in and to check back later if they don't immediately see all of their information. See the Source Sync Status page for more information on querying the API about connected sources and the status of Human API data syncs with the external source.

πŸ“˜

Historical Data Sync

The speed at which a historical sync works is based on two things:

A) Source Rate Limits
B) How much historical data the user has

Source Rate Limits dictate the amount of data we can query per user in a given period of time from the specific source. We have negotiated higher rate limits for nearly all supported sources, so we can sync data as fast or faster than any other third party. However, as a result of the limits, full historical data cannot always be retrieved instantaneously.

Additionally, when limits do need to be accounted for our system smartly schedules data retrieval to maximize these source rate limits without compromising our ability to stay on track of new data for the specific user from that source.

Overall, due to variations in data volume per user and rate limiting per source, there is no universal timeframe for a full historical sync, so it is best to rely on Notifications so that you can retrieve the data as soon as possible.

Medical Data

We pull raw Medical Record data directly from healthcare providers, process and normalize the data, and then make it available via the API endpoints. In addition, healthcare providers provide differing means of accessing patient medical data so some data takes longer to retrieve than others. As a result, we recommend implementing an asynchronous update process.

Best Practices: Asynchronous Update Process

Due to the time needed for retrieval and processing, after a user authenticates their medical records it is best to initially indicate the successful connection in your application and that their records are being retrieved. When the data is available, you can then follow up with a notification or email telling them when their records have been retrieved and can be viewed.

Specifically, once a user's historical data is available, we will send you a notification like the following:

[{
  "humanId": "777e72a36b060995a551e1d9d4fa37fc",
  "updatedAt": "2015-08-28T05:13:45+00:00",
  "type": "externalaccount",
  "model": "externalaccount",
  "action": "updated",
  "objectId": "54cd4697a7b875300999ff44",
  "endpoint": "https://api.humanapi.co/v1/human/sources"
}]

If you then query https://api.humanapi.co/v1/human/sources/<objectId> using this information, you will see a payload similar to the one below that will indicate via historySync.status whether or not the historical synchronization is complete.

{
  "id": "54cd4697a7b875300999ff44",
  "source": "54dc427aaa6b4cb7d6203061",
  "connectedSince": "2015-01-31T21:18:15.672Z",
  "devices": [
  	"54dc427aaa6b4cb7d6203061"
  ],
  "historySync": {
  	"status": "completed"
  },
  "sourceName": "Cleveland Clinic",
  "organization": {
    "id": "53c050ac51c69003200aa998",
    "name": "Cleveland Clinic",
    "href": "/medical/organizations/53c050ac51c69003200aa998"
  }
}

πŸ“˜

You should also check this endpoint and others you are working with as soon as you receive the user accessToken. In the event that the account data has been previously retrieved, the data will be immediately available.

How long will it take for all data to be available?

Similar to the Wellness API, this depends largely on the capabilities of the source and the amount of medical data that the user has authenticated. We retrieve the information as fast as the particular source will allow and notify you of its availability via Notifications once it is fully processed and available for your application via the API.