Skip to main content

Sync integrations

Integrations are the main source of the Context Graph. Each integration connection reads the objects it can see, such as repositories, services, teams and people, and stores each one as a source record. The resolver then links each record to one canonical entity.

What each integration brings​

IntegrationSource recordsLinked toEventsDaily reconcile (UTC)
GitHubgithub_repository, github_team, github_personrepository, team, personYes06:00
GitLabgitlab_repositoryrepositoryNo06:00
Bitbucketbitbucket_repositoryrepositoryNo06:00
Azure DevOpsazuredevops_repositoryrepositoryNo06:00
PagerDutypagerduty_service, pagerduty_team, pagerduty_personservice, team, personServices05:00
Slackslack_person, slack_team, slack_channelperson, team. Channels are their own entities.Yes07:00 people and groups, 08:00 channels
ServiceNowservicenow_service, servicenow_team, servicenow_person, servicenow_cmdb_ciservice, team, person. Configuration items are their own entities.Hourly changes04:00
Datadogdatadog_service, datadog_teamservice, teamHourly poll03:00
Jirajira_service, jira_personservice, personNo09:00
Your organization's users—personWhen a user is invited07:00

A GitHub repository with a service custom property, or a .autoheal/context-graph.yaml file, also creates or links a service. See Declare entities in a repository.

When a sync runs​

Every integration connection syncs in four ways. All four use the same path, so they produce the same result.

Four triggers, one path
Connect
Backfill: list everything
Event
Read the changed object
Schedule
Daily: list everything
Sync now
UI or API: list everything
Source records
Filtered. Missing objects are marked stale.
Resolver
Link records to entities
Context Graph
Entities and relationships

Connect​

When you connect an integration, or change its connection or sync settings, Autoheal starts a backfill. The backfill lists every object the connection can see, after your filters. Open the integration's Sync tab to follow it. A large organization can take several minutes.

Events​

GitHub, Slack and PagerDuty notify Autoheal when something changes. The change reaches the graph within minutes.

IntegrationEvents
GitHubRepository created, renamed, archived, deleted or transferred. Repositories added to or removed from the app installation. Team created, edited or deleted. Team membership and organization membership changes. Custom property changes.
SlackUser joined, profile changed, user group changed, user group members changed, channel created, renamed or archived
PagerDutyService created, updated or deleted
ServiceNowRecords changed since the last run, read every hour
DatadogService definitions and the APM service list, read every hour

An event says only which object changed. Autoheal reads that object again from the integration before it updates the record. Events that arrive late, twice, or out of order do no harm.

Schedule​

Every connection runs a full reconcile once a day, at the time in the table above. The reconcile lists every object, updates records that changed, and marks records that are no longer in the list as stale. Webhooks can be lost. The reconcile catches what they miss.

Sync now​

Run a sync immediately in three places:

WhereWhat it syncsPermission
Settings → Integrations → integration → Sync → Sync nowEvery entity type of that integration's connectionsUpdate the integration
Context Graph → Setup → Sources → Sync nowOne entity type, such as People or Repositories, from every integration that supplies itEdit entities
POST /api/v1/context-graph/syncsA connection, an entity type, or bothAs above

If the same sync is already running, Sync now joins it instead of starting a second one.

Choose what to sync​

Each integration connection has sync settings. Open Settings → Integrations → integration → Sync → Settings.

GitHub · acme  ›  Sync settings

Entity types
Repositories [Keep in sync ▾]
Teams [Keep in sync ▾]
People [Sync once ▾]

Filters
Organizations [acme] [acme-labs]
Skip archived [x]
Skip forks [x]
Only topics [prod] [service]

Target project [Payments ▾]

[Preview changes]

Entity type modes​

For each entity type a connection brings, choose a mode:

ModeBackfill on connectEvents and daily reconcileSync now
Keep in sync (default)YesYesYes
Sync onceYesNoYes
OffNoNoNo

Use Sync once for data that rarely changes, or for a one-time import that people will maintain by hand after. Use Off for an entity type that another tool owns. For example, turn off PagerDuty people when your organization's users and Slack already supply everyone.

Turning a type Off removes that connection's source records for the type at the next sync. A canonical entity stays while another source, a person, or an agent still supplies it.

Filters by integration​

Filters decide which objects become source records. An object a filter excludes is treated as missing: its record becomes stale, then removed.

IntegrationFilters
GitHubOrganizations, include topics, exclude topics, skip archived repositories, skip forks
GitLabGroups, skip archived projects, skip forks
BitbucketWorkspaces, projects, skip forks
Azure DevOpsProjects
PagerDutyTeams (services, members and schedules of these teams only), include disabled services
SlackInclude private channels, include archived channels, user groups
ServiceNowCMDB classes (default cmdb_ci_service and cmdb_ci), an encoded query for each table, active users only
DatadogEnvironments (env tag), teams
JiraProjects

Target project​

Each connection writes to one project. By default, that is the project the integration is connected to. Choose another project under Target project to send the records there instead. Entities, labels and relationships never cross projects.

Preview changes​

Every settings change shows a preview before it applies. The preview runs the same fetch and resolver steps as a sync, and changes nothing.

Preview · GitHub · acme

+214 repositories
+38 services, from the repository property service
31 of those link to existing PagerDuty services
4 need review, for example checkout could be PagerDuty's "Checkout API"
2 repositories archived since the last sync will be marked stale

[Cancel] [Apply]

Select Apply to save the settings and start a backfill. Suggestions from the preview go to the review queue.

Stale and deleted objects​

An object can disappear from an integration: a repository is deleted, a service is disabled, or a filter now excludes it. Autoheal removes it in stages, so a short outage at the source cannot empty your graph.

  • A record that is missing from a full list becomes stale. It stays in the graph, marked Not in latest sync.
  • A stale record that is missing from the next full list is removed.
  • A failed or partial list never removes anything. A list less than half the size of the previous one is treated as partial, unless you removed the connection or changed its filters.
  • Removing an integration connection removes its source records at once.
  • A canonical entity is removed only when its last source record is removed and no person declared it. Until then it stays, marked No live source if no integration reports it.

Sync status and history​

WhereWhat it shows
Context Graph → Setup → SourcesEach entity type's schedule, sources, entity count, last sync and status. Each connected integration and the entity types it brings.
The integration's Sync tabThe connection's entities, last sync, status, counts and errors, and its settings
An entity's Sources tabThe source records linked to the entity, each field's value from each source, and the original payload from the integration
GET /api/v1/context-graph/syncsEach sync's trigger, status, start and end times, counts and errors

A sync has one of these statuses: queued, running, succeeded, partial (Autoheal could not read some objects, so it removed nothing), or failed. A failed sync keeps the previous records as they were.

Last change on the Sources page is the newest update to any entity of that type. It includes manual edits, so it can be newer than the last sync.

Push from your own systems​

For tools that Autoheal does not integrate with, such as an internal CMDB, a deployment tool or a spreadsheet of service tiers, push entities yourself. Each system is a named source.

  • Push API. Start a full or delta run, send entities, then complete the run. A full run marks entities the source sent before and left out this time as stale. See Push API.

  • CLI. Send a JSON Lines file:

    BASH
    autoheal context-graph push --source internal-cmdb --full services.jsonl

    Each line is one entity:

    JSON
    {"blueprint": "service", "name": "billing", "title": "Billing", "properties": {"tier": "tier-2"}, "relations": {"owner": "finance-eng"}}

Pushed values are declared: they rank above integrations, and a sync does not overwrite them.

Declare entities in a repository​

A repository can declare the entities it builds in .autoheal/context-graph.yaml on its default branch:

YAML
# .autoheal/context-graph.yaml in acme/checkout-api
entities:
- blueprint: service
name: checkout
title: Checkout
properties:
tier: tier-1
language: go
relations:
owner: payments
depends_on: [ledger, fraud-score]
  • Autoheal reads the file on each push to the default branch (GitHub) and at each daily reconcile (all code hosts).
  • The repository becomes part_of each service the file declares.
  • Values in the file are declared. A change to the file is a change to the graph, reviewed like any other code change.
  • A monorepo can declare several services in one file.

Limits​

IntegrationLimit
GitHub, GitLab, BitbucketUp to 10,000 repositories per connection. A larger connection fails the sync. Use filters to narrow it.
PagerDuty100 API requests per minute per connection
ServiceNow120 API requests per minute, and up to 200,000 rows per table
SlackUp to 500,000 channels per workspace
Push APIUp to 1,000 entities per request

Each source record keeps the original object from the integration's API, compressed. Open it from the entity's Sources tab to see exactly what the integration sent.