Datadog Integration

Connect Datadog to enable agents to query metrics, logs, APM traces, and incidents during investigations.

Capabilities

Once connected, agents can:

Capability	Description
Query Metrics	Fetch time-series metrics for any Datadog metric
Search Logs	Search and analyze log data with filters
APM Traces	Retrieve distributed traces for request analysis
View Incidents	Access Datadog incident data
Check Monitors	Review monitor status and alert history

note

Write tools on this integration (mute and unmute monitors, create and update incidents) are available only to agents granted write access to it. A governance policy on write operations pauses each write for a person to approve, and every call is recorded in the Audit Trail.

Prerequisites

• A Datadog account with API access
• An API Key and Application Key
• Read permissions for the data you want to query

Setup

1

Get Your API Keys

1. Log in to your Datadog account
2. Navigate to Organization Settings → API Keys
3. Create or copy an existing API Key
4. Navigate to Organization Settings → Application Keys
5. Create or copy an existing Application Key

2

Add Integration in Autoheal

1. Go to Integrations in Autoheal
2. Click Datadog
3. Enter a name (e.g., "Production Datadog")

3

Configure Credentials

Enter the following:

• API Key: Your Datadog API key
• Application Key: Your Datadog application key
• Site: Select your Datadog site (e.g., datadoghq.com, datadoghq.eu)

4

Test and Save

Click Test Connection to check the connection, then click Save.

Alert Source Setup

You can also use Datadog as an alert source. Datadog forwards monitor alerts to Autoheal via webhook, and they trigger investigations.

1

Copy the Webhook URL

After saving the Datadog integration in Autoheal, copy the Webhook URL shown on the integration page.

2

Create the Webhook in Datadog

1. In Datadog, go to Integrations → Webhooks.
2. Click New and give it a name (e.g., Autoheal).
3. Paste the Webhook URL copied from Autoheal.

3

Replace the Default Payload Template

Datadog's default webhook payload does not include the fields Autoheal needs to route monitors ($ALERT_ID, $ALERT_TITLE, $ALERT_TRANSITION, $ALERT_PRIORITY, $TAGS, etc.). Replace the default PAYLOAD with the template below:

{
  "id": "$ID",
  "alert_id": "$ALERT_ID",
  "alert_title": "$ALERT_TITLE",
  "alert_transition": "$ALERT_TRANSITION",
  "alert_type": "$ALERT_TYPE",
  "alert_priority": "$ALERT_PRIORITY",
  "alert_cycle_key": "$ALERT_CYCLE_KEY",
  "alert_status": "$ALERT_STATUS",
  "alert_metric": "$ALERT_METRIC",
  "alert_query": "$ALERT_QUERY",
  "alert_scope": "$ALERT_SCOPE",
  "body": "$EVENT_MSG",
  "body_clean": "$TEXT_ONLY_MSG",
  "host": "$HOSTNAME",
  "tags": "$TAGS",
  "link": "$LINK",
  "date": "$DATE",
  "last_updated": "$LAST_UPDATED",
  "event_type": "$EVENT_TYPE",
  "priority": "$PRIORITY"
}

The same template is shown on the Autoheal integration page under Expected Payload with a copy button.

4

Save the Webhook

Click Save in Datadog to create the webhook.

5

Attach the Webhook to Monitors

Open any monitor you want to forward to Autoheal. Scroll to the Notify your team section. Add @webhook-YOUR_WEBHOOK_NAME (e.g., @webhook-Autoheal) to the notification message. Save the monitor.

Datadog now forwards alerts to Autoheal when the monitor triggers or recovers.

warning

If you keep Datadog's default payload template, required fields like alert_id and alert_title will be missing. Autoheal will then reject the webhook with a 400 error. Always use the template above.

Required Permissions

The API key should have at least these permissions:

Permission	Why It's Needed
metrics_read	Query metrics data
logs_read	Search log data
apm_read	Access APM traces
monitors_read	View monitor status
incidents_read	Access incident data

tip

Create a dedicated API key for Autoheal with only the permissions listed above. Avoid using keys with write permissions.

Datadog Sites

Select the correct site based on where your Datadog account is hosted:

Site	URL
US1 (default)	datadoghq.com
US3	us3.datadoghq.com
US5	us5.datadoghq.com
EU	datadoghq.eu
AP1	ap1.datadoghq.com
US1-FED	ddog-gov.com

Example Queries

Once connected, you can ask an agent questions like:

Show me CPU usage for the payment service over the last hour

Search logs for errors in the checkout service since 2pm

What monitors are currently alerting?

Show me the slowest traces for the /api/orders endpoint

Troubleshooting

403 Forbidden Error

• Check that both API Key and Application Key are correct
• Check that the keys have the required permissions
• Check that you're using the correct Datadog site

No Data Returned

• Check that the time range includes data
• Check that the metric/log query syntax is correct
• Check that the API key has access to the requested data

Connection Timeout

• Check your Datadog site selection
• Check network connectivity to Datadog APIs