> For the complete documentation index, see [llms.txt](https://docs.itoc360.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.itoc360.com/integrations/inbound-integrations/log-management/mimir-integration.md). # Mimir Integration ### Overview [Grafana Mimir ](https://grafana.com/oss/mimir/)is an open-source, horizontally scalable, highly available, multi-tenant time series database (TSDB) for long-term Prometheus metrics storage. When alert rules defined in the Mimir Ruler are evaluated and a threshold is breached, Mimir routes those alerts through an external Alertmanager, which then delivers a structured webhook payload to the platform. This integration supports automatic alert creation on firing events and automatic resolution when Alertmanager sends a resolved notification. ### Integration Flow 1. Mimir Ruler evaluates alert rules against stored metrics at a configured interval. 2. When a rule condition is met, the Ruler sends the alert to an external Alertmanager. 3. Alertmanager groups the alerts and delivers a webhook POST request to the platform endpoint. 4. When the alert condition clears, Alertmanager sends a `resolved` notification and the platform automatically closes the alert. ### Webhook Payload Schema The payload delivered to the platform follows the standard Prometheus Alertmanager webhook format (version 4). ```json { "receiver": "string", "status": "firing | resolved", "alerts": [ { "status": "firing | resolved", "labels": { "alertname": "string", "severity": "string", "env": "string" }, "annotations": { "summary": "string", "description": "string" }, "startsAt": "ISO8601 timestamp", "endsAt": "ISO8601 timestamp", "generatorURL": "string", "fingerprint": "string" } ], "groupLabels": {}, "commonLabels": { "alertname": "string", "severity": "string" }, "commonAnnotations": { "summary": "string", "description": "string" }, "externalURL": "string", "version": "4", "groupKey": "string", "truncatedAlerts": 0 } ``` *** ### Setup #### Step 1 — Create an Alert Source on the Platform 1. Navigate to **Sources** → **Add Source**. 2. Search for **Mimir** and select it. 3. Give the source a name and click **Save**. 4. Copy the generated **ITOC360 URL** and **Token**. #### Step 2 — Install and Configure Alertmanager Mimir does not include a built-in Alertmanager. You must run an external Prometheus Alertmanager and point Mimir's Ruler at it. Install Alertmanager using your preferred method (binary, Docker, Helm). Then configure it to forward alerts to the platform: **`alertmanager.yml`** ```yaml global: resolve_timeout: 5m route: receiver: itoc360-webhook group_wait: 10s group_interval: 1m repeat_interval: 4h receivers: - name: itoc360-webhook webhook_configs: - url: "https://api.itoc360.app/functions/v1/events?token=" send_resolved: true http_config: headers: x-itoc360-token: "" ``` > `send_resolved: true` is required for automatic alert resolution on the platform. #### Step 3 — Configure the Mimir Ruler Point the Mimir Ruler to your external Alertmanager in `mimir.yaml`: ```yaml ruler: alertmanager_url: http://:9093 rule_path: /etc/mimir/rules ``` #### Step 4 — Create Alert Rules Create rule files in the configured `rule_path` directory. Each file defines one or more alert groups. **Example: `rules/production.yaml`** ```yaml groups: - name: mimir-critical interval: 1m rules: - alert: MimirIngesterUnhealthy expr: | min by (cluster, namespace) ( cortex_ring_members{state="Unhealthy", name="ingester"} ) > 0 for: 15m labels: severity: critical annotations: summary: "Mimir cluster has unhealthy ingesters" description: > Mimir cluster {{ $labels.cluster }} in namespace {{ $labels.namespace }} has unhealthy ingesters. ``` > The `severity` label in `labels` is used by the platform for priority mapping (see table below). #### Step 5 — Verify the Integration After starting Mimir and Alertmanager: 1. Check Alertmanager UI at `http://:9093` — active alerts should appear there first. 2. Trigger a test rule (e.g., `expr: vector(1) > 0`) and wait for the `group_wait` period. 3. Confirm the alert appears on the platform under the source you created.
### Sample Payload The following is a real payload captured during integration testing. **ALERT (firing):** ```json { "receiver": "itoc360-webhook", "status": "firing", "alerts": [ { "status": "firing", "labels": { "alertname": "MimirIngesterUnhealthy", "cluster": "prod-eu", "namespace": "monitoring", "severity": "critical" }, "annotations": { "summary": "Mimir cluster has unhealthy ingesters", "description": "Mimir cluster prod-eu in namespace monitoring has unhealthy ingesters." }, "startsAt": "2026-03-09T22:28:43.863Z", "endsAt": "0001-01-01T00:00:00Z", "generatorURL": "http://mimir:9009/graph?...", "fingerprint": "34e164e9af873ac1" } ], "groupLabels": {}, "commonLabels": { "alertname": "MimirIngesterUnhealthy", "severity": "critical" }, "commonAnnotations": { "summary": "Mimir cluster has unhealthy ingesters" }, "externalURL": "http://alertmanager:9093", "version": "4", "groupKey": "{}:{}", "truncatedAlerts": 0 } ``` **RESOLVE (resolved):** ```json { "receiver": "itoc360-webhook", "status": "resolved", "alerts": [ { "status": "resolved", "labels": { "alertname": "MimirIngesterUnhealthy", "severity": "critical" }, "annotations": { "summary": "Mimir cluster has unhealthy ingesters" }, "startsAt": "2026-03-09T22:28:43.863Z", "endsAt": "2026-03-09T23:01:00.000Z", "fingerprint": "34e164e9af873ac1" } ], "status": "resolved", "version": "4" } ``` *** ### Field Mapping Reference | Payload Field | Description | | ----------------------------------- | --------------------------------------------------------------------- | | `status` | Top-level event type: `firing` → ALERT, `resolved` → RESOLVE | | `alerts[0].fingerprint` | Unique identifier per alert label set — used for fingerprint matching | | `alerts[0].labels.alertname` | Name of the alert rule that fired | | `alerts[0].labels.severity` | Severity label from the rule definition — used for priority mapping | | `alerts[0].annotations.summary` | Short human-readable alert title | | `alerts[0].annotations.description` | Detailed description of the alert condition | | `alerts[0].startsAt` | ISO 8601 timestamp when the alert started firing | | `alerts[0].endsAt` | ISO 8601 timestamp when resolved (`0001-...` means still active) | | `commonLabels` | Labels shared across all alerts in this group | | `commonAnnotations` | Annotations shared across all alerts in this group | | `groupKey` | Alertmanager group key used for deduplication | ### Priority Mapping The platform maps the `severity` label from the alert rule to an internal priority level. | Mimir `severity` Label | Platform Priority | | ---------------------- | ----------------- | | `critical` | CRITICAL | | `error` | HIGH | | `warning` | MEDIUM | | `info` | LOW | | *(not set)* | MEDIUM (default) | > You control the `severity` label in your alert rule definitions. Use consistent values across your rule files for predictable priority routing. ### RESOLVE Detection The platform automatically resolves an alert when Alertmanager sends a payload with `"status": "resolved"`. This requires `send_resolved: true` in your Alertmanager webhook configuration (set in Step 2). The resolved event is matched to the original alert using the `fingerprint` field, which Alertmanager generates deterministically from the alert's label set. As long as the labels do not change between firing and resolution, the fingerprint will match and the alert will be closed. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.itoc360.com/integrations/inbound-integrations/log-management/mimir-integration.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.