Klaudia <> Grafana Integration

Overview

Klaudia is Komodor's AI SRE agent. The Grafana integration lets Klaudia pull production telemetry from your Grafana environment during an investigation. As part of a chat or RCA session, Klaudia can trigger a dedicated Grafana subagent (the "Ask Grafana" skill) that queries Grafana for the metrics, logs, traces, profiles, dashboards, alert rules, incidents, and automated investigations relevant to the issue, so its root cause analysis reflects your observability data alongside the Kubernetes and Komodor signals it already uses.

Unlike the Datadog integration, which connects through a vendor-hosted MCP server, Grafana is added as a custom MCP server - https://grafana.com/docs/grafana/latest/developer-resources/mcp/introduction , on the Klaudia Integrations page. Komodor supports two ways to reach that server, described under Deployment options below.

Note:

• Komodor AI SRE agent ("Klaudia") uses a large language model (LLM) to generate responses. LLMs have the potential to produce inaccurate responses. Always verify critical information before acting on it.
• We recommend running the integration read-only. Klaudia reads context from Grafana to inform its investigation and does not need to modify your Grafana configuration. You control this by selecting only read tools in the wizard (see Configure the integration).

How it works

1. Register a Grafana MCP server with Klaudia from Organization Settings > Klaudia Integrations, using the Add MCP Server wizard.
2. Klaudia investigates an issue as usual, from the Komodor UI, from Slack, or triggered automatically.
3. Klaudia triggers the "Grafana" skill (which needs to be created as part of this process) when production telemetry would help the investigation. This happens as part of the chat or RCA session, alongside Klaudia's other specialized skills.
4. The skill queries your Grafana environment through the Grafana MCP server for the relevant signals: metrics (Prometheus), logs (Loki), traces and spans (Tempo), profiles (Pyroscope), dashboards, alert rules and routing, incidents, Sift investigations, and annotations.
5. Klaudia folds that context into its root cause analysis, so the conclusions and remediation guidance reflect what Grafana is seeing in addition to Komodor's own data.

What Klaudia can do with Grafana

The "Ask Grafana" skill is context-gathering, not action-taking. During an investigation Klaudia uses it to answer questions its Kubernetes and Komodor data alone cannot, for example whether an error-rate spike is visible in the metrics behind a firing alert, what the logs show around the time of a restart loop, or whether a latency tail traces to a specific service or endpoint. Queries are scoped to the services and time window relevant to the investigation.

Deployment options

You can connect Klaudia to Grafana in one of two ways. Pick the option that matches how run Grafana and how you prefer traffic to reach it.

Most use Option A, because it works for any Grafana flavor and keeps traffic inside their network over the Komodor Agent Tunnel. Even when the underlying Grafana is a Grafana Cloud stack, the self-hosted MCP server can point at it (the connection target is set by HTTP headers, described below).

Prerequisites

Common to both options:

• A Komodor account with access to Organization Settings > Klaudia Integrations.
• A Komodor agent installed on the cluster you will use for the Agent Tunnel (Option A).
• A Grafana instance (OSS, Enterprise, or Cloud) with the datasources you want Klaudia to query (Prometheus, Loki, Tempo, Pyroscope, and so on) already configured and receiving data.
• A Grafana service account and token with read access to the datasources and resources in scope. Use a Viewer-level or narrowly scoped token for a read-only integration.

Additional for Option A:

• The ability to deploy the open source Grafana MCP server (grafana/mcp-grafana - https://github.com/grafana/mcp-grafana) into the cluster, for example via its Helm chart or Docker image.

Additional for Option B:

• A Grafana Cloud stack, and the Grafana Cloud MCP server enabled for that stack. This feature is in public preview.

Option A: Self-hosted Grafana MCP server (recommended)

Step 1: Deploy the Grafana MCP server in the cluster

Deploy the open source grafana/mcp-grafana server into the cluster where your Komodor agent runs. Run it in Streamable HTTP mode so Klaudia can reach it over the network (the default port is 8000 and the endpoint path is /mcp).

Recommendations:

• Run the server read-only by disabling write tools, so Klaudia can query but never modify Grafana.
• Expose it as an in-cluster Service. A typical internal address looks like:
  http://mcp-grafana.mcp-grafana.svc.cluster.local:8000/mcp
• If you use Tempo and wants trace tools, enable Tempo's built-in MCP server and add a Tempo datasource in Grafana. The Tempo tools then appear automatically with a tempo_ prefix. If Tempo is not enabled, those tools simply do not show up, and the rest of the integration is unaffected.

---
# Grafana MCP server for Klaudia RCA.
#
# Runs grafana/mcp-grafana in HEADER (multi-tenant) mode: it takes NO Grafana
# credentials of its own — each request carries the target instance + token via
#   X-Grafana-URL:     https://<stack>.grafana.net
#   X-Grafana-API-Key: glsa_...        (Grafana service-account token)
# which is exactly what Komodor's "Static Token (Headers)" auth method sends.
#
# Reach it via Komodor's "Agent Tunnel" connectivity: the Komodor agent in this
# cluster proxies to the ClusterIP Service below, so NO public Ingress/TLS is
# needed. Deploy this into the same cluster as the agent tunnel (evaluation-poc-gcp).
# In Komodor, set Server URL to the in-cluster address:
#   http://mcp-grafana.mcp-grafana.svc.cluster.local:8000/mcp
apiVersion: v1
kind: Namespace
metadata:
  name: mcp-grafana
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mcp-grafana
  namespace: mcp-grafana
  labels:
    app: mcp-grafana
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mcp-grafana
  template:
    metadata:
      labels:
        app: mcp-grafana
    spec:
      containers:
        - name: mcp-grafana
          # Pin to a released tag in production instead of :latest.
          image: mcp/grafana:latest
          args:
            - "-t"
            - "streamable-http"
            - "--address"
            - "0.0.0.0:8000"
            # Host-header allowlist (DNS-rebinding protection). "*" is safe only
            # behind a trusted ingress/proxy that controls the Host header;
            # otherwise set this to your public host, e.g. mcp-grafana.example.com
            - "--allowed-hosts"
            - "*"
            # Non-browser clients (Komodor) send no Origin; "*" disables the check.
            - "--allowed-origins"
            - "*"
          ports:
            - containerPort: 8000
              name: http
          readinessProbe:
            tcpSocket:
              port: 8000
            initialDelaySeconds: 5
            periodSeconds: 10
          livenessProbe:
            tcpSocket:
              port: 8000
            initialDelaySeconds: 15
            periodSeconds: 20
          resources:
            requests:
              cpu: 50m
              memory: 64Mi
            limits:
              cpu: 500m
              memory: 256Mi
          securityContext:
            allowPrivilegeEscalation: false
            runAsNonRoot: true
            runAsUser: 65532
            capabilities:
              drop: ["ALL"]
---
apiVersion: v1
kind: Service
metadata:
  name: mcp-grafana
  namespace: mcp-grafana
  labels:
    app: mcp-grafana
spec:
  type: ClusterIP
  selector:
    app: mcp-grafana
  ports:
    - port: 8000
      targetPort: 8000
      name: http
# No Ingress needed: Komodor's Agent Tunnel reaches this Service in-cluster.
# (If you ever switch to "Public" connectivity, add an Ingress + TLS and set
#  --allowed-hosts to that public host.)

Step 2: Create a Grafana service account token

In Grafana, create a service account with read permissions on the datasources and resources you want Klaudia to query, then generate a token for it. For a read-only integration, a Viewer-level role is usually sufficient. Keep the token handy for Step 3.

Step 3: Register the MCP server in Komodor

1. In the Komodor platform, go to Organization Settings > Klaudia Integrations, open the Integrations tab, and click Add MCP Server.

   

2. On the Connect step, fill in Connection Details:
   • Server URL: the in-cluster address of your MCP server, for example
     http://mcp-grafana.mcp-grafana.svc.cluster.local:8000/mcp
   • Transport Type: Streamable HTTP
   • HTTP Headers: add the two headers that tell the MCP server which Grafana to query:
     • X-Grafana-URL = your Grafana base URL. This can be an OSS or Enterprise URL, or a Grafana Cloud stack URL such as https://<your-stack>.grafana.net.
     • X-Grafana-API-Key = the service account token from Step 2.
3. Under Authentication, set Auth Method to No Authentication. In this topology, authentication to Grafana is handled by the headers above, and access to the MCP server itself is protected by keeping it internal to the cluster and reaching it over the Agent Tunnel.

4. Under Connectivity, select Agent Tunnel, then choose the Tunnel Cluster: the cluster where the MCP server and the Komodor agent run. This routes traffic through the Komodor agent so Komodor never connects to the server over the public internet.

   

5. Click Next to discover the available tools from the server, then continue to Configure the integration below.

Option B: Grafana Cloud hosted MCP server

Use this option when the you run Grafana Cloud and prefers not to deploy anything. There is nothing to install: Grafana hosts the MCP server, and Komodor connects to it over the public internet.

Keep these constraints in mind before recommending this path:

• It works only with Grafana Cloud environments.
• It is currently in public preview, so support is limited and behavior may change.

To register it:

1. In Komodor, go to Organization Settings > Klaudia Integrations > Integrations, and click Add MCP Server.
2. On the Connect step, fill in Connection Details:
   • Server URL: the Grafana Cloud MCP endpoint for your stack.
   • Transport Type: Streamable HTTP.
3. Under Authentication, choose the method required by the Grafana Cloud MCP server (OAuth 2.1 or a token, depending on the stack configuration).
4. Under Connectivity, select Public. Komodor connects directly to the MCP server over the public internet.
5. Click Next and continue to Configure the integration below.

Configure the integration

This step is the same for both options.

1. On the Configure step, set the Integration Name, for example grafana mcp.
2. Under Skill, select the Ask Grafana skill, or create a new one. The skill defines how Klaudia uses the integration: when to reach for Grafana during an investigation, which use cases it applies to, and its cluster scope. You can review or adjust the skill text with Edit Skill.

3. Under Available Tools, enable the tools Klaudia is allowed to call. For a read-only integration, leave write tools disabled (for example add_activity_to_incident and create_annotation). Use Enable All or Disable All to set a baseline, then adjust. The header shows how many tools are enabled, for example "62 of 73 enabled".

   Note on Tempo tools: tools with a tempo_ prefix only appear if the target Grafana has a Tempo datasource and Tempo's MCP server is enabled. If you do not see them, that is expected when Tempo is not configured.

   

4. Click Next.

Scope and clusters

1. On the Scope & Clusters step, review the Use Cases inherited from the linked skill:
   • Chat: available in Klaudia chat sessions.
   • RCA: available in Root Cause Analysis investigations.
2. Set the Integration Mode:
   • Active: the integration is enabled and used in responses.
   • Shadow: the integration runs silently for evaluation only, without affecting responses. Useful for validating a new integration before rollout.
   • Inactive: the integration is disabled.
3. Review the Cluster Scope inherited from the linked skill (All Clusters or Specific Clusters).

4. Check the Summary (integration name, server URL, tools enabled, mode, use cases, and clusters), then click Update Integration (or Create Integration for a new one).

   

Verifying the integration

• On the Klaudia Integrations > Integrations list, the integration shows a Connected status and its Mode (for example Active).
• The tool count on the Configure step reflects the tools discovered from the server (for example 62 of 73).
• Run a chat prompt that requires production telemetry, for example asking about the metrics or logs behind a recent alert, and confirm Klaudia returns Grafana-sourced context.

Permission model

• Klaudia's actions are scoped to the invoking user's Komodor RBAC. Data the user is not authorized to see in Komodor is not surfaced.
• On the Grafana side, access is bounded by the service account token you provide. Use a read-only, least-privilege token so Klaudia can query context but cannot modify Grafana configuration, dashboards, or alerts.
• Keeping write tools disabled in the wizard adds a second layer of read-only enforcement on top of the token scope.

Audit log

Klaudia's Grafana actions are recorded in the Komodor audit log, including:

• Action taken (for example "queried Grafana for metrics context")
• The user whose investigation triggered the action
• Timestamp
• Triggered by: Klaudia or User

FAQ

Which deployment option should I choose?
If you run Grafana OSS or Enterprise, or wants traffic kept inside their network, use Option A (self-hosted MCP server over the Agent Tunnel). If you run Grafana Cloud and is comfortable with a public-preview feature and public connectivity, Option B (Grafana Cloud MCP server) avoids deploying anything.

Can the self-hosted MCP server query a Grafana Cloud stack?
Yes. The MCP server's connection target is set by the X-Grafana-URL and X-Grafana-API-Key headers, so a server running in the cluster can point at an OSS, Enterprise, or Grafana Cloud instance. Option A therefore works for any Grafana flavor.

Why use Agent Tunnel instead of Public?
Agent Tunnel routes traffic through the Komodor agent in the selected cluster, which is the right choice for an MCP server that lives inside a VPC or is only reachable internally. Public is for MCP servers that are reachable over the internet, such as the Grafana Cloud MCP server.

Does Klaudia change anything in my Grafana?
Not when configured as recommended. Use a read-only service account token and leave write tools disabled in the wizard, and the integration only reads context.

A query returns no results. What should I check first?
The most common cause is a wrong or missing datasource UID, since Grafana queries are routed to a datasource by UID rather than searching everything. Confirm the datasource exists and is healthy, and that the service account token has read access to it.

I don't see the Tempo (tempo_) tools.
Those tools only appear when the target Grafana has a Tempo datasource and Tempo's built-in MCP server is enabled. They are an optional add-on and are not required for the integration to work.

What is Shadow mode for?
Shadow mode runs the integration silently for evaluation without affecting Klaudia's responses. Use it to validate a new Grafana integration before switching it to Active.