Troubleshoot Azure Monitor pipeline

This article provides guidance for common issues encountered when deploying and using Azure Monitor pipeline.

Monitor pipeline health and performance

Track the health and performance of your pipeline deployment using Azure Monitor metrics and logs.

View metrics in the Azure portal

In the Azure portal, open your Azure Monitor pipeline resource and select Monitoring and then Metrics. The following metrics are available:

Collect and view logs

Create a diagnostic setting in Azure Monitor to collect resource logs for the pipeline. You can send these logs to a Log Analytics workspace, a storage account, an event hub, or a partner solution. If you send logs to a Log Analytics workspace, you can query them in the AzureMonitorPipelineLogErrors table.

Collect logs from cluster pods

To investigate issues not visible in the Azure portal, collect logs directly from pipeline pods on your Kubernetes cluster.

Retrieve pod logs:

kubectl logs <pod-name> -n <namespace>

#Example:
kubectl logs my-pipeline -n my-ns

Retrieve logs from a previous pod instance (useful if determining why a pod crashed):

kubectl logs <pod-name> -n <namespace> --previous

# Example:
kubectl logs my-pipeline -n my-ns --previous

Stream logs in real time:

kubectl logs <pod-name> -n <namespace> -f

# Example:
kubectl logs my-pipeline -n my-ns -f

Retrieve logs from all pipeline pods:

kubectl logs -n mon -l app.kubernetes.io/name=collector -f

# Example:
kubectl logs -n mon -l app.kubernetes.io/name=collector -fs

These logs often contain detailed error messages that can help identify the root cause of deployment, configuration, data collection, or connectivity issues.

No data arriving in Azure Monitor

If data isn't appearing in your Log Analytics workspace from Azure Monitor pipeline, the issue might be at any stage in the collection process. An Azure Monitor pipeline group processes data through the following stages:

Receiver → Processor → Exporter → Data Collection Endpoint → Data Collection Rule → Log Analytics Workspace

Work through the following checks in order to locate the source of the issue.

Verify pipeline group provisioning status

Before anything else, confirm the resource deployed successfully.

az resource show \
  --ids "/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Monitor/pipelineGroups/{name}" \
  --query "properties.provisioningState"

• If not Succeeded, check the deployment error details in the Activity Log.
• Confirm extendedLocation.type is "CustomLocation" and the custom ___location resource ID is valid and points to an Arc-enabled Kubernetes cluster.

Validate exporter to Log Analytics configuration

This is the most common area for mistakes. The exporter must specify three things correctly:

Common mistake: Using the DCR resource name instead of its immutable ID.

Example: Correct exporter configuration

{
  "name": "myExporter",
  "type": "AzureMonitorWorkspaceLogs",
  "azureMonitorWorkspaceLogs": {
    "api": {
      "dataCollectionEndpointUrl": "https://logs-ingestion-eb0s.eastus-1.ingest.monitor.azure.com",
      "dataCollectionRule": "dcr-6ed97ddc4a3f45b991ad89ea6930a949",
      "stream": "Custom-MyTableRawData",
      "schema": {
        "recordMap": [
          { "from": "body", "to": "Body" },
          { "from": "severity_text", "to": "SeverityText" },
          { "from": "time_unix_nano", "to": "TimeGenerated" }
        ]
      }
    }
  }
}

Check the record map schema mapping

The recordMap in the exporter's api.schema must:

• Include a mapping to TimeGenerated — Log Analytics rejects records without this column. Example: { "from": "time_unix_nano", "to": "TimeGenerated" }
• Map from fields correctly — Valid sources include body, severity_text, time_unix_nano, and attributes.{fieldName}.
  • Multi-level dot notation like attributes.foo.bar is not supported. Use bracket notation: attributes['foo.bar'].
• Map to fields that exist in the destination table — Column names must match the DCR/table schema.

Validate the custom table schema

// Check if the custom table exists and has expected columns
CustomTable_CL
| getschema

Verify the pipeline wiring in the service section

In the service.pipelines array, confirm:

• Every receiver, processor, and exporter referenced in the pipeline actually exists in the top-level arrays. A typo in a name (e.g., "otlp1" vs "OTLP1") causes MissingReceiverReference / MissingExporterReference errors.
• No duplicate references within a single pipeline (e.g., listing the same receiver twice).
• All defined components are referenced in at least one pipeline — orphaned receivers/exporters are rejected.
• Pipeline type matches the data — Use "Logs" for log data flowing to Log Analytics.

Example: Correct pipeline wiring

{
  "service": {
    "pipelines": [
      {
        "name": "MyPipeline1",
        "type": "Logs",
        "receivers": ["otlp1", "MySyslog1"],
        "processors": ["batchproc1"],
        "exporters": ["myExporter"]
      }
    ]
  }
}

Check receiver endpoints and network connectivity

• Endpoint format: Must be host:port (e.g., 0.0.0.0:514). IPv6 uses [::]:port.
• No port conflicts: Two receivers cannot bind the same port. Check for ReceiverPortConflict errors.
• Firewall or NSG rules: Ensure the source (application, syslog forwarder, OTLP client) can reach the receiver endpoint on the Arc-connected Kubernetes cluster.
• For Syslog receivers: Verify transportProtocol is "tcp" or "udp" (not "http"), and allowedFormats includes the format your source sends (e.g., syslogRfc5424, cefRfc3164).

Supported receiver types

Test connectivity from source

# For TCP syslog
nc -zv <pipeline-endpoint-ip> 514

# For OTLP gRPC
grpcurl -plaintext <pipeline-endpoint-ip>:4317 list

Validate the data collection rule and endpoint

Even if the pipeline is configured correctly, the DCR/DCE layer can block ingestion:

• DCR stream name must match the exporter's stream value exactly.
• DCR must be associated with the correct Log Analytics workspace.
• DCE must be accessible from the Kubernetes cluster — check firewall/private endpoint configuration.
• For private link setups: Confirm the DCE is added to the Azure Monitor Private Link Scope (AMPLS) and DNS resolves correctly.

Check DCR ingestion health

_LogOperation
| where Category == "Ingestion"
| where TimeGenerated > ago(1h)
| where Detail has "error" or Level == "Warning"

Check for ingestion limits with the daily cap

Log Analytics enforces a daily ingestion cap. Once hit, all data collection stops for the rest of the 24-hour window.

1. Go to your Log Analytics workspace → Usage and Estimated Costs → Daily Cap.
2. Check if the cap was reached. If so, increase it or reduce ingestion volume.

Check current ingestion volume

Usage
| where TimeGenerated > ago(1d)
| summarize TotalGB = sum(Quantity) / 1000 by DataType
| order by TotalGB desc

Validate transform statements

If your pipeline uses a TransformLanguage processor:

• The KQL transform statement must be syntactically valid.
• When targeting standard tables (e.g., Syslog, CommonSecurityLog), the transform must preserve all required columns — particularly TimeGenerated.
• Invalid transforms produce InvalidTransformStatement or InvalidTransformStatementForStreamSchema errors.

Check persistent storage configuration

If the exporter has persistent storage configured:

• A persistentVolumeName must be specified in service.persistence. Without it, you'll get PersistenceRequiredForExporterPersistence.
• The retentionPeriod must not exceed 2,880 minutes (2 days).
• Verify the Kubernetes PersistentVolume exists and is bound.

Check TLS configuration

If receivers reference a tlsConfiguration:

• The certificate and private key must be valid and accessible.
• Private keys must be stored in a Kubernetes Secret (not a ConfigMap).
• Expired or mismatched certificates will prevent data sources from connecting.

Inspect Arc extension health

The pipeline group runs as an Arc extension on Kubernetes. Verify the extension status and the health of the pods on the cluster:

# Check extension status
az k8s-extension show --name <extension-name> \
  --cluster-name <cluster-name> \
  --resource-group <rg> \
  --cluster-type connectedClusters \
  --query "provisioningState"

# Check pod health on the cluster
kubectl get pods -n <pipeline-namespace>
kubectl logs <pipeline-pod> -n <pipeline-namespace>

Common issues and resolutions

Deployment and configuration issues

Data collection issues

Connectivity and reliability issues

Performance and resource issues

Still need help?

If you're unable to resolve the issue using this guidance:

1. Collect diagnostic logs from the cluster and pipeline pods
2. Review pipeline extension versions for compatibility. See Extension versions
3. Check Supported configurations to ensure your deployment is supported
4. Contact Azure support with collected logs and deployment details