WAF-AGN-070 – Agent Observability

Säule: Agentic | Schweregrad: Mittel | Kategorie: Logging Monitoring | Automatisierbar: High

Beschreibung

Agent activities must be visible and measurable. Telemetry MUST be collected for all agents including tokens, latency, GPU utilization, and decision metrics. Metrics MUST be ingested into a monitoring system. Distributed tracing MUST link agent operations end-to-end.

Begründung

Risiko	Beschreibung
Risk	Invisible agent activity delays detection of misuse, drift, or attacks.
Risk	Without metrics, agent behavior cannot be governed or continuously improved.

Risiko

Beschreibung

Risk

Invisible agent activity delays detection of misuse, drift, or attacks.

Risk

Without metrics, agent behavior cannot be governed or continuously improved.

Anforderungen

Agent activities must be visible and measurable.
Telemetry MUST be collected for all agents including tokens, latency, GPU utilization, and decision metrics.
Metrics MUST be ingested into a monitoring system.
Distributed tracing MUST link agent operations end-to-end.

Implementierungsleitfaden

Document the requirement in a version-controlled policy or standard.
Assign ownership and establish a review cadence (e.g., quarterly).
Implement the required configuration as code in Terraform or equivalent IaC.

Reifegrade

Level	Name	Kriterien
L1	Ad hoc – control not consistently applied	* Manual evidence exists for some workloads, but coverage is unknown.
L2	Standardised – policy defined and communicated	* A version-controlled policy, ADR, or standard describes the requirement.
L3	Integrated – automated validation in CI/CD	* Terraform checks run automatically on pull requests and report results.
L4	Auditable – metrics and exceptions tracked	* Control coverage, failures, and approved exceptions are reported quarterly.
L5	Optimised – continuous improvement based on data	* Control effectiveness is measured and iteratively improved.

Level

Name

Kriterien

Ad hoc – control not consistently applied

* Manual evidence exists for some workloads, but coverage is unknown.

Standardised – policy defined and communicated

* A version-controlled policy, ADR, or standard describes the requirement.

Integrated – automated validation in CI/CD

* Terraform checks run automatically on pull requests and report results.

Auditable – metrics and exceptions tracked

* Control coverage, failures, and approved exceptions are reported quarterly.

Optimised – continuous improvement based on data

* Control effectiveness is measured and iteratively improved.

Checks

waf-agn-070.tf.aws.metrics-configured

Engine: terraform | Provider: aws | Automated: Ja

Agent metrics must be configured (tokens, latency, GPU)

Agent telemetry must include token counts, inference latency, GPU utilization, and error rates. Metrics must be exported to CloudWatch or a third-party monitoring system.

Scope: resource — aws_cloudwatch_metric_alarm, aws_bedrockagent

Assertions: * alarm_name not empty — Metrics alarms must be configured for agent monitoring.

Remediation: Configure Bedrock metrics export to CloudWatch. Use CloudWatch Insights to query agent metrics. Set up dashboards for token usage, latency, and error rates. Configure alarms for anomalies.

Compliant

resource "aws_cloudwatch_metric_alarm" "agent_latency" {
  alarm_name          = "agent-latency-alarm"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 2
  metric_name         = "Latency"
  namespace           = "AWS/Bedrock"
  period              = 60
  statistic           = "Average"
  threshold           = 5000
  unit                = "Milliseconds"
  alarm_description   = "Agent latency exceeds 5 seconds"
}

resource "aws_cloudwatch_metric_alarm" "agent_tokens" {
  alarm_name          = "agent-token-alarm"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 1
  metric_name         = "TokenCount"
  namespace           = "AWS/Bedrock"
  period              = 300
  statistic           = "Sum"
  threshold           = 100000
  unit                = "Count"
  alarm_description   = "Agent token usage exceeds limit"
}

Non-Compliant

# No metrics configured
# WAF-AGN-070 Violation
# No visibility into agent performance

waf-agn-070.tf.aws.distributed-tracing-enabled

Engine: terraform | Provider: aws | Automated: Ja

Distributed tracing must be enabled for agent operations

Agent operations must be traced using AWS X-Ray. Traces must include agent ID, operation type, latency, input/output size, and decision path. X-Ray must link to CloudWatch Logs for full context.

Scope: resource — aws_lambda_function, aws_bedrockagent

Assertions: * environment.variables.AWS_XRAY_TRACING_NAME not empty — Distributed tracing (X-Ray) must be enabled.

Remediation: Enable X-Ray tracing for Bedrock invocations. Use AWS SDK with X-Ray integration. Instrument agent code with X-Ray SDK for custom segments. Configure trace sampling.

Compliant

resource "aws_lambda_function" "agent_traced" {
  function_name = "agent-traced"
  environment {
    variables = {
      AWS_XRAY_TRACING_NAME = "agent-service"
      AWS_XRAY_DAEMON_ADDRESS = "127.0.0.1:2000"
    }
  }
}

resource "aws_cloudwatch_log_group" "agent_trace_logs" {
  name = "/aws/lambda/agent-traced"
}

Non-Compliant

resource "aws_lambda_function" "agent_untraced" {
  function_name = "agent-untraced"
  # No X-Ray tracing
  # WAF-AGN-070 Violation
}

waf-agn-070.tf.aws.log-integration-configured

Engine: terraform | Provider: aws | Automated: Ja

Agent logs must be integrated with monitoring system

Agent logs must be integrated with a centralized monitoring system. Logs must include agent ID, session ID, timestamp, operation type, input/output size, and result status.

Scope: resource — aws_cloudwatch_log_group, aws_cloudwatch_log_metric_filter

Assertions: * name not empty — Log integration with monitoring must be configured.

Remediation: Configure CloudWatch Logs Insights queries for agent log analysis. Set up dashboards showing agent health, error rates, and performance trends. Configure alarms for error patterns.

Compliant

resource "aws_cloudwatch_log_group" "agent_logs" {
  name = "/aws/bedrock/agent-logs"

  retention_in_days = 90
}

resource "aws_cloudwatch_log_metric_filter" "agent_errors" {
  name = "agent-error-metric"

  pattern = "{ $.level = \"error\" || $.error != null }"

  log_group_name = aws_cloudwatch_log_group.agent_logs.name

  metric_transformations {
    metric_name      = "AgentErrors"
    metric_namespace = "AgentMonitoring"
    metric_value     = "1"
  }
}

Non-Compliant

resource "aws_cloudwatch_log_group" "agent_logs" {
  name = "/aws/bedrock/agent-logs"
  # No metric filter configured
  # WAF-AGN-070 Violation
}

waf-agn-070.tf.azurerm.app-insights-agent

Engine: terraform | Provider: azurerm | Automated: Ja

Azure Application Insights must monitor agent operations

Azure AI agent components must be monitored via Application Insights to collect latency, token usage, error rates, and distributed traces.

Scope: resource — azurerm_application_insights, azurerm_linux_function_app

Assertions: * workspace_id not empty — Application Insights must be linked to a Log Analytics workspace. * app_settings.APPINSIGHTS_INSTRUMENTATIONKEY attribute exists — Agent host must reference Application Insights instrumentation key.

Remediation: Deploy azurerm_application_insights and connect it to agent hosting services (App Service, Container Apps, Functions).

Compliant

resource "azurerm_application_insights" "agent_ai" {
  name                = "agent-insights"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name
  workspace_id        = azurerm_log_analytics_workspace.ai_logs.id
  application_type    = "web"
}

resource "azurerm_linux_function_app" "agent_host" {
  name                = "agent-function"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name
  service_plan_id     = azurerm_service_plan.agent_plan.id

  app_settings = {
    "APPINSIGHTS_INSTRUMENTATIONKEY" = azurerm_application_insights.agent_ai.instrumentation_key
  }
}

Non-Compliant

resource "azurerm_linux_function_app" "agent_host" {
  name = "agent-function"
  # No Application Insights integration
  # WAF-AGN-070 Violation
}

waf-agn-070.tf.google.monitoring-agent

Engine: terraform | Provider: google | Automated: Ja

GCP Cloud Monitoring must collect agent telemetry

GCP Vertex AI / Cloud Run agent services must have monitoring dashboards and alerting policies for latency, errors, and token usage.

Scope: resource — google_monitoring_alert_policy

Assertions: * conditions attribute exists — Cloud Monitoring alert policy must define conditions for agent metrics. * notification_channels attribute exists — Alert policy must route to notification channels.

Remediation: Configure google_monitoring_alert_policy resources for agent services and google_monitoring_dashboard for agent metrics.

Compliant

resource "google_monitoring_alert_policy" "agent_latency" {
  display_name = "Agent high latency"
  combiner     = "OR"

  conditions {
    display_name = "Latency > 5s"
    condition_threshold {
      filter          = "resource.type="cloud_run_revision" AND metric.type="run.googleapis.com/request_latencies""
      duration        = "60s"
      comparison      = "COMPARISON_GT"
      threshold_value = 5000
    }
  }

  notification_channels = [google_monitoring_notification_channel.team.id]
}

Non-Compliant

resource "google_cloud_run_service" "agent" {
  name     = "agent-service"
  location = "europe-west3"
  # No Cloud Monitoring alert policies configured
  # WAF-AGN-070 Violation
}

waf-agn-070.tf.stackit.agentic-observability

Engine: terraform | Provider: stackit | Automated: Ja

Agent activities must be visible and measurable for stackit

Agent activities must be visible and measurable must be documented in IaC via a agentic-observability tag/label on stackit_project. This validates the control is tracked as code.

Scope: resource — stackit_project

Assertions: * labels key exists — stackit_project is missing the agentic-observability tag/label.

Remediation: Add a agentic-observability tag/label to the stackit_project resource to show the stackit implementation of WAF-AGN-070 is tracked in IaC.

Compliant

resource "stackit_project" "example" {
  labels = {
    agentic-observability = "true"
  }
}

Non-Compliant

resource "stackit_project" "example" {
  # Missing agentic-observability labels
}

waf-agn-070.tf.ovh.agentic-observability

Engine: terraform | Provider: ovh | Automated: Ja

Agent activities must be visible and measurable for ovh

Agent activities must be visible and measurable must be documented in IaC via a agentic-observability tag/label on ovh_cloud_project. This validates the control is tracked as code.

Scope: resource — ovh_cloud_project

Assertions: * tags key exists — ovh_cloud_project is missing the agentic-observability tag/label.

Remediation: Add a agentic-observability tag/label to the ovh_cloud_project resource to show the ovh implementation of WAF-AGN-070 is tracked in IaC.

Compliant

resource "ovh_cloud_project" "example" {
  tags = {
    agentic-observability = "true"
  }
}

Non-Compliant

resource "ovh_cloud_project" "example" {
  # Missing agentic-observability tags
}

waf-agn-070.tf.hcloud.agentic-observability

Engine: terraform | Provider: hcloud | Automated: Ja

Agent activities must be visible and measurable for hcloud

Agent activities must be visible and measurable must be documented in IaC via a agentic-observability tag/label on hcloud_server. This validates the control is tracked as code.

Scope: resource — hcloud_server

Assertions: * labels key exists — hcloud_server is missing the agentic-observability tag/label.

Remediation: Add a agentic-observability tag/label to the hcloud_server resource to show the hcloud implementation of WAF-AGN-070 is tracked in IaC.

Compliant

resource "hcloud_server" "example" {
  labels = {
    agentic-observability = "true"
  }
}

Non-Compliant

resource "hcloud_server" "example" {
  # Missing agentic-observability labels
}

Evidenz

Art Erforderlich Beschreibung

Art	Erforderlich	Beschreibung
`iac`	Erforderlich	Terraform configuration implementing the control.
`governance`	Erforderlich	Policy, ADR, or standard documenting the requirement.
`config`	Optional	Cloud console or API output confirming runtime state.

iac

Erforderlich

Terraform configuration implementing the control.

governance

Erforderlich

Policy, ADR, or standard documenting the requirement.

config

Optional

Cloud console or API output confirming runtime state.

WAF-AGN-070 – Agent Observability

Beschreibung

Begründung

Anforderungen

Implementierungsleitfaden

Reifegrade

Checks

waf-agn-070.tf.aws.metrics-configured

waf-agn-070.tf.aws.distributed-tracing-enabled

waf-agn-070.tf.aws.log-integration-configured

waf-agn-070.tf.azurerm.app-insights-agent

waf-agn-070.tf.google.monitoring-agent

waf-agn-070.tf.stackit.agentic-observability

waf-agn-070.tf.ovh.agentic-observability

waf-agn-070.tf.hcloud.agentic-observability

Evidenz

Verwandte Controls