Observability architecture
Decision
Adopt a standalone, subscription-scope Azure observability package as the tenant operations control plane. The package is owned by Platform Engineering and deployed independently from workload Bicep. It uses a cost-first hierarchy: native metadata and standard metrics first, selected data collection second, and preview or hybrid features only after a defined activation decision.
Scope
The package observes Azure resources and supported workloads. It does not own application business logic, model deployment, user data, or workload lifecycle. It creates only its own monitoring, alerting, governance, and FinOps resources.
Capability model
| Plane | Required capability | Azure implementation | Default state |
|---|---|---|---|
| FinOps | Budget, actual and forecast notifications, anomaly investigation, scheduled reports, allocation | Cost Management budgets, scheduled actions, exports, Advisor, tag views | Budget on; reports and exports opt-in |
| Inventory | Resource type, location, owner, project, lifecycle, expiry, and unmanaged items | Azure Resource Graph query library and dashboard panels | On |
| Governance | Required tags and lifecycle metadata | Parameterized Policy assignments and tag queries | Audit-first, enforcement by approval |
| Control plane | Deployments, deletes, policy changes, and failed operations | Activity Log, selected Activity Log alerts, optional diagnostic export | Activity Log alerts opt-in |
| Azure health | Service incidents, maintenance, advisories, and resource transitions | Service Health and Resource Health Activity Log alert definitions | Opt-in, narrow scopes |
| Workload metrics | Availability, requests, errors, throttling, latency, token and image use | Azure Monitor metric alerts and dashboards | Opt-in per supported metric |
| Diagnostics | Root-cause logs, traces, and application telemetry | Selected diagnostic settings, Log Analytics, Application Insights, scheduled query alerts | Off until approved |
| Experience | Cost, inventory, health, and workload visualizations | Azure Monitor dashboards with Grafana plus portal Cost Management | Dashboard shell on; definition opt-in preview |
| Response | Notification, routing, suppression, ownership, and runbooks | Action groups and alert processing rules | Notification path on; routing opt-in |
| Hybrid | Azure Local, Arc Kubernetes, and Foundry Local visibility | Azure Monitor Workspace, later DCRs and managed Prometheus | No collection by default |
| Health rollup | Business dependency and service health state | Azure Monitor health models | Preview pilot only |
Architecture boundary
Platform observability package
creates: monitoring resource group, workspaces, action group, budget,
alert rules, policy assignments, dashboards, optional data controls
Workload deployment
creates: application, Foundry account, model deployments, networking, data
Private overlay
supplies: recipients, scopes, resource IDs, budget values, thresholds,
retention, approved logs, destinations, and profile switchesThe diagram is intentionally a textual boundary map. A deployment topology drawing will be added as draw.io when a concrete multi-subscription landing-zone topology is approved.
Data and cost flow
- Cost Management and Resource Graph answer cost and inventory questions without enabling workload log ingestion.
- Azure platform metrics provide workload behavior evidence without exporting diagnostic logs to Log Analytics.
- Activity Log provides control-plane evidence for the default retention period.
- Action groups receive budget and approved Azure Monitor alerts.
- Only an approved diagnostic, tracing, synthetic test, or Prometheus definition writes data to a workspace. Every such definition has an owner, retention, daily volume estimate, cost envelope, and review date.
CAF and tag contract
The package derives resource names from the CAF pattern <type>-<workload>-<environment>-<region>-<instance>. Its required tags are Owner, Project, Environment, CostCenter, ManagedBy, Lifecycle, and ExpiresOn for temporary or sandbox resources. Policy assignments are parameterized so the consumer selects approved built-in or custom policy definitions, scope, exclusions, enforcement mode, and remediation identity.
WAF decisions
| Pillar | Design response |
|---|---|
| Cost Optimization | Budgets and low thresholds are foundational; Log Analytics caps, short initial retention, selected categories, and opt-in Prometheus and tracing prevent telemetry spend by default. |
| Operational Excellence | Separate deployment, source-controlled alerts and dashboards, runbooks, lifecycle tags, query library, and response ownership make operation repeatable. |
| Reliability | Service and Resource Health, deployment-failure alerts, metric alerts, action groups, alert suppression, and health-model pilot provide layered failure evidence. |
| Security | Private recipients and destinations remain in private overlays; diagnostics and tracing require data classification, least-privilege access, retention, and local-auth decisions. |
| Performance Efficiency | Standard platform metrics precede detailed logs. Metric dimensions and alert windows are selected per workload rather than treated as universal defaults. |
Profiles and activation gates
| Profile | Included | Gate before activation |
|---|---|---|
| Foundation | Resource group, Log Analytics workspace, Azure Monitor Workspace, action group, budget, tags, dashboard shell, query library | Private parameter file and approved what-if |
| Solution core | Foundation plus Cost Management actions, Activity Log and health alerts, metric alerts, alert processing, dashboard definition | Reviewed scopes, thresholds, alert owner, and runbook |
| Solution diagnostics | Selected diagnostic settings, Application Insights, availability tests, and scheduled query alerts | Data classification, source-category decision, RBAC, retention, sampling, daily estimate, and cost owner |
| Hybrid extension | DCR definitions and managed Prometheus readiness | Azure Local or Arc workload, scrape-volume and label-cardinality estimate, privacy review |
Hybrid and local design
Azure Local standard metrics and Foundry Local metadata can participate in the same inventory, tag, cost, and response model. The package creates an Azure Monitor Workspace but does not create a data collection rule, Arc extension, scrape target, or diagnostic stream by default. Foundry Local remains local-first: Azure-connected telemetry is metadata-only and opt-in, with no prompt, response, secret, or model-input collection by default.