Home / Azure

Azure Local | Day 2 Operations

Azure Local | Day 2 Operations


Reading Time: 7 minutes

Getting Azure Local deployed is only the beginning.

The real work starts after the installation team has left, the workloads are running, and the platform becomes part of normal operations. That is where Day 2 Operations begins. It is the point where you need to understand whether the platform is healthy, whether capacity is still sufficient, and whether workload placement is becoming unbalanced.

I wanted a single view that answers those questions without jumping between different blades, log queries, and spreadsheets. So I created an Azure Monitor Workbook that brings the most important Azure Local operational signals together.

The complete package is available with this post. It contains the workbook JSON, a supplemental Data Collection Rule, a PowerShell deployment helper, validation queries, and installation instructions.

What should we monitor after deployment?

For me, Azure Local Day 2 Operations starts with six continuous monitoring areas:

  • CPU utilization trends
  • Memory consumption
  • Storage growth
  • Network throughput
  • Host utilization balance
  • VM density

These metrics are related. High CPU utilization on one host may not be a cluster capacity problem. It may be a placement or balance problem. Increasing storage consumption may be expected, but the growth rate tells you when the next expansion decision needs to be made. VM density may look acceptable until you consider the capacity that must remain available when a host is unavailable.

Looking at each counter in isolation is not enough. The goal is to create operational context.

Why use Azure Monitor Workbooks?

Azure Local monitoring is integrated with Azure Monitor. Insights collects health and performance information, stores logs in Log Analytics, and uses KQL and workbooks to visualize the results. Microsoft also makes the built-in Insights experience customizable, which means an existing workbook can be used as a starting point for a platform-specific operations view.

A workbook is a good fit because it can combine queries, parameters, charts, tables, and operational guidance in one interface. It can also be shared with the team and deployed as code.

That last point is important. An operations dashboard should not become a manually maintained portal artifact that nobody can reproduce.

The workbook structure

The workbook in the download contains eight tabs.

Overview

The first tab provides a quick cluster summary. It shows the number of nodes, the total number of virtual machines, average CPU utilization, average memory utilization, and the latest host health and utilization state.

This is not intended to replace detailed troubleshooting. It is the view you open first to decide where to investigate.

CPU

The CPU tab shows utilization over time for each host. It also calculates average, peak, and P95 utilization.

Peak utilization alone can be misleading. A short spike may be perfectly normal. P95 and historical trends make it easier to distinguish a temporary burst from sustained pressure.

Questions this tab should help answer include:

  • Is CPU demand increasing over time?
  • Is one node consistently busier than the others?
  • Are high peaks isolated or recurring?
  • Is observed CPU demand leaving enough headroom for maintenance or a host failure?

Memory

Memory is often the first resource that limits virtualization density. The workbook calculates memory utilization from the Azure Local SDDC management snapshots and shows both the current state and historical trend.

This makes it easier to identify nodes that are structurally under pressure. It also shows whether the problem is local to one host or visible across the cluster.

The workbook collects additional memory counters through the DCR, but its main memory visualization uses the Azure Local management event data. This keeps the host balance calculation aligned with the same node snapshots used for VM density.

Storage

Storage needs more than a percentage gauge. The important question is not only how much capacity is currently used, but how quickly it is growing.

The storage tab shows used, free, and total capacity over time. It also provides a per-volume view with utilization, IOPS, throughput, and latency in milliseconds when those values are available in the SDDC management events.

This tab is intended to support capacity planning. It gives you the raw trend required to discuss when expansion is needed, rather than waiting for a threshold alert to fire.

Network

The network tab queries the Perf table for network interface and network adapter throughput. RDMA activity is shown separately.

Separating the two matters because not every environment exposes the same counters. Hardware, drivers, operating system language, and the actual network design can affect counter availability. An empty RDMA chart does not automatically mean that the workbook is broken.

The data quality tab helps verify whether the required counters are arriving in Log Analytics.

Host balance

This is one of the most useful parts of the workbook.

A cluster can have sufficient total capacity while still being badly balanced. One node may run hot while another remains lightly loaded. That can affect performance and reduce operational headroom.

The workbook compares each host with the current cluster average for CPU, memory, and VM count. It also calculates a custom balance score based on the spread between hosts.

The score is an operational indicator. It is not a Microsoft support threshold and it should not trigger automatic VM moves without context. Affinity rules, maintenance, failure domains, workload criticality, and reserve capacity still matter.

Use the score to start a conversation and investigation, not to replace engineering judgement.

VM density

The VM density tab shows running, stopped, failed, and other VM states per host, together with the number of logical processors and the calculated VM density.

VM density must always be interpreted together with workload size and availability requirements. Twenty small management VMs are not the same as twenty memory-heavy database servers.

The most important operational question is whether the remaining hosts can absorb the workloads after a node becomes unavailable. This workbook does not perform a failover-capacity simulation; a density number without failure capacity is only half the story.

Data quality

Every monitoring solution needs to monitor its own input.

The data quality tab shows the newest records in Perf, Azure Local SDDC events, health events, and Heartbeat for the selected cluster. If a chart is empty, this is the first place to look.

A beautiful workbook with stale telemetry is worse than no workbook because it creates false confidence.

The Data Collection Rule

The package contains an ARM template for a supplemental DCR. It deliberately avoids the event logs and baseline counters already collected by the Microsoft-managed Insights DCR. It adds:

  • Memory % Committed Bytes In Use
  • Network Interface Bytes Received/sec and Bytes Sent/sec
  • Network Adapter Bytes Total/sec, Bytes Received/sec, and Bytes Sent/sec

The Microsoft-managed Insights DCR remains responsible for Azure Local SDDC management events 3000, 3002, and 3003, Azure Local health events, Processor counters, Memory Available Bytes, Network Interface Bytes Total/sec, and RDMA activity counters. Windows events are written to the Event table and performance counters are written to the Perf table in the selected Log Analytics workspace.

Microsoft recommends enabling Azure Local Insights and retaining the DCR created by Insights because that DCR contains a special data stream required by the Insights experience. Do not configure the same event log or performance counter in both DCRs: overlapping sources can create duplicate records and additional ingestion charges. Check the existing Insights DCR before associating this supplemental rule.

Sources:

That distinction is deliberate. It keeps the Microsoft-managed monitoring experience intact without collecting the same telemetry twice.

Deploying the DCR

Start by enabling Azure Local Insights on the cluster. Confirm that the Azure Monitor Agent is healthy and that data reaches the Log Analytics workspace. Install the Az.Accounts and Az.Resources PowerShell modules, for example with Install-Module -Name Az.Accounts, Az.Resources -Scope CurrentUser, and make sure the subscription you pass to the helper is the intended one.

Deploy the supplemental DCR with the included PowerShell helper:

Connect-AzAccount

./Deploy-DCR.ps1 `
  -ResourceGroupName "rg-monitoring-prod" `
  -SubscriptionId "<subscription-id>" `
  -WorkspaceResourceId "/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.OperationalInsights/workspaces/<workspace-name>" `
  -Location "westeurope"

The helper deploys the supplemental DCR only. It verifies the requested subscription and creates the resource group if it does not exist; it does not create node associations.

After deployment, associate the supplemental DCR with every Arc-enabled Azure Local server node, while leaving the Microsoft-managed Insights DCR associated:

  1. Open Azure Monitor.
  2. Open Data Collection Rules.
  3. Select the new Day 2 Operations DCR.
  4. Open Resources.
  5. Add every Azure Local Arc-enabled server node.
  6. Save the association.

Use the same Log Analytics workspace that you select in the workbook.

Importing the workbook

The workbook file is a gallery template and can be imported directly:

  1. Open Azure Monitor.
  2. Open Workbooks.
  3. Create a new workbook.
  4. Select Edit.
  5. Open Advanced Editor using the </> button.
  6. Select Gallery Template.
  7. Replace the existing JSON with the supplied workbook JSON.
  8. Select Apply.
  9. Save the workbook.
  10. Select the Azure Local cluster and Log Analytics workspace parameters.

The workbook offers time ranges for operational and capacity analysis. Longer ranges are useful for growth trends, but they also depend on the configured Log Analytics retention period.

Validate before you trust the charts

Run the following query to confirm that the required Azure Local events are arriving:

Event
| where EventLog =~ "Microsoft-Windows-SDDC-Management/Operational"
| where EventID in (3000, 3002, 3003)
| summarize Records=count(), LastRecord=max(TimeGenerated) by EventID

Validate performance counter collection with:

Perf
| where ObjectName in~ ("Processor", "Memory", "Network Interface", "Network Adapter", "RDMA Activity")
| summarize Records=count(), LastRecord=max(TimeGenerated) by ObjectName, CounterName
| order by ObjectName asc, CounterName asc

Finally, check heartbeat freshness:

Heartbeat
| summarize LastHeartbeat=max(TimeGenerated) by Computer
| extend MinutesOld=datetime_diff("minute", now(), LastHeartbeat)
| order by MinutesOld desc

Do not skip this step. First prove that the telemetry exists. Then validate the workbook.

Sampling, retention, and cost

The supplied DCR uses a 60-second sampling interval for performance counters. That is a reasonable starting point for many environments, but it is not a universal recommendation.

A lower interval produces more detail and more ingestion. A longer interval reduces ingestion but can hide brief peaks. The right choice depends on the workloads, troubleshooting requirements, retention period, and budget.

Azure Monitor platform metrics for Azure Local provide near real-time monitoring and are available out of the box, while logs in Log Analytics provide flexible KQL analysis and longer operational context. In practice, I use both. Metrics are well suited for near real-time charts and alerts. Logs and workbooks are better suited for correlation, historical analysis, and a shared operations view.

Review ingestion after the first week. Remove counters nobody uses, tune the sampling interval, and set retention deliberately. Collecting everything forever is not a monitoring strategy.

What this package does not try to do

This package is a starting point for platform operations. It does not automatically solve capacity management, workload placement, or alert ownership.

It also does not include fixed production thresholds. A CPU threshold that makes sense for one cluster may be too aggressive or too relaxed for another. Establish a baseline during normal operation, maintenance, backup windows, and peak workload periods. Then define thresholds based on observed behavior and required headroom.

The same applies to the balance score. Treat it as a signal that supports investigation.

I think

Day 2 Operations is not about collecting as many counters as possible. It is about turning telemetry into decisions.

Can we perform maintenance without creating capacity pressure? Is storage growing faster than expected? Are workloads distributed sensibly? Is one host carrying more risk than the others? Is the monitoring data itself still current?

A single workbook will not answer every question, but it creates one shared starting point for operators, architects, and service owners.

Download the package, deploy it in a test environment, and adjust it to your operational baseline. If you improve the queries or add useful Day 2 views, I would love to hear what you changed.

Download contents

The downloadable package contains:

  • The complete Azure Monitor Workbook JSON
  • A supplemental Azure Monitor DCR ARM template
  • A PowerShell deployment helper
  • A README with installation and validation steps

https://github.com/GetToThe-Cloud/Website/tree/main/AzureLocal-Day2%20Operations

Test everything against your own Azure Local release and hardware before using it in production.

Share and Enjoy !

Shares

Designer (23)

Stay close to the action—follow GetToThe.Cloud across social!
Deep dives and hands‑on how‑tos on Azure Local, hybrid cloud, automation, PowerShell/Bicep, AVD + FSLogix, image pipelines, monitoring, networking, and resilient design when the internet/Azure is down.

🔗 Our channels
▶️ YouTube: https://www.youtube.com/channel/UCa33PgGdXt-Dr4w3Ub9hrdQ
💼 LinkedIn Group: https://www.linkedin.com/groups/9181126/
✖️ X (Twitter): https://x.com/Gettothecloud
🎵 TikTok: https://www.tiktok.com/@gettothecloud
🐙 GitHub: https://github.com/GetToThe-Cloud/Website
💬 Slack: DM us for an invite
📲 WhatsApp: DM for the community link

We use cookies to personalise content and ads, to provide social media features and to analyse our traffic. We also share information about your use of our site with our social media, advertising and analytics partners. View more
Cookies settings
Accept
Privacy & Cookie policy
Privacy & Cookies policy
Cookie name Active

Who we are

Our website address is: https://www.gettothe.cloud

Comments

When visitors leave comments on the site we collect the data shown in the comments form, and also the visitor’s IP address and browser user agent string to help spam detection.An anonymized string created from your email address (also called a hash) may be provided to the Gravatar service to see if you are using it. The Gravatar service privacy policy is available here: https://automattic.com/privacy/. After approval of your comment, your profile picture is visible to the public in the context of your comment.

Media

If you upload images to the website, you should avoid uploading images with embedded location data (EXIF GPS) included. Visitors to the website can download and extract any location data from images on the website.

Cookies

If you leave a comment on our site you may opt-in to saving your name, email address and website in cookies. These are for your convenience so that you do not have to fill in your details again when you leave another comment. These cookies will last for one year.If you visit our login page, we will set a temporary cookie to determine if your browser accepts cookies. This cookie contains no personal data and is discarded when you close your browser.When you log in, we will also set up several cookies to save your login information and your screen display choices. Login cookies last for two days, and screen options cookies last for a year. If you select "Remember Me", your login will persist for two weeks. If you log out of your account, the login cookies will be removed.If you edit or publish an article, an additional cookie will be saved in your browser. This cookie includes no personal data and simply indicates the post ID of the article you just edited. It expires after 1 day.

Embedded content from other websites

Articles on this site may include embedded content (e.g. videos, images, articles, etc.). Embedded content from other websites behaves in the exact same way as if the visitor has visited the other website.These websites may collect data about you, use cookies, embed additional third-party tracking, and monitor your interaction with that embedded content, including tracking your interaction with the embedded content if you have an account and are logged in to that website.

Who we share your data with

If you request a password reset, your IP address will be included in the reset email.

How long we retain your data

If you leave a comment, the comment and its metadata are retained indefinitely. This is so we can recognize and approve any follow-up comments automatically instead of holding them in a moderation queue.For users that register on our website (if any), we also store the personal information they provide in their user profile. All users can see, edit, or delete their personal information at any time (except they cannot change their username). Website administrators can also see and edit that information.

What rights you have over your data

If you have an account on this site, or have left comments, you can request to receive an exported file of the personal data we hold about you, including any data you have provided to us. You can also request that we erase any personal data we hold about you. This does not include any data we are obliged to keep for administrative, legal, or security purposes.

Where we send your data

Visitor comments may be checked through an automated spam detection service.
Save settings
Cookies settings