During a recent design session at a client site, our team had the opportunity to participate in something cool. We had the opportunity to create a custom DSL (Domain Specific Language) in YAML for an automation framework we wanted to build. This blog will help provide a high-level overview. The client wants to use Azure Automation to create a self-healing framework for their infrastructure. The environment is a mixture of both PaaS and IaaS, with certain VMs being used to host app and database servers. Unfortunately, because of potential performance issues, it is important to regularly perform healing actions on the environment, which so far is being done manually. Now you may be asking, “Why don’t they use something like Chef or Puppet?”. It is because these technologies were either in the process of being on-boarded or were not available. You may also ask, “Well, if you’re going to use Powershell, just use Powershell DSC!”. While I agree that Powershell DSC is powerful and could potentially power the engine for this application (explained below), DSC itself is not very user-friendly. One of the major factors for this session was to allow any user to edit or read the definitions, and YAML is a much more robust option for usability.

The design session itself was fascinating. I had never spent time creating my own configuration language complete with definitions and structure. It was a great learning experience! I had used YAML briefly when demoing out things like Ansible for personal projects, but never directly to solve an issue. YAML had always been another markup language for me. This usage of it, however, showed me the power of the language itself. The problem that we ended up solving with the below snippet was creating an abstraction layer for Azure Monitor Alerts.

After the session completed, we were left with something like this:
Create and Abstraction Layer

Now what?

We must take this YAML File and convert it into an Azure Monitor Alert. The first problem we ran into is that this file is a YAML File. How can I take these configuration values and convert them to be used in Powershell? Here is where Cloudbase’s powershell-yaml module comes into play. As we all know, Powershell was written on top of and created to be an extension for the .NET Framework, so people at Cloudbase created the wonderful powershell-yaml, a module that is a wrapper around the popular .Net Library YamlDotNet.

With powershell-yaml you can create something like this:
Converting the yaml File

In this example, I am not only converting the yaml file into a JSON file as an example, but I am returning the yaml as a PSObject. I find this much easier to use because of the ease of dot notation.

In this case, you can now reference variables such as this:
Corresponding Powershell cmdlets

I was able to follow up with the corresponding PowerShell cmdlets available in the Az Modules and programmatically create Azure Monitor Alerts using YAML.

The eventual implementation will look roughly like this:
Azure Alerts

Each Azure Alert will trigger a webhook that is received by the Engine that is running in Azure Automation. This Engine will then do all business logic to find out whether the piece of infrastructure in question needs healing. If all the previously defined reasons are true the automation runbook will perform the recovery action.

The exercise itself was eye-opening. It made me much more comfortable with the idea of designing a solution for our specific scenario rather than trying to find out and wait to find the product or framework that would solve the problem for us. Also, this PoC showed me the power of YAML and how it can turn something incredibly monotonous, like configuration values, into something that can be part of your robust solution.

A Single Place to Manage, Create, and ConsumeAzure Monitor and OMS

The integration of the Operations Management Suite (OMS) into Azure Monitor is completed for both Azure Commercial and Azure Government. This change by Microsoft has given Azure Monitor/OMS users a single place to manage, create, and consume Azure Monitoring solutions. No functionality has been removed and documentation has been consolidated under the Azure Monitor documentation. With this consolidation of services, there have been some terminology changes that will impact the way one talks about Azure Monitor components. The consolidation of OMS and other Azure services into Azure Monitor is simplifying the way you manage the monitoring of your Azure services.

Updated Terminology

Microsoft has updated some of the terminologies for the Azure Monitor components to reflect the transition from OMS. I have highlighted some examples:

  • The log data for Azure Monitor is still stored in a Log Analytics Workspace, but the term Log Analytics in the Microsoft documentation is now Azure Monitor Logs.
  • The term log analytics now applies to the page in the Azure portal used to write and run queries and analyze log data.
  • What was once known as OMS Management solutions have been renamed Monitoring solutions (items like Security & Compliance and Automation & Control)

Azure Monitor — Your 1 Stop “Monitoring & Alerting” Shop

Azure Monitor is now pretty much the one stop shop for your monitoring and alerting needs (the exception here would be Azure Security Center is still the place to go to for most of your security and compliance needs).

Azure Monitor is broken out into four main categories in the Azure Portal:

  1. The main components of Azure monitor
  2. Insights
  3. Settings
  4. Support + Troubleshooting.

The main components include the Activity log, Alerts, Metrics, Logs, Service Health, and Workbooks.

Under Insights, there is Application, Virtual Machines, Containers, Network, and “…More”.

The Settings category includes Diagnostics settings and Autoscale.

And finally, under Support + Troubleshooting, there is Usage & estimated costs, Advisor recommendations, and New support request.

Check out the below table that provides an overview of the Azure Monitor Components and Descriptions:

Azure Monitor Component Description
Overview Overview of Azure Monitor
Activity Log Log data about the operations performed in Azure
Alerts Notifications based on conditions that are found in monitoring data both metrics and logs
Metrics (Metrics Explorer) Plotting charts, visually correlating trends, and investigating spikes and dips in metrics’ values.
Logs (Azure Monitor Logs) Useful for performing complex analysis across data from a variety of sources
Service Health Provides a personalized view of the health of the Azure services and regions you’re using
Workbooks Combine text, Analytics queries, Azure Metrics, and parameters into rich interactive reports.
Applications Application Performance Management service for web developers
Virtual Machines Analyzes the performance and health of your Windows and Linux VMs and monitors their processes and dependencies on other resources and external processes.
Containers Monitor the performance of container workloads deployed to either Azure Container Instances or managed Kubernetes clusters hosted on Azure Kubernetes Service (AKS).
Network Tools to monitor, diagnose, view metrics, and enable or disable logs for resources in an Azure virtual network.
More Replacement for the OMS Portal Dashboard.
Diagnostic Settings Configure the diagnostic setting for Azure resources (formally known as Diagnostic Logs)
Autoscale Consolidated view of Azure resources that have Autoscale enabled
Usage and estimated costs Consumption and cost estimates of Azure Monitor
Advisor Recommendations Link to Azure Advisor
New support requests Create a support request