Let’s monitor that cloud!

It can’t be just a coincidence that recently many of my friends expressed interest in monitoring how their teams are using their cloud subscription. And yet, all of them have a valid problem to worry about! Let’s talk about this problem, scenario, and solution in this last blog post of 2018 :)

Problem:

A subscription is shared with the team or each member has a separate subscription. Anyone can create any kind of resources (e.g. Virtual Machines, Storage Accounts, Network Security Groups, Web or Application Services, etc…). Even though there are policies and access restrictions which can be forced in Azure Portal, sometimes Administrators or Support engineers require (or literally they ask for) ‘God Mode’ with full access to play around, explore services or fix some things. In this case, monitoring these activities happened in ‘god mode’ & quick remediation or reversal is important before any damage happens.

Scenario:

Sometimes, developers often need to create resources to try out things. For example, they may want to create a Storage Account. Now, we want to enforce rules around Storage Account and make sure that the created Storage Account is accessible only via HTTPs protocol. Or let’s say Network Admin creates a Network Security Group (NSG) and in that case, the policy is to disable or enable specific ports or protocols (UDP/TCP/etc).

Solution:

I consider Azure Services as Lego blocks. You can mix and match and use them together as and when required. In this case, there are multiple ways to handle this scenario and possibly many more. There could be operations team managing this infrastructure using their custom tools. However, being a developer, I’ll focus on two solutions from a developer’s point of view. And I can see there are two easy solutions:

(1) Whenever a resource is created (or executed like Logic App trigger) or modified or deleted within a subscription, a record is maintained at subscription level and resource group level. This log can be further exported to different Azure services to take action. For example, it can be exported (or streamed) to Event Hub for event publishing.

(2) Using Azure Monitor set up a trigger to such records (create, update, delete resources) and take action (Notify by Email/Text, call Logic App, Functions, etc…)

Once these pipelines are set up, these services (Event Hub/Activity Log) will send data in JSON format which can be used to track down what has happened.

	{
	"schemaId": "Microsoft.Insights/activityLogs",
	"data": {
	"status": "Activated",
	"context": {
	"activityLog": {
	"authorization": {
	"action": "Microsoft.Storage/storageAccounts/write",
	"scope": "/subscriptions/<subcription-id-hidden>/resourceGroups/SubscriptionMonitoring/providers/Microsoft.Storage/storageAccounts/submonstore"
	},
	"channels": "Operation",
	"claims": "<removing-details-for-brevity>",
	"caller": "<email-id-hidden>",
	"correlationId": "ca9bcd00-ecd2-42f3-9bfe-8aafe7ef9b21",
	"description": "",
	"eventSource": "Administrative",
	"eventTimestamp": "2018-12-25T03:27:00.3969244+00:00",
	"httpRequest": "{\"clientRequestId\":\"6a543b9d-601d-48d2-b4a5-295c95f67003\",\"clientIpAddress\":\"103.86.183.248\",\"method\":\"PUT\"}",
	"eventDataId": "72b33715-169f-483a-af00-0f06d47ddc12",
	"level": "Informational",
	"operationName": "Microsoft.Storage/storageAccounts/write",
	"operationId": "ca9bcd00-ecd2-42f3-9bfe-8aafe7ef9b21",
	"properties": {
	"statusCode": "OK",
	"serviceRequestId": "f833dc6e-4a65-4c74-a923-4bf896b1e53f",
	"responseBody":"<removing-details-for-brevity>",
	},
	"resourceId": "/subscriptions/<subcription-id-hidden>/resourceGroups/SubscriptionMonitoring/providers/Microsoft.Storage/storageAccounts/submonstore",
	"resourceGroupName": "SubscriptionMonitoring",
	"resourceProviderName": "Microsoft.Storage",
	"status": "Succeeded",
	"subStatus": "OK",
	"subscriptionId": "<subcription-id-hidden>",
	"submissionTimestamp": "2018-12-25T03:27:22.0940445+00:00",
	"resourceType": "Microsoft.Storage/storageAccounts"
	}
	},
	"properties": {}
	}
	}

view raw AlertFormat.json hosted with ❤ by GitHub

This JSON has required information like resource id and resource type. In this case, I’m using Logic App to parse this JSON and accordingly call respective Azure Function which will take the action (by calling Azure Functions). I’m considering two scenarios here. (A) when new Storage Account is created and (B) when new Network Security Group is created/modified with Rules. But there can be many such scenarios!

Now according to the case (NSG/Storage Account), it will call the Azure Function. To modify the resources in subscription, I’m using Fluent APIs which makes life much easier. For example the following code creates a VM using Fluent APIs in US East Region within rgName provided:

	var windowsVM = azure.VirtualMachines.Define("myWindowsVM")
	.WithRegion(Region.US_EAST)
	.WithNewResourceGroup(rgName)
	.WithNewPrimaryNetwork("10.0.0.0/28")
	.WithPrimaryPrivateIpAddressDynamic()
	.WithNewPrimaryPublicIpAddress("mywindowsvmdns")
	.WithPopularWindowsImage(KnownWindowsVirtualMachineImage.WINDOWS_SERVER_2012_R2_DATACENTER)
	.WithAdminUserName("tirekicker")
	.WithPassword(password)
	.WithSize(VirtualMachineSizeTypes.StandardD3V2)
	.Create();

view raw CreateVM.cs hosted with ❤ by GitHub

Using these Fluent APIs, let’s write Azure Function which will monitor newly created Storage Account and make sure that it is accessible only with HTTPs protocol.

	using System;
	using System.IO;
	using System.Linq;
	using System.Threading.Tasks;
	using Microsoft.AspNetCore.Mvc;
	using Microsoft.Azure.WebJobs;
	using Microsoft.Azure.WebJobs.Extensions.Http;
	using Microsoft.AspNetCore.Http;
	using Microsoft.Azure.Management.Fluent;
	using Microsoft.Azure.Management.ResourceManager.Fluent;
	using Microsoft.Azure.Management.ResourceManager.Fluent.Authentication;
	using Microsoft.Azure.Management.ResourceManager.Fluent.Core;
	using Microsoft.Azure.Services.AppAuthentication;
	using Microsoft.Extensions.Logging;
	using Microsoft.Rest;
	using Newtonsoft.Json;
	namespace SubscriptionMonitoringFunctions
	{
	public static class StorageHttpsPolicyFunction
	{
	[FunctionName("StorageHttpsOnlyPolicy")]
	public static async Task<IActionResult> Run(
	[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
	ILogger log)
	{
	string requestData = string.Empty;
	Rootobject rootData = null;
	string subscriptionId = string.Empty;
	string storageAccountName = string.Empty;
	string resourceId = string.Empty;
	string outputStatus = string.Empty;
	try
	{
	log.LogInformation("StorageHttpsPolicy: Begin to read activity log");
	outputStatus += $"\n StorageHttpsPolicy: Begin to read activity log";
	requestData = await new StreamReader(req.Body).ReadToEndAsync();
	rootData = JsonConvert.DeserializeObject<Rootobject>(requestData);
	if (rootData.data.context.activityLog.resourceType.Contains("storageAccounts"))
	{
	subscriptionId = rootData.data.context.activityLog.subscriptionId;
	log.LogInformation($"StorageHttpsPolicy: Found Subscription Id {subscriptionId}");
	outputStatus += $"\n StorageHttpsPolicy: Found Subscription Id {subscriptionId}";
	resourceId = rootData.data.context.activityLog.resourceId;
	log.LogInformation($"StorageHttpsPolicy: Found Resource Id {resourceId}");
	outputStatus += $"\n StorageHttpsPolicy: Found Resource Id {resourceId}";
	storageAccountName = rootData.data.context.activityLog.resourceId.Substring(resourceId.LastIndexOf("storageAccounts") + 16).Split("/")[0];
	log.LogInformation($"StorageHttpsPolicy: Found Storage Account: {storageAccountName}");
	outputStatus += $"\n StorageHttpsPolicy: Found Storage Account: {storageAccountName}";
	var token = Authenticate().Result;
	var credentials = new AzureCredentials(new TokenCredentials(token), new TokenCredentials(token), string.Empty, AzureEnvironment.AzureGlobalCloud);
	var azure = Azure.Configure().WithLogLevel(HttpLoggingDelegatingHandler.Level.Basic).Authenticate(credentials).WithSubscription(subscriptionId);
	var storageAccounts = azure.StorageAccounts.List();
	var storageAccount = storageAccounts.FirstOrDefault(x => x.Name == storageAccountName);
	storageAccount?.Update().WithOnlyHttpsTraffic().Apply();
	log.LogInformation("StorageHttpsPolicy: Policy application successful");
	outputStatus += $"\n StorageHttpsPolicy: Policy application successful";
	}
	else
	{
	outputStatus += $"\n StorageHttpsPolicy: Couldn't find Storage Account";
	}
	}
	catch (Exception x)
	{
	log.LogInformation("StorageHttpsPolicy: Policy application failed");
	outputStatus += $"\n StorageHttpsPolicy: Policy application failed with {x.Message}";
	log.LogInformation(x.Message);
	}
	return outputStatus != null
	? (ActionResult)new OkObjectResult(outputStatus)
	: new BadRequestObjectResult("Error in execution of this policy");
	}
	public static async Task<string> Authenticate()
	{
	var azureServiceTokenProvider = new AzureServiceTokenProvider();
	var accessToken = await azureServiceTokenProvider.GetAccessTokenAsync("https://management.azure.com").ConfigureAwait(false);
	return accessToken;
	}
	}
	}

view raw StorageHttpsPolicyFunction.cs hosted with ❤ by GitHub

In the above code

storageAccount?.Update().WithOnlyHttpsTraffic().Apply(); 

makes sure that Storage Account is accessible with HTTPs protocol only.

One last thing before executing this function or enabling Logic App, give permission to this Azure Function to modify resources. It can be done by creating and passing service principal or by using Managed Identity option. More details are here on this blog post.  Managed Identity helps to avoid storing credentials in code.

Once this setup is complete, whenever Storage Account or Network Security Group is created, modified & updated, the Logic App will execute and call the respective Azure Function.

Conclusion

Using these various Azure Services, it is possible to monitor the subscription and perform actions on resources as per the rules. This makes governance easier. The sample and code used for Azure Functions in this blog post is available on GitHub here. Clone it and follow the steps there to deploy and run it in Azure subscription.