Let’s monitor that cloud!

Cloud everywhere.
Pic: Cloud Everywhere, Singapore. Shot on Samsung Galaxy S8

It can’t be just a coincidence that recently many of my friends expressed interest in monitoring how their teams are using their cloud subscription. And yet, all of them have a valid problem to worry about! Let’s talk about this problem, scenario, and solution in this last blog post of 2018 :)

Problem:

A subscription is shared with the team or each member has a separate subscription. Anyone can create any kind of resources (e.g. Virtual Machines, Storage Accounts, Network Security Groups, Web or Application Services, etc…). Even though there are policies and access restrictions which can be forced in Azure Portal, sometimes Administrators or Support engineers require (or literally they ask for) ‘God Mode’ with full access to play around, explore services or fix some things. In this case, monitoring these activities happened in ‘god mode’ & quick remediation or reversal is important before any damage happens.

Scenario:

Sometimes, developers often need to create resources to try out things. For example, they may want to create a Storage Account. Now, we want to enforce rules around Storage Account and make sure that the created Storage Account is accessible only via HTTPs protocol. Or let’s say Network Admin creates a Network Security Group (NSG) and in that case, the policy is to disable or enable specific ports or protocols (UDP/TCP/etc).

Solution:

I consider Azure Services as Lego blocks. You can mix and match and use them together as and when required. In this case, there are multiple ways to handle this scenario and possibly many more. There could be operations team managing this infrastructure using their custom tools. However, being a developer, I’ll focus on two solutions from a developer’s point of view. And I can see there are two easy solutions:

(1) Whenever a resource is created (or executed like Logic App trigger) or modified or deleted within a subscription, a record is maintained at subscription level and resource group level. This log can be further exported to different Azure services to take action. For example, it can be exported (or streamed) to Event Hub for event publishing.

(2) Using Azure Monitor set up a trigger to such records (create, update, delete resources) and take action (Notify by Email/Text, call Logic App, Functions, etc…)

Once these pipelines are set up, these services (Event Hub/Activity Log) will send data in JSON format which can be used to track down what has happened.

This JSON has required information like resource id and resource type. In this case, I’m using Logic App to parse this JSON and accordingly call respective Azure Function which will take the action (by calling Azure Functions). I’m considering two scenarios here. (A) when new Storage Account is created and (B) when new Network Security Group is created/modified with Rules. But there can be many such scenarios!

Now according to the case (NSG/Storage Account), it will call the Azure Function. To modify the resources in subscription, I’m using Fluent APIs which makes life much easier. For example the following code creates a VM using Fluent APIs in US East Region within rgName provided:

Using these Fluent APIs, let’s write Azure Function which will monitor newly created Storage Account and make sure that it is accessible only with HTTPs protocol.

In the above code

storageAccount?.Update().WithOnlyHttpsTraffic().Apply(); 

makes sure that Storage Account is accessible with HTTPs protocol only.

One last thing before executing this function or enabling Logic App, give permission to this Azure Function to modify resources. It can be done by creating and passing service principal or by using Managed Identity option. More details are here on this blog post.  Managed Identity helps to avoid storing credentials in code.

Once this setup is complete, whenever Storage Account or Network Security Group is created, modified & updated, the Logic App will execute and call the respective Azure Function.

Conclusion

Using these various Azure Services, it is possible to monitor the subscription and perform actions on resources as per the rules. This makes governance easier. The sample and code used for Azure Functions in this blog post is available on GitHub here. Clone it and follow the steps there to deploy and run it in Azure subscription.

Advertisements

Published by

Mayur Tendulkar

Working as a Technology Solutions Professional in Microsoft at Singapore. More details at: https://about.me/tendulkar

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.