[Episode 383 – Securing Azure: Monitoring and observing your Azure estate](/content/episode383/index.html)

Episode 383 – Securing Azure: Monitoring and observing your Azure estate

by [Scott](/content/author/scottmsclouditpro/ "Posts by Scott"/index.html) | Aug 29, 2024 | Podcast

Blubrry Player

Auto Scroll

Welcome to episode 383 of the Microsoft Cloud IT Pro podcast recorded live on August 23, 2024. This is a show about Microsoft 360 5 and Azure from the perspective of IT pros and end users, where we discuss a topic or recent news and how it relates to you. In this episode, we continue our discussion on security as we transition from Microsoft 365 to Azure. We kick things off with Azure Security Logging and Auditing before moving into Azure Monitor for collecting, analyzing, and acting on telemetry data. We also explore how it can help you identify trends and anomalies to help with your threat detection. We got the Logitech MX keyboard, the MX mini. Are you enjoying it? I don't know. I have a problem. Sound very clicky clacky. It's not clicky. It is definitely not a clicky keyboard. It is better than the macOS keyboard except that it doesn't have my fingerprint reader. I do like the fingerprint reader on my Mac keyboard. Like, they need to just sell a standalone fingerprint reader for logging in to, like, desktops. I too wish they would get there. But it's backlit, but it's definitely not like a NuPhy or a Keychron or a Clicky Mechanical. It's yeah. I would say it's a little bit more tactile than, like, the macOS keyboard, Little bit more key travel, but not a mechanical keyboard. So if you like backlit macOS keyboards that are not, I would say better, and if you want space gray, they do not sell a space gray mini I don't want the number pad. They don't sell a space gray one without the number pad. They only sell white. They don't have a black one with the number pad? They have a black one with the number pad, not a black one without the number pad. Go figure. Baby will get one in September. Only pros use number pads. That's your problem. You you gotta upgrade. You gotta be more you gotta put more pro in the pro of Apple Pro. Joshua Sharfstein: What do I need a number pad for? Joshua Sharfstein: What don't you need a number pad for? It's there to like mishmash navigate. Turn it into like hot keys so that you can do, like, window management or something with it. When I hit 1, it goes to lower quarter. When I hit 9, it goes to the upper right quarter. No. That's what keyboard shortcuts are for. I'm telling you. Who needs extra keys when I can push 5 keys at the same time? Use the numbers as a macro pad, and you'll be okay. That's what my stream deck is for, and it gets in the way of my mouse. Oh my goodness. Okay. Today. Yes. Real quick before we do that. I should be recording the correct mic. Do I not sound like I'm on the right mic to you? No. You're not. Really? Yeah. I blame Teams. I have an issue with Teams, though we're not getting into the topic today. It has to do with its ability to select the right audio device even though the right audio device is selected. That's the Teams audio driver, Core Audio d. Horrible name because it's not actually Core Audio for, like, the system, but they named their audio driver Core Audio d for Teams, which is like, I don't know why you have a daemon with the same name as the other daemon that does the thing and Microsoft got a Microsoft. But let's just talk about other ways that Microsoft is gonna Microsoft. I don't know what other ways Microsoft will Microsoft. Loop 2.0 is out. That still doesn't allow you to secure a workspace with any form of group whatsoever. You're asking the wrong questions. The question isn't, can I secure it with a group? It's, why would I ever wanna do that? And then once you get over the hump of, you can't, you'll find another way through. Yes. You know what my other way through is gonna be? I'm gonna go create an Azure Automation runbook that loops through all of my teams or my groups, my Microsoft 365 security groups. So when to look at the name, it's going to create a corresponding workspace and loop, it is then going to iterate through every member in my group, compare it with every member of my workspace, and then add or remove users appropriately running every 10 minutes in an Azure automation runbook. No. I'm not gonna do that. Super easy. Barely an inquiry. Workspaces should be securable by group. End of story. Especially given that there are now notes for Teams meetings. If I'm gonna do a Teams meeting and my meeting notes are going into a loop component, said loop component 1 should be able to be assigned to a workspace so that said meeting notes can be in the same workspace with all my other loop things for said client. It means the way channels work and they can aggregate meetings and you can have multiple meetings in a channel, but it's technically still all in the same team and still Yes. Joshua Sharfstein: still in the same channel? Wouldn't that make sense? Joshua Sharfstein: And then secured by the same group security group, yes. Joshua Sharfstein: You tried to tell them how to fix the problem, not that it was actually going to be fixed, but let let me know how much your Azure automation costs you to run every 10 minutes. I think I can get it in for free. I figured out the math ones. 500 minutes, if it runs for 1 minute, I can run it how many times a day. We're gonna have to figure that out as Copilot. So have you no. You're Scott, I've been up since 3 AM teaching, and I warned you I'm gonna be all over the place. I keep trying to get you back into it. Thoroughly enjoying this. Back on topic. So back on topic, we chatted in the past about security, and we started getting into securing the modern workplace through the lens of identity and things that are available within intra ID. So we talked about things like security baselines for identity within intra and conditional access and some other stuff, and that kinda naturally leads into a conversation, which we had last time, about Microsoft 65 Office 365 workloads. Just because you're in that identity space, you're all tied in, you're already in SaaS land, and in that software as a service land. So you've got identity as a service, and you have software as a service things that exist out there like SharePoint Online and Exchange Online that are dependent on this identity as a service in the form of IntraID. The other thing that IntraID governs and becomes the identity store for is also Azure. So we should really talk about the Azure side of the conversation, which isn't going to be constrained so much to identity. Identity is a component about it. Right? So, like, something for conditional access, the way conditional access can apply to your SharePoint online tenancy. Conditional access can also apply to things like the Azure portal or potentially workloads that you stand up, say, like a custom website that you secure with entry ID and you're driving, like, OAuth authentication through there, then that's available to you as well. So if you're interested in, like, the ID side of it, you can go back and listen to that episode. But for this one, I wanted to just talk more about Azure Security in general, which starts to get a little bit weird because security is an open ended conversation anyway. And then it gets even weirder because it's Azure, and what is Azure, but a set of components that all sit behind this governance scheme of things like subscriptions and management groups and ultimately that identity construct in IntraID. So, yeah, we should talk a little bit about Azure and some of the other things that sit over here. So things like security we talked about security baselines for identity. We should talk about security baselines for Azure and what those mean, and how as we start to decompose out of identity as a service and software as a service, and we get more into, like, platform as a service, infrastructure as a service components that are out there, what do we do, and how do we think about that? Yeah. And real quick, because you mentioned identity too, it made me think of this. Just a heads up with identity, there is also a new identity in Azure. Starting in October, so, like, a month from when you'll probably hear this episode if you listen to it on release day, Microsoft is actually making MFA mandatory for all Azure users. Your break glass accounts. Come up with a plan for your break glass accounts because they are going to have to be MFA'd. A great way to think about that is our friend, mister Yubike or a Fido key. Since you do have to have MFA, tying that to my admin today's phone number might not be the best way to go about that. Consider things like Fido keys and plan for that because now you're going to have that for your break glass accounts as well. That I think that's probably more the automation and how am I getting in on the back end if something does go wrong scenario. Yep. Because this is also then gonna roll out early 2025, gradual enforcement of MFA for Azure CLI, Azure PowerShell, Azure Mobile app, infrastructure as code tools, like, everywhere. And with the YubiKeys, like, I would say see, this gets a little tricky because you also don't wanna just hide to 1 YubiKey because have you ever misplaced a YubiKey and then you can't log in? It's let's do this for multiple YubiKey and probably have multiple administrators of your Azure environment have these YubiKeys. Because I'm with you. I've had it tied to a phone or tied to my authenticator app in my phone and then you lose your phone or you reset your phone. That MFA does not come back reliably in my experience with the authenticator app. I don't know if you start switching to passkeys. Yeah. I like YubiKey for it. Phishing resistant, secure. But that is one thing I did want to mention that people should absolutely start preparing for kind of in that whole Azure security and identity vein that has been announced. To your point of dual enrollment, even with a single key, you might wanna consider dual enrollment depending on, like, the devices that you might need to actually use to access. For example, I have a USB c UB key with NFC on it. I've enrolled both the NFC component and the USB c component, and that lets me use USB c, like, on my desktop or my laptop when I need to get in. But if my desktop or laptop aren't available and I still have the key on me because it's on my Keyring, then I can still get in via my phone, which does support NFC. And it has to listen. There's the call out to multiple keys also to minimum every time. Any piece of hardware can die. It it can crash out. It can crap out. I've had the YubiKey, particularly like the nano YubiKeys, the ones that stick in, the little nubbin. Those things seem to die on me all the time, and I don't know if it's just cause they get so tight in the USB c ports and something goes wonky when you're trying to pull them out. Whatever it is, like, those things have not been reliable for me. So I've completely moved over to, like, USB c plus NFC kinds of things so that I can make my life just a little bit easier. Also, consider that if you're doing FIDO keys, they're going to come in various forms of connectivity, not just USB C and NFC, but also USB A. And what's the device I need to log in on? So you might actually want to have 2 or 3 of these, especially in the cases of, like, break glass accounts because it is super important kind of stuff. Like, it's worth investing $30 in a couple of keys and then distributing them out there and getting them to where they need to be. 100%. I'm with you. I have 3 keys that I enroll in almost all of my services when I use FIDO keys, and they're for sure all enrolled in Azure AD. Same thing as you. One's USB a, couple are USB c, one of them has NFC as well. Multiple options, they're not all in the same spot. So if I lose one, I can go look for 1 in another spot and all that. So moving on from identity and security to where you started to take me is security all up in Azure. Where do you wanna start with? Is this a broad, big, large topic? I think it's helpful to start at the top, consider what's the landscape of things that's available to you. So I I think one of the reasons this topic becomes so broad is you could say, I don't know, something like I'm a Azure SQL customer. Yep. And I'm running SQL inside of Azure Virtual Machines. I'm not running like SQL as a PaaS service, I'm running it as IaaS in a virtual machine. And in that world, you're running virtual machines. And how do you secure virtual machines, and what does that look like? And that's a pretty constrained conversation. Maybe that takes us back to the CrowdStrike thing. But if you take a step back from that and you say, hold on, like, before I was a virtual machine customer, what was I? Oh, I was an Azure customer. So what are the things that are available to me in Azure that I should go think about lighting up and turning on and enabling based on my scenarios? So in my mind, there's no reason that you would treat Azure in any way differently than you might like your Microsoft 365 environment. Right? So when you go into your Microsoft 365 environment, you don't first go into Exchange Online and start configuring security around Exchange Online. You start at the top and you say, oh, I'm I'm an m 65 customer. What can I turn on that gives me logging across the suite? And, hey, there's the audit log. So let let me go start to light that stuff up. What can I do to get visibility into the wider swath? And I think that's a good way to start with Azure as well because we've tied into the identity piece. So we talked about that and getting in, you can report on your sign in logs and start to understand that stuff. And then you take that next stop down to Azure, and before you even get into the services, there's this common base layer for Azure customers in the form of Azure Resource Manager and like the modern API surface, which is different than that old Azure service management service. At the ARM layer, you have these things like activity logs that are available automatically. Right? So for every Azure resource that's deployed, that you interact with, in certain cases that you perform like listing operations with or things like that, you've got the activity log there by default. And the activity log is integrated directly into a service, called Azure Monitor. And what does Azure Monitor give you? Azure Monitor gives you insights into metrics about your environment. So metrics being numbers. Right? Number of API calls, number of errors, number of sign in attempts, like whatever it happens to be given the the service you consume. So we start at the Arm layer, we start at the controllable plane. I think the first place we look is activity logs in general, because as you're getting into Azure, like you light it up and you have no resources, the very first resource you create let's say you create a virtual machine. So you're gonna go into the marketplace, and you're gonna click, I wanna create a new resource, you're gonna search for virtual machines, and you're gonna hit that blade. Ultimately, you're gonna submit that deployment, and that deployment, the act of submitting that deployment and having it ingested by Arm, captures that entry in the activity log, and then it's there for you. And then later when you come back and you interact with that virtual machine, let's say you come back and you add a disk to it, you're modifying that deployment, then that's captured in the activity log for you. And like I said, there's a whole slew of information that's captured in the activity log in general, but it's there for 90 days, 93 days by default. It's basically 3 months of data, both Azure Monitor for metrics and your activity log as well that you just have out of the box, ready to go. You don't have to turn anything on. You don't have to configure anything, but you do have to know that it's there and that it's available to you because what's going to happen is, for that activity log, any interactions with the control plane, so things that happen through the API surface of Azure Resource Manager, that would be your CRUD operations for these resources, your creates, your reads, your updates, your deletes. All that stuff is going into the Azure Activity Log, and it's just there, ready to go for you, which is nice. You didn't even have to turn it on. It's always weird to me that you have to turn on the audit log in Office in M365, but I get why they do it, but it's well, just turn it on by default. It's Right. They had a security incident. Right? That compelled them. That made them turn it on, but you're right. For a long time, multiple years, it was not on by default. And I said the same thing. I'm like, that's stupid. It should just be on. Anyways, yes, it is. It's on. It's there. Use step 1 is recognize the activity log is there. One of the things that I mentioned was the activity log is integrated with Azure Monitor. Azure Monitor is this whole Azure service that is a crosscut, and it allows ingestion from a bunch of other Azure services. But the goal is to give you access to that observability data. That comes in the form of the activity log, which is effectively strings, right? We're talking about like words that go in, this was a create event, here was the name of the resource, this was the UPN that initiated that call, so you can do things like track, k. Was this resource created by, a service principal? Was it created by a real human? Which human was it created by? And how does that start to tie back to your identity environment? That's all there for you. And the other thing that it gives you is the observability part of the platform is it also gives you access to metrics. So you're probably, like, once you start deploying Azure services, and and it's not directly related to security, but it's important to know that it's there, is you're probably gonna wanna think about ways that you can tie metrics into that and things like metric alerts. Let's say I deploy a storage account. So I deploy a storage account, one of the things that I might want to monitor as a storage customer is how many transactions or how many errors am I driving? Like, how many transactions are erring out? What's the class of the error? Is it is it a 5 0X error, like a a throttle? Is it a 4 0X? Maybe it's 4 0 threes or 4 0 fours, things like that. So that's all tracked in monitor by default. You can just go into monitor, and you can select a storage account, or you can even graph multiple storage accounts if you want to. I think you can put up to 200 individual storage accounts in a single Azure Monitor workbook for crosscut reporting, and you can immediately get those insights in a visual form. Or you can go the next click stop and you can say, hey, here's the things that are really important to me. Like, I might wanna know when I have more than n five zero three errors in a given period. So in the last hour, if I had more than a 1,050 threes, maybe I've got some segment of throttling going on that's a little bit higher than I want it to be. So you can do things like configure alerts on top of that metric or those set of metrics, or you can start to combine the logs and metrics inside of alerts as well, and and you can start to layer that information together a little bit along the way, if that makes sense. Some people are like, yeah, platform metrics, my storage account, my CPU resources, that type of stuff. Maybe not security, but I also think as you start developing a baseline for how these should behave, a storage account or a VM that you're monitoring these platform metrics on, and you set those alerts to alert you when you stray outside of your baseline, it could indicate a security event. Like, someone got access to your storage account and started uploading and downloading a whole bunch of junk to it or somebody got into a VM and is running a bunch of processes on it that are and you see it because your VM CPU spiked and is running way higher than it normally runs. The SQL resource or SQL platform metrics, all of those things, I think, are very valid, and it's something that people would monitor anyways on prem. If you're running a VMware host or some other servers on prem, you're typically monitoring CPU consumption, memory consumption, storage, disk, IO, all that type of stuff. So why would you stop monitoring it when you go to Azure? Because, ultimately, Microsoft doesn't care if you fill up a storage account or if you incur a bunch of costs or have some type of security incident that's really affecting your data. Right? It's still that shared responsibility model of I'm running a VM in Azure. I'm still responsible for patching my OS. I'm still responsible for antivirus on the v maybe even storage accounts, like, getting into some of the other security options maybe that we'll talk about is what files are in my storage account that I have up in Azure. So I think from that perspective, like, platform metrics can still very much be an indicator of security issues or other issues. I I have a lot of conversations with customers in this area, is there's both the control plane and there's the data plane. So there's interactions that are driven through Azure Resource Manager, like I want to in the case of a storage account, I wanna create a storage account. Now once your storage account is created and you do things like you stand up a container and you upload an object into that container, the upload of that object, the creation of that container, those are data plane operations. So we've we've actually crossed over to a new API surface. But the the the thing to remember here is, like, in the context of, say, a storage account and just about every service I can think of off the top of my head that's in Azure, even though it's got a data plane to it, that data plane information was still available and monitored to me alongside my control plane information. So it's a kinda have your cake, you need it too scenario, and it's important to recognize, like, where those things are coming in and how they're getting there and where they're coming from. Because in that scenario where I described monitoring a storage account for things like number of errors, those errors, those those HTTP error codes like that 404, 403, those 5 0 x errors, those are ultimately data plane metrics. But I didn't have to do anything to turn them on. Like, they were just there for me out of the box, and they were presented to me through that observability layer in Azure Monitor where it was automatically combining my control plane and data plane. So while Azure Monitor itself is not a security service, it has these components that are giving you all the observability and alerting capabilities, which leverage the right way, becomes something that's a component of your strategy to thinking about how you monitor and secure your environment. Because a lot of security constructs do come down to things like observability. If you think of even like the basics of an AV client, an AV client is there to watch for heuristics and new things to load and try and block them when they're malicious. So it's own kinda like little observability container. And, yeah, once you start to walk down that path and and recognize that crosscut is there for you, I I think it it makes things a little bit easier, and it opens up that mindset of, how do I get out of just observability mode into translating my security needs into things that observability is going to inform and where does this pull me to? Do you feel overwhelmed by trying to manage your Office 365 environment? Are you facing unexpected issues that disrupt your company's productivity? Intelligent is here to help. Much like you take your car to the mechanic that has specialized knowledge on how to best keep your car running, Intelligent helps you with your Microsoft cloud environment because that's their expertise. IntelliJunk keeps up with the latest updates in the Microsoft cloud to help keep your business running smoothly and ahead of the curve. Whether you are a small organization with just a few users up to an organization of several 1000 employees, they want to partner with you to implement and administer your Microsoft Cloud technology. Visit them at inteliginc.com/podcast. That's intelliginkdot com/podcast for more information or to schedule a 30 minute call to get started with them today. Remember, Intelligink focuses on the Microsoft cloud so you can focus on your business. So that covers activity logs, platform metrics, those both collected by default have access to. There's one other type of kinda core log. I would say you probably also want to monitor high I would highly recommend you start monitoring, but this one is not on activity logs is I'm going in and I'm creating storage accounts, I'm creating VMs, platform, the metrics we just talked about. Resource logs are actually then insights into operations that are performed within the Azure resource. So actions that are performed within a storage account, uploading, downloading files, actions within a VM. You can pull out event logs from VMs, actions within the SQL database. You have to turn on your diagnostic settings in these individual resources or potentially use something like Azure policy or something else to turn these on and then figure out where do I want these logs to go. And these are going to very much vary resource to resource based on what types of logs you're collecting. Logs within IIS are very different than logs within storage, are very different than logs within VMs. You can pick and choose too. There's multiple logs within all of these. You can go in and say, I just want all the logs from this resource. Or based on the resource, you can go in and pick, I just want this certain type of information, these certain logs from this particular resource. And usually, you have different options. You can go into storage accounts with these. Scott would love it if you put all your logs in his storage accounts. Typically, I think what people tend to do, and this is what I tend to do, is send these into log analytics. You can also I can't remember. What are the are there 3 or 4 options? Log analytics, a storage account, queues? There's 3. Log analytics, storage account, and event grid. Yes. So you can send them back through through eventing. Taking a step back, when we talk about monitoring and observability and then tying observability back to things like what do I want to monitor to think about the security of my environment, You need to start answering those questions like service by service or area by area. So for a storage account, I could be really interested, like I said, in things like number of errors on a given subset of transactions that exist out there. So that's my class of, hey, what are my 4 0 x errors for GitLab requests? Like, how many unknown requests am I getting? Is somebody randomly hitting me? Once you start getting to the place where you're like, show me the user who did that thing, or show me the URI for the resource that was impacted by that event, those are logs. So a good way that you can rationalize this in your head, or at least it's a way that I found to work for me, is metrics are always numbers, and those are always gonna be in Azure Monitor, and they're always gonna be there for free for the those 3 months. Outside of that, it's on you to retain them for longer. Once you start talking about things like, oh, was it a UPN? Was it a URI? What was the operation that was called? Things like that. Those are logs, and logs are strings. And if it's a string, it's always gonna be in a resource log if it's coming from, like, a native service perspective. So if you're just looking to rationalize, hey, do I need to turn on resource logs? If the thing you want to interrogate sounds and feels like and it is a string, yeah, you need resource logs to make it work. We talked about the activity log for Arm. That absolutely is a bunch of strings that you can go in query through Azure Monitor and light up scenarios around that without needing to enable resource logging on a given resource and standing up a log analytics thing or dumping your logs to a storage account and then worrying half, how do I download it? What's the scheme of the JSON file? All those kinds of things that come along with it along the way. Yes. Absolutely. Your audio wasn't going through the Discord. That was my fault. Yeah. Teams. It goes back to my Teams issue at the beginning. But, yeah, absolutely. Excellent points on all of that. Trying to think. Activity logs. We got platform metrics. We have resource logs. Which did you know? We have all that log data, Entra. Yes. And I'm gonna tie this back. The log data in Entra has been there. I noticed it with global secure access and some of the enhanced logging you can do now, because this is a requirement for enhanced logging, is you can actually configure Entra diagnostic settings. So these resource logs we just talked about, Entra has resource logs where you can go set up Entra to send its essentially diagnostic logs into log analytics as well. Mhmm. I don't know how long that's been there, but I haven't always had that set up because I haven't had a need to until I was playing with Global Secure Access. So for these things like monitor, entry ID, or like the activity log, entry ID logs, things like that, where they're there by default, but maybe they're not pumping their data out to to another source like Log Analytics. The reason that you start to pump the data out to Log Analytics, or maybe you go and you look at security specific offerings like Sentinel or things like that, that can ingest that data for you, or they can act as a target or a sync source for that data, is because you wanna retain it longer, or because you wanna start to get that crosscut visibility and insights into, do I need to correlate an event that happened in enter ID to an event that happened in the activity log to ultimately an interaction on the data plane in one of my resources. That might be something like, do we want the ability to track the malicious sign in? Say there was like a phishing hack, and somebody got in with some admin credentials to your enter ID tenant, And then from there, they created another user, they created a service principal, and they gave that some kind of elevated rates in your environment, and then they connected with that user. Great. The login attempt up here is in Entra, and then the other interaction now is down in the activity log, and then all the way down to what they do while they were there. Oh, I can see that they went into my storage account, and they changed the configuration of a container on my storage account, and they enabled it for public access, like public and on access. And then I tie that back to a metric, and I see that they egress some data or something like that along the way. You know, so it's, like, a little convoluted. I I I think, if you're not, like, deep and and, like, familiar with it and you haven't touched it all. But once you start to touch it all, it it starts to make sense because you're just decomposing it into the various layers. So really, you're walking through, like, the various API surfaces, right, like, Entra and Graph, down to Azure and Arm, and the control plane, ultimately maybe back to, like, data plane within an Azure service, if it happens to have a data plane. Not all services have data planes. Some of them only have control plane interactions. It's hit or miss depending on the service, so you do have to know what you're going for service by service. And someone in Discord was just asking to, like, being able to track who created, like, a function app and when they created it and stuff and where those would be. That would be more of that Azure activity log. That's gonna be logging more of those, and we had talked about a little bit more of those control plane creating those. But I do agree everything you just said about logging, and that's why I personally logging all of these to log analytics. We'll probably start talking about more security stuff in future episodes. To your point of it's these different APIs, it's these different control planes, but then somebody gets into NTRIC, gets into a storage account, finds a file out there that has other login data, and then they go log in to SharePoint and pull information out of SharePoint. It in my opinion, it's really important when you have this wide range of things. Like you said, bring all of those logs into one central spot. So when you are investigating an incident or doing some type of threat hunting or if you wanna set up some type of alerting, all these logs are together, and you're not jumping between the entry ID sign in log and then over to the SharePoint audit log and then over to the Azure activity log and then jumping back into your diagnostic logs for your storage account. It gets really hard to correlate everything when they're all in these disjointed places, and that's where I think log analytics and Sentinel, if we talk about Sentinel in the future, really give you a good security benefit when it comes to being able to pull all of this together. So it's an interesting conversation. I have it with customers quite a bit. There's the class of customers that is scrappy. I absolutely understand the motivations, where they're coming from, and they want these things to be free. In that world where they want them to be free, they kind of stick with the free offerings. So they do these things where they end up building bespoke tooling, say, to use the graph PowerShell commandlets to query the the sign in logs and things like that from Entra. And then they go and they wire up a CLI command to extract metrics from over here, and they build these. It's gonna come across as, like, derogatory. I don't mean it to be that way. You end up building these, like, Rube Goldberg machines that that are just these mismatches of things. And they accomplish their goals in many cases, like, they figured it out and they learn how to get there, and they think in their heads they did it on the cheap. But the reality is they had to invest all the time into building that bespoke tooling and doing all those things. And ultimately, you still need to spin up the compute. And and the compute is what costs you the money in these scenarios is, k, I need to wire something up that can be online and available with networking and disks so it can talk to that other thing and bring it down and ingest it. I caution folks about they get into their heads very quickly. They look at the pricing of Log Analytics or Sentinel or things like that, and they go, too much money. I can just go send my engineer off, and they can take 3 months and build it for me. Your engineer that just built that thing for 3 months, depending on their salary, you might have been able to buy a year of Log Analytics at your ingestion rate. It all depends on your environment, like what rate you have and rate of change and churn and retention and those things that you want along the way. But, I I think if you take a step back and you peel it back, like some of these things that look like they cost a lot of money when it comes to security, they're like a net wash when it comes to resourcing and operations and things like that. You do have to keep those things in mind. Oh, yeah. Another interesting question in the chat. So if you use a third party SIEM instead of Sentinel, would you consider log analytics aggregation doubling up on a function? So you don't have to Maybe. Install. I I I think is the yeah. But I think this is another misnomer is, like, customers feel that they're locked in, and I I get it. To a certain degree, you are locked in. Right? If you're an Entree customer, absolutely. You're using Entree. You really don't have another choice there. That is the identity provider for Azure and M365. So you you can do things like have your IdP, Okta, or whatever, as the relying party, but the reality is you're using Entra at the end of the day. Some stuff you can't fight. Some of these things you can. Do you have to use Sentinel? No. Do you have to use Log Analytics? Absolutely not. You can do things like build out that integration for those diagnostic logs, which can include metrics in them. They're not just resource logs. You can pump out the metrics as well. You can build out those integrations around things like Event Grid and pump those out to any system that you want to. I have no expectation that every customer who comes to me who says, I'm a Splunk shop, is immediately going to go and turn the switch and learn Kusto and start doing Sentinel or log analytics tomorrow. That's not good for them. They're already a Splunk shop. Or you might have another solution that's out there. So 90 ish percent of the time, and it's almost a 100% of the time with, like, the really large providers out there, they're going to have some integration that is going to be able to hook into Event Grid and the eventing system. That's a core service within Azure. Right? It's available. It's scalable. It it will do all those things for you, and it it would allow you to put that data out to where it needs to be. Your answer of maybe classic consulting 101, but very true, there might be pieces of functionality that you could absolutely replicate and do yourself. It's just, to my point earlier, it's not worth your time, right? Like the engineering effort that you'll invest to duplicate some of these solutions when they've already been done and built for you, I I really do think you need to have that kind of rationalization moment of, do I want my engineer to take a day, a week, a month, a year, whatever it is to build this thing, or can I just buy it and be done with it? Let's be honest, like you're in a pay go service anyway, everything's not gonna be free for you. Be smart and pick and choose where you're spending your money. Yeah. And, right, it depends too is we're looking at this from a security lens. So I would say from a security lens, if you are sending everything to Log Analytics and everything to a third party SIEM, yeah, you're probably doubling up on a lot of stuff that you don't need to be. That being said, Log Analytics is also not just security. So I actually have some clients I've worked with as well that have 2 log analytics instances. 1 that they send everything to from a security standpoint or even certain, resource logs that they wanna look at from a security perspective, and then I have another one for, like, operations because, for instance, Azure app services. If you're hosting a website in there, part of the resource logs or those diagnostic logs are also used for app insights for doing things like tracking website visitors and response time on your website and things that operations developers, your SEO folks, if they care about response time, certain, depending on their role, may actually want some of these logs. So you may still want a certain aggregation in log analytics, not from a security perspective, but from a operations monitoring perspective or from a debugging perspective or from a we just wanna know if there's an error on one of our Windows servers, again, going back to some of the operations. So I think from a pure security perspective, yes. There can absolutely be use cases for sending data either to both or even, again, sending sometimes data to 2 different Log Analytics workspaces. We mentioned the diagnostic logs. When you configure those, you can send them multiple places. You can send them to 3 or 4 different locations. It's not, I'm only going to send these logs to 1 or the other. Yeah. It's like level of effort and things like that, and I was saying you've inquired the whole time. It's EventOps, mea culpa. So the other thing to keep in mind too is if you're a customer who's coming to Azure with an existing solution and you're, like, looking at this space and you're going, yeah, I know there's some native stuff, but maybe you are that customer who has a third party SIEM or something else. Maybe you use, like, a a different firewall. Like, you you're out there using FortiGates and you're, like, Azure Firewall is not my thing. Those options are often available to you as well. So it's not like you have to completely ditch the ecosystems that you're in today. There there there's a pretty wide swath of partners and ISV solutions that are available to you. So like in the monitoring space, Elastic's there. So you can do like Elastic Integrations, Elasticsearch, things like that. Datadog is another one that I run into a lot. Like, I end up working with a lot of our cloud native customers and things like that, like custom dev shops who are spinning things up. So I see a lot of that. You can do things like Event Hubs has native Kafka integrations. So if you're pumping data out and you wanna pump it through Kafka and then ingest that over in, say, Databricks or do some Spark analysis on top of it, that's just all available to you. It's all possible. It's there. The hooks are there. It's worth checking through the documentation and seeing, hey, is there something that's already here that's in my wheelhouse that I'm familiar with so that you don't need to reinvent the wheel. And I think that's a big consideration, right? Because if you're reinventing the wheel or you're at it net new and you've never done it before, it's not like an immediate security hole, but it's a gap for you. And the and the more gaps you have in that observability, the less comfort factor you have, and you start to go down the weird rabbit hole in your own head sometimes about it. Ultimately, like, the cloud is about plugging a a bunch of pieces together. I I always think about it as like Lego bricks and every if I gave you a bucket of Lego bricks and it had a 100 different colors in it and I just spilled it on the floor and I said build me a house, you're gonna build something that looks like a house. Your house might have 4 walls, and the one my overachieving son builds is gonna have 6, whatever it is. But it's still gonna be a house, and that's okay. You're not gonna necessarily land on the same solution that somebody else did. I see a lot of customers that are like, just tell me how the other customer did it. I'm like, somebody else did. I see a lot of customers who are like, just tell me how the other customer did it. I'm like, I I really don't want to. We're not gonna be bespoke, and it's not like everything's in, you know, a unique snowflake that fell on the ground, but at the same time, we're all gonna be a little bit different. And that's okay. That's the place we're gonna land. Yep. Agree. Scott, when we said at the beginning of this episode, we're gonna keep an eye on time and quit out 30 minutes. Yeah. And we didn't even make it past, like, metrics and resource logs. Who saw that coming? Not me. Maybe we need to rename this to, like, the Microsoft Cloud Security Podcasts. Something observability. Yeah. We can probably cut it here. I think that's a good kinda grounding and an overview of Log. Some of the observability pieces and some things to think about there. We should definitely come back and revisit some of the native tooling that's there. I I think it's worth talking about. Sentinel and and some of the other things there, Defender, which we've talked about Microsoft Defender in context of Lakeham 365. I don't know that we've ever talked about Defender for Cloud and some of the integrations that come on the Azure side. There's a bunch of third party stuff out there. So, yeah, we can just keep running with this one for a while. I think that might be the plan. We'll see how long it goes. Not how long we can drag it out. We don't wanna drag it out, but how long it takes us to cover it to our satisfaction? 2025. Here we come. Perfect. Alright. Scott, go enjoy your weekend. Sounds good. Rest. Relax. As always. Thanks, Ben. Thank you. Alright. Have a good one. If you enjoyed the podcast, go leave us a 5 star rating in iTunes. It helps to get the word out so more IT pros can learn about Office 365 and Azure. If you have any questions you want us to address on the show or feedback about the show, feel free to reach out via our website, Twitter, or Facebook. Thanks again for listening, and have a great day.

Donate

X (Twitter)

Facebook

LinkedIn

Apps

Welcome to Episode 383 of the Microsoft Cloud IT Pro Podcast. In this episode we continue our discussion on security, transitioning from Microsoft 365 to Azure ( Episode 382 – Securing the Modern Workplace: Exploring Microsoft Entra ID Security Defaults, Conditional Access Policies, and Microsoft Secure Score) to Microsoft Azure. We begin with Azure security logging and auditing, discussing the importance of capturing and analyzing security logs to detect and respond to potential threats. Next, we delve into Azure Monitor data sources and data collection methods. Azure Monitor provides a comprehensive set of tools to collect, analyze, and act on telemetry data from your cloud and on-premises environments. Finally, we discuss how the components of Azure Monitor can be used for managing and analyzing the vast amounts of data generated in your Azure environment. We’ll explore how this platform supports advanced analytics, helps you identify trends and anomalies, and enables proactive threat detection.

Whether you’re a cloud architect, security professional, or IT admin, this episode offers practical advice and strategies for leveraging observability to enhance your security posture in Azure. Tune in to discover how you can better protect your organization by implementing effective observability practices.

Like what you hear and want to support the show? Check out our membership options. (more…)

Episode 377 – Microsoft Copilot for Security

by [Scott](/content/author/scottmsclouditpro/ "Posts by Scott"/index.html) | May 23, 2024 | Podcast

Blubrry Player

Auto Scroll

- Welcome to episode 377 of the Microsoft Cloud IT Pro podcast recorded live on May 21st, 2024. This is a show about Microsoft 365 and Azure from the perspective of it pros and end users where we discuss a topic for recent news and how it relates to you. This week, Ben and Scott discussed a recent Google Cloud event where a customer account and all of their data was completely wiped out without notice and we share some thoughts we have around customers protecting their cloud deployments. We also have some updates and thoughts around co-pilot for security after Ben has been able to get some hands-on experience with it. We talk about pricing, approaching the experience and how to think about leveraging it in your environment. We've had a bit of a chaotic week, so we're gonna get through today, but I should go bring up this article. This was an interesting one and we don't like throwing other cloud providers under the bus because, oh what am I doing? Because stuff happens. Microsoft has stuff happens, Google has stuff happen and Amazon has stuff happen. But this was just, there's some thoughts that I had that came out of this news article that I ran across. I actually had a friend that sent this to me the other day and I'm curious to see what you think Scott. So this was on ours, Technica, and you can actually go read this article as well on the company's website that this happened to. But the news headline is unprecedented. Google Cloud event wipes out customer account and it's backup and then it's UniSuper, I think it's UniSuper as how you pronounce it, UniSuper, UniSuper, UniSuper, $135 billion pension account details, its Cloud compute nightmare. - This is a rough one. Reading - Through this essentially sounds like there was some configuration change, something that happened as Google was standing up the private cloud for this client and somehow like they wouldn't have been standing up a brand new one because they already had a bunch of data up there, but it essentially wiped out UniSuper GCP account including all of its information in all of its backups that were stored in GCP. It's like they went in and just said Delete UniSuper from GCP .- Yeah, that's one way to think about it. So you sent this one over and I was kind of reading through it and trying to think about it in context. It really, I think it'll be interesting to see if an RCA ever comes out for this one like root cause on what actually happened here. Yeah. But from the outside looking in, what it looks like is this customer had an existing set of workloads and an existing account with A GCP and with Google Cloud and for some reason that existing account was deleted. I would be willing to bet like weird series of circumstances like that. It was something like really crazy like the customer's account got flagged for fraud because of something that was happening maybe due to like an automated fraud detection system or something like that. And then due to a series of errors in the fraud detection system, it just went completely off the rails and potentially took out their data. And the other things that sidecars alongside with, hey, this is a customer's account and weird things happen. So you hear about this all the time. I think like think about it in context maybe of like, let's take a step back from cloud provider and think about like social media providers. Like I don't know if you've ever been on a platform where you've had your account banned just unilaterally. Yep. Hey, this thing goes away and you can't do anything. You know, for me, I had my Google Ads account banned and I'm banned for life for Google ads and there's no remediation forum, there's no one for me to go talk to. This is a permanent decision and it's been made kind of thing. I was reading about one last week with a venture capitalist MG Siegler where his Instagram account got banned by meta. Same thing like hey, you've been banned unilaterally. There's nothing that you can do about it. In that case, they ha, you know, MG happened to have contacts at meta and can turn it back on. And in this case, this provider at UniSuper, so it's a superannuation fund in Australia, effectively think like 401k provider here in the United States, but government mandated retirement fund kind of thing. Like not a good look to you have it all go away and get blown up that way. So you quite often do hit these very weird code paths and things that you don't think you're, that you don't know that you were going to encounter until you encounter them. And unfortunately in this case it took down an entire customer along the way and not just like the entire customer. It took down their entire estate because they were so all in on Google and GCP for their workloads. Like so all in to the point where they've been quoted in joint press releases in the past of saying, Hey, we're all in on Google for their VMware engine, you know, their migration capabilities for getting us on-prem to the cloud, all these kinds of things. So what a not great week that company had , right?You know, be it from their CEO all the way down to their folks who were the ones who made the decision to go all in on Google and then for Google itself, like these are the kinds of things that follow you around as a company for a while. And I wouldn't be surprised if this one doesn't get more mainstream press or hasn't had more mainstream press beyond ours. I think ours Technico was just the first place that you had seen it and sent it over my way. - The part that blows me away about this and Google said this should never have happened, which is kind of apparent, but one, they said someone should never just be able to delete their account. But it was deleted to the point that in this joint statement from it was a joint statement from UniSuper and Google, both their CEOs or the Google cloud CEO was that UniSuper had a bunch of redundancy in place. They had two geographies to protect against outages and losses, but none of that protected them. What protected them was they had their backup in another cloud provider. It wasn't even like Google could go back to some backup that even they had internally in restore it. To your point about like this unilateral decision, I would've expected that Google would've had some customer account backup or protection in place, even behind the scenes to where maybe UniSuper couldn't recover their own account because it was all deleted and everything. But it sounds like Google couldn't even recover their account that they had to go stand up a brand new GCP instance and restore all of their servers, all of their data from a backup they had with another cloud provider. - It depends on how that data is stored. So I can absolutely see how such a thing would happen. You know, if you think about all the things that we've talked about in the past with your data in the cloud, when it comes to things like data ownership, you know, here in the United States, if you think about things like NIST standards for shredding hard drives, right? If I have, if I have my data on a hard drive at a provider, how does that provider shred my hard drive and ensure that I'm the only one who has access to my data? Those kinds of things. Yep. There are these very real kind of kill switches in place that effectively once the key, the primary key is gone and it's severed from the data, there's really no way to bring that relationship back. And quite often the hyperscalers are doing things like doing garbage collection on data pretty aggressively. You know, it's not a bank error in anybody's favor to retain terabytes, petabytes, potentially tens or hundreds of petabytes of backups for customers. And you know, just have those sitting around for weeks and weeks and weeks waiting for a customer to go, oops, I didn't really mean to delete that kind of thing. And especially if it's a a back to the whole like you know, how could this happen? You know, think about it, especially if it's a fraudulent workload, like you don't want that stuff on your system to begin with. True. So you potentially just nuke it from above and call it a day, especially if you're very, very sure that it is in fact data that should be nuked or a workload that should be nuked an account, uh, a billing account, things like that. Whatever it happens to be. So pretty unfortunate. More, more than pretty unfortunate. Very, very unfortunate. Good lesson though, in kind of thinking about the multi-cloud thing and DR in a multi-cloud world, how you think about positioning yourself with other providers as a customer, right? Like you might be all in on Google, but then you might leverage say like enter ID for your identity and as your security token service you could be all in on AWS, but you might leverage a component of Microsoft or Google for something in your workload. You know, there's a whole bunch of customers that do those kinds of things too. It's rough. Make sure you got backups right. That whole rule of three thing becomes,becomes pretty critical here. And not just do you have backups? This was another good lesson in even though they had backups, recovery was still forever. Like it wasn't about just having RPO RTOs were extremely elongated in this case. And if you think about it for a financial firm, that's kind of a super critical thing when you have money flowing in out in the case of something like this, which is a effectively a pension fund, pensioners who are in that fund, like you still need to get your payment right, right. To be able to to buy food and survive and, and all those kinds of things too. So just a bad situation all around. Yeah. And hopefully they find a way through. - It was out two weeks here. It was, it was May two was when it started and they full restoration of services on May 15. So it sounds like everything's back up now as of about a week ago. But yeah, being down for almost two weeks and I agree. I think the biggest part that one of the biggest takeaways from me even thinking about like my Office 365 environment was that whole thing of having some of those backups. 'cause you are, you hear so much about this and I've talked about this a little bit more recently too, of, oh well Microsoft has multiple data centers or multiple regions. I mean AWS Google, everybody has multiple data centers, multiple regions redundancies in place and some people in I would say six, seven years ago I prescribed to this a little bit more than I probably should have of those redundancies are gonna protect me. Like I don't need to have my own backups because Microsoft is building in so many different backups that why do I need to go pay for another one? This one went in and highlighted of, it's not common. I mean this is a one-off in Google's case. I can't say I've heard of any accounts in the Microsoft cloud where somebody's gone in and just everything's gotten deleted. But of having those backups to your point somewhere else so that if you are the one that finds yourself in one of those one-off scenarios, you can get your backups. Like I can't imagine a company this size if they didn't have those backups in another cloud, how much worse this could have been for them. - Yeah, definitely detrimental. You know the other thing to think about here is so, so you know you're kind of calling out, Hey do I have the backups? Do I have the backups? Like let me think about that. Like absolutely like think about backing up your data. But I think it's also critical and if you go read through, I'll, I'll put the link in the show notes to UniSuper kind of timeline of what happened and if you read through that timeline and how it goes, one of the things that potentially delayed them was also not just having the backups but having the configuration right and the ability to stand it all back up on that side. So for you, let's say for you as an M 365 subscriber Yep. Are you using things like the community tools for like M 365 DSE to back up the configuration of your tenants on a regular basis? Are you testing that you can stand up a new tenant with a similar configuration beyond just kind of the data pieces and the backup and restore bits there? - You're gonna catch me. I'm not doing that with mine. I have clients that I'm doing that for but I don't do it with mine. - So I think that's the other click stop that folks need to consider. Like we talk a lot about kind of uh, application recovery and having backups and user data is absolutely a critical piece of that. The other thing that comes into play here very much is recovery and configuration and, and all those kinds of things. So to the degree you can with the providers that you have and the systems that you stand up, really do think about that stuff holistically if it's within your wheelhouse - Configuration - And it, for some people it is for some people it isn't. You know, if you're out there and you're listening to this and you go like, oh I pay an MSP or something to do all this for me so I don't have to worry about it. Yeah, maybe go ask 'em some questions, right? Just make sure that you've got the warm fuzzies about what they're doing and how they're actually providing you value in cases like this. - Yes, I would highly encourage you to ask your MSPs questions about this type of stuff and it is, it's your value. And I will say like the clients I'm backing up configurations for it's, it makes sense. I think in my case, like for my tenant personally, if I lost my conditional access policies, I wouldn't really care App registrations. I mean some of those you think about too, like all my app registrations that are tied into Azure ad, if I lose Microsoft NID, it is always going to be Azure AD Scott. Either way if I would lose that and lose all my app registrations and then not be able to authenticate to some of my third party apps, like do I have backup credentials saved for the native logins for those versus just my SSO logins and and it was a a good callout. There's a lot of stuff besides just do I have my emails and my files to think about when you're in these cloud DR scenario situations, do - You even know what the stuff is? is an interesting one.So yeah, the other thing I often think about is if you are building and deploying software, how do you think about recovery within and standing up assets again if you have to around things like build pipelines and deployment pipelines. Uh, you know, so if you're using like GitHub actions, do you have that YAML save someplace? Like what happens if somebody comes in and nukes that repo right? And it just goes away one day, like how do you get over that and how do you do it? So you know, I think taking a step back, having that good holistic view of your entire estate, not only what resides in your estate but how that stuff was built, how it's configured becomes extremely important. And in some cases, you know, you can't automate your way out of a job when it comes to doing recovery with some of these things. But I think it's important to just understand kind of where those rough edges are and that you've accounted for 'em in your runbooks and all the other things. - Yes, a hundred percent. Do you feel overwhelmed by trying to manage your Office 365 environment? Are you facing unexpected issues that disrupt your company's productivity? Intelligent is here to help much like you take your car to the mechanic that has specialized knowledge on how to best keep your car running intelligent helps you with your Microsoft cloud environment because that's their expertise. Intelligent keeps up with the latest updates in the Microsoft Cloud to help keep your business running smoothly and ahead of the curve. Whether you are a small organization with just a few users up to an organization of several thousand employees, they want to partner with you to implement and administer your Microsoft Cloud technology, visit them at intelligent.com/podcast. That's I-N-T-E-L-L-I-G-I-N k.com/podcast. For more information or to schedule a 30 minute call to get started with them today, remember intelligent focuses on the Microsoft cloud so you can focus on your business. All right Scott, should we move on to our next topic? - We spent a quick, we've spent - a quick 20 minutes, quick 15 minutes there.15, 20 minutes on this one. Yeah, but it's an important one. Again calling out, I think using this story to highlight the importance of some of these backups and things to think about. This next one is a little bit more of an update on a topic we talked about earlier. Diving a little bit more focused into the Microsoft space is security copilot. I don't remember which episode it was. It's been a few episodes ago when it first came out of GA and it was available to stand up and I had said that I went and turned it on in my environment. I went and created a security compute unit instance or an instance of copilot for security, which you can scale based on security compute units. It's $4 per hour per security compute unit. And I spun it up for like two or three days and didn't see anything on my Azure bill. I was like huh, I've only used it a couple times. Maybe it only bills when you ask it questions. Something like that. And people were like, well let me know when you find out what happens. So I turned it off for a while 'cause I started getting scared that it was just racking up a bill in the background and even though I had my quota on there and I just got nervous. So the other day I turned it back on again just to find out what would happen and it started charging me very quickly. Do - Me a favor. Yeah, just so everybody has context, flip your web browser back over to - You for those that are seeing this . Yeah,- Flip your screen back over to cost management here. - . Yes. So here is it.I went from, and if you can't see it, I had like a $59 that I had accumulated on May three, may four. I was up to $162 and then it kept going very linearly. So the first two days it looked like a massive hockey stick 'cause I went from increasing my bill like 12 bucks a day to increasing it by roughly $113 a day, a hundred dollars a day and then it continued and I do still have like my $150 limit on the subscription. So even though it shows my accumulated cost up around $1,400 now because I may or may not have forgotten to go turn it off once I saw it hit that it is very much one security compute unit at $4 per security compute unit per hour has absolutely nothing to do with how frequently you use it. Once you light this thing up and turn it on, you are going to get billed $4 per hour as long as it is created in in existence. So realistically security compo copilot is absolutely going to cost you, I think it's like $2,920 a month if you go do the math and multiply out how many hours and average days in a month over the course of a year. All of that math. And that's just for one security compute unit. Know when you go spin these up, Microsoft recommends a minimum of three security compute units. Again, not saying you need it, they let you still do one, but three would absolutely cost you $12 per hour per yeah $12 per hour over the course of a month, which you can do the math 3000 multiply by three or at like $9,000 a month for Microsoft's recommended minimum configuration for security copilot. So pricing it is absolutely if you wanna use this, going to cost you a minimum of three grand per month unless you do something different. And I had some conversations with some people the other day that are like, I'm starting to use like logic apps or PowerShell or things like that to ramp up their security compute units or actually to like, I don't know if they were actually going to the point of destroying security copilot and recreating it when they'd wanna use it. So again, if you're not gonna use it at all during the night, do you really need to have three or four or five security compute units provisioned from 6:00 PM until 8:00 AM the next morning or do you actually blow it away and recreate it? It led to an interesting discussion I had about different ways people are trying to manage costs of security copilot based on number of security compute units provisioned within that instance of security copilot or even just like, can we just blow it away? It's expensive Scott, which led us into other discussions about ROI or even some of the pros and cons of blowing it away versus ramping it down, things like that. - And other news AI be expensive , right, I guess isis the takeaway there. So while it's a loss leader in some places maybe think like co-pilot consumer versus co-pilot within versus M 365 co-pilot, co-pilot for security is definitely not a loss leader kind of thing. So you need these SCUs, these security compute units to actually be able to have the associated compute to run through an action on your queries within the underlying LLM. So you know, responding to a query in an LLM takes a bunch of CPU and and memory and other things on the host to go and actually like retrieve the data read out of the vector databases, do all that kinda stuff. If you're doing RAG or something like that along the way, retrieve augmented generation, well then you've also gotta go out and have the compute to be able to retrieve that data. Say it was like a Word document or a PowerPoint, something like that. Be able to parse that in an LLM and then be able to construct these meta prompts and and all the other things so it's not free to get there. It's also very kind of fuzzy as to what that looks like and how it manifests. So you know you do have some usage monitoring within copilot for security. So directly within like the security copilot portal, security copilot.microsoft.com and being able to go thing, go and see things that way or you have kind of this just raw view of costing within cost management and how that carries through. But it's an interesting thing. You're sitting at $4 per hour, at least in the hero regions out here in the US you know that equates to, like you said, basically three grand a month, three K 29 90 I I think it's safe to round up a little bit and and just call it three grand in that case. Yep. And then you have the kind of recommendation for compute units. So if you don't know what you're doing with these things and you just kind of look at 'em and you go out and read like hey where should I start? Well you can start with one, I believe the recommendation is three. So you're kind of sitting at a min cost of nine grand per month before you've really done much with it. So like all things that needs to be measured and weighed, right? Like what's the ROI there and what's the value for me as a customer? Like once you're starting to hit like nine grand, you know, is it worth having that for an entire month or should you just pay for an MSP pay for a consultant, something like that. Once you're doing a couple of these and you're getting up to maybe not like the nine grand marker but let's say you hit the point where you're at six SCUs and you're running those for an entire year, now all of a sudden you've gone from 90 plus KA year to 180 k plus pretty quickly and that's theoretically somebody's salary , right?Including benefits and and other things on top of it, at least here in the us. So now is having one person better than having a bunch of commute units, I don't know, you know, needs to be weighed out organization by organization. I think - This is where it does start getting interesting for me in some of our discussion even before we started recording was how do you start showing the ROI of co-pilot for security in that particular case? Because again me, I have myself a couple of contractors that are doing work for me. I am not gonna go out and spend 30 KA year for security copilot. I can go into my audit logs, I can go write power shell, I can go write KQL and it is probably not going to save me 30 grand of time a year to have co-pilot for security in place. I'm not gonna spend that much time asking security questions of my environment with the size of company I am. But to your point, this is where the scale gets interesting and can you show that ROI is now you start getting up into a hundred, 150 employees, 10,000, 20,000 employees, there's a lot more data. If I go write a KQL query, I may pull a lot more data that I have to sort through or if I'm running a PowerShell script, there's just a lot more data as you get a bigger organization. So does having copilot for security where I can go in and ask natural language type of questions, ask questions about my audit logs if I have Intune ask questions about devices and about events and Intune does having a copilot to go over that much data, give me an ROI or to your point, do I start ramping up now because I have that many more employees, I'm having to buy six 10 security compute units and now my cost is getting up into the multiple hundreds of thousands of dollars like you said now it's a salary and I can pay somebody or multiple, somebody's a full-time salary to go in and write KQL queries and build out other processes to detect the data. It's an interesting ROI discussion. I don't know that I have the answer on how you would calculate that but I think it's something that a lot of companies are going in looking through and to your point, it's expensive. I get it. I would love to see this be a little bit more per use and maybe if there's Microsoft maybe would add some auto scaling in the future where you can scale up and down your security compute units because there may be some validity to you just have one security compute unit during the night and then ramp it up to six or nine or 10 during the day. 'cause one of those things we talked about, if you actually completely blow it away, you are gonna lose the history of your queries. There's that conversational history and while not retraining of the models, some benefits that come from keeping that co-pilot instance up and having those queries that historical que in your copilot. But I could see where there could be some rationalization to having, especially a large company where they're using copilot all the time, having 10 security compute units during your working hours when you're actually diving in or if there's some type of incident that is raised in your environment, you need to ramp up those security compute units So you can get responses to these questions a lot quicker but then when you're not using it ramp it down to one security compute units, you retain your history, you retain that environment but you're not paying for all those security compute units 24 7. I - Think it depends a lot on the functionality volume like so one of the, I don't know, maybe you can help me out here. So one of the confusing things to me with the copilot for security thing is it's kinda like back to that like suite of suite stuffs, right? Like that we've been talking about with Intune and and other things. So there's the concept of the embedded experiences within copilot for security. So that could be like co-pilot for security as it's embedded inside enterra. It could be co-pilot for security as it's embedded in Intune, it could be co-pilot for security as it's embedded within defender. And then each one of those defended embedded experiences has its own set of like nuance in the way things like logs are stored, how you query those logs, how you write effective prompts around those and and how all that stuff goes. So there's that piece of it and then there's the things that actually happen kind of like automagically in the background, right? If I think about like hey I have an active incident and I'm trying to query for risky users in entra and I don't know how to do that today. You know there there could be value in having that stuff spun up right there and you probably don't need a whole bunch of SCUs and a whole bunch of compute sitting behind it just to effectively prompt and get a KQL query that then you can go and run and and bring that data back. The thing that you might need it for is something like, say you're an organization who experiences a lot of live sites and you're doing a ton around incident management. So like one of the capabilities in the embedded experience for my co-pilot for security with Microsoft Defender is doing automated incident summaries. So if you can start to automate incident summaries and distill those down and potentially automate summarization of RCAs, can you eventually get to the point where this thing can write salient RCAs for you? Maybe is it tomorrow? Probably not. Is it a year from now? Is it two years from now? I don't know but that's like an inflection point that's likely to be on the horizon. And then if you think about that like being able to do like really good crisp incident summary response and RCAs RCA summaries and then eventually write RCAs or potentially even automate incident response, super valuable. Like if you think about being like an on-call, uh, A DRI or something like that, like the burnout could be real if you're the human that's on call 24 7 versus having the AI bot LLM whatever thing that can do it for you. Like I don't see how most folks wouldn't kind of pine for that and grasp onto it but you know, you kind of need to get it all to the point where it's like ooh those incident summaries are really good and ooh those RCA summaries are really good and can I take this to the next click stop and get it to where you know, it's turning more into automate all the things and if you can like hey great, there's likely to be more than enough value that's kind of inflicted by the tool there that makes it worth, you know, whatever the cost is that's associated with it. So I don't know, we'll see where a lot of this stuff goes. It does feel a lot like automate yourself out of a job when it comes to things like copilot for security and I think there's a ton of angst there in general, you know you mentioned like writing queries when you're doing threat hunting like at some point you're probably just building a whole body of individuals who are either getting really good at prompting or they can be really good at the actual hunting experience. I think for now you still want them to be good at the hunting experience. You don't want them to just be good at prompting and being able to draw on that as kind of their, their superpower. So yeah, we'll see where all this stuff goes. It's gonna be interesting like and I don't know what the timeline is like it's very hard to understand right now like 'cause this stuff is just moving so fast. Is that tomorrow? I don't know. Probably not. But is it like six months from now? Is it a year from now? Is it two years? Is it five years? That's very hard to discern. - The other thing you still run into with copilots, and I saw this some even with copilot for security and to be fair I haven't had a ton of chance to play with it. 'cause frankly I can't keep it running long enough to play with it for very long. I need to like set out dedicated points of time where it's like I'm gonna spend this day really diving into it and go spin it up and spend 50 bucks to have it running for a day. But there's still the thing of hallucinations too, right? Like if you're doing threat hunting and you are saying show me all the attacks on my exchange online environment or on my Azure front door instance coming from this set of IP addresses or you are creating these prompts to bring back your investigation, you want that to be 100% accurate. You don't want to miss something because co-pilot misinterpreted something or hallucinated on something or anything like that. So I think that's still very much an aspect of especially co-pilot for security into your point about you still are gonna want good people that are experienced in hunting and not just prompt engineering because I think while it can still pull a bunch of data quickly, there's still a validation that you'd wanna take place, especially initially to make sure that whatever is happening in the background when you're asking copilot for security these questions that it's returning these the right information. Because I did see where like when I did play with it and I was asking it different questions about Intune or about different data where it, I wouldn't get the same responses all the time and my data hadn't changed. I would expect the same responses every time around some of those and even some of the co-pilot for Microsoft 365. When I ask it about different tasks or tasks, the coming due dates or different tables to summarize different things, it's not always the same. And I think that's still one risk with all of these co-pilots is people, and we've talked about it, people always treating it as this is 100% accurate all the time when it's maybe not. And I, it's going to continue to improve as they continue to improve models, continue to look at data and figure out how do we make these in a way that they're more accurate. I am not worried about it running me out of a job right now. If anything, there are days when I'm pouring through rows and rows and rows of data, I'm like man, if copilot could help me dig through all of this quicker so I can move on to the next task that I have because I have more to do than I have time to do it, I'm actually looking forward to the day where co-pilot can help me optimize my time a little bit better because I don't, I, I don't see IT security co or co-pilot for security running me out of a job anytime soon. I - Mean I think that's the right way to think about it is as an accelerator, like so walk into it with kind of some intentionality like hey, this is all fairly new stuff. It's early days. Can I use it as an accelerator? Yes or no? Can our business use it as an accelerator? Yes or no? Are we thinking about it the right way? Like do we need to think about it as something that's on for a year or do we use it for three months to upskill ourselves and get to where we need to be? Right? Build that kind of prompt book and you know, wikis and all those kinds of things that you potentially want to have in place. Like use this to augment and improve your process is a good way to think about it. And you know, from that lens, I've worked places where like dropping 200 grand on a consultant to have them come in and write a 20 page document for you is something companies do a lot , right?Like, hey come help me improve this process kind of thing. Sure, whatever we, we've got just the consultant for you that that can help you do that. And there's value in those kinds of scenarios and lifting that stuff along. But you do have to be kind of walking into it with that level of intentionality. You can't be just sitting here and saying like, what am I gonna use this for and how's it gonna go? - Should we call it a day about twenty, fifteen, twenty minutes on backup and 20 minutes or so on co-pilot for security? And I'm guessing we both have meetings coming up. We - Can do it. I'm running low on coffee. - I am out of coffee. My coffee's gone. I just have a few, ended up with a few grounds in my coffee. I have a few grounds in the bottom of my coffee cup, but that's about it. - Fire your barista. Yeah, - Another story, another day. I don't know , but well that Scott, enjoy.What day is it? Tuesday. Enjoy the rest of your week. We normally do this on a Friday. We've had some sickness, some crazy busy schedules and recorded a bit at an off time and an off day. So enjoy the rest of your week. - We'll get back on track here. Hope - Everybody is back healthy and we should be back to our normal schedule here. Well maybe soon. We've got, now we have summers and vacations coming up, Scott. - Yeah, we do. My kids end school on Friday last day for them. So woo-hoo. - Are you excited for all the noise to return to the house? - ? My kids are teenagers.There's no such thing as noise. They're gonna be sitting in their bedrooms and playing video games, let's be - Honest. Sounds good. Well congrats on the end of school summer coming up and enjoy your week and we'll talk to you again soon. - All right, great. Thanks - Ben. Thanks Scott. If you enjoyed the podcast, go leave us a five star rating in iTunes. It helps to get the word out so more it pros can learn about Office 365 and Azure. If you have any questions you want us to address on the show or feedback about the show, feel free to reach out via our website, Twitter, or Facebook. Thanks again for listening and have a great day.

Donate

X (Twitter)

Facebook

LinkedIn

Apps

Welcome to Episode 377 of the Microsoft Cloud IT Pro Podcast. In this episode, Ben and Scott talk about a recent incident at Google Cloud where one of their customer accounts was completely wiped out without notice. Then they dive into Microsoft Copilot for Security. Ben has been getting hands on with it and it is expensive. They discuss pricing for Copilot for Security, how to think about approaching the multiple embedded experiences in it, and how to think about building a corpus of knowledge and truly leveraging it as an assistant and accelerator for upping your security game in your Microsoft cloud.

Like what you hear and want to support the show? Check out our membership options. (more…)

Episode 259 – Kerberos and Azure AD, sitting in a tree, a-u-t-h-e-n-t-i-c-a-t-i-n-g

by [Scott](/content/author/scottmsclouditpro/ "Posts by Scott"/index.html) | Dec 9, 2021 | Podcast

Blubrry Player

Donate

X (Twitter)

Facebook

LinkedIn

Apps

In Episode 259, Ben and Scott discuss some of the latest announcements involving Azure AD, including new security features in Microsoft Authenticator and a new capability that allows Azure AD to issue Kerberos tickets which allows for SMB file shares in Azure Files to be accessed without line of sight to a domain controller. (more…)

[ Episode 181 – Deep[er] Dive on Azure Sentinel ](/content/episode181/index.html)