[Episode 367 – Azure Files vs a Server with an SMB Share](/content/episode367/index.html)

Apps

In Episode 367, Ben and Scott kick off 2024 with a discussion of Azure Files. They start out reviewing a customer scenario Ben encountered and how they would approach it, breaking down the options available with Azure Files, hosting traditional SMB shares in Azure, and how a hybrid deployment can be the best of both worlds.

Like what you hear and want to support the show? Check out our membership options. (more…)

Episode 361 – TLS updates in Azure

by [Scott](/content/author/scottmsclouditpro/ "Posts by Scott"/index.html) | Nov 23, 2023 | Podcast

Blubrry Player

Auto Scroll

- Welcome to episode 361 of the Microsoft Cloud IT Pro podcast recorded live on November 6th, 2023. This is a show about Microsoft 365 and Azure from the perspective of it pros and end users where we discuss a topic or recent news and how it relates to you. We are back this week with some more KQL goodness as Ben discusses how he used Azure Data Explorer with some exchange email analysis. And Scott shares some additional tips and tricks for custo. They also discuss some upcoming changes to various Azure services when it comes to TLS 1.0, 1.1, 1.2 and 1.3. Okay, so we don't need to talk about lights on my face anymore. Do you wanna talk about your suggestion that I followed because you gave yourself a bit of a pat on the back for this one and I have to admit you had a really good idea. Yeah, I - Think this is a first, so folks need to know that well, that you actually listened to me that Ben and I chat back and forth throughout the week and generally break things and make fun of each other with the different things we're breaking. So you were having a little bit of a conniption, you were dealing with some Excel files and you were kind of talking about how they were getting into the hundreds of thousands, the millions of rows and you needed to analyze some data in there and I threw out why not just use Custo. Yes. So tell me about your CSV journey and custo and how it did or did not work out for you. - Custo itself worked out great. My log files hit or miss a little bit, but essentially what I was doing, I have a client that we are working on an exchange migration for from on-premises to Office 365. And we were going down this whole debate of do we do hybrid? Do we do a cutover, bunch of details around it we don't need to get into. But essentially it was they have a bunch of devices, scanners, printers, services and they are in an unfortunate situation where they don't have a complete grasp on everything that's connecting to their on-prem exchange server. So it was how can we figure out all of the different services, devices, et cetera, that are relaying through this on-prem exchange server that we need to account for when we go to Exchange online. So we used, there's a handy commandlet, it is only exchange on premises. It is get mess, get message tracking log that essentially gets like a log of every email that goes through the server. So I ran this against a server and this is the text message that I sent you was it came back with a 350 meg ish file, I believe 350 megs, 434 rows in my CSV file of emails, 331 Megs, 450,000 rows .I was like, what could possibly go wrong? Trying to open the CSV file on a computer, - easy peasy and then let's complicate thingsby you're on a Mac and it behaves a little weird with Excel anyway, so, - But I have 128 gigs of RAM and a whole bunch of cores because why not? But needless to say, it still does not perform super great. And then after the fact I realized that wait a minute, this was just one exchange server that they told me about. They have two other ones and this get message tracking log only pulls it from the server you're on unless you pass a parameter to get it from a server at a time. So I went through the other three servers and this was like the middle of the road CSV. There was another one I got from another server that was like 500 megs and another one that was uh, 250 megs or something all combined. It was 1.6 million rows of data of exchange tracking logs and you're, you were like, to your point, throw it in KQL and huh you know what, Scott's actually a smart guy. Maybe I should pay attention to him and listen to him every once in a while. So that's what I did. I actually used Azure Data Explorer and the handy little data ingestion, which makes it super quick. I mean I had the advantage, all three files are from Exchange exact same format. So I just went and did a little Azure Data Explorer configuration, said go upload these three CSV files into this one table and I can add all three of 'em at once and hit go. And it took a little bit because I had to upload like 1.2 gigs of CSVs to Azure Data Explorer. But let me tell you, going through and starting to write queries to find distinct IP addresses and to find IP addresses that messages were coming from that only had a single sender which indicated like some other service or something that wasn't just here's the bulk email from everybody. It actually worked out really well Scott, to sort through it. There was not some data, there was some data I would've liked to see in there to better differentiate from something relaying through exchange versus Outlook, like to be able to see the actual client connection versus IP address and sender. And I'm still digging through some data but that had nothing to do with KQL and CSV. It's purely what get message tracking log contains and just not quite the level of data detail I wanted in it. But in terms of being able to analyze it and sort through it and all of that, Azure Data Explorer KQL was kind of fun. It actually, it works really well for doing this type of thing. It's - Super easy to do, especially if you only have a couple million, even a couple billion records. Like it's not huge or massive. You can stand up a a party cluster basically really quick and get going with it. So we did this on the other YouTube channel or or to the cloud channel. We kind of did a one click ingestion and looked at how easy some of this stuff is. But I love that you were able to take that and turn it around into something practical. Kusto is near and dear to my heart unfortunately. Well I, I dunno, fortunately, unfortunately I spend all my time in Custo Fridaylast week I was trying, we collect some telemetry, very similar kind of thing about clients like what's your user agent that's coming through? What are oss, are you on? Things like that. And being in Azure storage, we get billions upon billions of requests. And so I was trying to wrangle this query for one day of data where it was returning something like 5 billion records in it. It's kind of a fun thing 'cause it's actually able to kind of wrangle it and put it together. Especially if you can think about doing your queries the right way. I'm gonna give you a quick kind of pro tip. - Okay. - For custo specifically, so this is one that I just learned on Friday and it's super helpful when you're dealing with text in custo like a very long uh, a very long string record. So think like maybe something spits out a string where it's actually comma separated like in the string. So maybe it says like resource equals X comma client IP equals blah blah blah. And it's not broken out into different columns. So there's this operator in custo which is called parse ware. So you can do all sorts of parsing in custo. You can parse JSON and you can parse version numbers and parse text and but use this parse ware operator and you can actually point it at that column. That's the big string and you can throw RegX at it and it extracts the data out of that string as you need by RegX and it turns it into calculated columns on the fly and it's super performance. So the way you would use this is, like I said, you have this big long string that you need to decompose. Like maybe you're gonna, we're gonna extend it into multiple columns and do an extend extract, extend extract, extend extract. Yep. Which is typically how I go about it, which is super resource intensive, but this parse wear thing, you just throw a RegX at it or multiple RegX is even 'cause you can do like RegX per extracted column kind of thing. Super duper cool. I was having a lot of fun with that. Wonder - If this would work. I was just looking at my data because I have one column in here. Oh see it didn't pull that out. I don't see it in there but I have seen that even in my CSV where it's, and I've seen it in other CSVs that you get from Microsoft where it's absolutely that it's like ACSV. But then the last column is like the context or the data and it's, you have your curly brackets that are wrapping like a monster Jason string or an embedded, it's like ACSV embedded in the CSV. So - Another way I, I'll give you a practical use case. So I have a table that tracks storage account names. Yep. And as part of, so there's a column called account name and in that column the account name is not just the account name like the string like my account name, it's my account name semicolon. And then a Unix date time which is the date time that the account was created. Uh, so somebody made the conscientious decision when they stood up this table that they were going to include the create time of the account as part of the account name just so we could have cardinality there and maintain uniqueness. 'cause you know I could create an account today and then 30 days later and then I delete it the same day and then 30 days later you go account, create an account with the same name over in your subscription. Technically two different accounts. But I'm always going through this table and I'm doing this thing where I say, okay, take account name, split it on semicolon and then grab like the first part of the array and then grab the second part of the array 'cause I want account name and create time as two separate columns. And now with this parse wear thing, I just go in and I say pars wear account name and then I can just throw a regex in and automatically split on the semicolon all in one step without having to extend out into multiple additional calculated columns on the fly. Which it turns out is like a nicety that you just kind of have to type less but it's also more performant because you're basically relying on a wear C clause rather than calculating after the wear. And your wearers tend to kind of be compute heavy anyway. So why not do both at once and have your cake and eat it too - Nifty. I just looked through my data, my source context one is a mess in this exchange. So this would not really work for source context because I don't, you know what the data you get out is nice but let me tell you, some of the data is so inconsistent in some of these cases, like this source context, - I shall pretend I'm surprised - One of it's one long string and then another email has like semicolon delimited stuff in it. And then another one is just completely blank. And I mean I get it, your source context changes from email to email but it makes it a little bit challenging sometimes to parse through it when there's no uniformity to the data format in that column or none that I've can decipher super easily. - It's a little rough if you gotta wrangle it that way. So maybe you wanna know, like my other pro tip that I do in cases like that is custo has a case statement selector, I don't know if you've ever used this. So sometimes I'll extend columns and then based on case, because when you do a case statement, you're basically doing if this then that, if this, then that and all the way down to having a default. But the cool thing is your predicate, like your if statement can be different for each and every case within that case statement. So you could have in that case where the, like the data's a little bit different between the two. You could say okay, case A when it meets this predicate then do this. But then when it meets this other predicate then format it like this. But if it does this other predicate format it like this and then maybe do yeah and then maybe do unknown or dump something out the other side doesn't work with billions and billions of records. I've found like it is a little more than a little compute heavy but couple billion, couple hundred million like not a problem, doesn't even break a sweat. - So if I can figure out a way to define my cases, I could use that. - Yes. If you have a predicate and you can figure out like basically what that statement is, like that wearer statement and split it out, you could technically put it all in the same column. - Nifty more stuff to play with, more data to go dig through. Do you feel overwhelmed by trying to manage your Office 365 environment? Are you facing unexpected issues that disrupt your company's productivity? Intelligent is here to help much like you take your car to the mechanic that has specialized knowledge on how to best keep your car running Intelligent helps you with your Microsoft cloud environment because that's their expertise. Intelligent keeps up with the latest updates on the Microsoft cloud to help keep your business running smoothly and ahead of the curve. Whether you are a small organization with just a few users up to an organization of several thousand employees they want to partner with you to implement and administer your Microsoft Cloud technology, visit them at intelligent.com/podcast. That's I-N-T-E-L-L-I-G-I-N k.com/podcast. For more information or to schedule a 30 minute call to get started with them today, remember intelligent focuses on the Microsoft cloud so you can focus on your business. Oh, so where do we go from here Scott? We could go one or two directions, we could go talk about some changes to TLS or we could continue down client stories from Ben - .It's your show. Take your - Pick. No you, I picked the first topic you get to pick if you want to hear more client cases from Ben or if you wanna talk about TLS. - Why don't we talk about TLS? Okay. I've had some fun at work with TLS the last couple weeks. So there's a bunch of services in Azure. There - Are a bunch of services in Azure - .Yes, there are a bunch of services in Azure. I was gonna say, let's take a step back. So we have this kind of operating principle of being secure by default. So maybe at some point we should come back and talk about the memo that just came out that was published about Brad Smith and our security org that kind of talks about our principles for how we approach secure by default security first for our customers. So you're gonna start to see more and more of this I think across Azure, certainly Azure, like I'm close to it, you'll see it across like M 365, 0 365, the the entire stack. Things are gonna start to get more, not locked down but they're gonna get uh, put into a better default position, which could mean more work for you as a customer. So, so I think customers are gonna need to pay attention to this. So one of the first ones to kind of hit is that TLS is changing in a whole bunch of places with regard to minimum supported TLS versions first of all. So there's services today that support like T LSS one T LS 1.0 1.1, 1.2 and then you're also gonna be see kind of forward looking changes in what's coming down the pipe as far as having access to newer TLS versions. Well newer in that they've been out a while and we're kind of catching up on them right now. So particularly with TLS 1.3. So the first one that's near and dear to my heart is in storage land we have announced that TLS 1.2 is going to become the minimum TLS version for all Azure storage accounts moving forward after November 1st, 2024. And it's not just moving forward, it's also we're deprecating support for 1.0 and 1.1 for existing accounts as well. So that's something that folks need to potentially take a lookout for. So if you have pinned ATLS version like you've pinned to TLS 1.0 or 1.1 and you're doing that today, you have a year, you have until November 1st, 2024 to go ahead and flip the switch and get over to at least TLS 1.2 as a minimum supported version. - Yeah and I sent you a text on this one because I'm curious if somebody's listening and has this information. I have not been able to find it but there were some Microsoft Cloud workshops around high availability that I worked on and that are still out there and to the best of my knowledge, okay, backwards a step now I'm going a step backwards SQL server,- It's our thing today. Yeah - It is SQL Server. If you're doing a high availability cluster and you want a cloud witness for your SQL high availability, you can set it up with a storage count. It's all built right into SQL Server. However, if you do not do TLS 1.0, if you actually do 1.1 or 1.2 a SQL server does not use T LSS 1.2 or will not work with T LSS 1.1 or 1.0 No it won't work with 1.2 or 1.1 when using an Azure storage account for your cloud witness. I have not seen anywhere across the intro webs where the SQL team has updated SQL server so that it will work with T LSS 1.2 in a storage account for that cloud witness. Yet what that means is if there's somebody from the SQL Server team that's listening to this, I would recommend making some changes to SQL Server to support T LSS 1.2. Are you gonna have a whole bunch of high availability SQL instances in Azure using an Azure storage account for your cloud witness that are going to break come October 31st, 2024? And then along with that, I don't know which versions of SQL they would go back and add this into SQL's expensive depending on how you license it and how you set it up in Azure. I know a lot of people still running old versions of SQL Server, people tend to put SQL server up and as long as it's working they don't tend to touch it until it's out of support. I'm curious to see how one, I don't know how many people use a cloud witness for high availability SQL servers and Azure, but I'm curious to see what effect this has in that particular scenario. Just 'cause it's one in particular that I was aware of and have come across regularly where TLS 1.1 and 1.2 aren't supported yet, particularly around cloud storage. - I would've assumed this one would've been fixed a couple years ago. So there was a SQL server has lagged in some areas in the past, so, so I, it's been a while since I've done a cloud witness so I know for backups, so doing like backpack files over to SQL Server. Yep. That used to be problematic as well because that from a SQL to a storage account connection did not support TLS 1.2 but they've basically patched and cumulative updated and hot fixed their way out of that one. I would have to go play with a cloud witness and see, - Maybe we should do this, maybe we should go do a, - It's been a hot minute since I've tried one but maybe - We should go do a YouTube video. It's - A good call out. And fortuitous timing, I think we're going to talk to the sequel team in a week or two here to do our big powwow. So , I should bring it up with 'em,- Ask them about this one because again, I couldn't find anything and I have the same assumption as you. I'm like, I would think with sql, whatever we're on now 20, are we on 2022? 2019 SQL subscription edition because that's what we're doing with everything now ,I would've doubt they would've fixed this given to your point how much Microsoft has been trying to get rid of 1.0 1.1 and really use 1.2 for everything. But I am not a hundred percent sure 'cause I can't find anything documented nor have I necessarily gone and tried this with the most recent up-to-date patched version of sql. And I would be also be curious like how far back you can go before it would break. So let me know, go talk to the SQL team, ask them the hard questions. - I would tend to think most of this is patched most of the way out. So I'll throw a link over in the chat just so you have it and you can maybe take a look through and let me know if I'm off there. But it looks like all the current supported major versions have been patched up to TLS 1.2. Now that being said, no place in that article of known issues that it fixes. Does it call out cloud witnesses? So mileage might vary with that. May vary with that one. - Alright, we're gonna go do some testing sometime, but yeah I've, if you - Have the ability to spin up a cloud witness real quick, I wanna try it out. - I'm not gonna do it right now. We'll have to do it on a YouTube channel. We can go through the one of those MCWs or something or part of the way through it 'cause that one's a long one. All right. Okay, other TLS stuff. So - We've got TLS 1.2 for storage is going to become the minimum supported, per supported version. That is November 1st, 2024. The other TLS thing that is out there is you're gonna start to see services also flip over to TLS 1.3 and the first one that I saw announced so far is Azure App Service. So app service for web apps, which includes your apps service plans, which also run for things like functions and logic apps as well. So this is kind of rolling out slowly TLS 1.3 for app service, like your underlying kind of compute and hosting engine is starting to roll out now. It's going to be rolling out over the course of the remainder of what we have here in 2023 and 2024 and rollouts worldwide. I'll put a link in in the show notes for everybody to go and take a look and see what's going on there. But basically kind of it goes through canary staging and the end user acceptance, all that stuff. And it'll hit us clients first, which is an interesting one given the presence of some really big retailers and things and coming into the holiday season, who knows who runs preview and who runs on supported GA stuff. So we shall see. But I have seen customers here get into trouble like when they do pinning for things like TLS versions, when they do cert pinning, all that kind of stuff. Like I understand why it needs to be done, but it's a call out that your runway is potentially limited with some of these things. - Yeah, well it looks like they already started rolling it out the end of October. October 23rd. It began rolling out and like you said, kind of through the end of the year and that's the preview, we will continue rolling out TLS sometime early 2024. Yeah, but that's all under the preview. Will it just be in preview then yet through January, 2024 is the way I would read this. - That's the way I, I read it for now. So I think it's a smart move to do it that way. So one of the things that happens is now you're going to have access not just to TLS 1.3 but to new cipher suites within T LSS 1.3. So you might also have a dependency within your stack on a certain cipher suite that you've been waiting for. So now it gives you the chance to kind of potentially play around with those and see what's what and what's gonna work and what's gonna break. - So yes and I would expect, I mean app service for web apps and then functions and logic apps going at the same time kind of makes sense since they all run on the same service. - Funny enough, it's just web apps though. It it, so it's the kind of public compute side of web apps. It is not A-S-E-A-S-E-V one is still kicking around there. V two is out and about as well. And surprisingly TLS 1.3 is not there yet. So again, your mileage may vary huh? In the ever landing cha ev ever changing land of Azure previews. - Yes. And I would imagine we'll start seeing more TLS 1.3 stuff coming. We have ignite. Well by the time people hear this episode, unless we switch orders, Scott Ignite may have already happened. Well by the time they hear this one Ignite will have happened. Whether we have an Ignite episode before this one or not, we'll have to play with schedules because Ignite is happening a week from when we're recording this essentially. Yes, - Rapidly approaching. - We shall see. Maybe there will be some TLS 1.3 stuff at Ignite. I think there will be some interesting announcements coming out there. Any other TLS stuff you wanna talk about? Those - Are the big ones that I'm aware of so far. We'll see where the rest come in. So - We're gonna talk about one more deprecation that has kind of Ben's client woes. We don't need to get into all my client woes around this particular announcement. This is almost sort of an I told you so not to this particular client but to other people because I have been saying this for years, but there was an article today, Microsoft Deprecates three features in Windows 11 version 23 H two. And guess which one of those features is? Scott, you wouldn't guess if I wouldn't have told you this. - I've been waiting for this one to drop for a long. Like it was inevitable that this one comes like - It's totally, and I told you so it's going to happen. Yes, - The web client web dev is finally going away. Woe be to those who continue to use SharePoint and file explore views. - Yes. And I have been, there is one client I've been having this discussion with and they're like, but it works to open and file explore on one computer and not on another computer. And I'm like, why are we even trying this? Microsoft should have taken, I mean web dev is going away. To be fair, Microsoft should have taken this out of the SharePoint UI like the day after they came out with it because it's been giving me woes and heartburnand all of the bad things since the day it came out because of all the weirdness, particularly with SharePoint. There's always been weirdness around file modified dates on functionality because it's not really doing a sync then it's opening it like a file server and there are still people that try to use it and people that try to use it. I say stop. It's going to go away. Microsoft has at least labeled it Legacy up until today, November 6th. But you should not be using web dev to open files and SharePoint. And now I finally have something to point to and say you really should not be using web dev to open files and SharePoint because it has now been deprecated in Windows. There have been issues with it now for a while, ever since Edge came out because it is a feature, at least it's in SharePoint. I don't know. I would assume it would affect other places too because it's part of the HTTP, the hypertext transfer protocol for those of you that have always heard it and never actually known what HTP stood for, where you had to use Internet Explorer to even use web dev. So as Edge has come out on chromium and they've started pulling out some of the IE. Supportability stuff and IE has started getting deprecated and support for IE has gone away. Web dev in Evolve itself has already started getting a bit unpredictable because of losing browser support. - Just a little unpredictable. - Yeah. Yeah. A little more unpredictable than it's been since 2003. But yes, now it is gone. We can wave goodbye to it. And if you're still using it, please stop once and for all , even if it still works on Windows 10and you have no plans to upgrade to Windows 11, just - Stop. That's the plea, huh? Please, - That's - The plea. Just stop. - Please just stop before I start crying on the podcast. - But won't you think of the files come on. - I do not care for the files or the pain that goes with dealing with files that are opened over web dev or mapped network drives that are SharePoint because we've had, and we can go find other episodes. We have talked at length about this so we don't need to get into that of why SharePoint is not a network file share, but it is not a network file share, therefore WebDAV should not be a thing nor should interacting with SharePoint files in that way be a thing. End of soap box off the soap box. - End of soap box. Well so the interesting thing that happens here, ,I always kind of cringe a little bit. So this is a change that's coming in Windows 11. It's not something that's like being back ported. 'cause we're so far gone from Windows 10 and unfortunately there's still lots of people that run Windows 10 as well. So they're in the clear I guess - Is Windows seven still kicking around out there? - I imagine it is. I probably have one or two former employers who are still running fleets of desktops on Windows seven because they can, I mean Windows CE is still kicking around, right? Like I don't know how it is in other countries, but over here in the US it powers your favorite uh ATM. - Tim, what about Windows Me? We can get on the Windows. - That was the one with the good startup sound. - Yeah, Sean said in Discord. Oh it is unfortunately about Windows xp. So yes, windows XP is still alive and kicking as well. , that's a whole other topic of conversation.That one's been outta support for how many years? - For one or two hot minutes now. - Yeah, so that is I think our news, oh, one last bullet points Scott. We don't need to talk about this either. We mentioned in a previous episode, and I'm not gonna say last in case these get out of order with Ignite the automatic conditional access policies and Entra being created by Microsoft. And I didn't have a good reference article for it today on November 6th. Microsoft did come out with a blog post about it. So we'll throw a link to the, that blog post in the show notes if anybody wanted to see the Microsoft official announcement today rather than the stuff that I found on the socials when we recorded that other episode. Perfect. - I've been looking out for that one. Good catch. - You got it. And finally, girls Who Code Scott or Fundraiser Girls - Who code tis the season of giving help a young woman in your life or contribute to some other young woman's career to enter the wonderful field of information technology and give to Girls Who Code. So if you go to give dot girls who code.com/ms cloud IT Pro, you can contribute to our campaign or go ahead and contribute to another campaign for Girls Who Code if you so desire, if you get like a match at work or things like that. But yeah, go ahead and make it happen. I would love to see us raise some money for Girls Who code through the podcast once again this year. - Yes. And try to beat last year's goal. So by December 31 we're gonna see if we can better what we did last year with those donations to Girls Who Code. So see what we can do. And with that Scott, we can sign off for the day. - Excellent. . Well Idon't get to sign off for the day. I'm going to Redmond for a work trip. So I've worked for Microsoft for three coming on three years now and I've never met anybody I work with and I'm going on my first work trip. So that kind of came in hot and I had to book tickets the last minute. But uh, I am going to Redmond tomorrow and I'll be there for a week. So that'll be fun. It - Just clicked. Scott, you're leaving right before Ignite then. - Totally. So I think Ignite like the sessions start on the 14th through the 15th. I'll be flying back in a red eye on the 14th. . Yeah,- You should have just stayed two extra days. - Holidays are coming up. I I got another vacation. I gotta get to you gotta All right. My wife would be mad at me. Sounds good. My dogs will be mad at me. My kids will be mad at me. Hey - Scott, I won't be mad at you. - Okay, that's good. I appreciate it. - I'm here for you. Alright, well enjoy your trip. Safe travels. Say hello to all the folks in Redmond for us Bug the sequel team about cloud witnesses and travel safe and we will see you next week for Ignite when we talk about all the fun stuff from that. All - Right, sounds good. Thanks Ben. - If you enjoyed the podcast, go leave us a five star rating in iTunes. It helps to get the word out so more IT pros can learn about Office 365 and Azure. If you have any questions you want us to address on the show or feedback about the show, feel free to reach out via our website, Twitter, or Facebook. Thanks again for listening and have a great day.

Donate

Apps

Episode 350 – Take a look, it’s in a book, or in a Microsoft Learn exam

by [Scott](/content/author/scottmsclouditpro/ "Posts by Scott"/index.html) | Sep 7, 2023 | Podcast

Blubrry Player

Auto Scroll

Welcome to episode 350 of the Microsoft Cloud IT Pro Podcast recorded live on August 25th, 2023. This is a show about Microsoft 365 and Azure from the perspective of it pros and end users where we discuss a topic where recent news and how it relates to you teams, channels. It's one of Ben's recent frustrations with some of the feature ambiguity between the various types of channels available. Ben and Scott also talk about the new open book policy or Microsoft learn policy for taking Microsoft exams. Finally, they wrap up with a new feature release near and dear to Scott's heart, giving you a simplified experience for creating and managing C D N endpoints for Azure storage using Azure front door. We need to have a segment, Scott. So on our podcast, we should start a Ben's weekly rant. Stop. It's your show. Like you can do whatever you want. Like I joined to provide color commentary so. Or to get me all riled. Up, you wanna segment like I'll put some sound effects in there or something and we'll be good of. Ben's weekly rant. Do you know what Ben's weekly rant is today? I don't know why I didn't realize this before. Ben's weekly rant today. I feel like I should have known about this or I would've encountered this before. My weekly rant today has to do with an overview of teams and channels and Microsoft teams. And to be fair, I actually have lots of rants about teams and channels and Microsoft teams and the way this has been implemented. But I had an interesting one that I hit today and I honestly don't know why I haven't hit this one before or why I didn't realize this before. Microsoft is all about shared channels for like private channels. 'cause you can do your own security on it. It creates its own SharePoint site, blah blah blah, blah blah, blah. And did you like all the blah blah blah blah blahs And I loved it. I work with some contractors like I have a few contractors, different companies. I partner with all of that and we have started using shared channels because in that case a shared channel is nice. We set up all the A a D stuff and I share my channel with them so that they don't have to do all the tenant jumping stuff in order to get to content for a client that we might be working on together. Well, today I had such a right. Makes sense. Very reasonable. That's what I thought. So today I had such a channel and all of a sudden I had a file and I'm like, sweet, we're done with this. Now I can share it to a client. It doesn't work because if you go look at the channel feature comparison, shared channels work great for external participant B two B direct connect, all of that. But guess what cannot be added to a shared channel. A guest user. . You like wanna have your cake and eat it too. So.Right, but if my client is not set up for B two B direct connect and I don't wanna add with him and I wanna share a file with him that we've been working on in teams, I can't do it anymore. Now I have to go put it somewhere else or go back to, let me email you this file or something else. And I just get irritated by that today because I wanted to have shared channels for when I work with contractors where it's all hunky dory. But I also wanted to be able to add a guest to it so that I could share a file with a guest, which seemed very logical to me. And they have this overview of this channel feature comparison table. And I think it just even reiterated to me how I would say discombobulated, the whole channel infrastructure and setup really is when it comes to teams. And I feel like you should not have to think this much or put this much planning into what type of channel do I want? Like I should have, apps is another one. This is my other pet peeve. Like I can't add apps, I can't add planner if I wanna do task management in a private channel or a shared channel and I wanna use planner, I don't use planner for this reason, but if I did I should be able to add it. Like the differentiation between standard private and shared should absolutely be a security construct, not a, we have a list of 15 things and depending on what you wanna do before, you won't go create a channel. You should go through this list of 15 things and figure out exactly what you wanna do and if for some reason what you wanna do doesn't work out. Tough luck. .Well I mean arguably sharing with an external guest is a security thing. Yes. Okay. So it, yes, it's a security thing but it's also not like shared channels should implement B two B stuff but I don't feel like it should take away other stuff. So I guess. It's hard. Yeah, it's it's, it's a balance thing like, and there's weird stuff too. The the one I've run into with shared channels and I guess I always just thought it was like a gap and I didn't know why I didn't know it was documented was the analytics thing. Like if you ever play around with like team analytics, like I have a thing where we have a internal team where you, you know, we let like our field and technical communities come in and one of the KPIs we have is like, Hey, how much growth over time can we drive in this team? Like how many posts can we drive? What are the types of engagement we get? How many members do we add month over month and interactions and things like that. And like it's a nightmare to, to get the data and put it together and I have to do a bunch of it manually and it's painful and it and it hurts. TE teams are just weird in, in general. I ran into another interesting one, in SharePoint.So I, I was working on a project with someone and they had created some files like think like word documents, PowerPoints, things like that over in their OneDrive. Totally valid thing, right? Like you create it in your OneDrive, like that's quite often like where documents start for me and then over time I take them and I either physically move them or I do the save as a copy thing and I save it into another site and then I just delete the one in my OneDrive so I don't have two copies of it and multiple things running around. So I'm working on this project with this person, I say Hey can you come and you know, I see you have these files in your OneDrive, like they're pretty much canonical at this point. Like we've iterated on them, they're done, can you move them over to the team site so they're ready to go. Like I have a team site with a channel and that channel has you know, a folder inside the SharePoint site, things like that. So it's just go to this place in SharePoint and just do like bring the thing over there. And the way that they thought to do that was to you know, do the thing where you could just add a new link in the SharePoint site and it creates URL file and they just made that point to the stuff in their OneDrive and I was like, I can see how functionally you thought that was the same thing but they're very different things. .I had never seen somebody use URLs in a SharePoint site that way. And it was just, it was one of those moments where it's like oh users are gonna do weird things that you just can't account for. . Yeah they really do.I know and I think with all of this it's hard. Like I also feel for Microsoft in that respect, right? They created teams that created channels. Arguably they probably had no idea they were gonna do shared channels and private channels when they first stood up teams and then they had to try to shoehorn these in. But I think that's where a lot of the frustration comes up is that as you add on these features, and we've talked about it before where that initial dependency on groups for security for planner, even when it used to be there for power BI workspaces kind of backed Microsoft into a corner with some of this stuff. And it really is how do you get around some of the, it's technical debt, how do you get around some of that technical debt from when you first rolled it out to be able to implement these in a way that to be able to implement these in a way that they work and work for everybody. And I've seen like the planner one I is one I've seen a lot and Microsoft said like a year, year and a half ago. Yeah we're aware of it, we're working on it, we wanna get planner everything. But a year, year and a half is a long time. Personally I've gone out and found workarounds. Like I said, I don't use planner, I use Jira because Jira gives me a lot more flexibility to be able to actually pin tasks to any type of channel no matter which one it is because I got tired of waiting for Microsoft to do, it continues. To be a tough space to work in. You're moving at the speed of the cloud. Yeah, so there's my rant but I would also say for those of you that are looking at all these channels, like go look at this channel feature comparison before you stand them up. And unfortunately again to the dismay of end users don't even know to look at this, right? Like what happens if you have an end user go spin up a certain type of channel and put a bunch of content in it and build out a SharePoint site and six months down the road they need to do something that you can't do in that specific type of channel. You can't convert it like at that point in time you're doing a channel to channel migration. I don't even have any good advice. Like do you go tell people to look at this overview before they spin up channels? I mean there's some stuff as you as an admin that you can do with sensitivity labels or setting some standardization, turning certain channels off. But it's kind of a tough spot for end users to be in to have to know some of these requirements to pick the right channel if you let users do that in your environment. Yeah, end. Users are never gonna come over to this documentation. Yeah. Ever. It's like I'm gonna make the argument it doesn't happen if it's not on support, do microsoft.com. They never find it and you know the s e O just isn't there for things like this versus like support Microsoft and even then, like you're still asking the user to take that extra step. I know I'm not gonna do it .No. Well and again it's not something I spent a ton of time looking at and thinking about until I hit this one so I don't know. But. Hey now you know for next time I. Do. And speaking of documentation, Scott, do you know who can start going to this documentation now in learn.microsoft.com? There is a new subgroup, a new group of people that can start visiting said documentation. . I see what you did there. Do you like that?That was actually one of your Yeah, yeah. It would've been a good transition if I hadn't called out that I saw what you did there so I, I ruined it for you but it was one of your better ones. That's okay. Yeah. So we've talked a bunch about certifications on the show in the past and I dunno about you like they're near and dear to my heart, right? Like I used to write books for these things. I certainly trained a bunch of people on 'em like it was my bread and butter for a little bit while a little while and I paid the bills. But the role-based certifications for Microsoft exams, you know those things where you go to a Pearson View site or you know you do 'em at home and you sit there with your camera on and you got the moderator and they're like, show me what your desk looks like and you don't have any like paperclips or post-it notes on your desk do you? And then you know, your kid walks in behind you and you're disqualified from your test. Those exams are now moving to a model where you will have access to learn@microsoft.com. So effectively the docs for the services that you're dealing with, like back to the example that we just had here where you were looking at a comparison of features in teams like that comparison was sitting over and learn@microsoft.com. So that stuff is now going to be available for test takers directly through the testing interface. So you know, if you've ever sat down for a Microsoft exam, you know like your moderator comes over, they spin you up into this environment and it's, it's locked down and like all you're in is the exam environment now they'll have links in there where you can optionally open kind of a split view with a browser tab and go over to learn and search through learn and potentially get to the information that you need to to which I think is good uh, for a whole bunch of reasons. One is Microsoft exams are notorious for testing you on the esoteric things that frankly you don't do day to day. It's like oh I'm going to show you a multiple choice answer and each one of those answers has a different set of parameters that all look very much the same on a PowerShell commandlet. You're like I don't know, I haven't touched that PowerShell commandlet in two months and even if I had touched it I was gonna do GI help or something else on it to work my way through it and get it to where it needs to be. So now you can potentially leverage the docks uh, to go ahead and look that up. So I think that's all good. I think there's still limitations here. So one thing that I think I would struggle with personally is I don't like Microsoft search. Like I don't think binging search is a great product and it doesn't return results with the fidelity that I want that I'm used to in other search engines like Google. Like I'm a, I'm a Google user and I'm used to queries syntax and Google and the way it returns things for me. So you're not allowed to go to Google and search and use a random blog, you're stuck to learn.microsoft.com and kind of the the builtin binging search that comes along with that. The other thing that I think is going to potentially be painful and trip people up is the certifications don't get updated as often as the documentation is .So there is a very real possibility that, again, I'll take that example of you're sitting on a multiple choice question and you've got four answers and they all have different PowerShell parameters that actually two of those answers could be right, but it turns out there's only one answer that's right for the test because maybe the parameters on that commandlet changed over time and the ones that are surfaced in the dock are what's right today. But the real answer is what was right six months ago when this test was written thing. I think that's gonna be a struggle for folks as well. I'm a big fan of this 'cause I've had the same thing to your point I like and I, I think this is gonna hit a lot more, at least for me when it comes to like PowerShell scripts or CLIs or like even some of the intricacies of maybe how things are named because that is what I do tend to rely a lot on the docs for I am also a very much a tab complete type person , I remember .100%. Just enough so that tab complete works but I don't know the whole commandlet name and to come into a test and remember it. So I see a lot of advantages to it here. But I also agree or I also think that you're not going to be able to rely on this to just go in and pass the exam without knowing your stuff for a couple reasons. One, there is still absolutely the exact same amount of time. So if you were running into time limits before when you took these and now you're going to go try to look up all this stuff and learn to validate your answers, you're going to run out of time. And I had a professor that did this to us even in college, he would say okay this test open book everything. I don't care what you take in, you can take in all the textbooks, all your notes, you can take in anything you want to but if you don't know it you're not gonna have enough time to pass it. And I wonder if even as some of these tests get updated, if Microsoft will start taking some of those approaches with these two where sometimes open book is nice or open learn in this case because you can go look it up but are they gonna start writing questions where it's not quite so black and white? And again I still think you're gonna have to know your stuff. There's also gonna be a temptation to go in I think and validate that you're answering it properly which could cause you to run out of time. If you are in there and you're like, oh I don't really know, let me go look this up and then like 20 minutes later you find the answer that kills a lot of time in a test. So I still say you're gonna have to know your stuff but I like that it's available because that's how a lot of people work today. Most people don't have power show command. Let's memorized this chart of team functionality or some of the skews on what's included in what licensing. I would venture to guess 95% of the people that do that on a day-to-day basis don't memorize it, they just know where it is. But you're also gonna have to get used to being able to just search using learn and not search using your favorite search engine of choice or relying on like the community stuff. Like it's only gonna be the docs, it's not gonna be some of the forums and tech community stuff that's under learn either. 'cause there are some like blog posts and stuff that also all fall under that U R L. So it's not YouTube videos, it's also not GitHub repos, which is interesting depending on the examination that you're taking. So one place I would say like having access to GitHub could be helpful would be some of the developer exams because lots of the samples for development are actually over on GitHub. Like the high fidelity samples that are gonna maybe get you to that answer sometimes. So T L D R is, you do have access to this but just because you have access to this, I don't having not sat and taken one in the new format yet 'cause like this was just released, I don't think this makes things any easier. I think it potentially takes away some of the jitters and anxiety that might come with test taking but it doesn't take away from the need to still understand the material, the core concepts. Like you're still gonna have to study and do all of those things. 'cause even having access to learn means you still need to go know where and learn to look like am I looking up a conceptual thing? Am I looking up a how to thing? Is it more a referential thing? And you're gonna need to know that. Yeah am I looking at the service docs? Do I need to go look at the calf or the WAF or some other weird place? Like all all that stuff kind of comes into play. You're also not gonna have access to other learn properties like Microsoft q and a and all that stuff as well. So uh, you know, just keep it in the back of your head. It'll be interesting. I think it is something that potentially eases things out. I do wonder how the proctors are gonna take it. I don't know. Have you ever done one of the online Pearson exams like back when they shut 'em all down for covid and everything. Like sit at home and do it? Yes. For like not a renewal but a brand new. Right, but. Actually doing, no I have never done it. Partly because I have way too much noise in my house and I've heard horror stories about proctors just like shutting it down as soon as they hear noise with all the family and kids I have home all day and the amount of tech in my office, like I literally have to go like sit on our bed in our bedroom or something to do it and even then kids can charging it. So I have never tried to do an at-home one. So that's a consideration is like background noise. You have to be in a quiet place. Your desk has to be cleaned off. Like they ask to, hey can you like move your webcam down and show a picture of your desk. Like for me I have like a mirrorless camera up here. I have to unmount it and like wave it around uh,or I just go grab like a U S B webcam from the kids so you know I can pick it up and make it easier. Yeah. But the other thing that they watch for is like they're watching your body movement and your eyes and things like that. And if you're looking to the side too much, if your eyes are flittering around, like they think you're getting into the mode where you're cheating, right? You could have somebody who wasn't there before and now they're in the room and they walked up behind you and they're over there. So I wonder how they're gonna do things like that. Like many of us are on, you know like multi-monitor setups, like I've got a 32 inch monitor in front of me. So for me to look in like one corner to the other corner, like my head is going all the way back and forth, right? Right. And I could be looking at something else even though there's only a wall behind me and there's really nobody or nothing there. So I wonder how they're treat that piece of it. 'cause it sounds like they're still gonna have the online moderators and, and that makes sense to me for for the at-home exams at least. So we'll we'll see how some of this stuff goes. But all in all I think it's good stuff. Yeah, it does only apply to the role-based exams and not the fundamentals. Yes, the role-based and the specialty exams. So things like Azure administrator, Azure developer, M 365 administrator, that stuff, those are all role-based exams and then you've got like networking, a V s exams, A V D, things like that are all specialties. Yeah, to your point too, I think even if you're reading on learn right your eyes tend to flitter, you tend to just get different facial expressions. If you're reading learn much like you would be cheating. So that will be interesting. I think I was thinking through this as you were talking to, I think my strategy and what I would recommend people do I think is go take the exam just like you normally would. Like don't look up, learn as you're going through it, go through answer the questions. The ones and I do this already, the ones you're not sure of, I mark 'em to go back and review later. So I think I'd go do the whole exam, mark the ones that I wanna review later and then come back and if there's time hit the ones I wanna review and look up, use, learn to see if I can help solidify those answers on the ones I wasn't sure of the ones I wanted to review versus trying to pull up learn as I'm going through each one of 'em. We'll. See, I think it'll work better at home for most folks in this model. Like the other thing that I uh, comes to mind is most Pearson locations are pretty trashy . Like they're in random places sometimes, right?Like hey go to this weird office building in the back of this office park and the door is locked and and it's like a skeevy place and you're like, uh,I don't know about this. Or they're in like college labs or things like that where there's a bunch of other tests and tests taking going on. Like I wonder about the infrastructure sometimes I think it was easier if it was just a shutdown down componentized thing. But now if you need to get out to the internet you're dependent on internet access the whole time. Like oh how quick can I browse through this? And and what's that look like? The machines you take exams on at appears and Exam Center are all like crappy thin clients. They're not like beefy machines like you have at home. So they run the exam software slow enough that I bet they don't run the internet any faster. Like when it comes to rendering in a web browser or things like that. I might have to go try and sit one just to see how it goes. Go figure one out Scott. We can compete on an exam. Indie, Do you feel overwhelmed by trying to manage your Office 365 environment? Are you facing unexpected issues that disrupt your company's productivity? Intelligent is here to help much like you take your car to the mechanic that has specialized knowledge on how to best keep your car running Intelligent helps you with your Microsoft Cloud environment because that's their expertise. Intelligent keeps up with the latest updates on the Microsoft Cloud to help keep your business running smoothly and ahead of the curve. Whether you are a small organization with just a few users up to an organization of several thousand employees they want to partner with you to implement and administer your Microsoft Cloud technology, visit them at intelligent.com/podcast. That's I N T E L L I G I N k.com/podcast for more information or to schedule a 30 minute call to get started with them today. Remember intelligent focuses on the Microsoft cloud so you can focus on your business. What else did we have? Did you have We had a couple other kind of interesting announcements. Oh I've got YouTube videos playing in my tabs now as I click through 'em. did you want to talk about one of yourannouncements this week around storage accounts? Yeah. We can talk about one of mine. So something that I've been working on with some of my partners over in Azure networking is improving the experience for our C D N customers. I don't know if you've paid attention to the C D N space at Microsoft and kind of what's been going on there, but there's a couple different resource providers for CDNs. Like there's Microsoft CDN and then within Microsoft C D N there's classic C D N, there's Akamai, there's Verizon. And at one point there was an offering called Front Door Classic. Uh, well it was called Front Door at the time Azure Front Door. And then that became Azure Front Door Classic and now there's Azure Front Door Standard and Premium. And we never really kept up with all that stuff on the storage side. Like many of our customers have ended up in a space where you know, they're, they're potentially relegated if they want to configure things through the Azure portal to leveraging like the classic C D N Akamai C dmm which is gonna be deprecated and and has been announced as gonna be retired in the next couple of years. So I want to like we set out on this kind of journey to see if we can improve the process for customers there and by improve it, not just hey let's make sure we're modernizing and giving customers access to like the right tool for the right job but also simplify the process. Like it, it was kind of an interesting thing for me as product manager to sit down and look at an existing experience that like I had nothing to do with originally in in storage and go like, oh yeah, this is kind of like not the greatest experience for customers. It's kind of complicated. It takes in excess of 20 clicks to create a C D N endpoint .Which is kind of crazy because really all you're doing is using the portal as like an arm template expression generator like we've talked about in the past. Like how do you simplify that, how do you make it quicker? So we cut it down to just a handful of clicks and you can deploy and manage your endpoints for a storage account. So for Blob as origin or for a static website as origin or both if you want to, you can basically go into the portal on a storage account and there's a blade in there that's now called Front Door and C D N and you just pump in a name for your front door profile and you choose a couple options like do you want a web application firewall and and some other security constructs like do you wanna enable edge caching and things like that. And you just click the button and it deploys and it just kind of goes, which is super slick. Like it's all good now. Now we have a unified management experience where you can come in. So if you're a customer who's managing uh, blob is origin under Azure front Door standard or premium or you're using any of the classic C D N constructs like Microsoft C D N, classic Akamai, Verizon, things like that, you have a one-stop management experience from within the storage blades. And if you want to like we don't do C D N as a a hobo service or a hosted on behalf of thing like you have a full access to front door and the associated C D M profile. So if you want to do other things in front door like okay great, I created my C D N endpoint and I spun things up but now I need to know go customize that. I wanna bind a custom domain, I wanna do custom ss, SS l, all that kind of stuff. I want to go look deeper at the features in the web application firewall and customize the the rule sets, the core rule sets associated with that. You have full access to that because the front door resource sets provisioned is something that lives in your subscriptions, you manage, you maintain, like we give you kind of the best experience for what you need as a storage customer in the storage side. But if you need more than that you just hop over to the, you hop over to the other side and go to your front door resource and you can manage everything through there as well. So I think it's really like a best of both worlds kinds of things And now that there's more native integration from our end from the storage side with Front Door, I tend to look at that as a little bit of like forward looking thing as well. Like what other kind of integrations can we build in to make things better for storage customers. Like you know I mentioned like custom domains, you can do custom domains in storage today but you can't do custom SS s L on a storage account. But if you front your storage account with front door, front door does both custom domains and custom S S L. So like is there a better native integrated experience that we could have there to guide customers in like hey if you need a custom domain without SS S L go this way. If you need a custom domain and SS SS l go over here to this other service 'cause it's the best thing and by the way it's natively integrated. Got. It. So this isn't necessarily adding any new SKUs or any new pricing or any of this. This is just taking those existing Azure front door standard and premium SKUs and creating some tighter integration between that and your blob storage. And our old C D N experience didn't even have front door in it. So if you were just coming over to storage you wouldn't have known that front door even existed or was an option for you unless you actually went out and did it from the front door side and came over. So now you kind of have this holistic view where you can start in front door and come to storage or you can start in storage and get to front door and have all that functionality between both stacks. Got it. I am gonna add this to my list and actually play with that 'cause I'm actually working on redoing my website right now as a static site hosted in Azure blob storage. So I might have to go throw Azure front door in front of it and then maybe go ask you for some Azure credits. .Front door's. Interesting. It's, it's, I had kind of lost sight of it being in storage Uhhuh ,it was good to get back up to speed on it like the last time I looked at front door, you know they had maybe like 150 pops and they didn't have as much of the rich WAF functionality and some of the other things like it's actually kind of come a long way. They've got like 192 plus pops today. They've got way better controls around caching and cash purge. Having a single front door endpoint with like multiple blob storage origins is super easy. Now we also took the time to make sure that it works with kind of all storage account types. So you know we have some like new endpoint types with these Azure D N S zone accounts in in preview today. And even though that stuff's in preview like hey we baked it into this experience and it's ready to go. So I'm very happy with the way it turned out and as you called out in the chat like hey look at the author of that blog post. I've been in Azure storage for like three years and I'm kind of happy like this is the first feature that like I got to not only support and like help engineering build out but I own it and launch it and it's the area of storage that I own. So that's kind of fun too. Congratulations on that achievement. It's kind of fun seeing your name at the top of all that but can you get Azure front door pricing cheaper? Like if I go do this I'm gonna have to make sure I stay on the standard plan because there's no way I'm paying for the premium one. stick around and watch the space and we'll see what I can do. You'll.See what you can do. Alright there's your next i I sent you after your next project because I mean the standard one isn't bad right? $35 a month for something like this. But yeah that premium one at 330 a month, that starts to get a little steep for my lowly little website. Hopefully I don't need anything in there. I think most folks like if you're just starting out you don't need premium to be brutally honest. Like the things that it adds in, you know is there really value add for there? Like do you need a web application firewall on day one? Do you need additional DDoS protection between like what Azure, beyond what like Azure Standard DDoS? Like probably not, right? Right. Especially if you're talking about like a blog or even like a simple like company website like like your website intelligent.com like likely doesn't need a bunch of those features. Like those features are there for the folks that need them and Standard does do a whole. Lot Yeah about production. Like I wouldn't need private link, I wouldn't need threat intelligence, security analytics, you know, especially if you're hosting a static website, right? Like there's not a whole lot that it's public already so youreally don't have a lot of threat detection you need or security stuff. It's just a bunch of H T M L files. Maybe you care about somebody getting in there and somehow changing your files. But I would agree like you look through the premium features and hosting a static website blob storage, you can probably get away with most of 'em with just that standard skew of front door and even. Like web static websites in storage are super basic things where I would say like for most folks, like you know you should start probably unless you know you really need it. Like you should start with Front Door Standard. I go the other way with things like static websites, like you probably think you wanna start simple, but uh, from what I do with static websites and what I see many customers doing, frankly storage isn't always the right tool for them. I I get, I'm supposed to sell more storage but I, I would almost,I would caution you and potentially guide you towards other services like Azure static web apps where it has rich native integrations with GitHub actions and all those kinds of things out of the box, right? I think about like most static websites are actually statically generated based on other content like things like Hugo and Ghost and all that. Yeah. So you have to run through like a build process if you wanna do that with a static web app just on storage. There's a ton of bootstrapping that you need to go do on your own. And if you just started out with back to that conversation about the right tool for the right job, if you just started out with Azure static web apps, you would potentially be in a better place 'cause it has all those integrations natively from day one and you don't have to do all that heavy lifting. We. Should go back and talk about those 'cause those have changed quite a bit too. I know like Blob storage used to be the way. Blob storage used to be the way, that's another area in storage that I look after. But now it's those Azure static web apps have gotten. I don't know that it's the way for everybody these days anymore. I'm very mindful of like in Azure we have multiple services that do the same kinds of things and just from what I see customers doing and from what I know that you probably do like you're probably like a Hugo or a ghost person kind of thing versus a I'm gonna open VS code and just actually manually write some H D M L here. I am actually using, now I'm completely blanking on the name of it. I'm not using either of those. I am using, oh my Edge just crashed on me. , we'll just call it a day. Yes.It to do too much. Why. Am I blanking on the name of it right now? But yes, I looked at a bunch of 'em. I'm not using Hugo or Ghost. It's a newer one that came on the scene that I actually like better 'cause it's not it, it just made sense to me. That's the thing with the static website generators, right? You just pick one that makes the most sense. But. I think the key word there is generator. Yes, there's static web apps and then there's statically generated websites. I absolutely use a generator. I think that's the distinction. So once you start to get generator in the name, like you might wanna be looking at something else. Yeah. So you go Pelican uh, it's in here. We'll come back with that one. I'll throw it in the chat, we'll throw it in the show notes. But I actually do have a meeting at four o'clock so I should probably call it a day with that. All right. Well as always thank you and we'll chat again next week. Alright. Thanks Scott. We'll talk to you next week. Yep. Thanks Ben. If you enjoyed the podcast, go leave us a five star rating in iTunes. It helps to get the word out so more IT pros can learn about Office 365 and Azure. If you have any questions you want us to address on the show or feedback about the show, feel free to reach out via our website, Twitter, or Facebook. Thanks again for listening and have a great day.

Donate

Apps

In Episode 350, Ben and Scott talk about some of the ambiguity in features between several types of Teams channels, a new model that includes access to Microsoft Learn for those taking role-based Microsoft certification exams, and a simplified experience for creating and managing CDN endpoints using Azure Front Door for Azure Storage customers.

Like what you hear and want to support the show? Check out our membership options. (more…)

Episode 325 – How do I migrate from SharePoint Online to Azure blob storage?

by [Scott](/content/author/scottmsclouditpro/ "Posts by Scott"/index.html) | Mar 16, 2023 | Podcast

Blubrry Player

Auto Scroll

Welcome to episode 325 of the Microsoft Cloud IT Pro Podcast recorded live on March 10th, 2023. This is a show about Microsoft 365 and Azure from the perspective of it pros and end users where we discuss a topic or recent news and how it relates to you. Microsoft Loop is finally coming in public preview later this month. So naturally, Ben and Scott share some of their experiences with Loop and in particular loop components to date as well as a few oddities that they've run across using Loop components. From there, they move into an interesting scenario they recently encountered around moving a hundred plus terabytes of data out of SharePoint and into Azure blob storage and some of the challenges that one might encounter in such a scenario. I have a question, Scott. When can I start playing with Loop ?I don't know. Off the top I can say that Loop is a confounding product to me. The more you make me interact with it, the more confused I get by it. So example, like real practical example, the other day we were recording the stream for YouTube and we're putting that together. And normally when we do the YouTube streams we go back and forth inside of just teams chat with, with random ideas and kind of, you know, status if we see somebody's dropped out or you know, their video froze or something like that. And there was this need to put in just a bulleted list for things. And it just so happened that in the tendency that you and I do, teams in like Loop is all lit up and ready to go. So as a, you know, great smart ass, I went in and said, Ooh, rather than creating some rich text here, I'm gonna create a loop component cuz it's gonna be great. So the scenario is you and I are in a teams chat, we're in the same exact teams chat. We both have permission to the chat, we're like in a live meeting, right? We're, we're going through and we're doing it. I wanna. Put this on top of it. We're in, we're logged into the same tenant too. Yes. Like we're both using IDs in the same tenant. This isn't external guest user or uh, I invited an external user to the meeting. This is you and I logged into teams into the same tenant in a chat in a meeting. Yep. So I went two of us in a chat in a scheduled meeting. This isn't even like a one-off teams chat or anything like that. I don't know if that would make a difference, but just to like, this is a fairly structured thing and, and kind of a common workflow I would think. So we're in the chat and I create a loop component which is just rich text and it's got a couple bullet points in it and I write up the text that I'm gonna put in there and I hit send the send button in Teams first problem. The loop component does not render for you at all and you can't even see it rendering there. You get basically a link to click on and go out and view it online at which point you get an access denied message. So you have to request permissions to the loop component that I just sent to you in that teams chat, which in and of itself I'm just gonna say like that is a horrible experience. Like yeah, teams should know that you and I are chatting with each other and that I explicitly created that. I didn't send you a preexisting loop component so it wasn't like I was sharing a document or a random link that you, you shouldn't have had access to. It's like I created this loop component as a part of this chat. So you had to go open up, you know, the web view of my OneDrive request access. I had to go and know where to go in my OneDrive to respond to that access request or wait for the email to come through from SharePoint that says, oh you know Ben requested access to your thing. I give you access. I have to tell you in real time, mind you, this is like minutes later as we've been dancing around and trying to figure out what's going on here, I have to tell you that I gave you access to that component. And even then, because teams is relatively static when it comes to chat and things like that, it's not like auto refreshing in the background for messages that were previously sent. Then you have to click out of the chat in the meeting and come back in, which I'm gonna argue doesn't really work like it works if you're somebody who always chats outside of your team window. Like you and I are in a meeting right now and we're doing video back and forth. So I tend to chat out of the same meeting, uh, the same window that the video's in and the only way to refresh that would be to leave the meeting and then come back like what a horriblebroken, awkward experience. Like it really makes me question if the people who want us to use loop components have ever tried to use them themselves. This is a little odd to me. So I have seen this done and I thought I had done this successfully in the past and I've seen people do this successfully in the past where they've all been in a meeting or in a chat and they've gone and added a loop component and everybody's been able to access this. So I'm also a little curious is, is this, do I have a setting in my tenant that I missed? Is this a timing thing? Is this like why is it doing this? Because I have, in all fairness, I have seen this work, but to your point, it shouldn't be this complicated is to why it doesn't work. Like. I'm gonna argue if there's a setting in your tendency that you need to set to make that work, it shouldn't, again, that should not even be a consideration if there's something out there that you need to go ahead and crank on a knob like that. Sorry, like that's just a, a non-starter And I might be a little salty about it cuz we've been doing like planning and a bunch of like product wor work this week and I've been spending a lot of time with customers and thinking about customer asks and thingslike this. This is just, I, it's one of those ones where I go, did anybody ever try this? I wonder, I just tested this. So I put a link in, I put a screenshot in the discord of what this looks like and it showed a loop list in draft and it gives me like a message at the top. Some recipients won't be able to view or edit this and this was in a chat with me and someone else in my tenant. I didn't actually send this one to you Scott, it was with somebody else. And I waited a little bit and I finally clicked send and then I got a little button that said refresh. And now that I refreshed it and I look at access, it looks like they have access. I'm speculating. I wonder if there's a timing thing because it did just work for me, but it was not as quick as you and I did it before, like it was seconds while you were talking where I started it up and maybe waited 20, 30 sec 20 or 30 seconds or so and it did work. Like is it just slow in going in and granting access to it? Where had we taken longer? Again, to your point, I would argue it should probably be quicker and a little bit more straightforward of ex explaining why it didn't work, but it did just work for me. But it was slow. We. Waited what like minutes the other day cuz we did a whole big dance back and forth of going out to OneDrive and you requesting permissions. So I'm gonna put on my hey I ideation hat thing, right? Yeah, like how, how should that work? So your argument that it might take a little bit for permissions to propagate, you should tell me that in the client. Like you shouldn't even display the component, put a spinner over for it or something and say like working on it and you need to wait like asynchronously until the other end gives an act and says, Hey, all the permissions have propagated and we're ready to go. If there's some type of weird asynchronous flow that's going on where like when you click a send button, it's creating the loop component and then you're waiting on another async call to do the permissions later. Uhhuh .Like there could be a weird timing issue there. Like don't make that an asynchronous call, make that a synchronous thing. Create loop component set permissions display, right? Like and then display block. Yeah, just block the display of it. don't make me have the cognitive load andthe overhead to think about that. Especially in a world, uh, where you and I are in the same exact tenancy, same a d tenant, same set of permissions, like all the, like all those things. You're not logged into that tenant right now, are you? Cuz I can't even send you one in the call we're in now. Probably not. I. Wanna go back and try this again anyways. We can go play with it more. But I think yes, it was one of my, that's only like the second or third time I've interacted with Loop and it has been a showstopper frustrating experience each and every time I've done it that it makes me question sanity as a user, which isn't a good place for any product to be. Yes. I would agree with that, but Scott, you can play with it more now because we've had loop components. But I did see that the Microsoft Loop app, so now we have a standalone web app for Loop is coming to public preview this month is the current plan. It's gonna start rolling out over this month. Go check your message center because there's a whole bunch of stuff in here. I actually saw it on Twitter before I saw it in the message center and they haven't figured out here. Here's a shocker for you Scott ,they haven't figured out the compliance stuff for it yet. So while it's coming in late March, it is not enabled by default coming late March. They are rolling it out, but it is going to be disabled in tenants by default and you actually need to go over to your cloud policy service. So the, what is it? config.office.com. Yes. config.office.com and go tweak your policies and config.office.com to enable the Loop app so that when it does come, so you can go enable it now, like it's in the policy. I already went and enabled it so that when it does get deployed to your tenant, you will be able to use it and play with Loop even more because I know that's what you wanted to do. Scott .I really don't know that I want to, it's .It's just another, it's another one of those things. I think it's going to be a, it's gonna be a very SharePointy experience and I often feel the rough edges on SharePointy experiences. Like I always want them to do more. And I think we talked about this the other the other day too, but uh, we, we did it a little bit more privately. We can do it again here. Uh, Purpose-built tools are purpose-built for a reason.Like they're very good at doing that one thing. So if I look at say, component or a piece of SharePoint, like lists, lists is like, okay, but it carries just all this overhead and baggage of being ultimately just a SharePoint list, right? It's not really a Microsoft list, it's a SharePoint list and Carries, that's a SharePoint list, all the constraints that come with that. So rebranding aside, like it's got a lot of cruft and it's got a lot of overhead and it's got some weirdness to it because it was designed to live in SharePoint. I fundamentally believe that, right? Like there, there's constraints in the underlying platform that drive to, you know, how these things come out and, and are manifested for end users. So if I think about like Microsoft lists and then Uhhuh ,maybe you go use Airtable, sorry, air tables a hundred million times better like . It, it is,but it's purpose built. Do I get the document integration and Power Automate and all those things? Like absolutely not. Like I don't get that whole ecosystem, but if I don't need that ecosystem, like a purpose-built tool might actually work a little bit better for me. And the reason this is kind of coming to mind for me is that lots of people compare Loop and especially kind of the standalone loop, like, hey, let's go to loop.microsoft.com as a notion replacement and notion is for all its flexibility, a very purpose built tool. Like it does have a line where it stops trying to be the everything for everyone kind of tool and it knows its limitations and it just, it wears those words right out there. Like you can find it in the documentation and you just know you go like, oh, this isn't gonna work. Versus something in SharePoint land and Microsoft 365 Land, quite often the answer is, well like that could work if you did this and this and maybe it'll work if you did this and what if you integrated this thing over here and you did this and you're like, I I I don't want to do that. I just kind of like want to to work and I want to do work, so enable me to do some work. Please. Yeah, and I think I totally agree. Like I think Loop could have been or could be, it'll be interesting to see when it comes out and we continue to play with it more. But in your, to your point like was SharePoint and OneDrive fundamentally the right place to store all of these files that are going to be your loop components, your loop pages, all of that. Ultimately those are gonna live in SharePoint. Just like you said, Microsoft lists are SharePoint lists, OneDrive document libraries, they are SharePoint document libraries. And there are absolutely fundamental things that, to be fair, technical deck carried all the way through from the early days of SharePoint. For those of you that have dealt with lists, maybe you haven't dealt with SharePoint lists and specifically, I still run into this all the time, is the whole 5,000 items in a query and in a list is like, I don't know if it was 2003, 2007, but fundamentally it's because all of this stuff lived in a sequel database. When you would go to query more than 5,000 items from the sequel database and would put locks on the database, that would cause the entire platform to run slow carry it forward15 years. And we're still dealing with it because of that technical debt. And when you do things like Loop and SharePoint lists and you try to treat a SharePoint list like a database, you inherently run into these things. To your point, other places are much better to build our li large lists because that's what they were designed for. And when you try to make a SharePoint list a database or even this like are we trying to make SharePoint the storage space for Loop when it should be like, oh, living in blob storage or living in some other backend service that isn't going to introduce some of the constraints. Now I also get the fact that SharePoint makes sense because you already have all the identity stuff, the permissioning stuff, the sharing stuff. There's a lot of stuff they don't have to build by putting it in SharePoint, but they also do incur that technical debt and those technical limitations that live in SharePoint really because of how it was built 20 years ago. Yeah. .It's interesting. Do you feel overwhelmed by trying to manage your Office 365 environment? Are you facing unexpected issues that disrupt your company's productivity? Intelligent is here to help much like you take your car to the mechanic that has specialized knowledge on how to best keep your car running Intelligent helps you with your Microsoft Cloud environment because that's their expertise. Intelligent keeps up with the latest updates on the Microsoft Cloud to help keep your business running smoothly and ahead of the curve. Whether you are a small organization with just a few users up to an organization of several thousand employees, they want to partner with you to implement and administer your Microsoft Cloud technology, visit them at intelligent.com/podcast. That's I N T E L L I G I N k.com/podcast for more information or to schedule a 30 minute call to get started with them today. Remember intelligent focuses on the Microsoft cloud so you can focus on your business. You also experienced a question going back to using things for what they're built for and challenges that can come with it. What about if you try to copy stuff out of SharePoint with AZ copy? I had a question from a colleague come in and I think it's an interesting scenario. Uh, so first let's kind of start with the question. I have a colleague that's working with a customer and that customer is looking to migrate a couple terabytes of data from SharePoint online to Azure blob storage. Like they clearly wanna pull some of those files, documents, whatever they happen to be out of SharePoint and push them over to Blob in and of itself. Like, okay, fine. There were some things that didn't even cross my mind in thisconversation when I was chatting with a person. So the initial ask, like just to the number that they asked for was higher than this. Well, like let's peg a number to it, right? Like, so I wanna migrate a hundred terabytes of data, which means I have a tendency with approximately 10,000 users give or take to get like the requisite licensing, right? It would be someplace in that order of magnitude, right? You, you're more than five licensed territory or, or you've bought a, a whole big chunk of storage from Microsoft, but they wanna migrate a a hundred terabytes of data from SharePoint online to Azure Blob and they're already trying to do it. And the, and the way they're trying to do it is using a storage explorer, which I have to imagine, like, I didn't even ask the question, but I have to imagine what they're doing is either using synced libraries that are already synced through OneDrive for business or they're using the, like the file explorer view and kind of the mm, the, the wonkiness that comes along with like that, that weird web a not web d a thing that they have going on these days. That thing that should have never been in SharePoint to begin with. Never should have been around to start. I I guess a couple of things here. So one is, uh, storage Explorer as a tool uses AZ copy under the hood for its underlying copy operations. So you can think about like dragging and dropping a file from say, your desktop into a storage explorer window that's mapped to a storage account in Azure as an operation where Storage Explorer is acting as a nice wrapper and a nice proxy and it's gonna build out and it's gonna say, Ooh, I had a file dropped in here, so I need to call AZ copy CP with this source, this destination and push all those things through. And if you actually look in the, the logs for Storage Explorer, you can see that it's calling the commands for AZ copy in the background for those kind of core copy operation things, right? The the puts and the GIS and, and all that kind of stuff. So. Just quick question, is there any performance loss by using storage copy versus just AZ copy in the command line? Or is it really gonna be about the same where Storage Explorer doing that type of stuff as just a friendly cover on top of AZ copy, kinda like the Azure portal is a friendly cover on top of rest calls? Depends on the operations you're doing. Like there's, okay, there are things that Storage Explorer does that are not functionality that's provided by AZ copy really we're talking about like the underlying data movement engine of storage Explorer. Is just AZ copy is. AZ copy and a standalone, uh, thing for it. So there might be instances where Storage Explorer is great for you, like you need a gooey, you need to browse it, you wanna do something like calculate folder statistics, anything like that. Like awesome, go ahead. You wanna view apples on like ADLs, gen two storage account ton of file or folder like great use storage explorer for that. Like that's native functionality to it. And then, you know, you wanna copy data, use AZ copy if you have a large data copy to do, you might not want to use Storage Explorer as an extraction layer because it's just adding more overhead at that point, right? It's, it's an orchestrator for you, but ultimately it's orchestrating a single instance of AZ copy. So if you're trying to do, you know, a large copy operation, and I'm gonna define large in like the hundred million plus file range, like say you're trying to copy half a billion objects, 500 million of those, you might wanna orchestrate that with like multiple running versions of AZ copy or even AZ copy running for multiple machines or things like that. So Storage Explorer may or may not cut it for you in that scenario, you know, as, as as kind of an orchestration or or or proxy thing that's sitting on top of, on top of easy copy. Got it. So back to our scenario. We've got a a hundred terabytes of data in SharePoint online, not very clear where it sits. Does it sit in, you know, a, a set of libraries in a single site? Is it multi-site, like whatever, I don't know that it matters, uh, a whole ton, but just from off the cuff from like the question I got, it's like, ooh, it sounds like they're maybe trying to do something unnatural here. So I was just trying to think through like what are their options And back to the, I don't always think about scale thing. I think the way you might, I saw a number that ended in TB and I was like, oh, it's terabytes. This is easy, right? Like you can totally use AZ copy, you can use AZ copy to do multi petabyte migrations. So like, we're good there we're golden. So my first thought was like, let's just sync it back to OneDrive for business. Like pull the data down locally in a client and then go ahead and continue to use Storage Explorer or just point AZ copy at that folder where OneDrive has synced locally and you know, don't do it on like OneDrive on like your client on your laptop while you're sitting on the beach. Spin up a VM in Azure that's in Microsoft's wan where it's gonna be able to do a semi performance synchronization through OneDrive for business. And then certainly like run easy copy from that op uh, from that VM as well. So you're all on the same backbone and, and everything's gonna kind of go a as quick as it can. And you quickly pointed out to me that there's some uh,some potential limitations just around OneDrive for business, whereas I'm like, I don't know, it's just a couple terabytes of data. Who cares? So. This was a really fascinating question. If there's a customer out there that's trying to do this and they want somebody to help them call me, cause I would love to do work through this, but this is not, again, from your perspective blob storage insignificant from a SharePoint perspective. So here's where my head instantly goes is continuing with our a hundred terabyte example, a hundred terabytes is one and a half. I just did the quick math 104 million megabytes SharePoint. Naturally, it's probably a lot of office files would be my assumption. I don't know for sure you'd have to go do some looking, but let's say your average file size is one meg, maybe even two megs PowerPoints. Excel could be bigger Word documents could be smaller. You're looking at like 52 million files where my head instantly went when you said sync it with OneDrive. Is OneDrive in the TSH or in the service description document limitations. And I've seen this, uh, play for optimum performance. Microsoft recommends storing no more than 300,000 files in a single OneDrive team or library site that you're going to sync. And if you are going to use sync, the same performance issue can be seen. If you have more than 3000, 300,000 items across all libraries, you are syncing even if the libraries themselves are smaller. So if you're talking 52.5 million files and you can never sync more than 300,000 items, assuming 300,000 items are evenly dispersed across all your document libraries and sites and they're not all in one, it's a lot of syncing and unsyncing to get that amount of files onto a local computer where you could even begin to use something like AZ copy to copy them up. I mean maybe you go stand up, what, 300,052 million divided by 300,000. You're doing it 175 times. No, maybe you're a large enough company. Again, a hundred terabytes, you're looking at probably 10,000 ish licenses, 175 people to do this. If you can get a bunch of people working together, maybe not the end of the world, but now you're trying to coordinate who's thinking what, who's copying what, not missing anything. This is not an insignificant amount of work. Fair enough. So one of my other thoughts and I and I tend to go to this back from like my old migration days cuz I was used to always going from say like an on-prem SMB file share to SharePoint. You really didn't come back the other way too much, right? But you definitely had this machination, you went up where you, you went a certain direction and I think Microsoft has built out tooling over time like the SharePoint migration tool, uh, thing, things like that that can help you out. So my other thought was like, well hey let's go to our old friends at an ISV or like a migration vendor like sharegate and just point sharegate at that thing cuz it's got a script ability engine, it has PowerShell support, all those kinds of things. Mm-hmm .So let's say it wouldn't matter a hundred terabytes of data if it was one terabyte per SharePoint site. That's only a hundred sites of data you have to pull down, script it all out in something like a sharegate and let SHAREGATE just run again. Let it do its thing. So again, let's spin up a VM in Azure that's in the region probably with my storage account and just install sharegate on it, right? Point it at it and let it crank, let it download everything. Once it's all downloaded, then I'm just gonna push it up the other way with AZ copy and let it go. How viable is that? That's probably where I would go first. Although I was just looking so I couldn't remember. Cuz again, I've always gone the other way with you. Go get a share gate, go get a AvePoint, go get Metal Logics, which is now Quest I believe, which is actually now Dell, right? Dell owns Quest owns Metal Logics.It just keeps going. Yeah. So. Sharegate itself, I'd have to go look at Questa Metal Logics, sharegate, I head handy. Sharegate does not support a local file system as a Target. It supports it as a source. So you can go to your point from local drives up to SharePoint, but you can't go back down. If you're gonna go to a Target, it's SharePoint on prem online. One of the other third parties may do it Mover, which is now Microsoft Migration Manager maybe used to support it. They were pretty flexible. Though. I know. Uh, SPMT only goes one way. It's it's on-prem up kind of thing. Mover. Is too, cuz they've started pulling functionality out of the old mover. Do IO when moving it into Migration Manager, which migration manager is different than the SharePoint migration tool. Yes. for those who are listening in audio, like Ben's a big,my head just. Mind. Exploded, mind blown emoji right there. Yeah. I'm. Legitimately trying to think how would I do this? Like there's Logic Apps and Power Automate that let you Connect. That's. The next place I went to. I said, hey, if it's ongoing or even point in time migration, I don't know that it necessarily matters. Logic Apps, power Automate and Azure Data Factory have connectors. Or at the very least I can just do an HDP call and make everything work through the HDP connector and get it to where it needs to be. You might have some scaling issues in there given number of calls, number of API calls, things like that that you have to consider along the way. But it should be doable. Should be maybe. So I tried to do this in essence, I have a client that we're doing, we're not doing this, I was going to SharePoint from SharePoint, right or wrong, as this may sound. Their process and the way they did their work was they had certain projects, these were large projects, think like large, relatively speaking, not large compared to what you're used to storing large projects of a folder in SharePoint that would contain four or five, six gigabytes of data, uh, thousands of files. So much smaller than millions of files in terabytes of data. And their workflow was, once this project reached a certain point, it moved from one SharePoint site to another SharePoint site and then it had another status and moved again. And then eventually the part project got archived off and it moved again. So we did, I tried to do this with Flow originally where it was doing somethingsimilar to that where they could go flag a project, flag a folder, update metadata, whatever it may be to get flow to actually do the move in the copy. It didn't work. The data was too big. Ultimately what we ended up doing is I have a hybrid runbook in Power Automate that actually uses sharegate on a server sitting in Azure. So they trigger a Flow logic app. The Flow Logic app runs, runs a hybrid worker in Azure Automation runs the PowerShell on the local server to use sharegate to move the data from one site to another site because flow and logic apps couldn't natively handle it, nor it could even just running PowerShell in Azure automation. I think it's really the SharePoint endpoints, it's those APIs you start handling hitting throttling limits. That same article that talks about, and I completely lost it on all my browser windows when it talks about the limitations of moving, let's see if I can go pull it up here. Uh, moving and copying files across sites. So copy move operations. A single operation has three requirements. No more than a hundred gigabytes of total file size. No more than 30,000 files in each file must be less than 15 gigs. Now maybe maybe moving and copying down locally or using Azure Data Factory or something like that could work around it. But you're, I think you're still gonna start hitting thresholds and constraints with some of that stuff. That's probably your best bet. The other thought that is kind of in the back of my mind, but again it's still using rest calls and API endpoint that can get throttled would be just to use PowerShell to download the files at a SharePoint. Just write a huge PowerShell script or write one that you can point at a site and just go aside at a time, run the PowerShell script, download 'em all locally and then use AZ copy to move 'em up. Yeah. I, so, so that was another thought that I had was you could be slightly transactional in nature about this. So AZ copy has things like command line, pr, uh, parameters where you can point it at a say like a folder of files that are sitting locally for you. But ultimately what it has to do is enumerate those and every upload operation is gonna be kind of a, a separate call and a separate API call. So if you're processing a objects to a certain degree in a linear fashion, you know, you're processing them, you know, in a, in that way versus a highly concurrent model where maybe you're gonna process like 10 million at once kind of thing. Like that might be hard for you to pull off something like Casey copy. It would be a long running operation. But depending on where the files are hosted in SharePoint, like let's say it is a hundred sites, go into something like Azure Automation and create a runbook in Azure automation where you don't even have to use AZ copy. You could actually use something like just the Azure CLI or Azure PowerShell to do this as well. Like if you want to be PowerShell driven, like you said, hey, let's go ahead and download through PowerShell using the SharePoint APIs, whether you're doing that with PMP or just native kind of SPO commandlets and then just call an upload operation on the other side and script it out that way, right? Like call this site, get everything in it, download one, upload one, download one, upload one, and then concurrency. You'd have to figure out how to kind of spread that out. Like how do you get peanut butter all over the place so that you can get the scale and throughput that you want. Like it is kind of an interesting problem. I, I kind of thought about it as easy in my head again, I started with, it's got tb so it, so it's easy in it, right? Like earlier today the, the very first customer conversation that I was working with a customer on was a uh, six petabyte data lake . So I saw 200 terabytes. I was like,you're my easiest problem today, right? Like yeah, you got nothing going on. . Yeah. See and so you mentioned Azure automation.We may have done a podcast on this a long time ago. You would something this big, you would have to run it in VMs if you wanted to script it with Azure automation, it would have to be hybrid worker or you'd have to run it just natively a VM and Azure because Azure automation, you would start hitting the fair use limits where if you start consuming too much memory, too much CPU or your script runs for too long, they essentially boot you out of the queue and restart your job length of time. I've had it happen to me as quick as 30 minutes I've gotten away running something as long as two hours. This amount of files, you'd be way beyond two hours. And because you're dealing with files, I would assume your CPU and memory consumption in whatever's doing the processing and the backend, they boot you out way sooner than two hours. So I don't think you could even do this with Azure Automation successfully, uh, without doing a hybrid worker, without running it natively on a server. So, gotcha. So at that point you just orchestrated on a server anyway, it's just a power sell script, right? Run it, do the Rone thing, write it and then set it, forget it and let it run and do its thing in the background. I think that's what I would do. I think my approach would be scripted out, run it on a server. If you wanna run it in parallel to your point, if you have a hundred sites and you are okay spinning up a hundred servers in Azure, just spin up a bunch of servers, dedicate a server to it, it might help with some of the throttling. I don't know what they watch from the SharePoint side in terms of throttling when you're hitting the api, if you're coming from a hundred different servers, maybe you don't get throttled as quick cause you're coming from a bunch of different IP addresses. Maybe they recognize that it's coming from the same tenant and going to the same blob storage and they're gonna start throttling you. But I think PowerShell on a server to download it then AZ copy to upload it is probably your best bet. All right. Well I, I mean if anybody else has any good ideas, uh, , let,let us know. This is almost where I want like the bulk export tool from SharePoint. You know, like kind of like we have like Azure import export where I can just send them a disc or have them send me a disc. I almost want you to just be able to send me a disc of my stuff, just do a dump and, and send it out the other side, put it in the mail. And if it gets to me in two weeks, do. They have that? I don't know. I don't, I don't think, I don't think for SharePoint such a thing exists. There's bulk import in the form of these tools, right? Like, like SPMT and mover.io and some of the other things you mentioned, but I don't think there's a bulk export tool. Like, not even on the compliance side, right? There's nothing that I can say to go in and gimme all my files. I do believe I was, I was doing some quick Googling on the side, like I did find, like there's CIS tools makes a document downloader. So maybe there's somebody else out there that makes, you know, something similar if that wasn't your jam. But it does look like there are a couple, at least a few kind of paid tools out there that would let you do it. So yeah, I, I don't know. I thought it was an interesting question though. Yeah. Absolutely. As, and I would be curious if anybody else has any other insight into it. Because one, I've, to your point, I've never gotten this question before about, I have all this data, but I guess it kind of makes sense, like if you're looking to archive stuff, we had some episodes too where you talked about SharePoint as a files server. It's an expensive place to store a bunch of files if you're just storing them there and never touching them again. Like I can see where this could become a request. As companies create lots of data and they have older files, they don't delete 'em, they wanna archive 'em somewhere. It is way cheaper to store 200, a hundred, 300 whatever, hundreds of terabytes and blob storage than it is SharePoint. SharePoint you're paying, again, I figured it out. I did the math. You get 10 gigabytes for every user. So every user license gives you 10 gigabytes. One terabyte is a hundred user licenses. So you're either paying for a bunch of licenses if you're going out and buying file storage, it is still 20 cents per month per gigabyte. 20 Cents per month per gigabyte. Eh, it starts to add up. It starts to add up compared to blob storage, especially if you can go into cool or archive. That's fra Is it like fractions of ascent? Yes. Yeah, .Yeah. SharePoint storage is 20 times more than archive blob storage. There could be a valid use case for this. Maybe Microsoft should go build a feature for archive to blob storage out of SharePoint when you get old files. That would be kind of a cool tool. .I I mean I, I certainly like the idea of it, right? Like, uh, pay me money, . That'd be, that'd be great. Yeah.Huh. Fascinating. All right. I'm curious, like Scott said, if you have feedback, let us know. Twitter, YouTube comments on the blog post, associating the podcast, whatever you want to do. If you're part of the membership, just give us feedback and Discord and we can share it on a future podcast. But. I was gonna say, if, if you were a a member, you could come along and participate as we record real time. You could see our yes, our made for Radio faces while we're here streaming off into the ether. Yes. Give us live feedback and input on all of our crazy ideas that we come up with. Indeed. All right. Oh, I think that's it for me today. I thought we'd get through that one in 15 minutes and here we are onceagain, you know, 35, 40 minutes into it and still debating. Still debating. We do have a question we need to get to it. We do have a question in Discord from one of our members that we have not gotten to that we should put it on the list, Scott, in the next week or two to answer that one cuz we can answer it in Discord. I've thought about going in and typing it up, but I think it's another interesting one that would be good discussion in a future episode. Next week. First thing, I'm gonna write it down on my Post-It note of things that I need to make Ben do, uh, we'll discuss. We would do it. Listener questions from Discord first. All right, sounds like a plan, Scott. And with that, go enjoy your weekend. Perfect. Thanks Beth. If you enjoyed the podcast, go leave us a five star rating in iTunes. It helps to get the word out so more IT pros can learn about Office 365 and Azure. If you have any questions you want us to address on the show or feedback about the show, feel free to reach out via our website, Twitter, or Facebook. Thanks again for listening and have a great day.

Donate

Apps

Episode 308 – Microsoft Purview In-place Data Share for Azure Storage with Priya Shetty

by [Scott](/content/author/scottmsclouditpro/ "Posts by Scott"/index.html) | Nov 17, 2022 | Podcast

Blubrry Player

Donate