Speaker 1: Welcome to the deep dive. Today, we're going to be
Speaker 1: diving into the world of automated machine learning exciting specifically
Speaker 1: on Microsoft's Azure platform. You know, think of it as
Speaker 1: AI simplified. And to help us navigate this really complex world,
Speaker 1: we have Dennis Sawyer's Practical Guide to Auto mL on Azure.
Speaker 2: Yes, this book is great.
Speaker 1: So who wants to start today is to give you
Speaker 1: a solid understanding of what auto mL is, what its
Speaker 1: benefits are, and then how you can actually use it
Speaker 1: on Ashure.
Speaker 2: Awesome, No, that's good.
Speaker 1: The book starts off with a pretty startling statistic. Eighty
Speaker 1: seven percent of AI projects fail.
Speaker 2: Wow, that's a lot.
Speaker 1: That's a lot of unrealized potential. Yeah, that is, especially
Speaker 1: considering the resources that are being poured into this field.
Speaker 2: It really is, and it's interesting.
Speaker 1: We'll see even more because the book.
Speaker 2: Digs into why interesting projects fail.
Speaker 1: Book digs into why it's not always they fail, and
Speaker 1: it's not all dramatic implosion.
Speaker 2: Right, dramatic inclusions a very slow bird. It's like a
Speaker 2: slow leak in a spaceship, just slowly draining resources and.
Speaker 1: Hope exactly, and as the book details, like the traditional
Speaker 1: machine learning workflow is incredibly time consuming incomplex so much.
Speaker 2: More than just building models.
Speaker 1: It's not just building the model, it's the data cleaning,
Speaker 1: the feature engineering, and then the dreaded hyper parameter tuning. Oh,
Speaker 1: tell me about it, which can be a real headache.
Speaker 2: It's like solving a Rubik's cube, yes, blindfolded while riding
Speaker 2: a unicycle.
Speaker 1: Absolutely.
Speaker 2: And then, as the book points out, many data scientists
Speaker 2: aren't actually trained and deploying these models in the real world.
Speaker 1: Oh.
Speaker 2: Interesting, So they end up with these really fragile okay,
Speaker 2: hobbled together solutions that are just waiting to.
Speaker 1: Crumble, right, And this is where and.
Speaker 2: So this is where AutoML comes in. Gota mL comes in,
Speaker 2: offering a potential solution to this ROI dilemma that exists
Speaker 2: within data science.
Speaker 1: Yeah that ROI prom Okay, so elevator pitch time. What
Speaker 1: is auto mL and how does it solve this problem?
Speaker 2: Okay? Imagine having an AI assistant that takes care of
Speaker 2: all of the heavy lifting in that machine learning process. Ooh,
Speaker 2: I like that AutoML can train multiple models simultaneously using
Speaker 2: the latest algorithms. Got it handles feature engineering, nice fine
Speaker 2: tunes those pesky hyper parameters and even offers built and
Speaker 2: explainability features which are really crucial for building trust and transparency.
Speaker 1: Wait, so does that mean data scientists are going to
Speaker 1: be out of a job.
Speaker 2: Not at all. Auto mL actually empowers data scientists. It
Speaker 2: frees them up to focus on the higher level aspects
Speaker 2: of a like problem definition, strategy and interpreting those results.
Speaker 1: So it's more of a collaboration exactly between human expertise
Speaker 1: and AI efficiency. Now let's talk Azure. Why is Azure
Speaker 1: so central to this auto mL story?
Speaker 2: So the book focuses on Azure because Microsoft's cloud platform
Speaker 2: offers a comprehensive suite of tools for auto mL, and
Speaker 2: that's all through the Azure Machine Learning Service AMLS as
Speaker 2: it's called.
Speaker 1: So it's like our auto mL playground exactly.
Speaker 2: And the book actually guides us I like it through
Speaker 2: setting up an AMLS workspace, and it gets us familiar
Speaker 2: with all the features we have at our disposal.
Speaker 1: Nice.
Speaker 2: One key concept is compute compute, which is essentially the
Speaker 2: engine towering your auto mL jobs.
Speaker 1: So compute is like the horsepower behind our AI engine.
Speaker 2: That's a great analogy, and the book distinguishes between compute
Speaker 2: instances which are for simpler tasks, and then compute clusters
Speaker 2: for those more demanding god the resource intensive workloads.
Speaker 1: So when you need to kick it into high gear exactly.
Speaker 2: And here's where it gets interesting.
Speaker 1: Okay, these compute.
Speaker 2: Clusters can autoscale. Ooh, I like that, meaning you only
Speaker 2: pay for what you use budget conscious, which is a
Speaker 2: major win for budget conscious projects.
Speaker 1: Smart. So we have our workspace, our powerful compute engine.
Speaker 1: What about the fuel, the data?
Speaker 2: Yes, data is the lifeblood of any AI project. MLS
Speaker 2: actually works with data sets, data sets which act as
Speaker 2: pointers to your data sources, whether they reside in a
Speaker 2: storage account, got it or a SQL database. And Azure
Speaker 2: even provides open data sets for experimentation.
Speaker 1: Free data to play with. I like that.
Speaker 2: Yeah, so like the diabetes data set it's mentioned in
Speaker 2: the book.
Speaker 1: So I give auto mL my data and it magically
Speaker 1: spins out a perfect model.
Speaker 2: Not quite magic, but AutoML does do a lot of
Speaker 2: the heavy lifting kind of scenes. So the book explains
Speaker 2: how auto mL starts with what are called data.
Speaker 1: Guardrail data guardrails.
Speaker 2: Which are automated data quality checks that can identify potential
Speaker 2: issues right out.
Speaker 1: Of the gate. So bad data equals bad outcomes exactly, garbaging,
Speaker 1: garbage out.
Speaker 2: So once the data passes inspection, what's next? Auto mL
Speaker 2: kicks into high gear with intelligent feature engineering okay, So
Speaker 2: it handles tasks like dealing with missing values, transforming categorical
Speaker 2: variable for example, using one hot encoding okay, which the
Speaker 2: book explains really well, and it even generates new features wow.
Speaker 2: And it tailors all of this to the specific algorithms
Speaker 2: that it will be using.
Speaker 1: So it's not just blindly throwing algorithms at the data,
Speaker 1: it's strategically preparing the data for each algorithm exactly. That's
Speaker 1: pretty impressive. And speaking of algorithms, seeking of algorithms, AutoML
Speaker 1: has a whole arsenal at its disposal.
Speaker 2: It has a whole arsenal.
Speaker 1: Ranging from classic regression and classification models to more advanced
Speaker 1: techniques like gradient boasting and deep learning. So how does
Speaker 1: AutoML decide which algorithm to use? Does it just pick
Speaker 1: one at random?
Speaker 2: It does not. It systematically tests multiple algorithms okay, parallel,
Speaker 2: got it. And while it's doing so, it also fine
Speaker 2: tunes the hyper parameters for each one, so it's searching
Speaker 2: for the best performing combination.
Speaker 1: It's like having a team of data scientists working tirelessly
Speaker 1: behind the scenes, exact optimizing every step of the process. Yes.
Speaker 1: So does this mean I can just sit back and
Speaker 1: wait for the perfect model to appear?
Speaker 2: Well, not quite. Ok The book emphasizes that while auto
Speaker 2: mL is a powerful tool, it's not a magic bullet.
Speaker 1: Okay, fair enough. So where does human judgment come in?
Speaker 2: You play a crucial role at it in defining the problem,
Speaker 2: choosing the right evaluation metrics okay, and most importantly interpreting
Speaker 2: the results.
Speaker 1: So there's still that need for that human touch, that
Speaker 1: understanding of the nuances of the problem that we're trying
Speaker 1: to solve. Absolutely, and speaking of getting our hands dirty, Yes,
Speaker 1: the book actually dives into building models using Azure Machine
Speaker 1: Learning Studio okay, which is a visual interface for working
Speaker 1: with mls exactly. It's designed to be user friendly even
Speaker 1: if you're not a coding guru. Yes.
Speaker 2: And the book walks us through creating our first classification
Speaker 2: model using the a classic Titanic passenger data ah.
Speaker 1: The Titanic data set a data science right of passage.
Speaker 2: Exactly, you know, predicting who survived and who didn't based
Speaker 2: on factors like age, gender, ticket class face, and the
Speaker 2: book makes it surprisingly straightforward. You upload the data, tell
Speaker 2: auto mL you're doing a classification task, and you let
Speaker 2: it work its magic.
Speaker 1: And while it's working its magic, we can monitor those
Speaker 1: data guardrails exactly see if any potential issues are flagged.
Speaker 2: Yeah, it's like having a built in data quality watchdog.
Speaker 1: I like that.
Speaker 2: Now, once automil finishes training the set of models, it's
Speaker 2: time to put on our evaluation hats.
Speaker 1: So we've got a bunch of models. Yeah, what do
Speaker 1: we do with them?
Speaker 2: Auto mL gives us a smorgasboard of metrics accuracy, precision, recall,
Speaker 2: and it gives us those helpful confusion matrices to visualize
Speaker 2: how each model is performing.
Speaker 1: I'll be honest, sometimes those metrics can feel a little overwhelming,
Speaker 1: especially if you're new to machine learning.
Speaker 2: I hear you.
Speaker 1: There's just the book so many numbers. The book clearly
Speaker 1: explains each metric, okay, and helps us to understand which
Speaker 1: ones are most important for different types of problems. And
Speaker 1: remember those explainability features, Yes, we can use them to
Speaker 1: understand why a model is making certain predictions. That's crucial,
Speaker 1: which is crucial for building trust and transparency.
Speaker 2: So it's not just blindly trusting the numbers. It's about
Speaker 2: understanding the reasoning behind them.
Speaker 1: Absolutely.
Speaker 2: Now, let's say we've found a model that looks pretty
Speaker 2: promising based on the metrics and the explainability checks. What
Speaker 2: happens next, then.
Speaker 1: It's time to deploy deployment. We need to make our
Speaker 1: model available for use either for a batch processing of
Speaker 1: large data sets or for real time predictions.
Speaker 2: So we're taking our model out of the lab and
Speaker 2: putting it into the real world.
Speaker 1: Exactly.
Speaker 2: But we've mainly been talking about classification models.
Speaker 1: What about problems where you need to predict a numerical
Speaker 1: value instead of a category.
Speaker 2: The book covers that took it dives into building regression
Speaker 2: models using auto mL on Azure. It uses the diabetes
Speaker 2: data set to actually predict the progression of the disease
Speaker 2: based on different factors.
Speaker 1: So regression is about predicting numbers, right, like stock prices
Speaker 1: or sales figures, or in this case, the severity of
Speaker 1: a medical condition. Exactly. Okay, And while the process is
Speaker 1: similar to classification, obviously the metrics and the algorithms used
Speaker 1: are different, but AutoML still handles all that heavy lifting
Speaker 1: it does. But we need to use the right tool
Speaker 1: for the job. Absolutely, I bet there are some tips
Speaker 1: and tricks for getting the best performance out of auto
Speaker 1: mL for these regression problems.
Speaker 2: There are, and the book offers some great insights.
Speaker 1: Okay, lay on me.
Speaker 2: Sometimes converting a regression problem into a classification problem can
Speaker 2: actually improve performance.
Speaker 1: Hold on, how do you convert a problem about predicting
Speaker 1: a number into a problem about predicting a category?
Speaker 2: It's all about binning binning. You're going to divide the
Speaker 2: range of possible numerical values into categories or bins. Okay,
Speaker 2: So instead of predicting the exact price of a house, okay,
Speaker 2: you might predict whether it falls into a low, medium,
Speaker 2: or high price range.
Speaker 1: It's like simplifying the problem to help AUTOMML find patterns
Speaker 1: more easily. Exactly, Okay? What are their tips does the
Speaker 1: book offer?
Speaker 2: It emphasizes the importance of experimenting with different primary metrics. Okay, So,
Speaker 2: for example, you might find that a metric like mean
Speaker 2: absolute error MAE gives you a more meaningful evaluation than
Speaker 2: something like root means squared error or RMS.
Speaker 1: Right, So it all depends on what you're trying to
Speaker 1: achieve with your model.
Speaker 2: It does.
Speaker 1: Okay, So there's still a lot of room for human
Speaker 1: judgment and decision making, even with AUTOMML handling so much
Speaker 1: of that complexity. Now I have to ask, what about
Speaker 1: those situations where you need to train not just one
Speaker 1: or two models, but one hundreds.
Speaker 2: The many models.
Speaker 1: Probablybe even thousands of models.
Speaker 2: That's the book really shines. It introduces a tool called
Speaker 2: the Many Models Solution Accelerator or MMSA. MMSA okay, think
Speaker 2: of it as an auto mL factory. You feed it
Speaker 2: your data I'm listening, and it automatically splits it up
Speaker 2: based on certain criteria, different stores, products, or regions, and
Speaker 2: then it uses the power of auto mL to train
Speaker 2: a separate model for each of those subsets, all running
Speaker 2: in parallel.
Speaker 1: That's a lot of models.
Speaker 2: It is a lot of models.
Speaker 1: So why would someone need to create so many models?
Speaker 2: It's incredibly useful when you need granular, customized models.
Speaker 1: So got it.
Speaker 2: The book gives an example of a retail chain that
Speaker 2: wants to forecast demand for each product in each store.
Speaker 2: By using MMSA, they can create thousands of custom models
Speaker 2: that account for all the unique factors that might influence sales.
Speaker 1: Wow. So it's like hyper personalized AI.
Speaker 2: It is.
Speaker 1: That's incredible.
Speaker 2: Imagine there are things to keep in.
Speaker 1: Mind, right, there's to be some catches.
Speaker 2: One of the key points the book highlights is choosing
Speaker 2: the right partition columns.
Speaker 1: Columns.
Speaker 2: These are the criteria that you use to divide your
Speaker 2: data into those subsets. You need to think carefully about
Speaker 2: which factors are most likely to influence the target variable
Speaker 2: that you're trying to predict.
Speaker 1: So it's like making sure you're slicing and dicing your
Speaker 1: data along the right lines so that the models you
Speaker 1: create are actually meaningful and relevant precisely. Now, once you've
Speaker 1: trained all these models using the MMSA, you still need
Speaker 1: to deploy them. You do, and that can get even
Speaker 1: more complex when you're dealing with thousands of models.
Speaker 2: It can instead is just one it does.
Speaker 1: So how do you manage all of that?
Speaker 2: The book really stresses the importance of automation here. Okay,
Speaker 2: It introduces a tool called Azure Data.
Speaker 1: Factory as your data factor, which can.
Speaker 2: Help orchestrate complex data flows, connect to various data sources,
Speaker 2: transform data, and even trigger those mL pipelines that we
Speaker 2: talked about earlier.
Speaker 1: So if those mL pipelines are like assembly lines for
Speaker 1: building our model. Yeah, then Azure Data Factory is like
Speaker 1: the logistics manager, making sure all the raw materials and
Speaker 1: finished products are flowing smoothly, perfect. And in addition to
Speaker 1: all this, the book is packed with helpful tips and
Speaker 1: tricks for getting the most out of AUTOMML. It is
Speaker 1: regardless of the type of problem that you're tackling. Yes,
Speaker 1: I love that practical advice. It's like having a seasoned
Speaker 1: auto mL expert looking over your shoulder, absolutely guiding you
Speaker 1: along the way exactly.
Speaker 2: And the book really encourages a spirit of experimentation. Okay,
Speaker 2: it wants you to be a data detective. Oh I
Speaker 2: like that, to try different approaches and see what works
Speaker 2: best for your situation.
Speaker 1: That's what makes data science so exciting. It's not about
Speaker 1: blindly following rules. It's about exploring, discovering, and finding creative solutions.
Speaker 2: Absolutely.
Speaker 1: We've talked a lot about az your machine Learning studio,
Speaker 1: which has that visual, user friendly interface, But for those
Speaker 1: who prefer to work with code, the book also covers
Speaker 1: the Azure Machine Learning SDK for Python. It does so
Speaker 1: for those who are comfortable covading, there's a way to
Speaker 1: get even more control and flexibility over that auto mL
Speaker 1: process there is, okay, and the book walks us through
Speaker 1: using Jupiter notebooks within Asure machine Learning, which is a
Speaker 1: very popular way to write and execute Python code for
Speaker 1: data science tasks. It is, and you can actually use
Speaker 1: those powerful compute clusters that we talked about earlier to
Speaker 1: run your code in the cloud, yes, giving you access
Speaker 1: to tons of processing power.
Speaker 2: Absolutely.
Speaker 1: It really is amazing how cloud computing has made these
Speaker 1: really complex tasks so much more accessible. It has. Now
Speaker 1: I'm curious about the different ways that you can actually
Speaker 1: deploy models once they're trained. We talked about batch scoring
Speaker 1: and real time scoring, yes, but the book also mentioned
Speaker 1: something called mL pipelines.
Speaker 2: mL pipelines are a fantastic way to automate your entire
Speaker 2: machine learning workflow, okay, from data preprocessing and feature engineering
Speaker 2: to model training and deployment.
Speaker 1: So it's like having an assembly line for your AI,
Speaker 1: ensuring that each step is executed in the right order,
Speaker 1: with the right settings. Absolutely, automation, efficiency, consistency. That's it
Speaker 1: sounds like a well oiled machine exactly.
Speaker 2: And to take it a step further, the book introduces
Speaker 2: us to Azure Data.
Speaker 1: Factory Azure Data Factory.
Speaker 2: This is a cloud based data integration service okay that
Speaker 2: can handle even more complex data flows, connecting to different
Speaker 2: data sources, got it, transforming your data okay, and even
Speaker 2: triggering your mL pipelines.
Speaker 1: So if mL pipelines are the assembly lines, that Azure
Speaker 1: Data Factory is like the logistics manager making sure that
Speaker 1: all the raw materials and the finished products are flowing smoothly.
Speaker 2: A great way to put it.
Speaker 1: Now, we've covered a lot of ground here, from the
Speaker 1: really medi gritty details of data preparation and feature engineering
Speaker 1: to the broader concepts of model deployment and automation. We have,
Speaker 1: but let's not forget about one really crucial aspect, the
Speaker 1: human element. Building a model is only part of the story. Yeah,
Speaker 1: what about gaining the trust and buy in of the
Speaker 1: people who will actually be using these models.
Speaker 2: That's a great point. You know, we can build the
Speaker 2: most sophisticated AI in the world, but if people don't
Speaker 2: trust it or understand how.
Speaker 1: It works, what good is it.
Speaker 2: It's not going to be very useful, right, And that's
Speaker 2: why those explainability features we keep talking about are so important.
Speaker 2: Extremely the book emphasizes the need to clearly articulate how
Speaker 2: a model is making decisions, especially in industries that have
Speaker 2: strict regulations, got it, or where the stakes.
Speaker 1: Are high, right, like healthcare of finance exactly. So it's
Speaker 1: not enough to just say the computer says this is
Speaker 1: the best course of action. We need to be able
Speaker 1: to back that up with insights and evidence.
Speaker 2: Absolutely, and auto mL gives us the tools to do
Speaker 2: just that. You can use feature importance scores to see
Speaker 2: which factors are driving the model's predictions. You can even
Speaker 2: drill down into individual predictions to understand why the model
Speaker 2: made a specific decision.
Speaker 1: So it's like having a transparent AI where we can
Speaker 1: peek under the hood and see what's going on exactly. Now,
Speaker 1: before we wrap up this part of our deep dive,
Speaker 1: I want to highlight something that really stood out to
Speaker 1: me while reading the book. It mentions that auto mL
Speaker 1: can actually be used in other Microsoft products besides Azure
Speaker 1: Machine Learning Studio. Yes, oh, so it's not just limited
Speaker 1: to this one platform, not at all.
Speaker 2: The book talks about integrating auto mL with tools like
Speaker 2: Powerbi Powerbi which is a powerful data visualization and business
Speaker 2: intelligence platform, So you.
Speaker 1: Could create these interactive dashboards that not only display the data,
Speaker 1: but use auto mL to generate predictions and insights. Yes,
Speaker 1: that's next level. That takes data storytelling to a whole
Speaker 1: new level. It does. And the book also touches on
Speaker 1: using auto mL with Azure Synapse Analytics, which is a
Speaker 1: cloud based data warehousing and analytics.
Speaker 2: Service exactly, and this opens up even more possibilities. Wow,
Speaker 2: for working with massive data sets and building these enterprise
Speaker 2: scale AI solutions.
Speaker 1: It sounds like the possible are pretty much endless. They are.
Speaker 1: We've covered so much ground, but it feels like we've
Speaker 1: only just begun to scratch the surface of what auto
Speaker 1: mL on Azure can do.
Speaker 2: We've just scratched the surface.
Speaker 1: So as we move into the next part of our
Speaker 1: deep dive, I'm curious where should someone start or if
Speaker 1: they want to dive in and explore this world of
Speaker 1: auto mL on Azure.
Speaker 2: Well, this book we've been discussing is an excellent starting point. Okay,
Speaker 2: Dennis Sawyer's has done a really great job of creating
Speaker 2: a practical and easy to follow guide. I agree, pact
Speaker 2: with real world examples, code snippets, and tons of helpful advice.
Speaker 1: And there are tons of online resources, including Microsoft's own
Speaker 1: documentation and tutorials exactly. And plus the Azure community is
Speaker 1: incredibly active and supportive.
Speaker 2: It is.
Speaker 1: It's great. So if you have questions or get stuck,
Speaker 1: you'll find plenty of people willing to help you will.
Speaker 1: I think the key takeaway here is that AutoML is
Speaker 1: not some futuristic concept.
Speaker 2: No it's not.
Speaker 1: It's here here, it's here now.
Speaker 2: It's now a powerful tool.
Speaker 1: It's a powerful tool that's available right now.
Speaker 2: And it's making AI more accessible.
Speaker 1: Than ever before. And with the power of auto mL
Speaker 1: on Azure, anyone can become an AI innovator. So to
Speaker 1: our listener, we challenge you, what problem will you solve
Speaker 1: with auto mL. That's a great question, that's something to
Speaker 1: think about. Think about it. We'll be back in just
Speaker 1: a moment with part two of our deep dive into
Speaker 1: AutoML on Azure. Welcome back to our deep dive into
Speaker 1: auto mL on Azure. Now, before the break, we were
Speaker 1: talking about how auto mL can be used in other
Speaker 1: Microsoft products. Yes, besides just as your machine learning studio.
Speaker 1: So it's not just limited to that one platform, not
Speaker 1: at all, Okay, So tell me more.
Speaker 2: The book talks about integrating auto mL with tools like
Speaker 2: powerbi powerbi okay, which is a powerful data visualization and
Speaker 2: business intelligence platform.
Speaker 1: So we can use AutoML to create these really interactive
Speaker 1: dashboards you got it that not only display d data
Speaker 1: but also generate predictions and insights.
Speaker 2: Exactly.
Speaker 1: It's awesome.
Speaker 2: It takes data storytelling to a whole new level.
Speaker 1: It does, Yeah, it does. And the book even touches
Speaker 1: on using AutoML with Azure SYNEPS Analytics, which is a
Speaker 1: cloud based data warehousing and analytics service. Wow, so many options.
Speaker 1: So this opens up even more possibilities for working with
Speaker 1: massive data sets, yes, and building these enterprise scale AI solutions.
Speaker 1: It sounds like AutoML is becoming this really integral part
Speaker 1: of the entire Microsoft ecosystem. It is, so it's not
Speaker 1: just this standalone tool, it's really integrated into all these
Speaker 1: different products and services exactly. Yeah, okay. Now, speaking of
Speaker 1: getting hands on with AutoML, the book goes beyond just
Speaker 1: talking about the concepts. It actually dives into using Azure
Speaker 1: Machine Learning Studio Yes, which, as we discussed earlier, is
Speaker 1: that more visual, user friendly interface for working with AMLS.
Speaker 1: It is, and it uses a classic example, the Titan
Speaker 1: Passenger Data.
Speaker 2: Oh, the Titanic data set.
Speaker 1: Predicting who survived and who didn't based on factors like age, gender,
Speaker 1: ticket class.
Speaker 2: Right, a classic problem.
Speaker 1: It's like a rite of passage for anyone learning data science. Exactly.
Speaker 1: I remember working with that data set in my early day.
Speaker 1: So the book actually walks you through the entire process,
Speaker 1: making it surprisingly straightforward.
Speaker 2: It does. It makes it really easy.
Speaker 1: So so you upload your data, tell AutoML it's a
Speaker 1: classification task, and let it do its thing.
Speaker 2: Yeah, let it work its magic, exactly.
Speaker 1: And while it's working its magic, we can keep an
Speaker 1: eye on those data guard rails to make sure nothing
Speaker 1: fishy is going on with our data exactly, right, We've
Speaker 1: got our data quality watchdog. We do. Now, once auto
Speaker 1: mL has finished training a whole bunch of models, it's
Speaker 1: evaluation time, right. Okay, So we have all these models
Speaker 1: all trained and ready to go. How do we pick
Speaker 1: the best one?
Speaker 2: So auto mL gives us a whole Smorgus board of
Speaker 2: metrics to consider. Okay, you know, accuracy, decision recall.
Speaker 1: So many metrics.
Speaker 2: There are a lot.
Speaker 1: Sometimes I feel a little overwhelmed, especially if you're new
Speaker 1: to machine learning.
Speaker 2: Sure, I understand that there's just so many numbers, right,
Speaker 2: but the book breaks down each metric, explaining what it
Speaker 2: means and when it's most relevant.
Speaker 1: Okay, that's helpful.
Speaker 2: Yeah, and it also reminds us to use those explainability features, oh,
Speaker 2: to really understand why a model is making certain predictions.
Speaker 1: Right. It's not enough to just blindly trust the numbers exactly.
Speaker 2: We need more.
Speaker 1: It's about understanding the logic behind those numbers. Yes, making
Speaker 1: sure the model is making decisions for the right reasons.
Speaker 1: That's right. Okay. So let's say we've found a model
Speaker 1: that looks good both in terms of its performance and
Speaker 1: its explainability.
Speaker 2: Okay, what's next deployment? It's time to take that model
Speaker 2: out of the training ground and put it to work
Speaker 2: in the real world.
Speaker 1: Okay, so we're moving from theory to practice. Yes, but
Speaker 1: how do we actually deploy a model?
Speaker 2: So the book gives us a couple of options. Okay,
Speaker 2: there's batch scoring, which is great for processing large amounts
Speaker 2: of data at scheduled intervals, and then there's real time scoring,
Speaker 2: which is ideal for when you need those instant predictions,
Speaker 2: like in fraud detection or personalized recommendation.
Speaker 1: So batch scoring is like sending out a monthly newsletter. Yes,
Speaker 1: while real time scoring is like having a live chat
Speaker 1: with your customers.
Speaker 2: That's a great analogy.
Speaker 1: Right, Different tools for different needs. Absolutely. Up until now
Speaker 1: we've mostly focused on classification models, right, where the goal
Speaker 1: is to predict a category. But the book doesn't stop there, No,
Speaker 1: it doesn't. It also dives into regression models, okay, where
Speaker 1: the goal is to predict a numerical value.
Speaker 2: Right, like predicting the price of a house exactly, or
Speaker 2: the number of sales you'll make next month, right.
Speaker 1: Or in the case of the diabetes data set that
Speaker 1: we mentioned earlier, predicting the progression of the disease exact.
Speaker 1: So the process is very similar, but obviously the metrics
Speaker 1: and the algorithms we use are different.
Speaker 2: Right, Different tools for different jobs.
Speaker 1: But auto mL still handles all of that heavy lifting
Speaker 1: for us. It does, which is great, but we still
Speaker 1: need to understand which tools are the best for the
Speaker 1: task at hand.
Speaker 2: Absolutely.
Speaker 1: Now, I bet there are some special tips and tricks
Speaker 1: for getting the most out of auto mL. There are,
Speaker 1: specifically for regression problems. Yes, so spill the beans. What
Speaker 1: are they? Well?
Speaker 2: The book shares some really interesting insights, Okay. For instance,
Speaker 2: sometimes it can actually improve performance to.
Speaker 1: Convert a regression problem into a classification problem.
Speaker 2: Hold on, how do you do that? How do you
Speaker 2: convert a problem about predicting a number into a problem
Speaker 2: about predicting a category?
Speaker 1: It's all about binning dah. Okay, you divide the range
Speaker 1: of possible numerical values into categories or vins. Okay, So
Speaker 1: instead of predicting the exact price of a house, you
Speaker 1: might predict whether it falls into a low, medium, or
Speaker 1: high price range.
Speaker 2: So it's like simplifying the problem a bit to make
Speaker 2: it easier for auto and mlifying those patterns exactly. Okay,
Speaker 2: I see any other tips.
Speaker 1: The book also emphasizes the importance of experimenting with different
Speaker 1: primary metrics.
Speaker 2: Okay, right.
Speaker 1: For example, you might find that a metric like mean
Speaker 1: absolute error or MAE gives you a more meaningful evaluation
Speaker 1: than something like root means squared error or URMSK. It
Speaker 1: really depends on what you're trying to achieve with your model.
Speaker 2: So it's not just about blindly applying the same metrics
Speaker 2: every time.
Speaker 1: No, not at all.
Speaker 2: You need to really think about what's most relevant for
Speaker 2: your specific problem.
Speaker 1: Okay, So there's still a lot of room for human
Speaker 1: judgment and decision making, even with auto mL doing so
Speaker 1: much of the work. Absolutely, Now I have to ask,
Speaker 1: what about those situations where you need to train not
Speaker 1: just one or two models, but hundreds, maybe even thousands.
Speaker 2: Ah, the many models problem?
Speaker 1: Right. It sounds daunting, it can.
Speaker 2: Be, but that's where the book really shines.
Speaker 1: Okay.
Speaker 2: It introduces this tool called the Many Models Solution Accelerator
Speaker 2: or MMSA for sure, MMSA. Think of it as an
Speaker 2: auto mL factory.
Speaker 1: Okay. I like that.
Speaker 2: You feed it your data and it automatically splits it
Speaker 2: up based on certain criteria like different stores, products, or regions.
Speaker 1: Right, so you can create really customized models for each
Speaker 1: specific situation exactly. Okay. So when would someone actually need
Speaker 1: to create so many models?
Speaker 2: It's ideal for when you need those really granular customized models.
Speaker 1: Okay.
Speaker 2: The book gives an example of a retail chain okay
Speaker 2: that wants to forecast demand for each product in each store.
Speaker 1: Ah, So they need a separate model for each product
Speaker 1: in each store.
Speaker 2: Yes, And by using the MMSA, they can create thousands
Speaker 2: of custom models that take into account all of the
Speaker 2: unique factors that might influence sales at each location.
Speaker 1: That's incredibly powerful. It is, but I imagine there are
Speaker 1: some considerations when working with this MMSA.
Speaker 2: Definitely, and one of the key points the book highlights
Speaker 2: is choosing the right partition.
Speaker 1: Columns, partision calls. What are those?
Speaker 2: These are the criteria you use to divide your data
Speaker 2: into those subsets. Okay, you need to think carefully about
Speaker 2: which factors are most likely to influence the target variable
Speaker 2: you're trying to predict.
Speaker 1: So it's like making sure you're slicing your data along
Speaker 1: the right lines exactly, so the models you create are
Speaker 1: actually meaningful and relevant. That's it. Now, once you've trained
Speaker 1: all these models using the MMSA, you still need to
Speaker 1: deploy them right you do, and that can get pretty
Speaker 1: complex when you're dealing with thousands of models. It's just one, yeah,
Speaker 1: so how do you manage all of that?
Speaker 2: The book really stresses the importance of automation here. It
Speaker 2: introduces us to a tool called Azure Data.
Speaker 1: Factory as your data Factory, which.
Speaker 2: Can help orchestrate complex data flows, connect to different data sources,
Speaker 2: transform data, and even trigger those mL pipelines that we
Speaker 2: talked about earlier.
Speaker 1: So if the mL pipelines are like the assembly lines
Speaker 1: for our models. Yeah, then data Factory is like the
Speaker 1: logistics manager.
Speaker 2: That's a great way to put it.
Speaker 1: Making sure everything runs smoothly, yes, okay. And in addition
Speaker 1: to all this, the book is packed with helpful tips
Speaker 1: and tricks for getting the most out of AutoML no
Speaker 1: matter what kind of problem you're working on.
Speaker 2: It is. It's full of practical advice.
Speaker 1: It's like having a seasoned auto mL expert looking over
Speaker 1: your shoulder guiding you along the way.
Speaker 2: Absolutely, and the book really encourages that spirit of experimentation, right.
Speaker 1: It's all about trying new things.
Speaker 2: Exactly, being a data detective.
Speaker 1: That's what I love about data science. It's not just
Speaker 1: about following rules, it's about exploring and discovering.
Speaker 2: Absolutely.
Speaker 1: Now we've talked a lot about Azure Machine Learning Studio,
Speaker 1: which has that more visual, user friendly interface, but for
Speaker 1: those who are comfortable with coding, the book also covers
Speaker 1: the Azure Machine Learning SDK for Python. It does, yes,
Speaker 1: so for those who are comfortable coding, there's a way
Speaker 1: to get even more control and flexibility over the AutoML process.
Speaker 1: Exactly okay, And the book walks us through using Jupiter
Speaker 1: Notebooks within Azure machine Learning Yes, which is a very
Speaker 1: popular way to write and execute Python code for data
Speaker 1: science tasks.
Speaker 2: It is very popular.
Speaker 1: And you can actually use those powerful compute clusters that
Speaker 1: we talked about earlier to run your code in the cloud, Yes,
Speaker 1: which gives you access to a ton of processing power. Absolutely,
Speaker 1: it really is amazing how cloud computing has made these
Speaker 1: complex tasks so much more accessible.
Speaker 2: It really is.
Speaker 1: Now I'm curious about the different ways you can actually
Speaker 1: deploy models once they're trained. Okay, we've talked about batch
Speaker 1: scoring and real time scoring, but the book also mentions
Speaker 1: something called mL pipelines.
Speaker 2: mL pipelines are a fantastic way to automate your entire
Speaker 2: machine learning workflow, from data preprofitsing and feature engineering to
Speaker 2: model training and deployment.
Speaker 1: So it's like building an assembly line for your AI,
Speaker 1: making sure each step is done in the right order, exactly,
Speaker 1: the right settings, automation, efficiency, consistency. It sounds like a
Speaker 1: well oiled machine exactly.
Speaker 2: And to take it even further, the book introduces another
Speaker 2: powerful tool as your Data Factory.
Speaker 1: As your data Factory okay.
Speaker 2: This cloud based data integration service can handle even more
Speaker 2: complex data flows. Okay, connected different data sources transform your
Speaker 2: data and even trigger your mL pipelines.
Speaker 1: So if mL pipelines are the assembly lines, then as
Speaker 1: your data factory is like the logistics manager making sure
Speaker 1: all the raw materials and finished products are flowing smoothly.
Speaker 2: Perfect analogy, right.
Speaker 1: It's all about streamlining and automating as much of the
Speaker 1: process as possible. It is now, we've covered a lot
Speaker 1: of ground here, from the nitty gritty details of data
Speaker 1: preparation and feature engineering to the broader concepts of Muddel
Speaker 1: deployment and automation. We have, but let's not forget about
Speaker 1: one crucial aspect, the human element. Building a model is
Speaker 1: only part of the story. It is what about gaining
Speaker 1: the trust and buy in of the people who will
Speaker 1: actually be using these models.
Speaker 2: That's a great point. You know, we can build the
Speaker 2: most sophisticated AI in the world, but if people don't
Speaker 2: trust it or understand how.
Speaker 1: It works, it's not going to be very useful.
Speaker 2: Exactly, and that's why those explainability features that we keep
Speaker 2: talking about are so important.
Speaker 1: Right. Transparency is key. It is, especially when you're dealing
Speaker 1: with decisions that could have a real impact on people's lives.
Speaker 2: Absolutely. The book emphasizes the need to be able to
Speaker 2: clearly articulate how a model is moving decisions, especially in
Speaker 2: those industries with strict regulations or where the stakes are.
Speaker 1: High, right, like healthcare or finance.
Speaker 2: Exactly.
Speaker 1: You need to be able to explain why the model
Speaker 1: is recommending a certain treatment or making a certain investment decision.
Speaker 1: You do. It's not enough to just say the computer
Speaker 1: says so right.
Speaker 2: We need more than that.
Speaker 1: You need to be able to back it up with
Speaker 1: evidence and insights.
Speaker 2: And AUTOMML gives us the tools to do just that.
Speaker 1: That's good. You know.
Speaker 2: You can use those feature important scores to see which
Speaker 2: factors are driving the model's predictions. Okay, you can even
Speaker 2: drill down into those individual predictions to understand why the
Speaker 2: model made a specific decision.
Speaker 1: So it's like having this transparent AI exactly where you
Speaker 1: can peek under the hood and see what's going on.
Speaker 2: That's it.
Speaker 1: Now, before we wrap up, I want to highlight something
Speaker 1: that really stood out to me while reading the book. Okay,
Speaker 1: it mentions that AUTOMML can actually be used in other
Speaker 1: Microsoft products besides just Azure Machine Learning Studio. Yes, so
Speaker 1: it's not just limited to this one platform, not at all.
Speaker 2: Tell me more So, the book talks about integrating AutoML
Speaker 2: with tools like powerbi Powerbio, which is that powerful data
Speaker 2: visualization and business intelligence platform.
Speaker 1: So you could create these interactive dashboards that not only
Speaker 1: display the data, yeah, but also use auto mL to
Speaker 1: generate predictions and insights.
Speaker 2: Exactly.
Speaker 1: That's amazing. It's like taking data storytelling to a whole
Speaker 1: new level.
Speaker 2: It is. And the book also touches on using auto
Speaker 2: mL with Azure synaps Analytics.
Speaker 1: Okay, which is that cloud based data warehousing and analytics service. Right,
Speaker 1: So this opens up even more possibilities for working with
Speaker 1: massive data sets, right and building these enterprise scale AI
Speaker 1: solutions Exactly. It sounds like the possibilities are pretty much endless.
Speaker 1: They are.
Speaker 2: The possibilities are endless.
Speaker 1: We've covered so much ground, but it feels like we've
Speaker 1: only just scratch the surface of what AutoML on Azure
Speaker 1: can do.
Speaker 2: We've just scratched the surface.
Speaker 1: So as we move into the final part of our
Speaker 1: deep dive, Okay, I'm curious where should someone start if
Speaker 1: they want to explore this world of AutoML on Azure.
Speaker 2: Well, this book we've been discussing. Is an excellent place
Speaker 2: to begin.
Speaker 1: Right, It's a great starting point.
Speaker 2: Dennis Sawyers did a phenomenal job creating this really practical
Speaker 2: and easy to follow guide.
Speaker 1: I agree.
Speaker 2: It's packed with real world examples, code snippets, and tons
Speaker 2: of helpful advice.
Speaker 1: And of course there are tons of online resources out there,
Speaker 1: including Microsoft's owned documentation and tutorials. There are plus the
Speaker 1: Azure community is incredibly active and supportive. It is.
Speaker 2: It's a great community, So.
Speaker 1: If you have questions, you'll definitely find someone who can
Speaker 1: help you will. I think the key takeaway here is
Speaker 1: that auto mL is not some futuristic concept. No it's not.
Speaker 1: It's here. It's here now, it's now.
Speaker 2: It's a powerful tool.
Speaker 1: It's a powerful tool that's available right now.
Speaker 2: And it's making AI more accessible than ever before.
Speaker 1: Exactly, and with the power of AUTOMML on Azure, anyone
Speaker 1: could become an AI innovator.
Speaker 2: Anyone can.
Speaker 1: So to our listener, we challenge you, what problem will
Speaker 1: you solve with auto mail?
Speaker 2: That's question?
Speaker 1: What will you create?
Speaker 2: What will you create?
Speaker 1: We'll be back in a moment with the final part
Speaker 1: of our deep dive into auto mL on Asure welcome
Speaker 1: back to the final part of our deep dive into
Speaker 1: auto mL on Asure. You know, this book has really
Speaker 1: opened my eyes to the potential of auto mL.
Speaker 2: It really is a game changer, and.
Speaker 1: Azure seems like the perfect platform. Oh, absolutely to really
Speaker 1: explore it. It is now for those who want to
Speaker 1: kind of delve a little deeper into those technical aspects. Yeah,
Speaker 1: the book actually doesn't shy away from the code, No
Speaker 1: it doesn't. It guides you through using the Azure Machine
Speaker 1: Learning SDK for Python. It does so for those who
Speaker 1: are comfortable with coding, there's a way to get even
Speaker 1: more granular control over the auto mL process.
Speaker 2: Absolutely, the book that shows you how to use Jupiter
Speaker 2: notebooks within Azure machine Learning, which is a really popular
Speaker 2: way to write and execute Python code for data science.
Speaker 1: And you can even use those powerful compute clusters that
Speaker 1: we talked about yes to run your code in the
Speaker 1: cloud exactly, tapping into massive processing power.
Speaker 2: And all that processing power.
Speaker 1: It's incredible how cloud computing has really made these complex
Speaker 1: tasks so much more accessible.
Speaker 2: It really has.
Speaker 1: I'm curious about the practical side of things. Okay, we've
Speaker 1: trained our models, but how do we actually put them
Speaker 1: to work right. The book mentions batch scoring and real
Speaker 1: time scoring, but it also talks about something called mL pipelines.
Speaker 2: Yes, mL pipelines are a fantastic way to automate that
Speaker 2: entire machine learning workflow. Okay, from data preprocessing and feature
Speaker 2: engineering to model training and deployment.
Speaker 1: So it's like creating an assembly line for your AI exactly.
Speaker 1: Each step is executed in the right order with the
Speaker 1: right settings.
Speaker 2: Yes, automation, efficiency, consistency.
Speaker 1: Yeah, it sounds like a well machine. It is. And
Speaker 1: to take even a step further, the book introduces us
Speaker 1: to Azure Data Factory. Yes, another great tool, this cloud
Speaker 1: based data integration service that can handle even more complex
Speaker 1: data flows. It can connect to different data sources, transform
Speaker 1: your data, and even trigger those mL pipelines exactly. So
Speaker 1: if mL pipelines are the assembly lines, then az your
Speaker 1: Data factory is like the logistics manager making sure that
Speaker 1: everything's flowing smoothly.
Speaker 2: A perfect analogy.
Speaker 1: Now, this book has given us a really fantastic overview
Speaker 1: of auto mL on Azure. It has, and it's clear
Speaker 1: that this technology has the potential to really revolutionize the
Speaker 1: way we approach AI.
Speaker 2: It really does.
Speaker 1: But as we wrap up our deep dive. Yeah, what's
Speaker 1: the one key message you hope our listener takes away
Speaker 1: from all of this.
Speaker 2: I think the biggest takeaway is that auto mL is empowering.
Speaker 2: It breaks down those barriers to entry for AI, making
Speaker 2: it accessible to a much wider audience. I agree, you
Speaker 2: don't need to be a data science guru to build
Speaker 2: powerful models on Azure. This book provides the guidance and
Speaker 2: the tools you need to get started, and with the power.
Speaker 1: Of auto mL on Azure, anyone can become an AI innovator.
Speaker 1: Absolutely so to our listeners, we encourage you dive in,
Speaker 1: explore the world of auto mL and see what incredible
Speaker 1: things you can achieve.
Speaker 2: Until next time, happy innovating.