Speaker 1: How'd you like to listen to dot NetRocks with no ads? Easy?
Become a patron For just five dollars a month you
get access to a private RSS feed where all the
shows have no ads. Twenty dollars a month will get
you that and a special dot NetRocks patron mug. Sign
up now at Patreon dot dot NetRocks dot com. Hey,
Carl and Richard here with your twenty twenty four NDC schedule.
Speaker 2: Will be at as many NDC conferences as possible this year,
and you should consider attending no matter what. The Copenhagen
Developers Festival happens August twenty sixth through the thirtieth. Tickets
at Cphdevfest dot com.
Speaker 1: NDC Porto is happening October fourteenth through the eighteenth. The
early bird discount ends June fourteenth. Tickets at Ndcporto dot com.
Speaker 2: And we'll see you there, we hope.
Speaker 1: Hey, guess what it's dot net rocks again. I'm Carl
Franklin and I'm Richard Campbell, and this is episode nineteen nineteen.
And Buddy, I went and I looked up you know
what happened in this year in history? So some significant
events included the collapse of the Habsburg and German empires.
Speaker 2: Yes, the end of World War One, end of World War.
Speaker 1: One, the redrawing of the map of Europe at the
Paris Peace Conference, the influence of Woodrow Wilson's fourteen points.
Speaker 2: Interesting they call it the Paris Peace Conference. It was
the Versiah Cords, and they were not They were not very.
Speaker 1: Peaceful, not very peaceful.
Speaker 2: Yeah, they were a very good setup for World War two, yep.
Speaker 1: And sowing the seeds of World War two, growing demands
for self determination by smaller nations, the growing involvement of
the United States and European politics and trade all that. Yeah.
Speaker 2: Yeah, one would argue this is the ess of the
nation stage. Yeah, basically the model we've been functioned ever since.
Speaker 1: I bet you didn't know Richard Campbell knew so much
about history. O. Who am I kidding? You knew you knew?
Go ahead and roll the music, because I got a
pretty good better no framework, awesome? All right? What do
you got? So you know, I'm cutting down on travel
to conferences. At the end of the year, I'm not
going outside the country. I'm not even sure that there
are any more domestic conferences that I'm going to at
the end of this year, but I still want to
do some Blazer hands on training.
Speaker 2: Right.
Speaker 1: So what I've done is if you go to APPVNX
dot com, slash training or nineteen nineteen dot po dot me,
you'll see the training page which links to a form
that you can fill out. I've decided I want to
do these online classes on Mondays. Oh yeah, mostly Mondays
from now until the end of the year, most of
them except for you know, some week off here and there.
And this is a form where you can select the
dates that you want to attend. Each Monday, we'll have
one starting at nine am Eastern and nine am Pacific,
so that you know, if more people want to do
in a Pacific time blah blah blah. But the deal
is that once we hit that magic number of ten,
then the class is going to happen, Right, So you
should select all of the dates that you can possibly make,
and then there's a greater chance that one of them.
Speaker 2: Will will take But online classes.
Speaker 1: Will online classes, but you know it's interactive. You'll be
able to do the hands on work and even though
we won't get to all the materials in one day,
you still get all the materials right so you can
go back on your own time.
Speaker 2: Well in the class size sounds like assuming nice as
small too, so people go be able to interact a lot.
Speaker 1: Well, ten is the minimum. I haven't really set a
maximum yet, but of course if worry about that problem
when it happened, I'll worry about that problem. So that's
what it is at Phoenix dot com slash training. Who's
talking to us Richard crawbd to.
Speaker 2: Comment on top of show nineteen sixteen, So just last
one published. To be honest, this is the show we
did with Mark Rendalls. We called how Simple is as
Simple as Possible? And in finding conversation with Mark, as
always he's a thoughtful guy as well as very funny.
I think we spent most of the show laughing. And
Rob King has this great comment. He said, once again
another great episode which reaffirms my position that everything old
is new again. I feel like there's a whole generation
of developers coming up with only known JavaScript frontends and
think the server is only for APIs or serving the JavaScript.
I've started going back to a server side first approach
of building apps, keeping get all in the server and
then build it out towards the client only required. Yes,
Lazer made this is really simple, and it's approached to
adding interactivity as required. I've even converted a Blazer WASM
app to pure service side because it's actually faster than
the first page and the app doesn't need any interactivity
beyond the forms. To that end, I've found that you
can integrate htmx something that Mark talked about right seamless
see into Blazer server and a built out of a
little demo repo which I'll include in the show notes
to Rob's GitHub repository. It's just a sort of hey,
these two things will play together, Blazer and HGMX.
Speaker 1: Yeah, there you go.
Speaker 2: You know, it's just sort of a sort of recognition
of yeah, we're kind of calling in a cycle. Like
there's clearly cases for putting more load on the client,
but one would argue we swung too far that way. Yes,
although I know as we get into these conversations about
large language models and machine learning so forth, there's now
a big push about how do you get more of
that workload on the client because folks are concerned now
that too much of that workload is on the server.
So I think it'll be some good talking points for
today's show.
Speaker 1: I'm really happy that a guy like Mark Render, who
you know, go back and listen to his stuff before
a Blazer came out. He was not really a service
side guy at all, like very much in the in
the composable c I CD pipeline kind of thing and
and all you know JavaScript, and a guy like Mark
comes out and says, you know what, I took Blazer
server for a ride and I really like it.
Speaker 2: Yeah, it's really great. Well, you know, and again his whole,
that whole how simple is simple is getting back to
the intention of what we're trying to do for the
customer exactly right, Like I do think we get enamored
of the technology. And again I brought this show and
these comments out because we're about to go down a
path about a whole other class of development, and I
think it's good to push back on those fundamentals and say,
what are we trying to deliver to customers?
Speaker 1: Yep, no pressure pre shant, Yeah for sure.
Speaker 2: So Rob, thank you so much for your comment and
a copy of Music co Buy. It's on its way
to you. And if you'd like a copy of Music
co By, I write a comment on the website at
dot at rocks dot com or on the facebooks. We
publish every show there, and if you comment there and
never read on the show, we'll send you a copy
of Music to.
Speaker 1: Go By, Music to Code by, Still Going Strong. I
just sold another Flak collection, which is good, and I
got inspired last night to do a new one.
Speaker 2: How many how many have you made?
Speaker 1: Is it like twenty one?
Speaker 2: Twenty one? Okay, so you know you're digging around for
another track? Huh, that's awesome.
Speaker 1: Well, I was listening to it last night. I just
decided to.
Speaker 2: I don't know if you've heard it. Stuff.
Speaker 1: It's pretty good. It is pretty good. Yeah, I've been
listening for a long time. Yeah, I mean I started.
I was listening to the more recent ones okay last
night and I was like, yeah, you know, this is
pretty good stuff. I think I got inspired to do
another one.
Speaker 2: Hey, you should meet that guy. He's pretty clever.
Speaker 1: Yeah, so we'll see. I'm not announcing it now, I'm
just announcing my intent.
Speaker 2: You got it?
Speaker 1: Some itch?
Speaker 2: I like when you have an itch? Yeah, stuff happens
when you have an itch. Yeah yeah.
Speaker 1: Well, anyway, you can also follow us on ex Twitter.
We've been there for years, but the cool kids are
hanging out. I'm Astdon, I'm Carl Franklin at tech Hub dot.
Speaker 2: Social, and I'm Rich Campbell at master do Social.
Speaker 1: Send us a two. That's another way you can get
a copy of music to code buy. Okay, let us
introduce Prashant Boyar. He is six times AI MVP, three
times Business Applications MVP, Microsoft Certified Trainer, speaker, author, and
leading AI architect from the Washington, DC area of the
United States. Currently, he works as an AI architect in
the Office of the Chief Technology Officer at Applied Information
Sciences That's AI at dot com. Braschant is on the
leadership committee for AI Fest, AI and mL User Group
and the Northern Virginia Data Platform User Group. As a
renowned international speaker, he frequently presents a tech conferences. Additionally,
Prashant was awarded the Antarctic Service Medal of the United
States of America for his outstanding service in Antarctica. You know,
I was going to tease you for not having enough
stuff on your CV, but let's go right to Antarctica.
Speaker 2: What was that?
Speaker 3: So, first of all, thank you very much for having me.
That was once in a lifetime experience for me. Like
I grew up in India and two thousand and seven
is the year I came to the United States for
my master's degree and I got I was really fortunate
that I got a scholarship in the university and as
part of that scholarship, I got a chance to work
with a lot of brilliant scientists. And those scientists were
studying the costmes raise and everyone in that room except
me was like really brilliant. So I had to, you know,
fit in and I know, and you.
Speaker 1: Might be a little unbiased towards yourself.
Speaker 3: Now, Like if you if you talk to any experimental
physics physicists or in theoretical physicists, not only they know
software development really well, but they also know all this
machine learning match and some advanced mathematics really well. So
I have like after that, I have really high respect
for all these physicists because for the work they do.
And if you compare the salaries against that, you will
be surprised.
Speaker 2: Yeah, they don't get paid much for yes, as hard
as they as they work. Antarctica is a great location
for doing stuff like measuring comic cause of rays. You know,
the atmosphere is a little thinner, and it's very very dark,
and there's a minimum amount of noise, but it's and
it's sad. That's like the best time to do that
work is midwinter of Antarctica where there's no sunlight for months,
so for the tough place to be for shann you
hung out there.
Speaker 3: Yes, So as part of that experiment, like we built,
it was a joint collaboration WITHWEN multiple universities, and ASA
was the main sponsor because typically NASA sponsors this, you know,
give this huge grant to the unisity so that they
can do research on their behalf. And as part of
the research, you know, the best results to get is
via satellite, but you know, sending any payload to satellite
is very expensive. So they came up with an approach
about how we send this really large duration balloons in
Antarctic continent which fly around you know the south you know,
South Pole for around twenty to eighty days, so not
really twenty to thirty days, collect as much data as possible,
and then we take years and years to analyze the data.
And not many people know, but US government has a
really big station there, like Unartic continent is huge and
a lot of countries have stations, but US in particular
has like huge stations over there. And the station I
went to, it's called Macmurdo Station. So when I was
there into like in the end of twenty eighteen, sorry,
two thousand and eight, I was there for almost a
couple of months. And that is like a typical summer
time in Antarctica, and that's the time where they will
have like around thousand people on the base and everybody
will be there to conduct some kind of scientific experiment
and to support those experiment. They also need a lot
of people, like they need cooks, they need chefs, you know,
they knew it, people who can like blow the roads
and all this kind of stuff. And I met a
lot of interesting people there who like, let's say a
software engineering by trade. They work here nine months in
US continent, and for three months they just go to
Antarctica and work as a chef or as a cook
because they just like it there and the pay is good.
Speaker 1: And the base has good internet access. I hear yes,
and I know that because I have a friend I
went to grammar school with who is messaging me from Antarctica.
I'm like, what you're aware, but he was at that
base and yeah, so pretty cool. How in the summertime
in Antaro would you think of it as bomby? Would
you go out in shorts? What's the temperature?
Speaker 3: So temperature is not that bad. You can go out
and go and go out in shots only for ten
to fifteen minutes, that's it.
Speaker 1: Wow.
Speaker 3: So like the during the summer time, you know there
the temperature was in the range of you know, like
minus let's say ten degrees celsius to plus five degrees celsius. Okay,
but there used to be some drays where you know,
it was like very very bad, Like we had like
one day where it was minus twenty degrees celsius and
things were pretty rough over there. But you have heated
rooms and you know, like food is awesome, there are
some recreational activities and you know, and that when you
go to those kinds of places, you kind of realize, okay,
how fast paced your life is back in like you know,
you know, the civilized world, and how much of a
time you got when you go to these kind of places.
Speaker 1: Did you ever have to slice open a banta with
your lightsaber to keep warm? No? No, that's the question.
Speaker 2: I was.
Speaker 3: I was fortunate enough. No, I went there on a
good good time over.
Speaker 2: There, Yeah, very good. You only had to smell them
on the outside.
Speaker 1: Oh my, sorry for that little diversion, but I just
had to know.
Speaker 2: It's great.
Speaker 1: So let's talk about Microsoft co Pilot Studio. I know
because I went to build with Richard, and we know
that it's for building your own custom agents and you
can connect them to a bunch of sources of data,
and we know that you can hook them up into
M three sixty five and all that. But what don't
we know?
Speaker 3: Okay, so let me go back to the history and
start with Microsoft first, foray into the cornosential AI. Like
if you remember in March twenty sixteen, I think it's
the first time when Satya Nadela went onto the stage
and he announced about Microsoft bought framework. Because still that time,
there was no good story when it comes to building
let's say, any kind of child bought or conversational experiences.
And after maybe two or three, after a month after that,
I know Microsoft is getting a lot of good price
right now because of their partnership with open AI. But
they also had their fair share of failures as well.
I think In March twenty sixteen, they launched a Twitter
bought called day oh yeah yes, yes yeah, and its
long form was thinking about you And they launched on
Twitter and that pot was from Microsoft Research and it
was their first public foray into the world of AI.
And people soon figure out on Twitter, you know, to
teach that bot to say some nasty stuff and Microsoft yes, yes, yeah, Microsoft.
Microsoft try to like you know again reteaching like good stuff,
but within sixteen hours they had to take it down
and they still now a lot of key executives at
Microsoft talks about that particular failure saying like, hey, we
learn a lot from that failure, like how what other
things not to do with AI stuff? And so they
continue to work on board framework for next three years.
But board framework, like is very complex.
Speaker 2: Yeah, we did a show, We did shows on bot framework,
like you said, like twenty seventeen. It's hard to think
about that now because the opening eye things overwhelmed everything. Yes,
and they made a lot of sense, like the fact
bot to me was totally logical that you could point
a piece of software at a fact and it would
give you sort of a natural language interface to it.
Speaker 3: But still, like Microsoft struggle to get really good traction
with a lot of customers and developers because it was
extremely hard to get you know, your conversational experiences right.
And then in November twenty nineteen they launch a low
code version of it called Power Virtual Asian and the
name Power itself can give you indication which product family
belongs to. It was from a power platform. So they
continued on that journey and then basically there was a
time where the entire investment on board framework stopped and
all the things like Microsoft was doing on Power virtualation side.
They had decent success. But then the real success or
real i'll call life changing moment for that particular product
or group was Ignite last year. Fen went on stage
and say, hey, Microsoft now is the co pilot company,
and if you would like to create a low code,
no code co pilots, then Copilot Studio is the product.
And I know Microsoft get a lot of flak, especially
their marketing department, for naming like weird products, but I
think this time really nailed it.
Speaker 2: To be clear, GitHub came up with the name, right,
That's why it's such a good name, because Microsoft didn't
think of it.
Speaker 1: We have a history of making really long names that
are very kind of academic sounding.
Speaker 2: Well they're named by Kimmittee.
Speaker 3: Yes, yeah, and someone I heard. I don't know how
true it is, but this product was supposed to have
another name. They already have a lot of printed name
and banners on it, but just three one week or
two weeks before Ignite, someone came up with this name
and they have to, you know, do a lot of
you know, last minute stuff.
Speaker 2: There's been a lot of name twitching, right copilot for
Microsoft three sixty five, Microsoft three sixty five copilot like it's.
Speaker 1: It's you know, the AI thing has been sort of
like a gold rush, hasn't it. Yes, like every company
wants to get their put their stake in the ground
and say we're doing AI or we're doing you know,
generative AI. And I mean I don't I don't want
our listeners who are dot developers to be thinking that, oh,
development is gone. Microsoft is just focusing completely from now
on on AI. Because if you went to build, there
wasn't a whole lot about dot net, you know what
I mean. There wasn't a whole lot of code, plumbing code.
It was all focused on AI. But you know, we're,
as Richard said in the beginning when he read that comment,
we need to stay focused on delivering value to our customers.
And the sort of AI thing just smacked this in
the face and came out of nowhere and says, hey,
guess what this is going to allow you developers to
provide better value to your customers. So really interested to
hear some scenarios about that, and you.
Speaker 3: Write a call like, you know, just don't think that
copilot Studio is the only product that I will need
to use, you know, when I had to create custom copilots.
There is another story in Azure world called AI Studio.
So if you are a professional developer and you don't
want to use let these drag and drop kind of
tools or low code no code tools and you would
have full control. Studio is your choice where since it's
part of Azure, you know it has a really good
pro developer story. You can be a dot net developer,
you can be a Python developer, and you can use
all the good services in Azure. And one other thing
I like about AI service a Azure EI studio is
the model selection. Like. One thing Microsoft has done which
not many people know is Microsoft launched something called Model
as a service where you can bring in some open
source model as well and host those in Azure and
use those in your Genia applications, so you don't have
to go with let's say models from open Ai or
models from let's say Microsoft for that matter. You can
use Lama hugging pace. You know, those models are also available.
So if you want to create a co pilot or
agent using that, you can do that now in Asure.
Speaker 1: But they still have GPT four GPT four oh dally
all that stuff from AI Open ai. But all right,
so wow, So that is really the fundamental difference between
these two is that co pilot studio is maybe for
draggy dropping business people and Azure AI studios more for us.
Speaker 3: There is one fundamental difference which not many people tay
pay attention. So when it comes to building a co
pilot or any kind of a application, a lot of
time people don't think about the UX part of it.
I know. That's where a copilot studio shines really well.
Like out of the box, it has really good capacity
of you know, having a really good UX. Let's say
I launch a co pilot for my existing enterprise app.
But then that copilot should able to understand or behave
like actual human right. The reason chad jipt got so
popular so fast is because we can interact with it
and you know, it can understand like at least what
I'm trying to say maybe ninety to ninety percent of time,
and it can give me good information back. And if
you compare that with old version of the chatboards that
more of these public facing website used to have. Those
used to be like, oh the major retailers or bank
used to have those, like used to be simple time
st where the first thing I used to google that
time is ourbing is what's the best way to get
to an actual human agent rather than you know, going
through their automated.
Speaker 2: Response because that's what everybody does.
Speaker 3: Yes, But now with this, after the generative AI and
some of the advancement, you know, you can at least
have some decent conversation. And that's where this co Pilot
Studios shine where it has a lot of widgets, it
has a lot of you know, building capabilities that if
I would like to build a good conversational experience, Copilot
Studio really shines as compared to Azure Eye Studio.
Speaker 1: Can I take the agents that I make in copiled
studio and move them over into Azure Ai studio and
enhance them.
Speaker 3: Right now, the create direct conversation story is not there.
But what you can do is, let's say, if you
have already an agent in Azure AI side, you can
directly call that from a Copilot studio, so you can
do integration. But there is no like Magic vand to
convert those.
Speaker 1: That's actually better.
Speaker 2: Yes, well, it also reminds you that there's more than
one team instead of Microsoft working on this stuff, right,
you know, that's that's sort of the real reality is
that there's lots of different places moving at once.
Speaker 3: And I'm a big believer of Fusion Team where hey, developers,
you focus on you know, doing really crazy stuff behind
the scene, you know, make the magic, but also gave
a lot of information worker or business users and ability
to you know, take the advantage of what they know
from the business and then you know, have an easy
way to interact with the back end system. So so
Copilo Studio also have that functionality way. Let's say I
have my logic app I would like to call or
I would like to call my EPI. I can do
that directly. There is an STTP connector available or I
can also invoke a power Automate cloud flow, and power
Automate has more than twelve hundred out of the box
connectors where you know, or I can build my custom
connector as well, which then can you know, call my
back end APIs and then can get the job done.
Speaker 1: I have a quick story here. I got an email
last week from somebody who said, you know, I see
that you have a YouTube channel and you've got some
great content there, and uh, you know, for a very
low fee. Actually it was pretty good. I will, I will,
I will generate SEO happy titles, descriptions, tags and stuff
for your YouTube content. So I said, okay, sure, I'll
try it. And what I got back was a RAR
file full of text files for and a lot of
them were for the dot net rocks videos. And they
kept mentioning the dot net Rocks videos are simply a
still shot with our podcast that plays on it. And
this thing went on and on about how you will
learn from these training videos, hands on experience and stuff.
They got to the they got to the songs that
my band has put up there, and they called, you know, Nachelle,
who's our background singer, but she does sing on some
of them, but on this one that I was singing
like all of it lead singer Nachelle Rollins blah blah blah,
this big, grandiose stuff, and I wrote it back. I'm
not gonna say it is by her own back and say, dude,
this is bullsh this is some AI generated crap that
doesn't know a still video from you know, from a
training video for crying out loud. So yeah, I won't
be using their actually incorrect, factually incorrect and being sold
as you know, the I don't know whatever.
Speaker 2: So I mean, you say, quietly said Fusion development there,
which is that power platform working with dot net dev
and so forth. So you see a sort of same
dynamic where we have domain experts, which I think is
the big biggest challenge when you're building a lot of
these language models, is you need a domain expert checking
for factual accuracy, knows enough to know that is right
or that is wrong. Yes, who actually sang on this track,
as well as some coding knowledge to be able to
pull the pieces together.
Speaker 3: Yes, that is, that is correct. And one of the
things is, you know a lot of people like really
got excited after the rise of generative EI, but there
are like legitimate you know application, A lot of organizations
built which doesn't use generative AI at all. Like one
of my customers, they did use Copilot Studio, and they
did launch a custom Copilot for their public facing website,
but they didn't use any of the generative AI features
at all, Like, because there will be some instances where
you don't want to give a sort of correct or
incorrect answer, right, You want to get give a factually
correct answer. And that are some of the things that
are there in Copilot Studio where for some of the
information you absolutely want to give the appropriate answers maybe
from your own knowledge pace or maybe from your own
database or you and that you can do. And there
are certain things where you think, okay, I can use
generative AI for some of the answers. You can do
that segregation wasy very easily as well.
Speaker 1: I'm just another story. I love telling these stories my
Facebook feed. I don't know about you guys, if you're
on Facebook at all, but every once in a while
I scroll and I'm seeing these accounts that are something
like tiny houses, right, and then this clearly generated beautiful
scene of this wonderful house with multiple staircases and flowing
waterfalls by a brook. And it's just like a place
that you'd want to be, right And the caption is
like peaceful place. It's not. Hey, this is where I
am right now, it's not. And to get millions of
views and hundreds of thousands of comments, it just makes
me mad. Welcome to the new world.
Speaker 2: Well, and again it's you used a bought to generate
the content. Who's to say you didn't use a bought
to generate the hits? Well, there you go, all right,
So let's take a break, and when we come back,
I've got a couple of questions.
Speaker 1: Great, did you know there's a dot net on aws community?
Follow the social media blogs, YouTube influencers, and open source
projects and add your own voice. Get plugged into the
dot net on aws community at aws dot Amazon dot com,
slash dot net. Hey Carl, here are you maximizing your
dot net applications potential? Rayguns tools not only catch errors,
they provide actionable insights to optimize your code experience the
power of detailed reports and real time monitoring to enhance
your applications performance. Join thousands of dot next developers who
trust Raygun to help them keep their applications running smoothly.
Visit raygun dot com, slash dot net rocks. That's raygun,
r A y g U N dot com, slash do
tn E t r O c ks for your free
fourteen day trial.
Speaker 2: We're back. It's dot at rocks. I'mrogard Campbell. Let's call Franklin. Hey, Hey,
I'm talking to our friend Forshett about the role of
as your AI studio as well as the co pilot studio,
because what's better than two?
Speaker 1: And I like how you can use them together.
Speaker 2: What's the call out for Patreon there, Carl.
Speaker 1: Yeah, And if you don't want to hear those ads
during the show, you can subscribe to our Patreon account
for five bucks a month. You can get a feed
that has no ads. So there you go.
Speaker 2: Awesome, Forsham, you mentioned like the ux of a co
pilot app. Isn't it just a textbox?
Speaker 3: Not necessarily? Sometimes it can be text paused, but some
times you may want to show a really nice adapt card.
Sometimes you may have to show an image or maybe
the user is looking for a video. So those kind
of things like if I go with traditional board framework way,
those kind of things are really hard to you know,
get it right. And another thing way where Copilot Studio
really shines is you know, like how I can support
my copilot across multiple devices or multiple experience. That is
like one of the biggest challenges I have seen with
multiple conversational experiences. Like, yeah, like I build it, it's
worse on my machine, or it works in the environment
where I have deployed, But what about now the customer
wants to use the same copilot on Microsoft Teams, or
they want to use it on a public facing website,
or the customer loves Slack they want to use it there.
Like how I can you know, now redo everything so
that you know, my custom copilot will work on that
experience as well or on that service as well. So
copilot Studio has a lot of these channels or services
already prebuilt where I don't have to redesign my experiences.
All I do is do some back end like changes
in the studio itself, and the same copilot then can
be accessible on let's say SLAG or Microsoft Teams.
Speaker 1: In both of these things, I noticed that you know,
you can connect it to your database. Yes, And apparently
the coolest thing is you know how many reverse threaded
left handed hemptas are left in inventory in Atlanta, Georgia warehouse, right,
and it'll just tell you that kind of stuff. But
the cooler things about generative AI for me are building
models from you know, training and unreal data. And is
that something that is not necessarily as valuable to regular,
run of the mill businesses as it would be to say,
you know, a chat gpt, which is the genie in
the bottle that knows everything.
Speaker 3: So, if I understand your question correctly, you're alluding more
to you know. How about I have my own data
and I would like to use that data into generative AI.
That's one way of doing it. Or maybe I have
my own data and I would like to generate my
own model using that data because I want the answers
in a specific format or in a specific way rather
than what you know, Chat Jupiter gives me. So there
are a couple of things you can do. And you
may have heard this a lot. Very few companies these
days are investing building their own model because it's very
hard to compete with companies like Amazon, Google, or open
AI or even Microsoft. So most common techniques which most
of the enterprises are using these days is either you
go with a rack pattern like retrieval augumented generation, where
you bring your own data and then feed that context
or data to your large language model and then get
the inside. Second very common is not very common, Less
common and more expensive out is fine tuning this models
like where as part of fine tuning, then I provide
series of prompts some training data. So then after a
fine tuned model is there, then it will be responding
in a specific way, and it will be also having
a specific information that let's say the generic large language
model or foundation on a large language model will not have.
Speaker 1: Now, when you say that, do you mean cleaning the data,
like taking out the things that aren't relevant, or do
you mean when you say fine tuning, do you mean
telling it what to ignore in the data.
Speaker 3: Yes, so you have to basically provide custom instructions. You
have to provide custom prompts, and you can also provide
your own data as well. Where okay, these are the
sete of things that you should be doing in this
particular way. Don't include that or don't include these, so
those kinds of additional things you can do. One thing
I do want to highlight is fine tuning is not
for everyone it's only you know, for companies where you
know you're going to have large workload or your consumption
is going to be very high. And let's say with
this large language model and or this large language model
will not work in its current capacity because under the hood,
if you read the licensing documentation from both open ai
or even from Microsoft on Azure side, not only you
had to pay money to find tune the model, but
once the model is fine tuned, you had to also
pay Microsoft money or open am money to host that
fine tune model. And then on the top of that
the charges for your tokens that you will be exchanging.
Speaker 2: It sounds like rags the way to go there.
Speaker 1: Yes, oh you know what you say that, Richard, But
we have some mutual friends in Europe, and one in
particular whose company decided to do away with RAG in
their company because the percentage of errors that came back
was too high. And I just asked this person literally
two weeks ago, has your opinion changed? And no, no
it hasn't. It's still still or prone. So what do
you think about that?
Speaker 3: So RAG, you know, on theory, it looks simple, but
one of the hardest challenge I have seen with the
RAG is let's say I'm looking for a specific information
from my company data. There is a limit on how
much of the day, like how much of a token
size or context size you can send to a large
language model. So most challenging part with the RAG implementation
is how I will make sure for that particular specific
question or conversation I'm having, how I can get the
least amount of data that is more relevant, like how
I can get that, And that's where you know, then
you have to have vector databases. That's where you need
to have your ash eye search and those are like
configuring those and getting those right are easier said than done.
And let's say if you're let's say, if you the
example you gave Carl is in case they're saying less errorrates, sorry,
more errorrates on RAG, maybe you know they will go
with fine tuning because there they have the build capacity
in terms of you know, fine tune the model. Because
fine tuning is not everyone's cup of tea, you know,
to have certain skills. And also they may be have
a use case where the cost will justify, you know,
aware or the cost the amount they had to put
in the fine tuning will justify in terms of the
errate they're getting on the right side.
Speaker 1: Yeah, how much would it cost for the rag of
the Franklin Brothers band to get the lead singer? Right?
Speaker 3: I don't know.
Speaker 1: Those are the questions you're gonna be asking, though, you know,
we want we want accuracy because sometimes you can ask
a question if you're if you're looking for information, right,
real information. Typically you're asking a very specific question. Yes,
the answer could be yes, or it could be no, yes,
or the answer could be, you know, one of a
hundred possibilities, and that is usually a fundamental question that
business decisions will be made on because if it wasn't,
what's the point of rag.
Speaker 2: Right.
Speaker 1: We need to ask these fundamental questions, and then you
need to base decisions on See, we have to be really,
really sure that that answer is correct.
Speaker 2: And one of these challenges I remember from you know,
decades ago, where you only have to generate a report
once that has inaccurate numbers, and everybody questions the numbers
forever after that. Yeah, that's right, So you know, you
can't afford to lose that confidence.
Speaker 3: And that's where you know, a lot of companies, especially
after the rise of Jenny I especially the big companies
where if they get out any wrong information or incorrect information,
if there is like a significant financial penalty against it,
the first thing they are doing is cleaning up their house,
like cleaning the data. Like what if your data you
have multiple versions of the data. One is saying good thing,
but another saying is bad thing, and then somehow your
AI search or whatever other two you're using picked up
the bad information and then use that to the for
you rag pattern.
Speaker 1: I remember Gary Short and Seth Warez both saying that
the majority of their job, and this was maybe ten
years ago, was cleaning data, you know, just to get
more accurate. And so the ability the fine tuning, as
you say, to omit things that are outliers or standard
deviations that are way high or something.
Speaker 2: And we now up here in Canada we have a
core precedent around this with Air Canada. So Air Canada
deployed a chat bot using an LM for providing advice
for travel, and a fellow used that tool to figure
out if he could change a flight for it was
a bereavement. Fares are called sover Yeah, a funeral or
something like that. This is a great story. Yeah, And
the software said, yeah, no problem, you can change that.
And so then he went to actually change it and
they said, no, you can't change that, said, well, the
bot told me, and then he was forced to buy
the ticket the last minute spend even more so forth.
But he's now gone through the courts and the court ruled, hey,
that you're using that chatbot as if it was an employee.
You have the same liabilities to that employee. So if
your employee gave incorrect information, you'd be held accountable for it.
Your software, you're also held accountable.
Speaker 1: This is a great time to be alive in the
you know, seeing the real genesis of AI taking off
and getting more and more accurate in the in these
kinds of stories really shape it well.
Speaker 2: And it's I'm happy to have those case laws in
place because it's for me as the architects, sitting down
with leadership saying we want to use this technologies, like
here is the risk we are assuming, right, right, And
so you know that's just why you spend money on testing.
Speaker 1: And that was a yes or no answer, right, It was.
Speaker 2: A little more complicated than.
Speaker 1: That, Can I get a refund? Yes or something along
those lines or something along those lines.
Speaker 2: And yeah, you're right. It pretty much breaks down it
exactly that, Carl.
Speaker 3: Yeah, and that's the reason. Like some of the customers
I work with, they are pretty large customers, and one
thing they're doing is hey, EI is great, but nobody
should be launching anything because it going through some governance
committee where they will be evaluating what exactly you're doing.
You know, what technology you're using, whether it's going to
be for internal users or for your customers, because if
you do any mistake, and let's say if it's for
your outside customers, you know, the implication of that can
be tremendous.
Speaker 2: Yes, now, definitely the motion at this moment here we are,
you know to a latter half of twenty twenty four
is all internal apps. It's an hr bot, Yes right.
It's those kinds of things where you're only upsetting the
employee when you lie to them about how many vacation
days they have.
Speaker 3: And again that's a good point. You got what, Richard,
because vacation days is something that you shouldn't be relying
on general to AI. You tell you, it should be
coming straight from your employee handbook, where where you know
you have full control over what information is getting from
there and what information is surfaced.
Speaker 2: If this isn't a project that makes sense Pshan, what
prout checks do like that seems like exactly, I'm not
going to read the handbook. Yes, I'm going to pester
HR for an answer, yes, and they're going to yell
at me. So we make the bot to solve that
problem and you.
Speaker 3: Can solve it easily. Now with Copilot Studio along with
Azure Ai Studio as well, but co Pilot Studio, a
lot of companies are also using it for customer centric
BOTE outward bots like Business to Customer as well. And
one of the reasons they are doing it because some
of the red teaming and other stuff the co Pilot
Studio team has done because not peny people understand when
you launch anything external, especially at childbot like Microsoft learn
It in twenty sixteen, where people will try to get
some nasty stuff from it. And if you launch a
public facing website chadbot, you know, anyone can start interacting
with it and start asking some stuff that they shouldn't
be asking or your company shouldn't be giving responsor to that.
Speaker 2: So business bring us back to outward facing is not
there yet. It's too high.
Speaker 3: Risk y's high risk. Yes, but if you know, if
you use the right tools, or if you know you
know you have a good team in house, you can
definitely use some of these tools where you can have
a more confidence, you know, launching this outward facing application
as well.
Speaker 2: I just debate that anyone has a good team at
this point.
Speaker 1: What are the things that I think would help And
it's definitely almost impossible with large language models that are
trained on language right and try to predict the next
word in their answer, but citing references. Right, if that
Air Canada bot, you know, when it said yes you
can get a refund and here's a link to the
policy that you can read and decide for yourself, right,
then it might have been a bit more accurate, and
not not necessarily accurate, but it would have allowed the
user of that bot to make a more informed decision.
Speaker 2: Yep.
Speaker 3: And I think KYL, like Microsoft co Pilot studior team
should hire KRL because now he's letting all their good
features indirectly via all these success stories. But yes, that's
one of the advantage where this co pilor studio giates
is if you are going with the rat pattern and
let's say you're using your knowledge based then while it
generate the information, it does give you information about the citation,
like what are the source of this information and so
that you can click on it and then you can
verify Okay, this information looks great or I can then
you know, use it with more confidence and then I'll
go on with the next stuff.
Speaker 1: That's great. It's so good to hear. Yeah, ask chat
GPT for a reference. Sometimes get that and on lessly.
It doesn't know the way, it's bunchet. It's a very
difficult thing to deal with. Yah.
Speaker 2: Yeah, I think this is where they it's just where
they use the term grounding. I don't know why this
term was necessary.
Speaker 3: Yeah, they use the grounding. Yes, Like, okay, we use
this information from grounding. It's not like something the lllum
made up and this is the source of it or
proof of it.
Speaker 2: Can we just always be grounded like thing? Right?
Speaker 3: Like really like the people who have kids, you know,
grounding me is totally different in their context.
Speaker 1: Yeah right, true enough. Or the people who like to
walk barefoot out in the lawn every day because it
helps their magnetism or something.
Speaker 2: Walking barefoot and grass is pretty good. Man. You don't
have nice.
Speaker 1: It's nice. It's a nice feeling, but it's all it
needs to be. There's some weird science that is associated
with it would say not soccer, but okay, pseudoscience anyway.
Speaker 2: Now I appreciate that. I don't want software speculating on
my van. I prefer it to say I don't know.
Speaker 3: Yes, and those kinds of things you can configure in
co pilot studio, like you can have Okay, I'm launching
this copilot and it's going to take care of only
four or five things. Anything extra, people like, anyone ask
it can just come back with I don't know or
I'm not allowed to talk about that. Go through this
link or talk to this person and then and do this.
Speaker 1: In chat GPT's defense, it does that, you know, when
it doesn't have an answer, it says you should call
the manufacturer or you know, contact the whatever the authority.
Speaker 3: One thing people don't understand is chigibt is a custom experience,
Like you're not interacting with an LM directly like as
an API. It's a customer application. A lot of guard rails,
like Opena has put into it. Now they're launching a
memory as well, so a lot of things are going.
It's a custom app, so yes, if you have a
team that can build a custom app like that, then
yes you will get similar answer. But let's say if
I'm a developer, I'm calling this LLM directly. You know
most of this LM don't have a memory as well,
Like the calls are stateless, so those kinds of things
it may not say, and it may just give you
completely like completely incorrect answer as well.
Speaker 2: Yeah, and I guess that that brings me back to
this whole. Like the HR bot makes sense is if
you can ground it on. I know you're not going
to read the user they employee hand books, so I'm
going to turn it into the ability for you to
ask questions and it's going to reference accurately. And when
I'm not when my certainty is low in the SAW
where let software certainty low, it's like you should talk
to HR about that. Because from an HR perspective, if
I can take eighty percent of the stupid questions and
just answer them so I'm only dealing with the twenty
percent that are more complex, I'm pretty happy. Like that's
a good outcome.
Speaker 3: You can use for HR, you can use it for
your IT support as well, because it has connectors where
it can talk to your own knowledge base and that
knowledge base can be in service.
Speaker 2: Now yeah, no, I just put I have an automated
responder just says you try turning it off and turning
it back on again.
Speaker 3: Like said, yeah, my favorite code is be smart restart.
Speaker 1: You know there you go. Nice? Nice works for phones too. Yep, yes,
all the time. And Pat and Dwayne Patrick Kines Douain
Laflat from security this week say you should be restarting
your phone at least once a week at least if not,
you know, three or four times a week. Yeah, all right,
So where do we go from here? Is there more
that we haven't discussed? Like how is it expensive? How
do we get started?
Speaker 3: Okay, so cost wise, even though it's a part of
Power Platform, its cost is a little bit different. It's
just like ash. Everything is based on the consumption. So
the minimum pricing point is you had to bot. You
had to buy a package for twenty five thousand messages,
which cost two hundred dollars. And again this is a
retail price, like if you have an enterprise agreement, you
may be getting some discount on that, but that brings
up a question is what exactly a message is? Right?
A message is basically anything like when I ask anything
to my copilot, if that's going to trigger any kind
of response or any kind of AI agent a call
behind the scene, that constitute as a one message. And
if I'm using generative AI features, then just for the
sake of simplicity, what Microsoft has done is any answer
that has generative AI in it, it is just going
to count as the two messages. So if I ask
a question you know how many PTOs are there, and
I'm using some of the generative EI features and then
that come back, then it just counts as the two messages.
Speaker 1: Now it's two hundred dollars a month for twenty five
thousand messages a month. Does that mean that if I
only use one thousand messages or you know, less than that,
am I only going to get charge point zero zero
eight cents per message?
Speaker 3: No? No, you have to, Like the minimum is two hundred.
Speaker 1: That's what you have, whether you use it or not.
Speaker 3: Yes, And that's one story which is really not good
in my opinion right now, because you buy that license
for per tenant. Okay, but what if I'm working for
a really big organization where I have thousands and thousands
of employee and ten departments are there. Every department has
their own copilot and how exactly we're going to manage
the pricing because those twenty five thousand message I can
be exhausting in one hour or two hours.
Speaker 2: Yeah sure, yeah, yeah, it feels like it's beta pricing,
Like it's just pricing because you're doing a beta test project,
not actually deployment level yet. And so now I'm starting
to feel like the relationship between copile studio and as
your AI studio is the same relationship between power platform
and visual studio.
Speaker 1: Correct, right, Yeah, that makes sense to me, Richard.
Speaker 2: So if you're the front end stuff, the place to experiment,
the price to prototype copilot studio when you want to
get into precision and tuning and more skilled people AI
studio knowing that anything you build an AI studio you
can surface in copilot. Yeah, and all right, I dig this.
This is huge in development. Like you said, we did
that show with vishuas earlier this year where we talked
about that same dynamic between power platform and dot net right.
Speaker 3: That the and I know yeah, which Fus is a
huge advocate of future development. In fact, he's the one
who taught me all this concept, and I agree with
it because the more and more I work with enterprise customers,
like there's always the case where developers, like good developers
are a rare commodity. Yeah, and you'll agree that, you know,
it's very hard to find good developers, Like how about
then we create some tools where you need developers only
when that tool cannot do certain things for you, right.
Speaker 2: Right, But the other part is like I never look
at the power platform folks, and I guess it's the
copilot studio folks. As these are the lessers, these are
the domain experts. They're actually going to save you time
because they know more about the problem space than you do. Right, Like,
my struggle with really great developers is how much time
they spend having to learn the domain. Where if you
have a good domain expert relationship going there, you can
focus more on breaking those bits down and go, okay,
well that's where some custom code needs to live. That's
there is no easy solution there. We need to build something,
take a spike, you know, those kinds of things. Like
that's a really constructive way to work quickly.
Speaker 3: And I'm seeing like so I'm talking about this conversation
AI or Microsoft Board framework since twenty seventeen and since
last Ignite. I'm saying like just a huge interest from
developers as well, like pro developers thinking about or are
eager to learn about co pilot studio. Like a couple
of weeks before, I was at dev Intersection or next
Gen EI conference in Vegas and I did session and
a workshop on copilot Studio and its attendance was more
than some of the popular you know, pro developer sessions
or confront or workshops were there.
Speaker 2: Yeah, that's great, but any Yeah, I appreciate this because
you start now, you start thinking about rethinking the UX
of an application, a lot less buttons and sliders and
a lot more texts and most of the screen taken
up with it presenting information to you based on your requests.
Speaker 3: Right, And it doesn't have to be only this. It
can be. Yes, for some things you'll be using this,
but for some more traditional or more things where you
definite need to have you like proper UI with multiple
buttons and multiple sliders, yes you'll have that, but this
can be your beginning point where I'm every day I'm
starting my work. I'm just coming here, you know, doing
my stuff. I asking copilot to do a bunch of
stuff for me. And I always give the example, you know,
Javis in Iron Man. That's a good example. Hey, there
are certain things if Javis can do. Let Javis handle that.
And I'm going to focus more on the complicated stuff
behind the scene.
Speaker 2: Yeah, I'm waiting for office to bet on this tap. Right,
Like you think about immediately think in terms of the
old Alan Cooper of the Sovereign app. Right, what's the
first app you go to? Now? For me? It's still Outlook,
you know. That's why I'm sad.
Speaker 1: Six five threads one for me.
Speaker 2: There's also teams, like a lot of folks, that's their
starting point. And you start thinking about the LLM interface
to this, how you would start your day? I mean
right away, your first step in either app is triage. Yes, right,
what's the most important thing? Right? What do I need
to deal with?
Speaker 1: Now?
Speaker 2: You know, where are the crises and so forth? Like
you could see a good agentic type tooling in this,
helping this or through those priorities. I mean, I would
argue the first triage is how many of these has
the bot already responded to?
Speaker 3: MM?
Speaker 2: Hmm right, I've not only sorted these out. Here's the
ones I think. Here the answers are, are you okay
with this? It's like yeah, yes, and that that that
that that that okay? What are the ones I need
a trioge like my wife could my life could be
way better?
Speaker 3: Yeah. I think that story is still evolving with Copilot
for Microsoft sixty five. Uh, and they do have one
capability there where in case you would like to extend
the functionality of out of the box co pilot there
you can use Copilot Studio and its license is already
covered as part of your Microsoft Pay sixty five Copilot license,
so I can listen.
Speaker 2: Prashan, I'm an old grizzled Microsoft observer. At this point,
you're not grizzled. I've come to appreciate disper he said
on a podcast. You will know a technology is being
taken serious by Microsoft when they bet their mark key
products on it. Yes, right, when that shows up in Outlook. Okay,
they're committed, like, no two ways about it. That's why
we bought into Active Acts or you know that whole
control surface. That's why we bought into Ola, and it's
also why we were suspect of WPF when Office never
took on WPF. It wasn't until Visual Studio took out WPF.
They're like, okay, right, Like that's there's a reason to
take the cues this way as to how far down
the path these products really are. Like I feel like
these are still experimental products because the Microsoft seems to
feel these things are still expound products.
Speaker 1: Well, they may be working on them, but they haven't
surfaced yet. Yeah, I have a question for you, totally
off topic, well not really. Apple's coming out very soon
with probably out by now, the new iPhone that has
AI Apple Intelligence not artificial at all, and I kind
of I feel like I get it. They're basically taking
the SII experience and exposing you know, GPT four oh
on the back end so that Siri will become more intelligent.
And I just wonder if you know anything about that,
because I, you know, if that is as easy as
it sounds. Man, I'm going to get in my car,
turn on car Play, and just start talking to my
car and it's going to be night rider.
Speaker 2: You know, what do you think for Shadow? I have thoughts.
Speaker 3: So I'm not an Apple person. I use most of
the Android stuff. But after saying, like some of the
demos from open Ei recently, the demo ware, the latest
version of GPT four oh is multimodel, and then we
can have multiple conversation and we can also change the
context in between. I think we are getting there as
long as they figure out how to run this LLLMS
or all these models cheaply on the devices. Yeah, I
think we are there, like maybe one or two years.
Once they put some SA safeguards in it. And let's
say I'm driving a car and if that LM can
execute on my car's hardware, we should be there.
Speaker 1: Are you saying that the LLLM stuff is on the phone,
they're not just calling an API out to GPT to
do the work.
Speaker 3: Yeah, Like once that gets mainstream, like where we will
have more and more this small language model is doing
all this stuff, but they're actually executing on the local stuff,
we'll have like much better experience, and we'll also see
more and more push from these companies as well, because.
Speaker 1: I don't know if that's the case now. It seems
to me that they're just calling out too with an API,
but that.
Speaker 3: Is a very expensive right now, Like if you look
at yes, they can do it for a few months
or a few years, but that's not really long sustainable
business model because all these lllms require really bf infrastructure.
That's the reason why I think. I don't please don't
quote me on this, but I heard even though openan
AA is making so much money, like billions of dollars,
they're still not profitable because they had to spend a
lot of money behind the scene on the infrastructure to support.
Speaker 2: That far and far more.
Speaker 1: Yes, what was your thoughts, Richard?
Speaker 2: I think Apple is late to the AI game and
needed to announce something, and the fact that they keep
pushing back the release of it is proof they don't
know what they just committed to. Ooh, and they're panicking.
You know, we expect Apple to come late with a
refined product that has clearly not happened. The fact that
they've already pushed the dates back shows they they said
they were going to jump before they jumped, and now
they're struggling because it is a very unrefined product. It
is not asked Apple asque at all, but they could
not afford to not say something good way to hear,
so they had You know, they only do one show
a year. They do that show in the WWDCS in June.
Were you really going to wait till twenty twenty five?
You couldn't, so you said it, and now you're trying
to make it come true. That's not very apple asque,
but all the hallmarks are there. The only reason we
might give a pass is because Zappli you don't do that.
It looks like they did this time.
Speaker 1: Well that feels like the end of our show, Prashant.
This has been very enlightening and I appreciate you putting
up with my digressions and you know, little voyages into
stories and things. But man, this is just chocolate block
full of great meaty goodness.
Speaker 3: So thank you, thank you, thanks for having me. And
it was really it was my pleasure, you know, pleasure
on all of my side for.
Speaker 1: This fantastic all right, and we'll talk to you next
time on dot net rocks. Dot net Rocks is brought
to you by Franklin's Net and produce by Pop Studios,
a full service audio, video and post production facility located
physically in New London, Connecticut, and of course in the
cloud online at pwop dot com.
Speaker 4: Visit our website at d O T N E t
R O c k s dot com for RSS feeds, downloads,
mobile apps, comments, and access to the full archives going
back to show number one, recorded in September two.
Speaker 1: Thousand and two. And make sure you check out our sponsors.
They keep us in business. Now go write some code,
See you next time.
Speaker 3: You got Jamal Vans
Speaker 2: And