How'd you like to listen to dot NetRocks with no ads? Easy? Become
a patron for just five dollars a month, you get access to a private
RSS feed where all the shows have no ads. Twenty dollars a month will
get you that and a special dot NetRocks patron mug. Sign up now at
Patreon dot dot NetRocks dot com. Hey Carl and Richard here with your twenty
twenty four NDC schedule. We'll be at as many NDC conferences as possible this
year, and you should consider attending no matter what. The Copenhagen Developers Festival
happens August twenty sixth through the thirtieth. Tickets at Cphdevfest dot com. Ndcporto
is happening October fourteenth through the eighteenth. The early discount ends June fourteenth.
Tickets at Ndcporto dot com. We'll see you there, we hope. Wow,
you did it again. You found the last build show at dot net
rocks. I'm Carl Franklin and I'm Richard Campbell, and Mark Brown's here.
We're going to talk to him very soon. But anything that you want to
tell the people about our week here at Build you know, it's fun to
be back in person. You know, everybody's getting better at conferences by bad,
Like we're remembering how the conference. Yeah but yeah, I get Microsoft
asked me to put together a podcasting space, so we brought in a bunch
of other podcasters as well. Yeah. Then I figured, you know,
we could make some dnrs while we were here. Yeah, and you know,
great line of guests, and this is the last of them. Yeah,
the published wise, it's actually get a published in order, so it's
all the last when we're recording. Yeah, third day, so it should
be fun. Yeah, but I don't have any whiskey here, which is
usually the last show of the day, the rituals. We're on the third.
I'm a little bruised. Well you are, Yeah, I'm not.
However, I've been a good boy. You're a responsible individual. I appreciate
it. I wouldn't say that. Let's not get crazy now, all right,
roll the crazy music for better? No framework? Man, what do
you got so better? No framework? For those who don't know, this
is it started out as I would, you know, point my finger at
a namespace in the framework. This you know, the any you know classes
in dot net that we may or may not know about that kind of thing.
But that got old real quick, and I started googling stuff and finding
eventually, do run out of framework? Do you kind of got through the
framework? Yeah, it took a while, but you got there exactly.
And nobody wants to hear, you know, a lesson on the bowels of
the common language run time. Well, anyway, so Mark Brown is here,
and Mark you can join in because I picked this toy for you.
Yeah. Yeah, So who's a toy boy now right? I know I've
become the toy boy, right. I know Mark is into a lot of
things, you know, technology wise, but also electronic music, and I
know this about you. So I found I'm sharing a device that I have
and I have a link for it on sweetwater dot com, which is where
I bought it. Did not find it on Amazon, but they're out of
stock. So this is going to be fun. When you know ten people
who listen to dot net rocks, I'll call up and slam the back orders,
slam the back lid. So this is a Kenton l N d R
MIDI line driver. It's basically two boxes. It's a ballan. You know
what a ballan is? A way to use one transport for a different protocol,
right, And there's a lot of Cat five, Cat six ballons where
you just string a network cable between two things. You can do for video,
for audio, for whatever. Well that's what this is. So MIDI
Musical Instrument Digital interface has a problem that when the cables get over even close
to twenty feet long, the latency gets high. It's a kind of serial
protocol. It's a very yeah, it's a serial protocol, old school,
and so this solves the problem. It's always been a problem for me because
in my studio, the piano is typically on the other side of the room
from the workstation where I'm recording, because I want isolation and everything, and
I've always had a problem with this, and so I've tried Bluetooth adapters and
they're two Like, the latency is too much. This is the only thing
that works. So I've basically hardwired this into my studio, one side on
the piano and the other side of my workstation. I'm taking a switch between
either. It's litter just because Ethernet lends at a higher frequency. Yep,
you could haliday a heckup a lot faster, that's right, okay, and
maybe you did put a switch and you'd be fine. You're still talking milliseconds,
yeah, exactly. So the piano that I have is the Yamaha Baby
gran that has made built into it, and there's a Selenite under every key,
so if you plug a MIDI keyboard into it and play, the keys
go up and down right. And it's also a great MIDI controller. So
I just have it now hardwired in so at any time somebody wants to play
the piano and record it. Let's say there's a drummer playing right and the
piano playing at the same time, I can just have the piano player record
MIDI and with no sound, and then edit it right to fix the mistakes,
and then play it back through the piano and record it. Wow.
So that's a trick that my clients love. Very clever, do that now?
Yeah. Anyway, it's one hundred and sixty nine bucks. And if
you struggle like me, yeah for the pair. If you struggle like me
with this problem, this is a solution. Whenever they get back and stock
and ask for Mike God Love, he's my guy at Sweetwater. Yeah,
and they have everybody orders a bunch of them then maybe they'll rush the back
order back. Yeah. Maybe, yeah, yeah, maybe they'll offer me
some free stuff. Who knows stuff is good. Free stuff is good.
Bought some stuff from Sweetwater. They're good people. They are all right,
all right. Richard, who's talking to us today, grabbed a comment.
Let's be clear, you found this comment, but yeah, grabbed a comment
off of mast done. Yeah, it's only because we've already read all the
COSMOSCB related kind of off of the of the website and so yeah, plus
other issues, but that's not important now. This comment comes from Hassen Savarn,
who said it looks like the AI theme is replacing data topics and conferences.
I get it. AI is everywhere lately. But AI depends on data.
Yeah, you still need good data models, optimized databases, data quality.
It all still matters. So organizers, please don't remove data topics from
your upcoming events or net topics for that matter. Correct. Why would they
do that? I no, no, no, everybody thinks it's Python now
with the AI think so on the other side is like his gigo ever meant
more than it means right now, garbage in, garbage out right like that
was an old joke of computing decades ago. But what is a large language
model but a garbage amplifier and garbage it spreads it everywhere, or a small
language model for that, Yeah, hassan, great comment, Thanks so much,
and I'll respond to you a masadon he send you a copy of music
go by, And if you'd like a copy of music, go buy,
write a comment on the website at dot at rocks dot com, or on
the facebooks or on masd on Twitter. All of those work. Well,
read them all and if I read them on the show, I'll send you
a copy of music. Go by. Yeah, blues guy, I'm a
guy. Yeah, I heard they just got DMS on there. There you
go. Well, you can find all the ways to get in touch with
me anyway at Carl Franklin dot com, including all those things that Richard mentioned.
So let's formally introduce Mark Brown. Been on the show several times,
but not in a while. He's been in the software industry for more than
thirty years old guy, when I was younger, ripersnappers. Today he has
worked for a wide array of companies, from ISPs to dot COM's building applications
that ranged from the largest of back end systems to the smallest the devices and
everything in between. Mark leads the growth team for Azure Cosmos DP. Welcome,
Thank you very much. Great to be back. It's been a while
too long. And as you've you just caught me here and said, hey,
we haven't talked about Cosmos debating around like you are correct, that is
right. Are you doing any electronic music these days? You know it's been
Uh no, I guess I'm just gonna say no, that's fine. Yeah,
it's been hard to takes all it takes, you know, it does
take time and dedication and yeah, and you've been busy, been very busy.
Uh, life has not been the same. And I guess about eighteen
months since the AI detonation. Yeah, like chat GPT landed on my head
like a ton of bricks right over that November twenty two. Yeah. Yeah,
and I spent all of my Christmas vacation looking at this thing and thinking,
hey, we should be there's there's a there's a there's a place for
us in all of this. So it's your fault, well not all may
some of it, because here it build it's all AI. I mean everything
in the keynote is AI and everything everywhere is all averything. Yeah, well
you, let's face it, Microsoft has a great platform here. Yeah,
yeah, jumped on right like I jumped in, and I started looking at
l ms in this whole Jenai space and the technology here, and thought,
you know, there's a place for Cosmos in building these types of applications,
particularly when you look at things like chat. Like chat seems to be kind
of the kind of the interface, if you will, a natural language interface
into that. And you know, customers have been using Cosmos to build just
regular chat bots and chat applications for years, and that was kind of the
angle I took. I'm like, what's our role in that? And things
have been snowballing. In fact, it wasn't just me that was thinking and
looking about this, but also our own engineering team and even the open Ai
folks. Very soon after they initially launched, before that really exploded, they
migrated over from existing postcrists. They were running all of their chat GPT,
all the chat history and everything on. There was running ont of some Postgrist
servers and they were struggling as a numbers just a hundred million users in two
months. Yeah, it doesn't just happen for free. It was overloading because
they'd have to go and reshard their databases like constantly to try to keep up
with the growth in there. And we reached out to him and said,
hey, I think we've got a way we can help you with this,
and we walked them through Cosmos and explained, hey, this is you know,
this is scale out architecture here right, so beause postcrists is still in
RDBMS. So did they rearchitect their data? They did not really need to
re architect. Migration was not wholly difficult, and they were able to do
it basically with the lights on. Wow for that. So they migrated over
to us very early after they launched, and then we helped them ride from
like a handful to I mean their peak was like over one hundred and eighty
million daily active users. Wow. I think I remember the day that went
into effect, because it went from you know, one word at time to
just like yeah, yeah, And they did all of that with a single
Cosmos instance. Wow. We just basically as users kept piling on and kept
basically adding chats and sessions to it. You know, Cosmos just ended up
just just scaling out. That's what it does. It's a scale out architecture
in there, so was that the largest customer that demand wise that they're not.
They're not our largest customer, not even not even close actually, but
they're I mean they're large, yeah, for sure. And I will say
this one thing that was unique was the rate of growth. For sure,
the fastest app in history to reach those kinds of consecutive or daily acting.
Us growing at that rate, I think it's it's pulled back from that one
eighty, but now with four to zero out, it's it's spiking again,
right, So they're they're likely going to go over that two hundred number,
I'm sure at some point. Now. We've talked about AI a lot on
this show since we've been a build and seeing all the great things coming in.
I've mentioned on more than one occasion that I'm so for developers embracing AI
and for you know, especially the offerings that you have get hub co pilot
and even the co pilot that is now set to allow us to sort of,
you know, reach into our enterprise and see, you know, and
ask questions in real language. That's great and and so, but I was
concerned on the first day. I'm looking at the keynotes and I'm saying,
you know, there's somebody out there that's saying, you know, I'm going
to use this to be my brain and you know, to make decisions for
me. I'm just going to trust it. And of course you know that
that's that's a problem. But then I was very excited to see on the
second day the security layer that Microsoft ads over these large language models, so
that you know, that's something that chat GPT doesn't have. If I'm not
mistaken, it's this thing that is looking for hallucinations and looking for inaccurate data
and even malicious use of it. So important, and I was very glad
to see that. You want to talk about that a little bit. That's
the whole responsible AI thing you're talking about there. That's a huge focus and
emphasis on everything we do. Everything we produce and release internally at Microsoft,
that's levernges of technology has to go through a pretty set of rigorous in seat
of testing and controls and all that stuff to make sure that it's adhering to
all those responsible AI principles that are within there. I'm really glad that we
out of the gate are applying this to that, and not just for ourselves
but also making sure customers that's applying those technologies, we have to keep them
safe. That's right as well, and keep their customers safe. We don't,
you know, think about the early debacles people misusing chat GPT. There
was a lawyer that had the tool right assummation that referenced cases that simply didn't
exist exist yah, and he didn't check. And now, oddly enough the
judge doesn't go for the oh the software did it answer? Yeap. You
know there was another case in Canada actually with their Canada Canada one. Yeah,
you remember you know that one? Yeah, no, I've paid close
attention to that. Where we'll tell this anyway. It was a chat bought
that uh, powered by an LM that was answering questions about somebody when needed
to make flight changes for a particular reason. I can't remember which reason it
was. And they said, oh, yeah, there's no feast for that,
that's fine. Yeah, And so they were one hundred bucks you could
yeah, yeah, anyway they went or he was supposed to get I think
he was supposed to get a refund, and then when he went to get
the refund, they're like, oh, you know, we don't rEFInd that.
It's like, well you told me, yeah, like, oh,
that's that's a mistake in the software. Yeah, And they actually went all
the way to court on it, and the court ultimately said they have an
employee had said that you would be liable. You just because you use a
tool to do something that employee might do, it doesn't mean you're not liable.
You've got to payback good on them. Yeah, it is good,
and you know, it kind of highlights an important aspect with building these types
of applications, which is, uh, you need to capture those interactions that
happen between customers and users and this technology, these lms, just like you
have and say any customer service scenarios that you always everybody's conversation be recorded.
It's trading purpose exactly right, you know what you must do the same.
Even more important because you're having a computer talk to you're experimenting this new technology,
right, Like I think we scrutinized customer service calls much more closely when
they were new than we do today. That this is new, so it
needs to be closely screwedin. You've got to make sure that your customer service
agents are on script and giving the correct information the same as you do with
your lms. And this is a key kind of workload that we're finding with
Cosmos these days and building these types of applications is Yeah, you can in
the past have used these things to build chat applications and chatbots and also just
chat assistance where you're talking and chatting with a human. Even more important now
where you have lms that are generating responses sometimes had a whole cloth if they're
not properly grounded to the data that they should be, you need to capture
those interactions. If for no other reason, then you need to be able
to audit that information. But you can take that information and also feed that
into maybe an mL model and do some kind of sentiment analysis. You can
feed that into some kind of clustering. You can do all kinds of analytics
on top of that after the fact, to make sure that your data is
better, make sure that the information it's better, the prompts are better.
That's super important, and you can't do it unless you capture that data to
begin with. Yeah, and so this has been a key workload for us
for Cosmos over the last eighteen months, is using Cosmos as part of building
that chat interface and capturing that information. And now with build, we just
announced our native vector search for Cosmos dB as well as something I'm super excited
about, which is this new suite of technologies called disc an N. So
an N is approximate nearest neighbor, right, and in the vector space.
That's essentially how you do these types of queries is you don't do a keyword
match or a full text match or search. You use vectors and vectorize data
and words and whatnot to to find the semantic meaning and al also find those
related terms within there. And so now we've got this capability built natively into
Cosmos. So customers that are using Cosmos to say, store and capture all
that chat interactions can now do vector search vector database correct, so you automatically
have a large language model or a small language. Yeah. And now not
just for the chat but also you know, vectorize your data like RAG pattern
is a big thing now, right. You want to be able to have
an LM that you don't need to necessarily retrain. You can provide it and
augment that information that it generates for a user using data out of your database.
And the best way to get relevant information is using these vector search technologies.
So take your data, your transactional or whatever data it is, vectorize
its stored in cosmos, and then do a vector search over that and then
have it enrich and augment. I like things like operations manuals, like documentation
about the business, about how we operate. Those kinds of things are easy
and obvious. But there are other things too that you might want to vectorize
that you might not be thinking of. Yeah, like things like operation manuals
and static documents. I mean you can do those, you can put them
in like a cognitive search or AI search. The harder challenge is how do
you do that over high volume transactional data. How do I do that?
If I'm trying to capture I don't know, actions in a shopping cart,
or what pages people are browsing or other things where you have an enormous volume
of data vector databases in the market today, and even search engines capable of
capturing that kind of data. You need an operational or a transactional data store,
especially a scalable one to be able to capture that. This is what
was unique about the announcement. And I'm not sure landed it build here,
but the disc and en technology with Cosmos really allows you to take and build
these types of scenarios over high volume transactional data and you just you can't do
that with existing like ahnsw types of data. We used to think that the
OLAP cube was like the end all be all of finding related data. That
wasn't you know, easily accessible by regular RDM myths. And now this takes
too all new level, doesn't it. You need? So really all you
have to do on that, I say all you have to do, but
if you use Cosmos dB as you're a data store, now all that data
becomes searchable, yes, and in fact natural language, and not just within
Cosmos. But also you know, we have a feature, you know,
fabric is a brand new YE or analytics one like data Live platform o.
The word fabric has been reused a couple of times, but I did notice
the logo on my shirt to yeah, that's the new fabric, the new
fabric. Yeah, but we have a we have this feature called Cosmos Mirror
which will take any data you write into Cosmos and then automatically etl that into
your one leg So if you want to take that data, yes it's already
vectorized, but you can store it in the one leg and then spin up
a new workspace and create new mL workbooks and you can do all kinds of
stuff like maybe do basic clustering in there, to do sentiment analysis over what
your customers are saying everywhere. You can go anywhere with this and do it
in a big data analytics platform to augment all of that in there and get
those insights and then use that to feedback in and that I think is the
promise and what's coming next is you can build these applications. Customers can come
and use them and they can you know, you can ground your data and
make sure it's providing good, sensible answers. But you really need is that
three sixty. So I want to take that data, I want to do
analytics on it, and then I want to apply that back to help refine
the system prompts that are within there, help refine all of the all of
the knobs and dials for how those things interact with customers in there, and
help optimize those things. Boy, that's just creating a whole bunch of new
jobs, isn't it. I mean, if you think about the expertise required
to do that, you know and do and hit those tweak those knobs and
dials. You know, that's that's like a whole new thing. Yeah,
but you know what it's Microsoft. What do we do is we take and
democratize things that are hard and make them accessible for developers. Right. Yeah,
you mentioned one like off the cuff there and we've been talking about a
run ass because we talk more about data analytics over there. But just as
it's one drive for data. Yeah, right, Like you think about when
you want to build a model like this, you need to pull from a
bunch of different data sources and so you know, you need different considentials for
every SQL server. Like just even finding all that stuff, it's a nuisance
and one like really comes at it. Of all the data is here,
and it's not a repository like you dump off a copy of the data here.
It's the data so it's being continuously updated as well. And it just
makes a consistent way for resources to get access to data. So like the
evolution of the data lake, isn't it. Yeah, And in fact it
doesn't have to just sit there, Like you can analyze and run jobs on
this data that's in there and then use something like a Spark to write it
back into like a cosmos or to a hot store or something. It should
also be available in the lake, right right, it is, right,
so it doesn't have to just sit there and just gather dust, right,
you know, take and analyze and process that data and then feed it right
back in again. Sure. Wow, yeah, mind is being blown well
here this week. Just wait, it gets crazier than that. I did
this show with a run already on run as and I'll put a link to
it. But you can put symbolic links into the lake to data stores that
are not in Azure. You can link to Oracle databases. You can look
up to S three, so all of the data appears visible in the lake,
even if he's even in the lake. But it has the credentials,
so if you have the rights to access that, then it deals with the
credential calling to go over to a WS and pull the data as part of
your search. Like oh so check out this big fish that I pulled out
of Lake Superior. Oh no that wasn't lake. You'reon whatever, like whatever
lake you want. You're good, right, You're fine. Before we get
to the break and we're it's coming up. But Cosmos dB when we talked
about in near early days was really about h geodistributed high velocity database and that
really to me said certain categories of projects are going to need this, Like,
so is that still the case? Like does it make sense for most
folks? You don't want to experiment with copilots that sort of thing that they
would go Cosmos, You know, it was never that was never the first
foot forward, right. It was always where you could take it if you
needed to write. That was always the promise of Cosmo says you could it
could you could use it for any size workload. But if if you needed
to do stuff that you could not do with a relational database, Cosmos could
take you there right right. You can always start small, and in fact,
a lot of the engineering an effort we've been putting focused on is really
designed to help improve that developer experience starting at a small scale. You know,
we have a serverless option now and actually just recently we announced the availability
for being able to take and seamlessly migrate from serverists to say an auto scale
and then if you want, you can go back again in there. But
that makes it easy if you start with like a serverless option. That's kind
of how you should do your dev test right, Like why pay for a
database? Why pay for compute if you're not using it? Right, it's
a dev right, or it's a test environment when you go into production.
Great. Then migrated over to an auto scale allow it to elastically scale in
and out, and that's great. Other things as well have always been I
guess, you know, for designing for a scale out database can be a
little tricky, you know, yeah, absolutely, Then I was perfectly happy
with that viewpoint that. Look, I've tried to scale document data stores across
multiple sides. That's hard, and you don't want to own that. It's
plumbing, right, So the idea that there was a knob in Azure around
causes Dbut that's put it in these geographies. Keep these syncs at this rate,
like you pay for it, but you don't have to own that complexity.
That's right. But it may begged the question like do I ever want
this in one node mode? But you're describing one node scenarios, you're at
least initially knowing you would go to multiple nodes they if the case arise,
it always starts small, yeah, right, So, and it needs to
be approachable and easy to use, and also it needs to be mutable in
a way, right. Like, one of the things that's been challenging or
is challenging for building distributed scale A databases is you typically have to make very
very complex design decisions very early on, like how do you partition data so
that it scales out? And in the past that was always a challenge because
in many cases you were asking a developer to predict how it was going to
be used once it was in production. Only thing worse than that prediction was
trying to change it afterwards. And that was the thing that we realized is
we needed to make it easy for a developer to change that because you guys
have all done this for a long time. You all know you get an
app into production and you're surprised by how customers use it, and then you
need to go back and back to the drawing board and I need to refactor
a bunch of this stuff. And that from a scale out point of view,
if you do not have your partition design correct, you cannot scale.
It is the key to scaling, and that's another recent innovation is we've made
it way easier for customers to go back and say, give me a big,
great easy button that I want to push and allow me to repartition my
data so that I can scale right and then maybe change it again. Like
absolutely, the data size grows and new features come in, you need to
repartition or have multiples. This is another innovation as well, is providing a
material easy way to do a materialized view on that data. This is common
with like say a CQRS type of pattern right where you have ingesting of lots
of data and you need to be able to scale that out. Well.
Rarely the case that you're going to read the data the same way you write
it. You may have data coming in you want to do reports by state,
or you want to do a report by some category, or you want
to do some kind of dashboard over here, and you can't run queries over
a large scale out database like that and get good performance on your repath.
You need to have that data kind of repactored and pivoted so that you can
do efficient dashboards, efficient reports, whatnot. So providing that capability just another
big read easy button for developers you don't and then so you you know,
I do not like pre optimizing. I've been burned by it too many times.
And I'm not saying I want to have the problem first, but I
kind of want to have the problem first, right because often those optimizations create
more problems than they solve. So it's better to watch the data flow in
and see where it's getting rough because you've got good instrumentation and go, okay,
well if you're protuction like partition like this will reduce the impact, and
then you try your benchmarkt like now you're describing as any area where I'm willing
to express now to say, okay, let's try a partition strategy and see
what the instrumentation. But this is key for a developer experience, right is
you need to have that flexibility to be able to come back and approach it
in a different way or solve it in a different way. Folks ask us
for a lot of certainty, and we don't have it. Man, Yeah,
yeah, I appreciate you state. It's like I don't have to decide
that right now. Let's wait till we see how the data flows before we
decide make a decision, and then have the flexibility to revise. Yeah,
we guessed here, it's a guess. We're going to measure it like this
and revisit this at some point and maybe adjust it. Yeah, yeah,
well I appreciate it. So make it easy for developers to run a small
scale they want to run on a single note. That's great, But help
them make it easier for them to achieve that kind of scale that you just
can't get that anywhere else. That's why customers use us. It's always been
Microsoft's mo And with that, I'm going to interrupt for one brief moment for
this very important message. Attention dot net developers looking for the ultimate SDK to
handle a lot electronic document processing. Meet tx text Control. Txtext controls your
go to solution for seamless PDF generation, secure electronic signatures, and efficient digital
forms processing, all within your dot net applications. Empower your products with robust
document management capabilities, boost productivity, and deliver top notch solutions to your clients.
Trusted by developers worldwide, including me and Richard, txtext control is the
SDK that makes a difference. Check out demos dot textcontrol dot com for live
online demos and see it in action. Hey Carl here, tired of the
debugging grind, Let's talk about a breakthrough that's here to help ray guns AI
error resolution isn't just another tool. It's like having a seasoned developer right by
your side, suggesting fixes and explaining errors in real time. If you're curious
as to how it can smooth out your debugging day, visit raygun dot com
slash dot net rocks that's raygun r A y g u n dot com slash
do O t n E t r O c k s for your free fourteen
day trial to explore the way reygun is revolutionizing debugging. And we're back.
It's not a Rocks. I'm Richard Canvill. Let's call Franklin Yo. Chat
with our friend Mark Brown in the latter stages of the twenty twenty four build
in Seattle. Gray day as usual, Seattle. That's how this place works.
It's I mean, it's cloudy and gray. It's just another day.
It's just another day. We call that Thursday. Well, I mean it
does seem like these LM workloads were made for you guys, Like, but
what's the shape of the data that they're typically managing. It's the vector data
stores, right, It could be anything, right because as a Jason data
store, we largely don't care. Yeah, you can take anything, right,
You can have hierarchy, you can do whatever. Yeah, it's it's
whatever. You need to go back to why we're in the space and what
we were doing. You know, vector databases are super hot, and there
are a lot of them out there now and they're getting a lot of attention,
and they're very important to building efficient workloads on these alboraks got an implementation
of a vector store inside of it now or will they will? Yeah,
So they've been doing a brute force thing lately. So you just basically put
an array of vectors into a into a into a column and then call a
cosign function on that thing til you can get basically it's it's an exact match,
right, or a brute force not doing approximate nearest neighbor. So I
got a story for it towards the beginning of the year, when you know,
before any of this was available and nobody even knew how to make a
vector database or what they were. I was approached by a colleague to put
together some training on AI and put you know, how to use chat,
GPT, how to use the tools, the APIs and and and it was
just because of was such a hot topic. And I said, after a
while when things changed so rapidly, It's like, you know, this stuff
is changing so fast, this is not the time. We just got to
wait, got to wait this out and see what major tools shake out,
because they will be available. And turns out I was right, Yeah,
yeah, it only gets easier. Yeah, all these things that we had
to Can you imagine going into a training class like in I don't know,
right before Windows came out, and somebody's teaching you how to write your own
windowing system, right, and the Windows comes out, It's like, well,
what the hell was all that then? You mean like prior to like
Jeff Richter's book or something like that, Like, please give me some MFC.
Yeah, I'm old me too with you. It's been crazy for the
last eighteen months. The everything, all the APIs for open Ai and everything
built on top of it and around it, whether it's things like a semantic
kernel or a lying chamber law index, these things are changing like weekly.
They would go and push new builds and you'd break half of everything that's right
there. And a lot of customers I was working with were frustrated because there's
a lot of center of gravity around using these orchestration SDKs for llms right,
like, they do provide a good kind of single sheene if you will try
and help connect all these pieces together. But man, they were changing so
fast, so fast that it was a lot of customers are like, you
know what, I'm just going to set those aside for a bit because I'm
trying to get something built here and if my billd breaks every single day because
you guys are changing, then I need to kind of take my own ownership
of that. Ryan McKay and I were doing a YouTube show called The AI
Bought Show, and it just kind of was like that, you know,
he would and in fact, he would go and test some things on against
I don't know if it was chat Gypt or just something in open AI.
Maybe it was like the you know, the playground, I think it's called
right, and he would test it all and then get the response, and
then we'd record the next day and the response was completely dead, and it's
like, oh, they fixed this because he was showing like how to break
it or whatever. Now it doesn't break. And the next day it was,
oh, well this kind of didn't work yesterday. It's amazing. It's
just just goes to show you, you know, just calm down. This
is a good time right now to investigate what Microsoft is doing because they they
are the tool builders. They are the tool buildings. I'll pull some links
for like lang chain and the other semantic kernel. This is something that we
haven't really talked about all that much on dot net rocks. Maybe we could
just give us an elevator pitch for what that is. Yeah, So Samanic
kernel is just among kind of these group of these orchestration SDKs for building l
ms and kind of the equivalent of these things that people may have heard of
would be lang Chain. It's really the big one I think out there.
La Index is another one of these things. And you know what they do
is Python, right, Yeah, they all support like Python and JavaScript for
the most part, like lang chain for sure. Samanic kernel offers a dot
net sdk. They also have a Python option in there as well. They
may have a JavaScript also that they're in there, but anyway, you know,
these things are changing so fast, they might have something else in there
as well. But the value of what these things do is they provide a
way to easily kind of plug in lots of different things. It could be
different lams. You can build kind of an agent's on top of these things.
They all have these notions of kind of planners or chainers within there,
so you can really build kind of these complex LLM workloads and do it in
a kind of an easier abstracted interface, right, so you don't need to
get down into the weeds of native SDKs to connect to all of these different
vector databases or llms or embedding models. You can just provide kind of that
single user interface for that. That's the promise of these things. The reality
for the last year and a half is they keep changing so fast that it's
hard to keep them and use them. But that's their promise, and I
think they do a good job of doing that. They are starting to solidify,
if you will, some level is a semantic kernel after these things that
were announced to build considered like a low level tool compared to like I don't
know, the co pilot studio or any of these other high level tools for
sure. I mean those are look at the copilots too. Is kind of
a low code way of going and building these types of of applications, for
sure. If you're going to go and roll your own and you wanted to
be able to bring together lots of different bits and pieces of these things.
You would use something like a semantic kernel or maybe a combination of like native
SDKs and semantic kernel to go and build these types of things. So that's
their value and being able to do that provide that pluggable interface, if you
will, for doing these things. I can feel people listening, going agents,
What what's that? Why do I need one of those or a series
of these? Are these like little experts in a particular silo that maybe talk
to each other or maybe have areas of ex ortise. Yeah, it's exactly
it it So let's just I mean, I could kind of give you some
a rough scenarios. If I'm in an e commerce type of application and I
want to do i don't know, product search or something or asked about products,
I may have an agent specifically tailored to doing product searches. Right.
I may have another agent tailored specifically to refunds or to some kind of customer
profile or updates or anything, I mean, anything that you need. Like,
all of these things kind of operate around the notion of you have a
system prompt that kind of defines the behavior and also grounds it to specific data.
Right, And So the idea of the agent as you've created this kind
of encapsulated whole that takes inputs and is specialized around a particular thing, maybe
grounded to some specialized data, and then provide specialized outputs for the thing that
Richard hates the most is that's an easy way to answer promorphize an expert software
thing. You can give them their own personalities and you can tell them to
answer every question like a sixties stoner or something. Yes, you know,
okay, dude, let me tell you what's going on. I did something
very similar. I created a demo that I've used at the Fabric conference that
was at actually it was a night last year, and it was built and
grounded to the old old Adventure Works twenty seventeen data based from SQL service.
So it's like a bike store, right. So I created this AI agent
and I called him Wheelie, and I went to the Being image generator and
I created an image looked like it looked like Clippy. Ironically, could have
like a little like a little cartoon avatar on a bike doing a big wheelie
or whatever, and he had a big smile on and stuff like that.
So I created a whole persona around this thing. The T shirt that said
keep on truck, and I should keep the T shirt for this thing keep
on wheeling or biking or whatever. But no, but it's a retail bike
store, right, I want to create an AI assistant that's really into biking.
So the system prompt for this thing is you were super enthusiastic about biking,
and you really just want to help people find all the biking and accessories
that they need so they can go have great biking adventures. Right, and
you should have all this energy and all that stuff. So what's interesting is,
you know, I go when you start testing this thing, and it's
exactly what it comes back with this And I find if in the prompts you
use word like brilliant you know, or just like master of you know,
it will actually be smarter, right in terms of you know, or the
perception will be smarter. So you can provide all of that stuff, you
can give it that personality, you can you know, it's a great way
to represent your brand if a way, right, if you have kind of
a specific vibe for who you are. As long as it doesn't tell you
this refunds that don't exist. Yes, that's right. Accuracy is a feature
too, and safety. Here's a coupon. Yeah, it's it's interesting to
you know, we did I did this great show with Steph A. Warre
as we talked about it. It's a language computer. It fits out language
for you based on your instructions. Yep. Ye, And it's just humans
that project understanding into that. Yes, you always have to remember your instructions
and also the data you give it as well. Right, it's only smart
enough for whither what's been trained for or the information that is provided. Well.
Yeah, and it needs testing. You know. The conversation over and
over again is that you need experts in the data sets to test it and
say, oh that is correct, that's incorrect, and sort of work your
way through it. It's it's it's certainly not a reason not to use it,
but it's like it's just like if you're gonna work in a dynamic language,
you need testing around it. Right, you know, you you you
have to do the whole thing, You have to own the whole problem,
and you can make these things work and be incredible labor saving devices. You
know, they talk about response times dropping down to seconds and you know,
making customers happy as long as it's correct. I had this argument with a
friend, you know, when I was talking about hallucinations and stuff and wrong
answers, and the argument was to defend this behavior. Well, I know
people like that too. It's like, yeah, my uncle Joe is a
complete bs er, but I'm not going to hire him to run my company.
Joe's not processing my refund. No, those aren't the folks you use.
So now fair enough, Like, what do you want from your software?
What are you expecting? I just wonder for you know, trying to
imagine the next couple of years in the transformation of the user interface, like
a lot less buttons and knobs and things and a lot more just to free
flowing texts. I think it's the natural language processing is kind of the going
to be the interface for these things, right, not typing your talking and
learning. Yeah, I also wonder about machine learning models for vision, Like
I've always thought it would be great to have an assessment of frustration on the
operator's part, right, but the software could see that you're struggling with it,
just you mean, like feed in a like a jiff of that guy
banging the hell out of keyboard or something anymore like most of these machines have
a camera on already. Yeah, Like, could I actually recognize your struggling
to achieve something in the software and then affect the software? Like do you
need help? Can I put up a prompt? You know? Or I've
noticed you only use these six features? Could I put them front the center?
Like? What if the UX was dynamic to the way the user used
it? Or please place that hamburger in the bag before I call and keep
this light for somebody to come over. Oh yeah, So I heard someone
on in one of the keynotes, and I can't remember who it was say
that many people would rather talk to an AI than a real person. That's
the first time I had ever heard that argument, because in my part of
the world, most people don't want to go to a website and chat with
a bot, you know, they want a real person. But is there
this growing population of people who would rather speak to an AI than a real
person. I guess it depends on the capabilities of that AAI. Let's start
with the initial thing is would you rather go to a website to get the
information? Would rather call the company? Right? Like I tend to not
want to pick up the phone anymore. Because it takes longer looking for the
least amount of time to get the things I need. And so you know,
the reason you've been avoiding calling tech support and things like that is it
takes a while and you have to keep explaining yourself in some ways. If
the software is the interface for that, it's less frustrating and you're talking usually
to someone who really doesn't understand what you're doing. But it's just taking your
answers and plugging them into an expert system and anyway, you know. Anyway,
so what you have as a transcribed version of a chatbod, like,
why is that absolutely right about that? Yeah? I think it's gonna whether
they want to do it or not is going to depend on how good the
experience is. Okay, well, here's a great example. Things that I
used to Google for I now ask chat gipt. So you basically believe in
the I Feel Lucky button on Google because you're just getting on answer. Yeah,
you're talking about school search. It's you know, you put in a
search term, you look at the array of answers, right, You're like,
I'll let me rEFInd their search term, right, but I will ask
a question like, you know, is there a coin operated laundry facility at
this hotel that I'm going to. It's much faster and easier for chatting through
a site for that, the calmbing through a site and trying to find it,
you know, or calling them or definitely calling them. Yeah, I
use the chatchipt on being like maybe five six times a day. Yeah,
it's replaced my search. Yeah, it's also replacing things like Google Translate,
for example, because now with four to oh and not for zero, but
four oh chpt four oh. You know, you can use your voice and
you can just talk to somebody and it will say, all right, translate
the English to Spanish, the Spanish English. We'll just do that. Yep.
It's powerful. It's powerful. Stuff's actually I always had a conversation with
someone's adjacent but not the same thing. Whose husband is waiting for a cock
clear implant. His hearing has gone that bad. He says, I don't
even want to talk to him or I have to shout and I always sound
angry. And I pulled out Live Trends, I grabbed her phone, installed
Live Transcribe on it, and then we just kept talking and she's like the
words are coming out. It's like just put this in front of him,
like, what are you doing? You've had the tool in your pocket the
whole time and you didn't know. Isn't that funny? Yeah, But the
fact the idea of walking into a shop in France when your frendsh is bad,
my friend is bad, and being able to just have it spew frenchise
Frenches just slightly off. Yeah. They maybe bring the world closer together.
Yeah. Yeah. It was a while ago that Google did some keynote thing
where they were where they were doing live transcription and translation and voice and and
you supposed to get a product for that, but it's really good. There's
a browser plug in for Google. Well that's the truth. I think it's
now no longer you're not going to buy dedicated hard for that. Your phone
will do it. Yeah. It just is a thing now, it's a
thing, a new capability being added to the phone again because it didn't stock
like, it didn't do enough. So the doomsayers will say, you know,
all these things are being replaced by a single app and a single interaction.
What you know is everybody else doomed? You know, is this so
disruptive that it's going to cause mayhem? I don't think it's anywhere disrupted than
anything else. Yeah, I agree, you're appreciating the choir, But you
know, there's certainly people out there who were You know, the story of
the lad Heites, you only go as far as the Lades were actually a
group of people who very technical people by the way making They were hand manufacturing
cloth, right with very simple tools. And in came the first steam powered
looms and they smashed them up. And the thing they never talked about after
that is after that the employer came back and they talked through how they were
going to train, and all those folks started running those automated looms. And
the biggest thing to happen is, yeah, you need a fewer people making
cloth, But cloth became so much less expensive that people bought more clothes,
right, yeah, and then many more people employed. Well, the term
ludite is thrown around as someone who doesn't understand or like technology, and that's
not the case. The Luddites were very technical. They were they were the
high tech, but they were they were frightened by the change, by the
change. I have a more recent example, and that is just the kind
of the emergence of cloud I used to see a similar kind of reaction from
your old I T pro guys that were out there in the loading dot grabbing
all those racks of hardware and then going and racking and loading all that stuff
in, and then they're like, wait a minute, now that's my job.
My job is doing that. Now you got some guy with a web
page pushing a button. You have friends that are fit that I think you
know, I'm talking. We have several front and I told him, I'm
like, look, I go that piece goes away. But there's a whole
ton of other stuff now that you were going to learn and do, and
you were getting to the bottom of your to do list, right, That's
true. It's not a thing. It's not a thing. There's so many
more problems to solve now. I got to I say this on every dot
and I rocks show we've done since we've been here. And I'll say it
again. The chances of you being replaced you as a developer being replaced by
an AI are slim to none. But the chances of you being replaced by
a developer that uses it are high. Not high maybe, but significant enough
so. And there's no excuse for developers there's no you know, there are
no safety concerns about using an AI to help you write code, to look
things up, to do research. You know, they just don't ask it.
Whether you should ask your boss for a raise, you know, that's
that's not ken If you want it's just the you still have to live with
the consequences, that's right. Yeah, you may get bad advice, but
you won't get bad code. And if you get bad code, just ask
again. Yeah, try it again. I've been here's a trick, ask
it to comment your code that it generates. I'm all in. Yeah,
the last eighteen months I even used it, and the good HELB copilot is
a great example. I needed to I needed to retrain learning Python. Yeah.
I've been a c sharp developer for you know, as long as it's
been around in two thousand and two, I guess right, and even before
then, of course other Microsoft technologies, and I go back to VB back
in the nineties, but me too. This AI space has put a lot
of focus on Python right because, and I see this with the customers.
A lot of these projects will start within kind of the data engineering and machine
learning space because it seems to be that's the center of gravity for a lot
of this machine learning and this space. It is a great language, but
I didn't I was not very fluent in it, but I needed to learn
it. And I've been using gethub, copilot and the tools in visual Studio
and vis code and elsewhere to help me retrain, to help me retool myself
so that I could be more productive writing Python. So you have this assistant
effectively, that's coaching you on writing your running better. Take this C sharp
code converted Python. Yeah, tell me comment it, tell me what it's
it's actually briant. You know what I do now is I write a comment
as to what I wanted to do, and then the Python comes out,
and then it shoots out Python. I'm like, yeah, that's great.
Yeah. Yeah. I can write the code and have it commented, or
I can write the comment. It'll write the code. Yeah, you have
a choice. Is that like schema first versus code first? Yeah. I
write the comment and it writes code that also has comments. Yeah, ma'am,
I used it, and I told you this. I used it for
you know. It would spit out link statements that were some not readable by
my customer, and so I basically asked it to you know, split this
out into multiple for loops or whatever and comment it. And I took that,
commented it. Yeah, and so you know, above the liuery,
above the link query, this is what this link query does. Great,
So not only is it solving a problem, but it's educating my customers,
right. Better documenting, yeah, much better documentation. Yeah. I think
it was twenty twenty two. My GitHub annual reports that the primary language I
was writing in that year was Python. Really, I didn't think I was
writing that much Python. As you know, I munch a lot of data
in my life and buy golly pythons a bunch of data and so just you
know, use the tool of them. I mean, the second was YAML,
which makes me sadder, and the third was c sharp. Thank goodness.
Yeah, I'm sorry for your YAML. Yeah. You know, I
run a lot of infrastructure too, So Yamil's my friends there, you go,
that's a good job. Yeah, oh well I got I got that
they complain about. It's all good, right. And then it's also so
this reality, it's like you are not the language you programming, right,
You're the person who solves problems, and you know, the more tools y
am your toolbox the easier gets that is the key is a ye. Yeah,
yeah, I'm It's been a long time since I cared what language I
was working. I'm just trying to get to the bottom of the problem.
And now the tooling is super good at helping me. You know, why
am I struggling with this? You know? And and it'll it'll get you
over the hurdles. This is why developers, if for no other reason,
should embrace this. Yeah, right, because it's a great robber duck.
Holy man, it's such a good rubberd to make so much at very least,
yeah, so much more productive. Yeah, I spent much time testing,
less time just debugging stuff. I'm getting it just front to end.
It's getting done faster. And here's the thing, like, you know,
I don't be doomsday, but you know, if you don't embrace it,
you may it may be to your detriment. So why not we work in
technology? Why should we not be embracing that? Sure? Why are you
using an it like you could just be? When you got into this gig,
you knew it was going to be changing fast, right, that's what
got you excited about it? Well, yeah, here it is again.
We're turning the crank and it's changing fast. Using your chromium tip tweezers to
light up electrons anymore either, right, like listen, shout out to right.
Not organizing electrons one at a time anymore. I can hold B zero
THEREO one zero one zero zero, just another tool, more automation, more
productivity. Right, and you do. Maybe we'll get a little further down
that list. The joke of courses. Every time we get these things,
list gets longer. Yeah, there's more to do. There's so much more
to do. Mark Brown, thank you. It's been a great pleasure talking
to you, and this is great stuff. I love talking to you guys.
Thanks for having me back on. I'm glad I could be your closer
for this build for it absolutely. Thank you friend, all right, and
we'll talk to you next time on dot net rocks. Dot net rocks is
brought to you by Franklin's Net and produced by Pop Studios, a full service
audio, video and post production facility located physically in New London, Connecticut,
and of course in the cloud online at PW O P dot com. Visit
our website at d O T N E t R O c K S dot
com for r s S feeds, downloads, mobile app comments, and access
to the full archives going back to show number one, recorded in September two
thousand and two. And make sure you check out our sponsors. They keep
us in business. Now go write some code. See you next time.
You got jad middle Vans the