Speaker 1: How'd you like to listen to dot net rocks with
Speaker 1: no ads?
Speaker 2: Easy?
Speaker 1: Become a patron for just five dollars a month. You
Speaker 1: get access to a private RSS feed where all the
Speaker 1: shows have no ads. Twenty dollars a month, we'll get
Speaker 1: you that and a special dot net Rocks patron mug.
Speaker 1: Sign up now at Patreon dot dot NetRocks dot com.
Speaker 1: Happy Chris Mahana Kwanzadon. It's dot net Rocks. I'm Carl Franklin, an.
Speaker 2: Amateur Campbell and we timeshift. So this is actually being
Speaker 2: published in January.
Speaker 1: That's right, January ninth. So we hope you had a
Speaker 1: happy holiday anyway.
Speaker 2: Yeah, I'm sure it was great.
Speaker 1: Yeah, we haven't had ours yet as of this recording,
Speaker 1: No December nineteenth, but you know it's going to be
Speaker 1: a good show. Our old friend Thomas Betts is with us.
Speaker 1: And before I get into better know framework, I just
Speaker 1: want to check in with you, Richard Campbell and see
Speaker 1: how your holiday is shaping up.
Speaker 2: Oh, now we're gonna we're shortly after this, like this weekend,
Speaker 2: we'll go to the city for the week Lots of
Speaker 2: friends to visit a few different parties, go to spend
Speaker 2: time with the girls. It's probably been the leak out
Speaker 2: by now. The youngest is pregnant, so she's like, hey,
Speaker 2: can I bourrow one of your old winter coats because
Speaker 2: mine don't fit anymore.
Speaker 1: Yeah. Kelly's daughter had a daughter and she turned one
Speaker 1: last week. Oh yeah, right, so we had, you know,
Speaker 1: the obligatory smash the cupcake in your face birthday party.
Speaker 2: That's the thing.
Speaker 1: It was pretty awesome. Yeah, all right, well, we've got
Speaker 1: a lot of stuff to cover in better know framework.
Speaker 1: It's history day here, so roll the music, all man,
Speaker 1: what do you got? So? I hadn't done this in
Speaker 1: a while, but you know, nineteen thirty two, which is
Speaker 1: our episode number, was such a jam packed year that
Speaker 1: I want to talk about some things that happened. And Thomas,
Speaker 1: I know you're there if you have anything you can
Speaker 1: jump in.
Speaker 2: You two.
Speaker 1: Richard, of course you are the history guy. So January fourth,
Speaker 1: British colonials in India arrested Mahatma Gandhi and put him
Speaker 1: in prison. January twelfth, US elected their first female senator,
Speaker 1: Hattie Wyatt Caraway, who represented Arkansas. February seventh, NASA astronaut
Speaker 1: Alfred M. Warden was born in Jackson, Michigan. He went
Speaker 1: on the Apollo fifteen mission, which saw the use the
Speaker 1: first use of a lunar roving vehicle, a little Buggy,
Speaker 1: Little Buggy.
Speaker 2: March first.
Speaker 1: Lindberg plays a big role In nineteen thirty two. The
Speaker 1: Lindberg kidnapping occurred, where the twenty month old son of
Speaker 1: famous aviator t Arleson and Lindberg were kidnapped from their home.
Speaker 1: Was kidnapped from their home in east Amwell, New Jersey,
Speaker 1: March fourteenth. George Eastman, the inventor of the Kodak camera,
Speaker 1: shot himself in the heart, aged seventy seven. March seventeenth,
Speaker 1: the German police rated Hitler's Nazi headquarters April's second. Famous aviator,
Speaker 1: Charles Lindberg, paid fifty thousand dollars as a ransom for
Speaker 1: his kidnapped son. In April nineteenth, President of the United
Speaker 1: States Herbert Hoover first suggested the five day work week.
Speaker 2: So you can.
Speaker 1: Blame him for that. What was it before seven days?
Speaker 2: Seventh? It was huh, the were six actually because he
Speaker 2: was supposed to rest on Sundays, right, okay.
Speaker 1: May fourth American gangster Al Capone entered the Atlanta Prison
Speaker 1: convicted of income tax evasion. May fifth, Japan and China
Speaker 1: signed a piece treaty. May twelfth, the body of famous
Speaker 1: aviator Charles Lindberg's kidnap son is found in New Jersey.
Speaker 1: Ransom didn't do any good, apparently. May twenty first, Amelia
Speaker 1: Earhart became the first woman to complete the transatlantic solo flight,
Speaker 1: having flown for seventeen hours from Newfoundland, Canada, to Londonderry,
Speaker 1: Northern Ireland. June thirteenth, Great Britain and France signed a
Speaker 1: peace treaty. July twenty eighth, Douglas MacArthur, acting against US
Speaker 1: President Hoover's orders, commanded several attacks on the Bonus Army,
Speaker 1: which was World War One veterans in their families, attempting
Speaker 1: to a victim from their encampment. At least two veterans
Speaker 1: died in the attacks, with fifty five injured. July thirty first,
Speaker 1: Nazis gained thirty seven percent in the Reichstag elections in Germany,
Speaker 1: becoming the largest party in parliament by a large margin.
Speaker 1: August second, American physicist Carl David Anderson discovered and photographed
Speaker 1: a positron the first known anti particle, and Thomas, do
Speaker 1: you have some other news about physics, don't you?
Speaker 3: Well? I saw that the nineteen thirty two Noble in
Speaker 3: Physics went to Werner Heisenberg. Ah, he got it a
Speaker 3: year before Schroedinger. So all your quantum mechanics and atonic theory.
Speaker 1: A lot of stuff happened in thirty two, all right.
Speaker 1: So August sixth the Venice Film Festival, the world's oldest
Speaker 1: film festival, opened for the first time. August thirteenth, President
Speaker 1: von Hindenburg refused Adolf Hitler when he asked to be
Speaker 1: appointed as chancellor, and instead offered Hitler the position of
Speaker 1: vice Chancellor of Germany. Hitler refused a position and announced
Speaker 1: he would oppose every government not headed by himself until
Speaker 1: he was chancellor. August sixteenth. John Lindberg, Charles Lindberg's second son,
Speaker 1: was born just five months after the kidnapping and death
Speaker 1: of his older brother, Charles Lindberg Junior. Almost done. Here
Speaker 1: is Stember twenty in his cell at your word. At
Speaker 1: jail in Pune, India, Gandhi began a hunger striking against
Speaker 1: the treatment of India's lowest classes, known as Untouchables. October two,
Speaker 1: the New York Yankees won their twelfth consecutive World Series game.
Speaker 2: Yay.
Speaker 1: As a Red Sox fan, I can still appreciate that.
Speaker 1: October third, Iraq gained full independence from Britain and joined
Speaker 1: the League of Nations. October ten, the largest hydroelectric power station,
Speaker 1: the niper Dam dni nieper nieper Dam, was first put
Speaker 1: into operation in the USSR. It's actually Ukraine, but well
Speaker 1: it is now, right, always was was it USSR?
Speaker 2: Then? Well, okay, he was always in Ukraine. Okay.
Speaker 1: November sixth, the Prime Minister of Italy, Benito Mussolini, introduced
Speaker 1: an amnesty decree freeing thousands of convicts. December fifth, Albert
Speaker 1: Einstein was granted a visa to enter the United States,
Speaker 1: and December twenty seventh, Radio City Music Hall first opened
Speaker 1: in New York.
Speaker 3: Sit.
Speaker 1: And that's just a little bit of what happened in
Speaker 1: nineteen thirty two. A lot of very famous people that
Speaker 1: you know were born in nineteen thirty two. But I'm
Speaker 1: not going to go down there. It's sort of the threshold, right,
Speaker 1: like that's a long time ago now.
Speaker 2: Yeah. Yeah, I'm working on the space geek out, which
Speaker 2: will have already been published by the time you heard this,
Speaker 2: and I'm still writing the script. And nineteen thirty two
Speaker 2: is also when Karl Janski created the first radio telescope,
Speaker 2: not because he was an astronomer, but because he was
Speaker 2: trying to build a directional antenna and he kept having
Speaker 2: this hiss no matter where he pointed the antenna, so
Speaker 2: the directional antennae could tell which way like thunderstorms and
Speaker 2: stuff were because lightning strikes have created a radio wave
Speaker 2: that he could say, Okay, well the lightning coming from there,
Speaker 2: but there was this hiss that was coming from everywhere. Wow.
Speaker 2: And it took a while to finally figure out that
Speaker 2: it was the hiss of the universe. Yeah, and it's
Speaker 2: actually the cosmic background radiation from the Big Bang. Wow.
Speaker 2: And that he ended up building the first radio telescope
Speaker 2: just trying to figure out what was going on, an
Speaker 2: entirely new class of astronomy. So cool.
Speaker 1: Yeah, you got to have an imagination to do that
Speaker 1: kind of stuff, can't.
Speaker 2: Just like he had a problem. Yeah, the problem is
Speaker 2: what's that noise? What is that? Okay, Richard, who's talking
Speaker 2: to us? He got a comment, Yeah, grabbed a comment
Speaker 2: on off the show eighteen fifty eight of Moth we
Speaker 2: do with Thomas back in August of twenty three, so
Speaker 2: a little over a year ago, and that was the
Speaker 2: leveling up your architecture game conversation. Seem to have a
Speaker 2: theme with your shows, Thomas. And there was a bunch
Speaker 2: of good comments on the show, and this one comes
Speaker 2: from Laslow. He says, the more shows I listen to
Speaker 2: about software architecture, the less I'm sure about what it's about.
Speaker 2: I've seen great examples of software architecture and designs, but
Speaker 2: I've never seen them evolving from an architect's point of view.
Speaker 2: A big bowl of mud never starts at the beginning
Speaker 2: of a software's lifetime, but a couple of years later,
Speaker 2: when the architect has left the project, new developers are
Speaker 2: not fully onboarded, and the clients realize they can push
Speaker 2: your requests that were previously quote too hard or too
Speaker 2: expensive to implement you and sometimes junior developers cross the
Speaker 2: line in previous design reels to be clients happy, making
Speaker 2: the application architecture rot. I'd like to see an architecture
Speaker 2: that stands the test of time. Hm hmm. You know,
Speaker 2: architecture isn't a thing right. It's a set of ideas
Speaker 2: that have to be implemented by people, and as long
Speaker 2: as people don't implement them, you can take any building
Speaker 2: and stick something horrible onto it and damage its architecture.
Speaker 2: Same with any piece of software. The only way architecture
Speaker 2: stands the test of time is that people choose to
Speaker 2: care for and pretty much leave it alone. Well, no,
Speaker 2: I mean challenge it, improve it. I could. It makes
Speaker 2: sense when you have a feature that's a problem in
Speaker 2: the current architecture, say, is the architecture wrong, Like, is
Speaker 2: there a way to press against the architecture to still
Speaker 2: get some of the benefits of it, of the extreme design? Well,
Speaker 2: we do that all the time anyway, and some of
Speaker 2: us just admit to it is compromise, right. No, you know,
Speaker 2: all ideal architectures can't be built. It's always a depression
Speaker 2: of less than I do.
Speaker 3: You always end up with an architecture. It's whether you
Speaker 3: made it explicit decisions or implicitly accepted the decisions along
Speaker 3: the way. And if you just keep letting it rot,
Speaker 3: as they said on the comment, then yeah, you'll end
Speaker 3: up with the big ball of mud and you won't
Speaker 3: have a nice, clean architecture, but you'll still have an architecture.
Speaker 2: Unfortunately. Yeah, nobody makes a ball of mud, it just emerges.
Speaker 2: So Las Well, thank you so much for your comment.
Speaker 2: And a copy of music co Buy is on its
Speaker 2: way to you. And if you'd like a copy of
Speaker 2: music code By, I write a comment on the website
Speaker 2: at dot net rocks dot com or on the facebooks
Speaker 2: we publish every show. There to be comment there and
Speaker 2: I read it on the show. We'll send you a
Speaker 2: copy of music code By.
Speaker 1: And we're also on all the social media's we're on.
Speaker 1: We've been on x Twitter for a long time, at
Speaker 1: Carl Franklin at Rich Campbell, and we're also on blue Sky.
Speaker 1: I'm at Carl Franklin bsky dot app, and I'm Richcampbell
Speaker 1: at besky dot app or and is it app? Is
Speaker 1: it app or is it social I don't even know
Speaker 1: whatever blue sky.
Speaker 3: Dot, bsky dot net or dot whatever.
Speaker 2: Yeah, bsky dot app. It is bsky dot.
Speaker 1: Okay, good because you know, I just pull it up
Speaker 1: in the browser and it's there.
Speaker 2: I don't really pay attention much anymore of the URL.
Speaker 2: But anyway, doing all that stuff through buffer these days,
Speaker 2: I you known't actually see the regular sites much. What's
Speaker 2: buffer pray tell She's a tool for looking across all
Speaker 2: the different social media's that you're dealing with, and that's
Speaker 2: how we get all the posts out for all the
Speaker 2: shows I need that. It's a good little product. Yeah,
Speaker 2: nothing bad to say about it. I switched to it
Speaker 2: attle while ago, and I've been very happy and it
Speaker 2: works with them all.
Speaker 1: And we're also on macedon. I'm Carl Franklin at tech
Speaker 1: Hub dot Social.
Speaker 2: And I'm rich Gamble at mass it on social, So.
Speaker 1: You know, get in touch with us, ask us questions
Speaker 1: and you may get a free copy of music to
Speaker 1: Code by as well. All right, Well, that last voice
Speaker 1: you heard there before Richard was Thomas Betts and he
Speaker 1: is a laureate software architect at Blackbod baud Blackbod, the
Speaker 1: leading software provider for social impact. In his spare time,
Speaker 1: he contributes to InfoQ dot com and helps organize Q
Speaker 1: on software development conferences.
Speaker 2: This is interesting.
Speaker 1: Credits dot NetRocks for inspiring him to give back to
Speaker 1: the software community as a writer, podcast host, and international speaker.
Speaker 1: Well that's nice. How you doing doing well?
Speaker 3: Yeah? I realized that we don't talk about the stuff
Speaker 3: I do with InfoQ, but it's kind of my side project.
Speaker 3: That's the job I've had the longest now for like
Speaker 3: eight or almost nine years.
Speaker 2: Wow.
Speaker 3: So yeah, being able to speak locally at a developer
Speaker 3: conference and now I spoken internationally and organize a conference.
Speaker 3: So kind of following you guys the example. You know,
Speaker 3: it started with writing into comments I think, like I
Speaker 3: don't know, two thousand and seven. Yeah, just to join
Speaker 3: the conversation and give back to the community.
Speaker 2: Yeah.
Speaker 1: Yeah, we've been friends for a long time.
Speaker 2: Yeah.
Speaker 1: And so your your topic these days is architectural intelligence,
Speaker 1: which you say is the next AI, meaning the next
Speaker 1: AI acronym, right, exactly.
Speaker 2: So let's define what that is.
Speaker 3: Well, I want to you know, right now, we've got
Speaker 3: everyone's wanting to put AI on everything. But I think
Speaker 3: we're in that Arthur C. Clark moment where any sufficiently
Speaker 3: advanced technology is indistinguishable from magic. Oh yeah, and that
Speaker 3: right now, any software we don't understand, we just call AI.
Speaker 2: Right.
Speaker 3: We don't know what it is, but everyone's asking for it.
Speaker 3: The CEOs are asking for it. The product owners are saying,
Speaker 3: I need a AI in my products. We look innovative
Speaker 3: customers aren't asking for it, but give it a couple
Speaker 3: of years and the expectation will be there. Like, why
Speaker 3: it doesn't this have AI? It must not be modern?
Speaker 1: So Thomas real quick. I play in a band and
Speaker 1: it's a ten piece band and it's awesome, and you know,
Speaker 1: after we do a particularly great tune, I will say,
Speaker 1: this band does not use AI.
Speaker 2: Yeah.
Speaker 1: It's kind of like when you get rice cakes and
Speaker 1: they say fat free. You know, it's like, well, duh.
Speaker 1: But everybody thinks that anything like any kind of pedal
Speaker 1: that we're using to change our voice or any kind
Speaker 1: of sound. You know, software is AI just because it's
Speaker 1: good and it might use digital signal processing or whatever,
Speaker 1: but it's not AI.
Speaker 2: Right.
Speaker 3: I think we call stuff AI until we have something better,
Speaker 3: and then we call it computer science.
Speaker 2: Ye.
Speaker 3: Right, we have a product and a tool and a name,
Speaker 3: and if we go back to you know, like AI.
Speaker 3: When we talk about AI, mostly what we mean is
Speaker 3: generative AI, not general AI, not artificial general intelligence. That's
Speaker 3: data and the terminator and other characters from science fiction.
Speaker 2: I don't think the average person even thinks that far.
Speaker 2: They're just looking at large language models right right.
Speaker 3: Right right, and genhi is large language.
Speaker 2: Model is one of them. But you're doing gen ai
Speaker 2: disservice if you can, you know, keep it scoped into LMS.
Speaker 2: It's a bunch of other things. It's been you know,
Speaker 2: that's been going on for a decade. LLLM is only
Speaker 2: kind of detonated with chat GPT in twenty two. Like
Speaker 2: this is all. This current storm is pretty recent. And
Speaker 2: anything I've learned from the shows we've done recently is
Speaker 2: that the actual smart machine learning people in this space
Speaker 2: are pretty offended. What do you do it because a
Speaker 2: lot of this stuff is sloppy machine learning.
Speaker 1: Yeah, and I'll I'll just talk about jen ai. In
Speaker 1: terms of images, I can spot a chat GPT generated
Speaker 1: image a mile away now.
Speaker 2: It just has a look to it.
Speaker 1: And I'm I'm offended when I'm scrolling through Facebook and
Speaker 1: I see a picture of this idyllic house, you know,
Speaker 1: with perfect lighting with waterfalls going through it, and you know,
Speaker 1: there's no comment. It just says ah or something like that,
Speaker 1: you know, and and there's a million views and a
Speaker 1: million likes, and it's clearly generated. It doesn't exist, and
Speaker 1: there's no there's no place, there's no date. It's just
Speaker 1: like a bucolic setting, right, and.
Speaker 3: Half of those million likes are the that are liking
Speaker 3: the thing that the other bot created, so.
Speaker 1: And people that use them to take pictures of themselves
Speaker 1: and turn them into AI pictures.
Speaker 2: No, stop that.
Speaker 3: I like that you brought up like traditional machine learning,
Speaker 3: like we used to call what's now just established machine
Speaker 3: learning and an mL model. Like for a while that
Speaker 3: sounded like AI, and then we move the AI a
Speaker 3: little further. It's that marketing term that just kind of
Speaker 3: is the umbrella.
Speaker 2: Yeah, I've always said it coined it as artificial intelligence
Speaker 2: is what you call it when it doesn't work. Yeah,
Speaker 2: so as it does work, it'll get a new name.
Speaker 3: But I think there is that correlation between a large
Speaker 3: language model and other machine learning models, Like the difference
Speaker 3: is in the algorithm inside. And if you're not a
Speaker 3: data scientist, you probably don't understand. But I still, because
Speaker 3: I don't understand the inside, I treat it as you know,
Speaker 3: a function box. I put it in an input, I
Speaker 3: get an output. So if I'm doing image recognition, I
Speaker 3: send it to my image recognition machine learning model, and
Speaker 3: it says this is a cat, this is a dog,
Speaker 3: or comments on the on a block well.
Speaker 2: And it's the whole point. I have an API. I
Speaker 2: don't want to know. I don't know how to create
Speaker 2: a cryptographic key, but I do know how to call
Speaker 2: an API that gives it to me exactly.
Speaker 3: And so these are the things that software engineers know
Speaker 3: how to use, Like this is some little function box
Speaker 3: and I can just call it, and here's my input
Speaker 3: and what's my expected output. I think what you need
Speaker 3: to understand with large language models is you give it
Speaker 3: a series of tokens, a bunch of words. We call
Speaker 3: them tokens, but token might be part of a word.
Speaker 3: That's some of the semantics need to learn.
Speaker 2: But I think it's a you know, the joke is
Speaker 2: a tokenization. I think it's the cleverer attack here, right,
Speaker 2: Like the stochastic parative spitting language back is not that impressive,
Speaker 2: But the fact that we've come up with a strategy
Speaker 2: for converting language, virtually any language, into a set of
Speaker 2: numeric symbols that, by the way, cross between each other,
Speaker 2: like it's the Babelfish man, Like you've almost cracked universal translation.
Speaker 3: Right, because that's that's what tokenization is. Like, here are
Speaker 3: all I want to take this string of characters and
Speaker 3: turn it into a bunch of floats, and it's not
Speaker 3: the asking number of like this is an A and
Speaker 3: this is a B. Like this word or part of
Speaker 3: a word has this representation in a multi dimensional array,
Speaker 3: and that's all it is. It's a lot of math, right,
Speaker 3: it's all statistics. And when you feedsie into a large
Speaker 3: language model, here is my series of tokens, here's my
Speaker 3: words and my sentence, and all it gives back is one.
Speaker 3: It just predicts the single next word. And I think
Speaker 3: that's the mystery people don't recognize, is it takes that
Speaker 3: one like, well, that's not useful. And you watch chat
Speaker 3: GPT and especially the early versions, you watched it type
Speaker 3: out very slowly. What it's doing is it's feeding that
Speaker 3: one token back in adding to you, and that keeps
Speaker 3: building up the context that auto regression eventually produces a
Speaker 3: series of words that say, oh, the next part of
Speaker 3: the sentence is likely to be this, yeah, and it
Speaker 3: looks like magic yeah.
Speaker 2: Well and them Importantly, it can be interpreted as intelligence
Speaker 2: where none exists. Yeah.
Speaker 3: And this goes to if we look at the you know,
Speaker 3: the learning aspect of it, like we're fine calling it
Speaker 3: machine learning because we start with training data. For any
Speaker 3: machine learning model, you give it a set of training data.
Speaker 3: Maybe that's your sales data for the last you know, quarter,
Speaker 3: or it's pictures of dogs and cats. In our case,
Speaker 3: it's just words. Have this thing read everything you can
Speaker 3: find and train it on that. So you give a
Speaker 3: lot of training data that's words, and then it's really
Speaker 3: good at understanding words, but it's still just predicting the
Speaker 3: next word. And when we see something that looks like
Speaker 3: a human probably did that, and we don't understand how
Speaker 3: a computer could do that, we think, I don't understand it, it
Speaker 3: must be intelligent, and we're applying that where it just
Speaker 3: doesn't exist.
Speaker 2: Well. Plus, and humans are prone to that sort of
Speaker 2: thing anyway, right, Yeah, heck, we think our dogs understand Yeah,
Speaker 2: and we talk to our cars, which especially weird this
Speaker 2: tendency to ANSWERPROMORPHI. It's like, it's oh, it's necessary. I mean,
Speaker 2: is it necessary or is it a weakness?
Speaker 1: Oh?
Speaker 2: I think so.
Speaker 1: I think it's necessary for us to be able to
Speaker 1: have some sort of relationship with the thing that we're anthropomorphizing.
Speaker 1: It's easier for us if we give it human at attributes.
Speaker 1: I think it makes it easier.
Speaker 3: Architects are good at coming up with metaphors to describe
Speaker 3: stuff and people. How is someone going to relate to
Speaker 3: the software? How do I translate this complex idea into
Speaker 3: a design that my engineers are going to be able
Speaker 3: to implement or the users are going to understand? And
Speaker 3: things like the desktop on my computer, well that used
Speaker 3: to be analogous to the desktop where I'm setting the
Speaker 3: computer down on and I can have a pile of
Speaker 3: papers here.
Speaker 2: Right.
Speaker 3: We have these ideas and then those just become things,
Speaker 3: and then a floppy disk icon sticks around for twenty
Speaker 3: years after we've gotten rid of floppy disks yep.
Speaker 2: Yeah, and now it is more better known as a
Speaker 2: save icon than it is as a physical thing, right. Yeah.
Speaker 2: People think it's weird that I three pre printed as
Speaker 2: save icon, like what's wrong with you? Why would you
Speaker 2: do that? But everybody recognizes it and that's why we
Speaker 2: still use it. Yeah.
Speaker 3: So let's get to the architectural intelligence part of this.
Speaker 3: I think it comes down to two questions. If we
Speaker 3: have jen AI or really just LLMS, the two questions
Speaker 3: are is this appropriate for my software like the scenario
Speaker 3: I have? And then if I decide it is how
Speaker 3: do I optimize it? And I think you can look
Speaker 3: at some of the examples of when it makes sense
Speaker 3: to using your software because it is a language model.
Speaker 3: It's good at doing things that are language based, right, right,
Speaker 3: like you want to have a natural language interface. These
Speaker 3: used to be things that people worked really hard on
Speaker 3: and turns out you can just throw stuff at it
Speaker 3: and it can translate it into something else. Like you
Speaker 3: said that universal translator. I can't write a good search query,
Speaker 3: but now I can just talk to it and transcribe
Speaker 3: and it just works.
Speaker 1: Yeah, and you might think of replacing certain complex patterns
Speaker 1: whereas you know, picking something from a series of drop
Speaker 1: downs and then maybe you know, in a grid in
Speaker 1: setting some things where you could just pop up a
Speaker 1: box and ask the user what they want, you know,
Speaker 1: what they want to see or what they want to do.
Speaker 2: Yeah.
Speaker 3: My company has a hackathon once a year and one
Speaker 3: of the teams that won actually people I worked with.
Speaker 3: They were looking at our custom ad hoc report builder, right,
Speaker 3: very customizable. You can choose all these things, and they
Speaker 3: realized that all you're doing in the UI is sending
Speaker 3: requests to the API to say, create a report with
Speaker 3: these characteristics. Yeah, so they train you know, basically wrote
Speaker 3: a prompt that says, here's the API, and you could
Speaker 3: ask it please create a report that runs every Monday
Speaker 3: morning and the week over week totals and gives these values,
Speaker 3: and it knew what to do and you simplified the
Speaker 3: complex part of the system, but you didn't replace the
Speaker 3: actual report generation, right right, just that interview.
Speaker 2: It's literally a ux change to UX change. Yeah.
Speaker 3: Yeah, And again it's that language level of the UX change.
Speaker 3: The language model is good at understand language where it
Speaker 3: starts to slide into the maybe we should or maybe
Speaker 3: we shouldn't. Like if you're asking for in product help
Speaker 3: and you want to find stuff like maybe that's a
Speaker 3: little better than searching your help and maybe and consummarize
Speaker 3: the results. We've seen stuff like that. This is where
Speaker 3: RAG comes in and does different things. But you know,
Speaker 3: some of the examples I've seen like, oh, we have
Speaker 3: a rules engine because we have all this complex logics,
Speaker 3: we added a rules engine, and now we have a
Speaker 3: rules engine to manage, and now we have difficulty figuring
Speaker 3: out like what are the rules that are applied? What
Speaker 3: if we just replaced the whole thing with an AI
Speaker 3: and it can just do the logic the rules engine.
Speaker 3: This I think goes to the example you give for
Speaker 3: it Air Canada that basically said someone's eligible for a discount, Yeah,
Speaker 3: because the AI said so, But there's no actual rule there.
Speaker 3: It just said the most likely answer, the most likely
Speaker 3: next word when you ask, am I eligible for a discount? Yes,
Speaker 3: it's just predicting a word, but it's not actually based
Speaker 3: on logic.
Speaker 2: Yeah. And more importantly, when then that went all the
Speaker 2: way to court because it's like, oh, our software failed,
Speaker 2: we're not liable, It's like, no, you presented that as
Speaker 2: a replacement for a human agent, and if human agent
Speaker 2: had said the wrong thing, you would be liable for it.
Speaker 2: And so you're liable for it, right, which is good.
Speaker 2: You know, let's get some case law in place there,
Speaker 2: and also pressing against employers to say, if you're doing
Speaker 2: this kind of utilization of this software, it comes with
Speaker 2: a price, so you know, test carefully, right. I think
Speaker 2: that's the bigger issue I have is a you know,
Speaker 2: I mentioned that the API for calling cryptographicky, but what
Speaker 2: if one in one hundred cryptograph keys were just invalid
Speaker 2: but you had no way of knowing like this is
Speaker 2: the problem I had with lllms is you're putting them
Speaker 2: in a critical workflow and you haven't really tested them.
Speaker 2: You don't know what the failure modes look like.
Speaker 3: Right, And I think anywhere you're going to use an LM,
Speaker 3: take the LLM out and put a person in that place.
Speaker 2: Yeah. Right.
Speaker 3: And if you asked a person to do something, would
Speaker 3: you trust them or do you have other checks and balances?
Speaker 3: Is something has to be reviewed by a supervisor or
Speaker 3: do you just let that person have full autonomy and
Speaker 3: they can do things or you have later auditing to say, hey,
Speaker 3: they messed up and we can fix it. But you
Speaker 3: can't just magically think that this LLM can do everything
Speaker 3: because it can write poetry.
Speaker 1: So, guys, as if United Healthcare wasn't already in the
Speaker 1: news enough, they are facing a class action lawsuit alleging
Speaker 1: that the company misused AI to deny specific insurance claims,
Speaker 1: and especially on elderly people. So there's you know, there's
Speaker 1: a I'm going to link to a news story about this,
Speaker 1: and there's an interview with people who were denied and
Speaker 1: clearly weren't shouldn't have been denied. An insurance claim and
Speaker 1: they basically suit and they said, yeah, your AI basically
Speaker 1: made this determination.
Speaker 2: Well I think that so, yeah, but they all so
Speaker 2: configured it to do that, right, It's no different than
Speaker 2: having a person say deny all acclaim.
Speaker 3: But I think that's also where we're seeing AI is
Speaker 3: thrown on as a label what was probably just machine learning. Right,
Speaker 3: they have all this could be historic claims data, and
Speaker 3: there are plenty of examples of it is.
Speaker 1: Actually it was predictive analytics that they used because the
Speaker 1: ices are pretting that if we allow this, you know,
Speaker 1: the chances are that they're going to I don't know,
Speaker 1: abuse it or whatever.
Speaker 3: Yeah, those those models are trained with biases because it's
Speaker 3: the day you gave them, and if the only data
Speaker 3: you gave them. I think the crime ones are horrible, Right,
Speaker 3: It's like, oh, there's all this crime because there were
Speaker 3: these arrests in these places. Therefore we're going to send
Speaker 3: more people, which then leads to more arrests because that's
Speaker 3: where they sent the cops. It didn't actually solve the
Speaker 3: problem of why is high why is crime high in
Speaker 3: these areas?
Speaker 2: Right?
Speaker 1: Or we found that where there are house fires, there
Speaker 1: tend to be a lot of firemen, So let's get
Speaker 1: rid of the firemen.
Speaker 3: So I think we're getting into my third category. We
Speaker 3: went from good to maybe to the really questionable uses
Speaker 3: of AI. This is where people think I should just
Speaker 3: you know, replace that report generation. If you need specific
Speaker 3: mathematics like added up your regulatory and compliance scenarios your
Speaker 3: quarterly reports, you really really really shouldn't do that because
Speaker 3: it will come up with numbers. They're not going to
Speaker 3: be the right numbers. It's based on statistics, based on
Speaker 3: words and language, not this is my accounting model. We've
Speaker 3: had accounting standards for like four or five hundred years
Speaker 3: for good reason, Like we know how to do math.
Speaker 3: Don't put it in those places.
Speaker 2: What if it can help you generate the query.
Speaker 3: Again, that goes back to the where does it fit?
Speaker 3: Put it in some place. That's the language aspect. That's
Speaker 3: what it's good for language aspect. Yeah, But I think
Speaker 3: what people need to understand is this is non deterministic software. Right,
Speaker 3: we are used to software is a series of if
Speaker 3: then else statements.
Speaker 2: Right, very deterministic.
Speaker 3: Yeah, AI is very non deterministic. It might give the
Speaker 3: same thing, but it might give something different. And this
Speaker 3: is where you can set temperatures and all the different
Speaker 3: things and move the sliders around. You can get more
Speaker 3: creative answers or less creative answers, or more specific, and
Speaker 3: you can get it to repeat the same answer. But
Speaker 3: it's always still based on a prediction model.
Speaker 2: The fact that you can get different answers to the
Speaker 2: same question, it's just just clear indication. It's like, it's
Speaker 2: not that determined. Yes, the determined model will give you
Speaker 2: the same answer for the same question.
Speaker 1: Yeah, I got to admit that I have used at
Speaker 1: GPT to generate store procedures or SQL queries where you know,
Speaker 1: I gave it the the data that I needed, and
Speaker 1: you know, I'm just I'm not the sequel guru that
Speaker 1: Richard is, and I don't know, you know, my group
Speaker 1: buys completely befuddle me. And it worked, you know, it
Speaker 1: turned out to return the right stuff.
Speaker 3: So you just said, you know, you gave it a
Speaker 3: sample of your data, like here's what it looks like.
Speaker 3: The more specific you can get, then the better the
Speaker 3: answer is. It will always give an answer. It will
Speaker 3: never come back and say I have a question, can
Speaker 3: you provide me more information? If you said, please generate
Speaker 3: a store procedure it would write one, it'd be completely
Speaker 3: useless for you. But if you said I need a
Speaker 3: store procedure that gets this data, that prompt is giving
Speaker 3: it more information and that narrows the context. Like AI
Speaker 3: is really good at broad general statements. It knows all
Speaker 3: this generic knowledge because it read the Internet. It doesn't
Speaker 3: know your specific scenario. And that's where architects come in.
Speaker 3: Is like architects take these design patterns we have and
Speaker 3: we figure out what is useful in this scenario and
Speaker 3: is AI one of those things that might be useful
Speaker 3: in my design at this time or does it not apply?
Speaker 1: But in this case, it knows the rules of tseql
Speaker 1: and you gave it everything that it needed in order
Speaker 1: to create the right select statement and it worked. Because
Speaker 1: of that narrow scope. You also have a testability aspect
Speaker 1: there too. Were able to try it and evaluate the
Speaker 1: results and decide if that you know was correct.
Speaker 2: The compiler gets to say yeah, as they say too right,
Speaker 2: like if the if the database didn't like it, it would
Speaker 2: have spat it back.
Speaker 1: I also like the ability to if I'm asking it
Speaker 1: to generate a method in c sharp that does X,
Speaker 1: and it smells a little funky, and I would say
Speaker 1: to it, can you try this again, but with less verbosity,
Speaker 1: can you maybe use link or do something like that
Speaker 1: to you know? And it'll say sure, and it'll try
Speaker 1: it and it'll work. On the other hand, I've had
Speaker 1: methods that use link and a lot of complex link
Speaker 1: and then I will say, hey, can you expand this
Speaker 1: to use loops and if then statements, And then I
Speaker 1: will take that and I will I'll test it, make
Speaker 1: sure it works, and I'll comment that for somebody who's
Speaker 1: reading it who really doesn't understand link, say this is
Speaker 1: what this link statement does.
Speaker 2: Yeah.
Speaker 3: I just had to give a demo of get ub
Speaker 3: copilot within my company and it was a five minute
Speaker 3: lightning talk. I said, we had this new feature we're developing.
Speaker 3: It was pretty complex, so we spent a little bit
Speaker 3: more time than usual doing a upfront design. Just wrote
Speaker 3: out some markdown, a few mermaid diagrams, here's some classes,
Speaker 3: and here's the API end points. And I'm like, what
Speaker 3: would happen if I just gave this to get up
Speaker 3: copilot And the first time I did it it wasn't great.
Speaker 3: It worked, but it wasn't it didn't It didn't fit
Speaker 3: the style of the coding that we had in our project,
Speaker 3: things like we used filescope name spaces because who needs
Speaker 3: extra curly braces. And then I changed my prompt and
Speaker 3: I said things like, please use filescope name spaces, and
Speaker 3: please create the interface for each of the classes and
Speaker 3: injected into the controllers like things that a developer would
Speaker 3: have known to do based on the design document, because
Speaker 3: they said, Okay, you only need to provide this level
Speaker 3: of detail. I can figure out the rest with Copilot.
Speaker 3: It could do better once I told it, please follow
Speaker 3: our guidelines, but I had to tell it what our
Speaker 3: guidelines were.
Speaker 2: Yep, just like a developer, same thing. Yeah, give better
Speaker 2: and better instructions, get more precise results.
Speaker 3: Yeah.
Speaker 1: Hey, I think it's time for break. So we'll be
Speaker 1: right back after these very important messages. And if you
Speaker 1: buy chance do not want to hear these messages in
Speaker 1: the future. You can get an ad free feed by
Speaker 1: becoming a five dollars a month patroon at Patreon dot
Speaker 1: dot NetRocks dot com. We'll be right back. Do you
Speaker 1: have a complex dot net monolith you'd like to refactor
Speaker 1: to a microservices architecture? The microservice extractor for dot Net
Speaker 1: tool visualizes your app and helps progressively extract code into
Speaker 1: micro services. Learn more at aws dot Amazon dot com,
Speaker 1: slash modernize, and we're back. It's dot NetRocks. I'm Carl Franklin,
Speaker 1: that's Richard Campbell, hey, and that's our friend Thomas Bets
Speaker 1: and we're talking about architectural intelligence. Should I shouldn't I?
Speaker 1: And if I should, where and how much? And why
Speaker 1: do we even need to do this?
Speaker 3: Yeah, I think we were leaving it off with AI
Speaker 3: being nondeterministic software, and I want to get the idea
Speaker 3: that that's a feature, not a bug.
Speaker 2: Right.
Speaker 3: It gives these good enough answers, like that's why it
Speaker 3: seems intelligent, Like it did a really good job. And
Speaker 3: there are times when the really good job is flat
Speaker 3: out wrong, but there are times when it's going to
Speaker 3: be okay. And find those places in your applications where
Speaker 3: it's like I can tolerate the good enough answers, Like
Speaker 3: if someone doesn't find all the help references, but I
Speaker 3: found enough and they got their job done, that's okay.
Speaker 2: Yeah. I like your reporting API wrapper scenario because you're experimenting,
Speaker 2: You're trying to come up with a way to look
Speaker 2: at the company's data in a way that'll presumably allow
Speaker 2: you to take an action, so you don't know exactly
Speaker 2: what you're asking for. The most frustrating thing I see
Speaker 2: with most people play with some kind of report builders.
Speaker 2: They gather get everything or nothing right. They always have
Speaker 2: scoping problems and so forth. So a tool that allows
Speaker 2: them to get to maybe do a little more iterative
Speaker 2: and improve an expression on that that might be an
Speaker 2: easier way to go.
Speaker 1: The anthropomorphizing problem is a big one, and Richard's been
Speaker 1: banging this drum for a long long time. Don't fall
Speaker 1: into that trap, or try not to. But it's kind
Speaker 1: of like having an educated uncle who sounds very smart
Speaker 1: and uses big words and never says um or like
Speaker 1: or you know uh, and you know you ask them
Speaker 1: the question, they give you a very intellectual sounding answer,
Speaker 1: and it may be completely wrong. They may have brain damage.
Speaker 1: You know, your uncle may be educated at Havad. However
Speaker 1: he got in a car accident a couple of years
Speaker 1: ago and hasn't been the same since, but he's still
Speaker 1: sounds very smart.
Speaker 3: I like, I've taken issue with some of the terminology
Speaker 3: like get hub copilot, great product name, it's not a copilot.
Speaker 3: Like we talked about this as being your AI assistant.
Speaker 3: It's going to help me do my job. I've flown
Speaker 3: on enough commercial flights they all have co pilots, and
Speaker 3: I'm pretty sure those people are fully qualified to fly
Speaker 3: the plane and are probably doing it. Also, on a
Speaker 3: long flight, they aren't flying the plane. They flip on
Speaker 3: the autopilot, So at some point you are trusting the
Speaker 3: computer to do the thing with monitoring.
Speaker 2: I think Getthub Copilot did a disservice to real copilots. Yes,
Speaker 2: although I did appreciate the name at least implied to you, Hey,
Speaker 2: you're still the pilot. It's still your fault.
Speaker 3: I think AI agents has been like changed to a
Speaker 3: gentic AI, which is better but harder to say. So
Speaker 3: no one says it.
Speaker 2: But that's the idea of the disease.
Speaker 3: The AI gets to make these decisions and we're not
Speaker 3: there yet. No, Like we shouldn't just let them run
Speaker 3: a muck.
Speaker 2: I don't want to. I don't want to get there.
Speaker 2: We're going to get there. Yeah, it's going to have
Speaker 2: This is the thing they're pitching now, and there's certain workflow.
Speaker 2: I mean, it's really not that different from any sort
Speaker 2: of stream based bit of software that has the ability
Speaker 2: to to act in some way, right, Like, I've played
Speaker 2: with plenty of prescriptive analytic models for email prompting. You know,
Speaker 2: you don't necessarily you don't want to be in the
Speaker 2: workflow of this person's been to the site, they put
Speaker 2: in some stuff in the car. If they didn't buy,
Speaker 2: we send them a tiicle email about the things they
Speaker 2: put in their cart, right, Like, all of that is
Speaker 2: automated now, and the fact that it's using a machine
Speaker 2: model to determine when to send that email, like, don't
Speaker 2: send it right away, that's creepy, right, it's a few
Speaker 2: hours later, and they're actually using all of the relative
Speaker 2: response data to feedback into the model to adjust the time.
Speaker 3: Ye.
Speaker 2: Now that these agentic models are going to play on
Speaker 2: that presumably take it further and maybe make it easier
Speaker 2: to build, because good prescriptive modeling is hard, Like, that's
Speaker 2: a tough thing to build, So maybe we're lowering the
Speaker 2: bar for how to make this stuff.
Speaker 1: You also have to beware of AIS or AI agents
Speaker 1: rewriting things that you wrote and making sure that if
Speaker 1: you do choose an augmented version that it's accurate, so
Speaker 1: that means you have to you right. Let's say, let's
Speaker 1: say you're on Facebook and you want to do an
Speaker 1: AD or something, so you write the ad copy and
Speaker 1: then it gives you three other options. Hey how about this,
Speaker 1: which sounds one of them sounds more exciting and stuff,
Speaker 1: but it left out details. It left out links, for example,
Speaker 1: it left out references to other Facebook pages, so makes
Speaker 1: me skeptical.
Speaker 3: Yeah, one of the other. There's actually a product that's
Speaker 3: out there now for my company because we do software
Speaker 3: for nonprofits, a lot of fundraising, a lot of donation,
Speaker 3: and the people who are good at running a nonprofit
Speaker 3: and doing the behind the scenes work might not be
Speaker 3: the best people at writing the message that little, you know,
Speaker 3: short text it's going to go out and say, hey,
Speaker 3: please donate for this cause or the we have just giving,
Speaker 3: which is kind of like go fundme. You have a
Speaker 3: small little thing. How do you put that little blurb
Speaker 3: out there that's not too long that gets people's attention.
Speaker 3: And so we have tools that allow you to, you know,
Speaker 3: click these buttons and change the tone and it'll help
Speaker 3: you write the message. Because again, it's just language. It's
Speaker 3: great at writing language that convinces people, and we've seen
Speaker 3: people donate more to causes that are using that product.
Speaker 3: So you know, there's a very clear correlation to well,
Speaker 3: it's good for our products, good for the nonprofits that
Speaker 3: are using it. So again, find the right places for it.
Speaker 3: I think the agents they're going to get better as
Speaker 3: we get them to be more specialized. And this is
Speaker 3: one of those weird things that the bigger is not
Speaker 3: always better. We've talked about these lms, right that have
Speaker 3: grown to billions and billions of parameters. Right, I can't
Speaker 3: remember what GPT three to GPT four like double the
Speaker 3: number of parameters, basically how big the model is and
Speaker 3: how many things it knows.
Speaker 2: It was more it was like one hundred and seventy
Speaker 2: five billion to a trillion. Yeah, yeah, you know, five
Speaker 2: times and the context size growth.
Speaker 3: Right.
Speaker 2: Yeah, Well, there's an argument there's no path forward because
Speaker 2: there's not more data, Like there's not four trillion parameters
Speaker 2: to be had.
Speaker 3: Yeah. Yeah, they can't train it on anything else because
Speaker 3: it'll start training itself on what it knows and then
Speaker 3: it'll be the cyclical. It'll just like go down the biospiral.
Speaker 2: You know, in science fiction when we talked about a
Speaker 2: superintelligence is that it would start self learning and getting better.
Speaker 2: So far, this software when self learning gets worse, like
Speaker 2: if you if you put it back out to train
Speaker 2: against its own data, it gets less effective, like it's
Speaker 2: a photocopy of a photocopy.
Speaker 1: But for the average user, they can build up a
Speaker 1: context over time so that it quote unquote no, or
Speaker 1: remembers things that you've talked about in the past. So
Speaker 1: you can just say, hey, remember that application I was
Speaker 1: telling you called blah blah blah. Yeah, I have a
Speaker 1: question about that, and it'll know well quote unquote no, yeah,
Speaker 1: it'll pull up that context.
Speaker 3: And that's the most effective way to use any LLM
Speaker 3: is good prompt engineering, And that's like we talked about
Speaker 3: rag go and find this data. But you've got to
Speaker 3: have someone who knows how to do that effectively, because
Speaker 3: you know it poorly, you get worse results. Here's how
Speaker 3: to index my data. But if you can stick everything
Speaker 3: that you could possibly want to know about this question
Speaker 3: into the prompt, you're gonna get great, great results. The
Speaker 3: problem is the corpus of knowledge that I need you
Speaker 3: to know about for my product won't fit into forty
Speaker 3: thousand characters or thousand.
Speaker 2: Now if either of you run into Windows Recall yet
Speaker 2: not yet.
Speaker 3: We have a new laptop that has it, but I
Speaker 3: haven't played with it yet.
Speaker 2: Yeah, I mean there's been a lot of This was
Speaker 2: an announced back in I think Microsoft did a terrible
Speaker 2: job of announcing it was only for the new laptops,
Speaker 2: the Copilot plus Com based coplout plus PC laptops. But
Speaker 2: it's basically taking a snapshot of everything you're doing the
Speaker 2: whole time. Like you talk about a way to generate
Speaker 2: knowledge about you for a system I could see, you know,
Speaker 2: at the moment, it looks like it's largely just a
Speaker 2: search engine. Hey what was that pair of pants I
Speaker 2: was looking at in the past? Right? And the fact
Speaker 2: that it can sort through all of that because it
Speaker 2: has a copy of everything you've done stored on your
Speaker 2: machine is interesting. But there is that larger idea that
Speaker 2: over years, you'd gradually be building a remarkable augmented retrieval
Speaker 2: set about yourself just because it knows everything you've done,
Speaker 2: given that you were only working from one machine. Because
Speaker 2: it doesn't and it's only stored in that machine, so
Speaker 2: you're within the constraints of what that computer can do
Speaker 2: in the name of security, but if you did anything
Speaker 2: on your phone, it's not going to know about it.
Speaker 2: If you have more than one can who would have
Speaker 2: more than one computer? I was drained? Well, Gmail is
Speaker 2: that experience for me now?
Speaker 1: Yeah, I mean, and it has been for a long time.
Speaker 1: It's your store of intelligence, it's my store of intelligence.
Speaker 1: But the problem with it, of course, is when you
Speaker 1: want to find something, you're going to hit on all
Speaker 1: of the spam that you've gotten first, and because there's
Speaker 1: more of it now, you have to really get creative
Speaker 1: with your search, your advanced search, and you know, that's
Speaker 1: kind of the problem. Whereas this recall thing sounds like
Speaker 1: it's a little bit more focused, and.
Speaker 2: There's lots of people freaking out about the security around it, like, oh,
Speaker 2: of course, and they were as soon as it was introduced.
Speaker 3: Yeah, I think any one who's taking this seriously understands
Speaker 3: there's a security aspect. I think you're going to see
Speaker 3: small language models become more popular because there isn't it's
Speaker 3: not going over the wire. I can the model is
Speaker 3: small enough. I can host it on my laptop, or
Speaker 3: I can host it on my servers. I don't need
Speaker 3: to be calling out to chatchy here. Open ai APIs right,
Speaker 3: and so I don't have the concern of sending the
Speaker 3: data over the wire if I can self host it.
Speaker 3: Now I'm paying the hosting costs of hosting a model,
Speaker 3: but maybe I get better results because I don't have
Speaker 3: to sanitize my data. So I'm able to ask better
Speaker 3: questions because I know that the data is not leaving
Speaker 3: my domain.
Speaker 2: I don't think the customer cares at all, right, like
Speaker 2: they're happily using chat GPT inside of companies even though
Speaker 2: they're specifically forbidden. I'm doing so, you know, on the
Speaker 2: administrative cybork are battling. Give them a path forward to
Speaker 2: use actually secure approaches. Right, people cared about security, we'd
Speaker 2: have a lot fewer problems. There's nothing you can do
Speaker 2: to me. People care about security. Cost there is something, right.
Speaker 1: Yeah, if you can if you can equate security to dollars,
Speaker 1: like if we don't take these secure initiatives, we're going
Speaker 1: to lose x amount of dollars, then you'll get some
Speaker 1: of these attention.
Speaker 2: Right. If you run this in the cloud, it costs you,
Speaker 2: you know, and dollars a month. And if you run
Speaker 2: it locally, it doesn't, right, you might actually win some
Speaker 2: folks over to Yeah.
Speaker 3: I think that's why Microsoft copilot. Again, the copilot name
Speaker 3: is now like any different products. So it's a biggest
Speaker 3: I've heard internally. It's over two hundred sure, wow, But
Speaker 3: it's the I want to search our SharePoint in one
Speaker 3: drive and teams and everything else. But it's like, here's
Speaker 3: the thing that we provide, like well, all the other
Speaker 3: stuff that we trust Microsoft to secure, that's going to
Speaker 3: take care of it. I'm sure Gemini is going to
Speaker 3: do the same thing. If you're a Google based company, right, you're.
Speaker 2: Doing Google workspaces, you're doing the same sort of thing
Speaker 2: more or less. The uh yeah, and I think you're
Speaker 2: talking about M three sixty five copilot in that case.
Speaker 2: But it looks like there's at least several flavors of
Speaker 2: M three sixty five copilot. The thing that'll give you
Speaker 2: a hints to make better PowerPoint slides is different from
Speaker 2: the thing that'll find stopping SharePoint from you, but it's
Speaker 2: all under the M three sixty five copilot matter.
Speaker 1: So I got bills from Microsoft co Pilot three sixty
Speaker 1: five and I never signed up for it. Did Dick
Speaker 1: automatically convert your office three sixty five subscriptions to copilot
Speaker 1: and tack on a couple hundred dollars for some reason.
Speaker 2: I don't know the answer to that.
Speaker 3: Yeah, anyway, you mean an AI to understand Microsoft billing.
Speaker 2: Yeah, I guess. One thing I want for a lawyer.
Speaker 1: One thing I want to mention to you, Richard is
Speaker 1: we should get test Ferrondez on the show talk about RAG.
Speaker 1: She has some different ideas than mainstream.
Speaker 2: Yeah, there's good tech in that space, no, no doubt. And
Speaker 2: it is the sort of balance between consumer level tools
Speaker 2: for this and this line that we play in developers
Speaker 2: where we're building code for our companies and they have
Speaker 2: other requirements, but it's also utilizing the data of the company.
Speaker 2: I mean, I would be highly resistant to wanting to
Speaker 2: build apps in this space with things like M three
Speaker 2: C five copilot around, Like when it comes to navigating
Speaker 2: through the data of the company, that seems to be
Speaker 2: the tool given you're in your company is in M
Speaker 2: three sixty five. Yeah, right, But the I mean, I
Speaker 2: wonder if it's just a UX feature when it comes
Speaker 2: to l MS, it's really just a UX feature. It's
Speaker 2: a different way to communicate with a p software to
Speaker 2: get results you want.
Speaker 3: Where we're at right now, that's the best use case.
Speaker 3: Everything below that starts getting into the questionable should we
Speaker 3: do it?
Speaker 2: Or is it good enough? And I think we did
Speaker 2: a show with vishuwaz On that long ago where he's
Speaker 2: getting involved with a startup where it's trying to write.
Speaker 2: It's trying to use the tool to write proposals more
Speaker 2: quickly than humans can, except the company then has to
Speaker 2: submit that proposal and comply with that proposal. So making
Speaker 2: sure that proposal is accurate is not a trivial problem.
Speaker 2: In the end, writing the software to gender proposal easy part.
Speaker 2: Testing to make sure the proposal is correct and is
Speaker 2: capable of doing what capable of executing on it. That's
Speaker 2: much harder and what takes longer for somebody smart to
Speaker 2: write the proposal himself or for somebody not quite as
Speaker 2: smart to generate the proposal and then have the smart
Speaker 2: people read it and make sure it's right. And the
Speaker 2: consequences of being wrong are is money right? You're going
Speaker 2: to submit a proposal for a you're not going to
Speaker 2: be capable of doing, or if it's grossly underpriced, or
Speaker 2: you know you missed a requirement, Like there's validating these
Speaker 2: things is not a trivial problem. I look at it
Speaker 2: almost like writing contracts like I've had. I've dealt with
Speaker 2: companies where we had service level agreements with other companies
Speaker 2: but had no way to measure where that we were compliant.
Speaker 3: And the problem is the level of effort to write
Speaker 3: a good contract and a bad contract is exactly the same,
Speaker 3: more or less.
Speaker 2: Yeah, if you're.
Speaker 3: Pressing the button for putting this in and the AI
Speaker 3: generates it, you don't know if it's a good or
Speaker 3: a bad one until you have an expert review that.
Speaker 3: And so you get the perception that, oh, we've saved
Speaker 3: this effort because look at how easy it was to
Speaker 3: press the button. But if you still have to have
Speaker 3: someone do a thorough analysis, and maybe it made it
Speaker 3: easier for them to do the review than to be
Speaker 3: writing it the whole time, and maybe they make a mistake,
Speaker 3: and so you still need a second editor to review it.
Speaker 3: Maybe you only need one qualified expert instead of too.
Speaker 3: But there's that perception that, oh, it's it's good, and
Speaker 3: you don't realize that it could also be just as bad. Yeah,
Speaker 3: And if you don't know how to tell that this
Speaker 3: was good and this is bad, then that's not a
Speaker 3: good implementation.
Speaker 2: You can get back to this. You really can't use
Speaker 2: this tool unless you're qualified to have done this work
Speaker 2: without the tool, right, because you need to evaluate its output.
Speaker 3: Yeah, Like I said, if you take the LLM out
Speaker 3: and put a human in that, what would you do
Speaker 3: to make sure they did it right? And we tend
Speaker 3: to do that because we don't trust people for really
Speaker 3: important stuff like I'm going to write this contract, Richard's
Speaker 3: going to review it to make sure that I dotted
Speaker 3: all my eyes and cross my t's.
Speaker 2: I would argue the LM's better in the checking role
Speaker 2: that you get the contract and then you run it
Speaker 2: through the LM to say what's been missed.
Speaker 3: Yeah, and that's like having copilot and GitHub copilot like
Speaker 3: write the tests.
Speaker 2: Yeah for my code?
Speaker 3: Should you have it write the code and write the tests?
Speaker 2: Well?
Speaker 3: Maybe maybe not. I have the developers do that as well,
Speaker 3: so we kind of accept it.
Speaker 2: So a little that's unnerving because it's like garbaging garbage
Speaker 2: out and it's like I couldn't write good code, but
Speaker 2: I can't write good tests either, So it passed. Well.
Speaker 1: There are people out there who can read and understand code,
Speaker 1: but aren't so good at writing it, especially when it
Speaker 1: comes to architectural decisions, right, so you know that may
Speaker 1: be a good role for them.
Speaker 3: And that's where the context matters, Like the architect is
Speaker 3: always going to say, it depends. So how do I
Speaker 3: know that this code is good code in this application?
Speaker 3: In this instance? So I can have it write the code,
Speaker 3: but is it the right code? Does it follow our patterns?
Speaker 3: And that got back to the idea that I learned.
Speaker 3: If I give it a better prompt and say here
Speaker 3: is how we write our code. Follow these standards, but
Speaker 3: that's still just basic stuff like how to write async
Speaker 3: and where to put your curly braces, it's not follow
Speaker 3: these design patterns.
Speaker 2: This also reminds me of how outsourcing actually was the mistake,
Speaker 2: because if you can't describe the problem well outsourced, it's
Speaker 2: not going to work any better if you did it locally.
Speaker 2: But as we got better at describing problems for U
Speaker 2: source work, we actually got code results. You get better
Speaker 2: at describing the in the prompt what I need you
Speaker 2: to do here, you know through that tool you're going
Speaker 2: to get usable results. I do like the idea of
Speaker 2: these tools are good at making going down the checklist,
Speaker 2: like taking an architectural design and feeding it to an
Speaker 2: LLM with a good prompt about what our expectations are
Speaker 2: around an architectural design and saying what have we missed?
Speaker 2: Like what would you correct?
Speaker 1: There's one thing I really hate about being the recipient
Speaker 1: of LLLM generated content, and that is being lied to.
Speaker 1: And I get an email that says, you know, hey,
Speaker 1: we looked at your podcast. We think it's amazing. We
Speaker 1: want to talk to you about blah blah blah, and
Speaker 1: I really love this episode with YadA YadA.
Speaker 2: And you know it's just all generated.
Speaker 1: And or here's another one I got recently. We'd really
Speaker 1: like to buy your company. We'd really like to buy
Speaker 1: your company. We think it's interesting and it's whatever. So
Speaker 1: I ignore it because it's obviously a bot. Yeah, and
Speaker 1: I send another email another week, Hey are you are
Speaker 1: you really? Are you interested in selling your company? Ignore
Speaker 1: that another one ham circling back. You know, blah blah blah,
Speaker 1: you kind of And so I wrote back and I said,
Speaker 1: why don't you tell me what you know about my company?
Speaker 2: Cricket of course not. They just the whole goal was
Speaker 2: to get you to respond it anyway, and then you
Speaker 2: just go on a list. Now that person will respond.
Speaker 3: They ignored it, you said saying about having it asking
Speaker 3: it to analyze your design documents. I think that's something
Speaker 3: that the architects can really benefit from, Like we can
Speaker 3: use these tools to our benefits. Another pair of eyes. Yeah,
Speaker 3: you know, metaphorically speaking, use it as your rubber duck.
Speaker 3: I don't always have someone around. I've got a whiteboard here,
Speaker 3: but that's you know, for me to like draw stuff out,
Speaker 3: and I want to know, is this good If I
Speaker 3: write up in an ad R an architecture decision record
Speaker 3: what did I miss? And I can ask it to
Speaker 3: you know, analyze that and find things messed.
Speaker 2: It's another thing the tool is good for is documenting
Speaker 2: what we did, right, Yeah, recording the meeting where you
Speaker 2: generated that ADR in the end and then having the
Speaker 2: tool summarize that as part of this is why this ADR.
Speaker 2: There was a meeting on this date. These are the
Speaker 2: people are there, These were the key talking points. This
Speaker 2: was the decision ADR fire the secretary. You're supposed to
Speaker 2: write that up. We just don't.
Speaker 3: Yes, yeah, we had the meeting, we made a decision. Yeah,
Speaker 3: but the why behind the decision gets in the discussion. Yeah,
Speaker 3: and so if they can capture that. And then there's
Speaker 3: also the I need to communicate with different people. This
Speaker 3: is the architect elevator idea. I need to talk to
Speaker 3: the CEOs and the CTOs all the way down to
Speaker 3: the engineers and the basement doing the work. Those have
Speaker 3: different audiences. I might have that same design document that
Speaker 3: I need to convey in different ways. I need to
Speaker 3: get more detail to the engineers, but I need to
Speaker 3: summarize really quickly.
Speaker 2: I could see as an architect, you'd build up a
Speaker 2: body of prompts. It's like, I'm allowed to take this
Speaker 2: adr to the CFO. Here's the CFO prompt. Yeah, they
Speaker 2: care about return on investment, they care about initial capital costs,
Speaker 2: like include these numbers, like that kind of thing, so
Speaker 2: that the tool would spit out a fairly well shaped thing,
Speaker 2: and you really use that over and over again. Yep.
Speaker 3: Yeah, I think we're going to see the I have
Speaker 3: my toolkit of here's the things I have, and I
Speaker 3: used to write macros am I going to write those
Speaker 3: those prompts that I reuse that stuff that I had
Speaker 3: to do that one time and said please use filescope namespaces.
Speaker 3: I shouldn't have to tell it every time, but it
Speaker 3: doesn't remember, so I had to tell it every time.
Speaker 3: But if I can have a macro that starts that,
Speaker 3: like start our code with this and it's pre injected,
Speaker 3: that's useful.
Speaker 2: Yep. Yeah, Lank, you're language scoping now. You know language
Speaker 2: of the Executive Committee, language of the security group, you
Speaker 2: know language of the vendor. Yeah, you know. I I
Speaker 2: have gone back with and re read previous interactions with
Speaker 2: a given vendor, like an ISP that I was working
Speaker 2: with on a project. They say, you know what, what
Speaker 2: were the ones that worked? Essentially when we talked this way,
Speaker 2: we got better results and you were almost cut and
Speaker 2: pasting from previous ones, like in a way building up.
Speaker 2: That prompt would be that same example, maybe a little
Speaker 2: quicker and a little clearer. I would also say, as
Speaker 2: someone who organizes conferences, I can tell when you use
Speaker 2: cht GPD to write write your abstract. Yep. So that's
Speaker 2: got seven hundred submissions in three hundred of them, the
Speaker 2: same opening sentence.
Speaker 3: G I think this is one of those things that
Speaker 3: QCon stands apart, we don't have a call for papers
Speaker 3: right as one hundred percent human curated content. We get
Speaker 3: together like six months in advance. I was on the
Speaker 3: program committee for QUCAN, San Francisco, and we say, what
Speaker 3: are the topics right now that I want to learn
Speaker 3: about that I think other engineers want to learn about that.
Speaker 3: Gave us our twelve or fifteen tracks. We find track
Speaker 3: hosts for each of those, and then they find they
Speaker 3: reach out to their network and find five people to
Speaker 3: talk about this topic.
Speaker 2: And that's great.
Speaker 3: That pole model means you're using the intelligence of the
Speaker 3: people who know what's relevant right now. It's not a
Speaker 3: push of police accept my talk.
Speaker 2: Yeah, yeah, doesn't mean they didn't also write the abstract
Speaker 2: with the machine.
Speaker 3: Well, I mean yes. I expect everyone to be using
Speaker 3: whatever tools are at the disposal. And if if an
Speaker 3: AI and an LM allows you to make a better presentation,
Speaker 3: that's great. I like the presentations that say I used
Speaker 3: Claude to help write this presentation. Here's where it sucked,
Speaker 3: because that's useful information to me, Like it just can't write.
Speaker 3: It can write a presentation, but I can't write a
Speaker 3: great one.
Speaker 2: Not concern is that catalog when when a potential attendee
Speaker 2: looks and sees the same sets of words over and
Speaker 2: over get every session, they're going to get the creeps right, Like,
Speaker 2: that's not good for business. So you definitely have to
Speaker 2: push on. Hey, I like your top I like you.
Speaker 2: I want you to speak. This is the topic you're
Speaker 2: talking on. But you use chet GPT to write this thing,
Speaker 2: so it looks like every other abstract. Like you have
Speaker 2: to write a better prompt or at least edit this
Speaker 2: into something that looks like it's you rather than a
Speaker 2: piece of software.
Speaker 1: Yeah, so especially here's here's a tell if the first
Speaker 1: three words are did you.
Speaker 2: Know they I've got a lot of in the in
Speaker 2: this fast based movie based technological world in a world
Speaker 2: right the clerk's in opening. Yeah.
Speaker 3: Yeah, we've started having to use because info has article
Speaker 3: submissions and we feed it through was this AI generated?
Speaker 3: Like my son's in college and all through high school
Speaker 3: and everything else, they've had plagiarism checkers and now they
Speaker 3: have AI checkers. Wow, LLLM checkers, ELM checkers.
Speaker 2: I'm doing it. I'm doing my best to discourage people
Speaker 2: from using AI because AI just tells me you don't
Speaker 2: know what it is.
Speaker 3: That's the problem. It's such an easy thing to say.
Speaker 1: Yeah, I'm going to start using NS, which is natural
Speaker 1: stupidity because it's kind of the same thing, just in reverse.
Speaker 3: But I mean again, I go back to it has
Speaker 3: value and when it's wrong, that's a feature, not a bug.
Speaker 2: Yeah. Well, now I've been pressing against folks. Is like
Speaker 2: when you say AI, what do you actually mean? Can
Speaker 2: you articulate it? Yeah, so that it becomes a slower term,
Speaker 2: a term that needs to be qualified. Right, Why don't
Speaker 2: you use the qualified term? Then maybe we can get going,
Speaker 2: And I think otherwise we're all talking magic, right, yeah, right,
Speaker 2: if you just wait, AI equals magic. Sorry, no magic allowed.
Speaker 2: Oh you're using a large language. Well well that's not magic. Fun.
Speaker 3: Yeah, And that's what the architectural intelligence is is figuring
Speaker 3: out when to use those actual AI elements. What are
Speaker 3: the real world things we can do? And that just
Speaker 3: comes down to good traditional tradeoff analysis. We make trade offs.
Speaker 3: Is this the right thing or the wrong thing? Oh,
Speaker 3: it doesn't fit here. For all these decisions, I can
Speaker 3: put it in my ADR and say I considered an
Speaker 3: LM and we decided to go with a traditional write
Speaker 3: the code approach. But you've got to get past the
Speaker 3: hype of hey, AI can do everything. Well, AI doesn't
Speaker 3: actually mean anything. It's just a marketing term. What is
Speaker 3: the tool and how can you use that tool?
Speaker 2: And you know the person doesn't actually know what they're
Speaker 2: saying when they say that, because when you ask a
Speaker 2: question like that, they really tails ben yep, well you know,
Speaker 2: you know, sorry, we don't have Jarvis. Jarvis isn't the thing.
Speaker 2: So what do you actually got here?
Speaker 1: Thomas? Is there anything we missed that you want to mention?
Speaker 3: I just wrap up. So I started with the quote
Speaker 3: from Arthur C. Clark. I think it's his third law.
Speaker 3: The thing I want to wrap up with is his
Speaker 3: second law that the only way of discovering the limits
Speaker 3: of the possible is to venture a little ways past
Speaker 3: them into the impossible. I think there's a lot of
Speaker 3: this hype around AI and LMS and what can they do?
Speaker 3: But we don't push the boundaries, we won't find what
Speaker 3: those limits are. So sometimes you have to basically believe
Speaker 3: the hype and go into that impossible and then we'll
Speaker 3: figure out where we connection get too.
Speaker 2: That's good, Well, thanks, Thomas.
Speaker 1: Has been enlightening to say the least, and it's always
Speaker 1: good to talk to you, so thanks again.
Speaker 3: Always great to say you guys.
Speaker 2: All right, we'll see you next time on dot neat.
Speaker 1: Rocks dot net Rocks is brought to you by Franklin's
Speaker 1: Net and produced by Pop Studios, a full service audio,
Speaker 1: video and post production facility located physically in New London, Connecticut,
Speaker 1: and of course in the cloud online at pwop dot com.
Speaker 1: Visit our website at d O T N E t
Speaker 1: R O c k S dot com for RSS feeds, downloads,
Speaker 1: mobile apps, comments, and access to the full archives going
Speaker 1: back to show number one, recorded in September two.
Speaker 2: Thousand and two.
Speaker 1: And make sure you check out our sponsors. They keep
Speaker 1: us in business. Now go write some code, See you
Speaker 1: next time.
Speaker 3: You got jam, Vans
Speaker 1: And