Hey, Carl and Richard here with your twenty twenty four NDC schedule. We'll
be at as many NDC conferences as possible this year, and you should consider
attending no matter what. The Copenhagen Developers Festival happens August twenty sixth through the
thirtieth. Tickets at Cphdevfest dot com. NDC Porto is happening October fourteenth through
the eighteenth. The early discount ends June fourteenth. Tickets at Ndcporto dot com.
We'll see you there, we hope. Hey, gis what it's dot
net rocks. I'm Carl Franklin and amateur campbell fishwes Lele is here with us
and we're excited to talk to him. How are you doing, my friend,
seen any hotters this year? Oh? Yeah, well, the otters
are everywhere right. It's been summer up on the coast, so it's hard
to be on happy up here. But we've got the kayak out, cleaned
them up, did some repairs, all the things you need to do,
and then you get to go, you know, meet the otters where they
are. How's that? Do they taste good? Well? They think no,
they don't. They smell and they smell terrible. But now, don't
eat otter. That's that's not a good eating carnivores in general, not a
good idea. That's a good idea. But no, they've got to stink
all their own. Man Like, it's not hard to find an otter den
you'll smell it first. Wow. Yeah, okay, I have no such
things to report. But summer into your place means barbecue, right like the
girls are running. I just bought two new twelve hundred watt subway first for
the band. Well, he's that kind of guy. You guys have been
playing up a tornado these days. I mean I only see it from your
calendar. I could seep seeing every every Friday, every Saturday, blocked out
through the summer. One of the guys that saw us at Ocean Beach just
sent me like fifteen videos that he took of us playing wow out there.
I don't know what to do with them. They're little clips. You know,
you put them on TikTok, dude, that's what the hip kids do.
I guess. Yeah, while it still exists, yeah, before while
it's look, it's not going away. It's just a question of who's going
to own it. Yeah, maybe it's the reality. All right, Well,
I have something really funny and very cool from our friend Simon Crop for
better no framework, roll the music. Alright, man, what do you
got? I love Simon Simon's and Assi and he comes up with really great
stuff. Every once in a while he crosses the line into comedy, which
is also always helpful. But he's got this tool called waffle generator for the
big waffles. Well, in a way produces text which on first glance looks
like real ponderous pros replete with cliches. So this is like you know your
lorem ipsem if you just want text to fill up the screen instead of using
Latin random Latin? Is this lo ifs in the large language model era?
Is that what this is? Oh? No, yeah, so this is
great. Look at the example content on the Githovery boat. This stuff,
I might say, the aesthetic of economico social disposition quote in this regard the
underlying surrealism of the take home message should not divert attention from the esthetic of
economic economic social disposition Humphrey Yokomoto in the Journal of the Total Entative Item.
And then there's some on any rational basis A particular factor such as the functional
baseline, the analogy of object, the strategic requirements, or the principle overriding
programming, provides an interesting insight into the complementary functional derivation. This trend man
dissipate due to the measurable proficiency complete and utter nonsense, But it sounds good.
This is LLM degeneration, you know, toombs for a purpose. It's
so awesome. So it's hilarious. Uh yeah, so he basically get this.
He has a blazer wrapper around it, of course, that's what it's
called blazing waffles. Blazing waffles. Oh my god, it's, oh,
simon, so awesome, brilliantly awesome. If you need to make text.
Yeah, he's also got an extension for bogus to use waffle generator, which
is great. Okay, so good. He must be stopped though. Sometimes
I got to ask him about this. What I'm usually the one he says
stuff like that, he must be stopped. What do you think of that?
Vishwaz Yeah, it's great, it's good. You know, whenever every
time we get something that's both a functional tool that we're going to use and
makes us laugh, it's like the perfect better no framework. It's a good
day. It's a good day, every day, no question. All right,
So who's talking to us? Richard grabbed a coma Hoff Shows nineteen oh
four, the when we shot back at build with Mark Brown. We were
talking about COSMOSDB and its role in large language models like chat, GPT and
others, you know the role of AI and Cosmos DUB in general, and
our friend Brent vander Mead had this comment. He said, oh yeah,
thanks for the post traumatic event. At minute thirty two, Carl talks about
trying to put together training on artificial intelligence and be patient for the tools coming
out. Oh yeah, And Carl says, you know, this stuff is
changing so fast, this is not the time. We just got to wait,
wait it out and see what the major tools shake out like. And
I said, it only gets easier. Carl said, you could imagine going
into training class like right before Windows came out, and someone teaching you to
write your own windowing system. YEP, exactly what I was thinking. And
Brent, that was my point. That's why I said we shouldn't be doing
this, just hang on. Yeah, and Brent comes back with I immediately
was thrown back into setting up a three t your team foundations on premises server
cluster. If only the rapid changes were occurring back then, and how people
were willing to suffer setting up Jenkins pipelines for automation into AWS. Oh man,
am I happy with aks Helm terraform as your DevOps and get hub actions.
Thanks for the great show. I mean it's a great example. Think
about how many shows we did talking about continuous integration back in the day,
sort of building your own tool set every Now this bespoke pipeline and today it's
set of services right, you turn them on right and you've got it set
up even as I AD as infrastructure as code, so you're able to just
really run a script. Is like players my pipeline, let's go. We
set up CI for many people in many companies back in the day. Yeah,
it was a whole business. You know, you're right the folks that
would specialize in just getting that right for you so you didn't have to figure
it out. I pretty sure you got a copy of music coe by,
but we'll hook you up with something cool. Give me a ring at Richard
Aguapp dot com. Thanks you so much for your comment and a copy of
music Cobe is on its way to you, and if you'd like a copy
of music, co buy I read a comment on the website at donnat Rocks
dot com or on the facebooks. We publish every show there, and if
you comment there and I read on the show, we'll send your copy of
music go by very good. And you know you can always follow us on
ex Twitter as I call it. But we've been there for many years.
But the cool kids are hanging out. I'm masked on I'm at Carl Franklin
at tech Hub dot social, and I'm Ridge Campbell at mass it. I'm
dot social. Send us a two Rudy two two Rudy two dy fresh and
fruity and uh weird. The last time vishwas Lele was on. We had
such a great conversation and I was under the impression at no. I said
to him, you were on just like a month or two ago. It
was nine months ago. Yeah, yeah, I don't feel nine months older.
But obviously I am been pretty crazy. And first show is all the
way back in two thousand and seven. I like he's been Oh yeah,
he's an og pretty sure we owe my sub sandwich at this point or something.
Yeah, so his title has changed. He is now co founder in
CEO at pwin dot Ai. He's also a noted industry speaker and author,
and the regional director for Washington DC. Welcome back, vishwas, thank you
both. Glad to be back a new job. Dude, what are you
doing. You've been AIS CTO for ten fifteen years? Yeah, almost sixteen
years. I've worked at AIS for almost thirty years, So quite a big
change here since the dast But it does seem to me like you're not alone
either. Some really great tech people I know are jumping into AI startups like
it's the hip thing to do right now. It's such a massive opportunity.
Yeah, so, Richard, I was not going to be started on this
idea. I was not envisioning. If you asked me like nine months ago,
what was your plan. The plan was whenever a new technology like Azure,
open ai or open ai comes along, We've always had this culture of
let's can we find a few people jump into it, try to understand it,
try to build something so that we can understand it better and take it
to our customers. I mean, that's what we're planning to do. And
with Azure open Ai, we went through a bunch of use cases and then
said, hey, can we work on a use case which is can help
our internal folks. We have a pretty sizable proposal team, so it can
we build something that can help you. So we build something, and then
we took it out to other people and one thing led to another and here
we are so really good because you were making things that people wanted. It's
like this is actually a product and we should do more of it. Yeah.
So what we did, Richard, was first we build an m VP
for our internal teams and got some feedback from them, some good, some
bad, all of it. And then we said, okay, let's do
this. Can we before we build any more functionality into this, can we
go to eight or ten companies? Ended up going to a dozen or so
companies and said, would you allow us to generate some content for you and
then we'll talk about this later. But this is a co pilot for proposal
writers. So we went to a bunch of companies and said, can we
generate the first draft of a proposal for you? Maybe eight or ten companies.
It's going to be called the Aesthetic of Economico Social Disposition. You're going
to love it, so sorry, that's great, dash waffle waffle false.
So we we said, okay, we took it to eight or ten companies
and they got some feedback. And what I learned from it, Richard,
to your point, A lot of people are jumping in, and as they
should. This is definitely a disruptive technology. Can I can feel it from
early on in the share Point days you remember two thousand and seven show Getting
into share Point, or remember the cloud show early days. This seems like
much much bigger than all of those things combined. And there's this idea that
you hear technology that can reason on your behalf. Forget the content generation pieces
right, forget that can summarize text for you, but the fact that it
can reason. Given some things, it can reason and give you an output
back. Think about it as applicability all across what we do in it.
But we took one use case and maybe in hindsight, if I think about
it, you can do a lot bit generative AI. But the generative AI
models are hard to sort of control their good But how do you get them
to focus in and generate one twenty thirty forty page document and that has an
ROI associated with that because you know come, sure are time consuming to build?
Yes, they are time consisting to build. So I think, in
hindsight, picking one use case which aligned well with the superpower of generative AI,
which is to generate content, but then to solve a problem where people
were spending struggling writing this content and then generating the first draft. And it
is important to note that it is a co pilot very much using the term
copilot. Yeah, so you're still the pilot. It's your fault. It's
still your fault. Yes, we're just getting you to this first draft.
And later on in the show, I can talk about as a CTO,
as an architect, I have supported proposal writing for at least two decades,
right, and I can talk about all of the trouble and trial and tabulation
supporting that, and then how that product helps with all of those challenges that
the organizations face. So let us go off. I mean, I'm all
about the summarization only existing content, like I get that from a large language
model. But you talked about reasoning. Yes, where is the reasoning needed
and how does the tool do? Yeah, so the reasoning is needed because
let's just literature Defied's okay, I'll take a step back and sort of describe
the problem and then we can talk about how we solved it. Right.
So, I remember nine months ago as you were ending that show. At
that time M three sixty five Copilot had not come out or it was a
yeah, it was being talked about. I think it available in a limited
review or something like that. And Richard, I remember vividly that you asked
a question that if you're generating the content based on your past responses, right,
and let's say you've submitted twenty thirty forty RFP submissions in the past,
you stored them into a SharePoint site. Perhaps now you're M three sixty if
I have co pilot has now scanned those documents, and you're now able to
go and say, hey, generate me an answer to this question. And
you at that point said, why would I not use M three sixty five
co pilot for that? Right? Why do you need this special purpose co
pilot to do what N three sixty five copilot seems to be doing out of
the box That that was a question proposed to do. Yeah, nobody really
knew at that point, right, And the answer to that question is that
M three sixty five Copilot is a very good general purpose chat You give it
tons and tons of content, could be any content in your share point one
drive teams, what have you. And now you can use the graph connector
to pull in content from anywhere else you want to putting data from Salesforce,
you can do that. But the fact is that it is a general purpose
chat bot where the content is broken up. And let's say there's some sort
of a chunking strategy. You chunk it by the page numbers, or you
chunked it by the discussion, or you chunk it by sections. You chunk
it some way. You store it in a vector database, and there are
lots and lots of them available. But then you are asking a question that
Hey, based on the information you have, can you answer this question right?
And it goes about answering that question. I keep in mind that chat
bought including chat gpt our System one chat butt And what do I mean by
system one. I'm really talking about the Daniel famous book from Daniel Canman System
one system I'm exactly I'm talking about, right, and it is can you
elaborate on that visual lot of it? Yeah, So there's a famous book
by by Nobel Laureate Daniel Canman called Thinking Fast and Slow. Highly recommend folks
read it very good. Book really talked about two things. And this book
came out many many years ago, well before CHATGBT was a thing. But
in this book they talk about system one and system to system one. Think
of that as the way your brain works that allows you to avoid a ditch.
Right, You don't have to think too much, You instinctively take a
reaction. System two is when you're thinking about reptile brain exactly. So chat
GPT and other chatbots today are system one. Right, you can go up
to chat GPT and say, I'm going to ask you a question, but
this is a really detailed question. I want you to think about it for
thirty minutes and then only respond. There's no way to do that. It
starts responding in three seconds or whatever the latency is. And so when we
talk about a special purpose chatbot in this case, in the case of proposals,
you really have to think about the proposal domain. Think about the important
entities in the proposal domain. As you look through past documents, you need
to collate them in a certain entities which are very domain specific. And then
generating a response to a question. There are some best practices that are available
out there in terms of how do you write persuasive content right and how do
you reason about So these questions can be long form questions, open ended questions
in RFPs, like we have this problem about modernization and this system needs to
be modernized, and these are the challenges we are running into. It could
be anything, right, I'm just giving you an IT example, could be
a healthcare example, could be something else. So these are questions that you
have to respond to. Their very broad based questions. But then there are
best practices in terms of how you write persuasive text in responding to that question.
And that's where the reasoning comes in. Where our our domain specific copilot
understands some of these best practices. When it ingest the content, it automatically
organizes them based on the entities things like that I can I can make an
analogy here if you go to chat GPT to ask it for a problem and
how to configure this middleware or whatever. Right, you're you're taking a chance,
whether it's going to assume that you're using startup CS or program cs right
a dot net six and before version of asp net core, or you know,
or the new, new or modern stuff. And if it understood this,
there would be another layer before it started spitting this stuff out that said,
oh, we have this contact that says it should know we were using
dot net eight and we're using the latest version of see sharp, and we
want to you know, frame the answer in that context. Right. So
that's setting up a system prompt and making sure that chat, GPT or any
language model has the right context so that it can give you an answer that
is very much aligned with your background and your needs. I'm also talking about
KYLA. I'm also talking about a scenario where in some cases you're trying to
answer a question. So let's just take that t T example, and I
think that'll appeal to the listeners here as well. So there's a question about
somebody, some government agency or some enterprise issued in RFP with a question about
modernization of some data platform, right, and they talked about, hey,
this platform needs to be modernized. How do you go about have you done
this kind of work before? How do you go about solving this problem?
Again, I'm over simplifying this, but let's say that's starf right. So
in this case, we would have to go find out if you have done
this kind of work as an organization, right, and then we will have
to search through your past RFP responses, your core competencies. Maybe you've written
some white papers, you've written blogs, you have transcriptions of podcasts and things
like that. We go look through that, and that's a common RAG pattern
that I'm talking about that people are familiar with. But we have to go
find a couple of examples, or maybe three examples of where we have done
this kind of data modernization. So you get that text which says, well,
these three examples match somewhat the requirement of the RFP. But then the
reasoning comes in, how good a match is this? Right? Should we
use this as an example? If it is not a good match, should
we be changing our prompt and looking for a different kind of an answer.
So all of that is happening dynamically in run time, and we are not
asking our users to write any of the proms, right. We are essentially
generating all of the prompts for them based on the best practices that I talked
about. Right, And there's an interesting inverted ratio that we have learned about.
So if we're generating a twenty page document, in many cases we end
up generating two hundred pages. That's the inverted ratio of proms, which includes
all of the prompts and the context that you're feeding into the language model.
I mean, that's not that weird when it comes to a good search string.
Even like when you focus in on a specific thing, you need to
know you have to be very descriptive to get the answer. That's right.
That's a good analogy. Yeah, I mean, I think it's useful for
folks t als to understand. You're in Washington, DC, like you guys
are in the business of writing RFPs for government, which tend to be lengthy
and have very specific rules, like it's not a trivial thing, it is
not a real thing. There is a multi hundred page document called the Federal
Acquisition Regulation which talks about all of the details of an RFP. What would
go in section M, what would go in section L, Section C,
things like that, so the rules around that. And I mean, I
think a language mode would be good at given this rule set and this goal
and these capabilities, write me the paragraphs right, and what we are finding
is as good as these language models are. Just like human beings, if
you tell them twelve things to do. Yeah, they will do the first
second, and then we will do the eleventh and twelfth and forget everything everything
in between. That's so, is that interesting? It's very interesting, But
it does sound like the in building this RFP generator, you literally want to
go section to section and write a separate prompt for each one. That's effectively.
Again, because the rule set's so consistent, you should be able to
generate almost all of this, right, right, So let me actually take
you back into the dane in life of a technical architect or an engineer like
me who is supporting RFPs, and then we'll come back to this point.
So I'll try to be very quick here. So let's say an RFP arrives.
It's a fifty sixty or one hundred page PDF which is montrivial to read.
Somebody has to read it. This is the request for you to propose
to them. This is the request. Your proposal will be larger, although
in some cases they want you to write it. They have very strict page
limits into how much you can write, right, don't know. They can
send you as many pages as they want, but you can only send them.
You can only and then you know, all the tricks about you know,
creating really dense diagrams to sort of save on words comes into play.
But let's just say you've got a fifty page complicated RFP. Now some human
being has to read that and understand what is happening in this RFP, and
then let's say just spend a day or two to figure out. And then
the organize is a meeting where they invite different people. There's a proposal manager,
there's a technical sme there is account team. Right, there is no
point in responding to an RFP if you don't know anything about that customer.
That is a general rule. That's not just in government. That's a general
rule. Right. Why waste you know, there's no price for the second
and third and fourth place based on your response, So you have to read
that. You organize a meeting and they present what is requested in the RFP,
and then somebody like me, an engineer, would join that meeting and
say, yes, we can do this. This is SharePoint related, this
is records management. Oh, we will use this tool and we will architect
these pieces. And somebody will say fine, you go off, and the
other architects go off and start writing the first draft. Now I don't want
to write anything because I'm already busy with my day job. And they give
me two weeks. I go off and start writing, and turns out that
I don't know about other my colleagues. I'm not a good writer. So
two weeks go by, and then we're expected to show back, come back
to this meeting with what we have written. And that's when the yelling begins,
because you know they don't like what we have written. Everybody has written
it differently, and you know, of course I am over emphasizing on all
of the projects that I've worked on, and the same is true with my
colleagues, and there's a rich set of experienced library, but I'm only focused
on the ones I've worked on, right, because that's what I know best.
So now two weeks have gone by, and really a draft is there,
which is like poor quality draft, which reads like ten people have written
it. And now you've only two more weeks left to the final submission.
Right. And it is often said that proposals are one either at the beginning
or then. What they mean is either at the beginning, you understand that
you're going to win, you put the right team in place, or at
the end, if you have enough time left, you want to take a
step back and think about what should be a creative pricing strategy. Who else
would be bidding this right? How can I ghost my competition? If you
do those things, your chances of winning goes up. But the sad reality
of proposal writing is you spend most of the time in the middle, chasing
the first draft, chasing the second draft. So the value proposition for the
tool, and especially designed for people like me, you have taken all of
your past performances, stored them in a library, and then when the RFP
arrives, our engine, peven engine, will take a look at the RFP,
parse it down into what is exactly being asked, and then rather than
having to write ten pages of pros and return it, you're really dealing at
a higher level. You're providing hints to our model. We call it flight
plan and very creative. Right if you're calling ourselves copilot, a flight planner
seems to be logical. So there, so you set up a flight plan,
which is really and this is Richard and cal This is where I think
this is a very key concept in my mind. Again, looking back,
it seems trivial, but this is one of the concepts that we landed on
earlier on that has served as well. So if you think of calling yourself
a co pilot, which means we are not here to replace the proposal writers.
You know, these people have gained experience and skill or many many years.
We are here to augment them. And how do they work with the
model without becoming a prompt experts themselves or without becoming experts in a RAG pattern.
What we ask them to do is we ask them to fill out a
flight plan, essentially tell us in great how should we be responding. So
these are the solutions we want to incorporate. Here are the win themes.
We think we are differentiated because we are the best Microsoft partner in the world.
What are the pain points? If you really know the customer, you
should know their motivation for issuing this RFP, So you provide that vision in
the form of a flight plan, and then our engine will take that flight
plan and then generate these prompts dynamically as we talked about, and then generate
a twenty thirty forty page response. And why twenty thirty forty pages, because
that's what's been asked, And that was the other innovation. If you go
to any other co pilot, you'll end up getting one or two pages.
Even chat GPT, if we ask a complex question, it will say,
hey, this is a complex question. Let me just tell you how you
can get to this answer. In our case, we can't tell our users
how you can get to an answer. They want to see twenty pages.
So we have two. Like I was saying earlier, these language models,
as good they are, they tend to forget things in the middle. So
it's our job to take two requirements, give it to the language model,
get them to generate something, and evaluate it at the end of that and
see if they really answered the question. If not, send it back again,
send it back again. Now you have section one, you do that
for section two, section three instead of you do that repeatedly. And that's
why people ask me, why is it taking you guys four or five hours
to generate a twenty page document? And when I ask a question of chat
GPT, it gives me an answer in seconds. Why is it taking you
four or five hours? Well, the answer is that we're having to iterate
over the content that's being developed, and at every step of the way we
are evaluating it for the sake of completeness. And by the way, we
also do hallucination checks as the content is being generated, right, you have
to without that human to check things out. Customers can't just expect you to
plug stuff in and copy and paste out of a chat bot, you know,
right, right, So I agree hallucination checks up. That's why very
important. And you know, going back to my example, Carl, the
RFP is asking for companies who have done a five thousand or fifty fifty thousand
notes sorry, migration to the cloud. And that's what the and now the
text that was generated says you have done five thousand note migration. Yeah,
we can't find a reference to a five thousand note migration in your entire past
history. Say, hey, you may want to go double check this.
Well, how effective has it been in generating proposals? So well, you
should ask our customers that. But one, well sounds like they're happy but
happy with it. Yeah, I think I'll lot more work needs to be
done, obviously. Language models building, Richard, Building a production grade application
with language models is hard because they have their own personality. When Microsoft goes
from four to four turbo to four omni. There's some personality aspect of these
language models can change, and we have to account for that, especially if
you're generating a multi layer prompt, you have to account for that. So
take some additional time. But so far we have had customers who said that
we've accelerated their timelines, given them more time. We said, you reached
out to a dozen customers, were they all pretty happy with what they got?
So I didn't complete the story. We built an MVP. We went
to a dozen customers and said, we just want you to try our product.
We didn't even have a UI at that point, right, And because
what's the point of the UI if the content is not of the quality where
it is helping you with saving time and improving the outcomes. Right, So
we went to them, got some really good feedback, added more capabilities,
got the UI, and of course our UI is SharePoint because it turns out
that eighty to ninety percent of proposal writers live in share points. So why
should why should we build something outside an environment that they're already Why blow against
the wind? Yeah? So then, Richard, what we did was back
in February or March. This has been a very fast moving thing lest in
nine months, we said, okay, we're going to offer this in a
commercial manner as a paid service, and we went to many of those original
data customers. Many of them signed up, so we were at this point
running several production customers who are paying customers of ours. And I'm really proud
to say that Microsoft's own proposal Center of Excellence. Microsoft has this group called
the Global Proposal Center of Excellence. We went through an evaluation with them and
they just onboarded as wow the tool. Wow, congratulations, congrats. Yeah,
that's great, and gentlemen, we should pause for one moment for these
very important messages. Now we're back. It's not that Rocks. I'm Richard
Campbell. That's Carl Franklin. Hey here with our friend vishchwas Lela now the
lead on this new Pewin company. And obviously you know your MVP has gone
well, you've done some initial testing. You know. I'll throw my sissedmin
hat on because I've talked to a lot of CTO CIO types who are looking
at these technologies and they keep being told by Microsoft, you should have your
quote data estate in order, like what I'm afraid of with the generalized tools
like this, is that your data state is not in order, and so
data is going to be pulled into the model and then subsequently revealed in prompts
that everyone's going to be very unhappy about. And there is no it's not
like a sign pop out of a server somewhere when your data state is in
order. You don't know, it's like your system should be secure. It's
just as impossible. Like, one of the things that appeals to me about
your approach to this is you're encapsulating the data set and arguably you're taking that
responsibility saying where the folks who will make sure that the data that appears in
the model is appropriate. I mean, I'm wondering. I'm just wondering.
How much of work is that for you? Yeah, that's a huge amount
of work. If I had to look back, we've spent thousands of hours
data science hours in building the multi layer prompt engine. I would say an
equal amount of time in worrying about ingestion of the data. And now,
fortunately for us, this is why a domain specific copilot made sense, because
you know you're limiting the data set, right. Some of the things that
you're talking about is you might have hundreds of thousands of documents in SharePoint and
stored somewhere on one drive, and that one document which nobody is found until
now. You can be sure that M three sixty five copilet would find it.
And somebody asks the question, that question will get answered inappropriately because that
document was available, but nobody knew about that document. In our case,
it helped that we limit the kind of data that goes into the engine.
Right And as I mentioned, your past RFP submissions, your white papers,
and things like SPARS, which is a government tissued document which says you've worked
on these technologies well structured. So we limit ourselves to those documents. We
do things like many of these proposals include resumes of people, and we exclude
those resumes because now you're experienced from a previous employer might pollute your current employer's
data, so we exclude that. So as part of the ingestion Richard,
we break the documents down into logical sections. We use the language models for
some classification. Even so we don't have human taggers to say this is a
management plan, this is an executive somebody. We're using language models to do
some tagging of the documents themselves, and I'm seeing this now inside of perview
Microsoft's and again this IT conversation where it's getting really good at calling out PII.
Even before you got around a tagging. It knows the shape of identifiable
information. So starting with a limited data set helped us. And you know
that may not be applicable to other people here, and frankly, that's the
challenge that Microsoft had with m TH sixty five copilot, because you have everything,
and of course they've added capabilities like hey, don't include these documents in
my index or leave this library out right. So obviously those governance controls have
come into play. In our case, we started a small data set,
but even then it was really important to parse it. I'll give you an
example. You might partner with a company when you're applying for a certain piece
of work. That company may end up teaming with a competitor reviewers in a
subsequent RFP. Now if you've taken their content included into your knowledge base,
chances are that that content will show up, so you have to be careful
about that in terms of how you're using maybe partner's data. So all of
those things, Richard apply and we're breaking that content down again. I talked
about entities and then we use those for our content generation. I got a
question, did you, and you may have said this early on, I
didn't hear it, but we're using Microsoft copilot Studio to create your're We're not
losing max of copallet studio. Have you used it, I've used it,
But this we started our development well before copiallet Studio. And secondly, our
response engine at this point is so complicated that we needed to go directly build
things in Python and asure machine learning notebooks and things like that. Gotcha.
I mean what I like about this is you clearly brought your own meta understanding
of RFP writing to the tool set that's right to then make into a product
that works well for individual companies against their data. But you know, there's
three pieces here. There is the language model as it stands. There is
the understanding of the rules of writing RFPs effectively and applying those. And then
there is the data set that is the companies for what their capabilities are and
how they want to respond to RFPs like those. You need all of those
things. It's not there's no magic wand here. Smart people had to work
really hard to make this work. But when it works, the fact that
you're going to be able to respond to more RFPs and to write better responses
like this is money in the bank if done why if done right? And
I should add one other thing which I should have mentioned earlier. I believe
that in order to develop any effective AI solution, you need to have two
good mL engineering. Of course that's a given, but then you have to
bring domain expertise. And you alluded to that that we brought some domain expertise,
yes, but we are not experts in proposal writing right. So what
we did Richard early on is we went to a company called Shipley Shipley and
you may not be familiar with them if you're not in the proposal space.
But Steven Shipley fifty years ago wrote the book on proposal writing right, and
really an influential book in terms of how should you be thinking about your RFP
responses? And that translated into a company. People can go to Shipley wins
dot com fifty year old company and their job it is to teach classes about
better proposal writing, teach nuances of writing compliant proposals and when you need a
surge capacity, they make their consultants available. Wow, okay, yeah,
so they literally the business around it. That's a business around it. So
we went to them and we said, you know, we are a tech
company. We of course write our own proposals, but we are nowhere near
the subject matter expertise and the name that you have, and we would like
to partner you to be co creators of this tool for us. Right and
Shipley had never partnered with a company, and I was clear that I'm not
just looking for a referral partnership that hey recommend us. I want you to
be joining the team so that you're constantly helping us improve the prompts. You're
bringing that domain expertise. And they went through their own evaluation, Richard,
for a period of time, they looked at other tools, and very proud
to say that they decided to join in as an exclusive partner, as a
conciliation partner. Right and having them on our engineering scrums and saying, generate
this content in this manner, because what is very interesting about proposal writing is
if you don't write it in a certain manner right there, you are restating
the problem. You're saying you've done these things in the past, You're reinforcing
the point. There's a certain format, and proposal writers tend to be very
personniketty about that, right, if you don't follow that format wrong or all
that's are off right, Yeah, and to the three of us, if
you read one page of highly curated proposal writing ready stuff and then some other
stuff, you would say, hey, reasonable people can argue about it.
Oh yeah, your sentence is here, your reference is here. But they
don't like that, right, They want the format to be exactly, So
how do you write content that is persuasive, that follows the structure. And
having a domain expert like them, so so fortunate to have them. They've
taught people the process, the methodology, the best practice is I'm betting they're
testing like that. They're evaluating the output. Absolutely did a show on run
as with Lyn Langett. Now that logo we call the hard part of machine
learning. It's like test test, test, Like you've got to evaluate these
outputs, right, and it takes experts to even know what's good and what's
bad. It takes experts and you cannot use just chat GiB to say which
section is better, right because it requires experience, skill, all of that.
So they're helping us with testing, they're helping us write this. So
that was an important partnership. And that's maybe one tip to the list us
out there that in order for an YI solution to be effective, you can't
just have five spart engineers go to any domain and say I have a solution
for your problem because I know as are mL and Python better than you.
Well, that does not matter. Now you'd know my problem set. Well,
there's plenty of smart developers that have built quality software that was wrong because
they didn't have the domain experts involved to make the right thing. This is
the same roles. You sit a different place in some respects. You're not
qualified to evaluate this output. You need an expert to evaluate this out.
So having that, I was just finishing that other point that you succinctly made,
which is it's really important to have a domain expertise, And I was
just adding that our expertise was not enough alone and we needed to partner with
the best in the business to do that, and we're fortunate to have them.
I mean, I've got to think there's somebody who's made their career to
writing great RFPs. This is still pretty threatening like that, at least but
admittedly you presented the right way. It's like it's only going to give you
a draft. We're going to need your help right every time, at least
to still be read. Yeah, Richard's that's a good point. And at
some level we are all concerned about the socizeful impact of generative AI and all
of the things that we do. But the way I presented two important points.
One is, remember the notion of flight plan that I talked about.
An expert proposal writer can quickly fill out the flight plan because think about that,
you're setting the vision, or you're setting the intellectual momentum for you to
win. Without a vision, nobody is winning any RFPs, right, And
this is where the proposal directors or proposal writers come in and they fill out
the flight plan. And we have been talking to them about how their job
changes with generative AI, where they are operating at the object layer. We
talk about them, We talk to them about objects, and then we are
dealing at the work words level. Right, Yeah, they're changing the objects
and attributes and methods maybe, but then we are generating all the code for
them. In other words, that we are generating all the words for them.
So that's one, and it is a model driven architecture in the sense
that you change the flight plan, you look at the content and you feel
like, hey, this came out pretty flat. Let me go back and
change the flight plan and regenerate and regenerate and regenerate. Right, So it's
a model driven architecture and the better they get it with that setup, they
can make these quick changes. That's one. The second important point is we
are not claiming authorship. Right. I get this request from some people that
hey, can we use your tool? Our knowledge repository is pretty sparse.
We have not done this type of a work. Can you help us generate
good content? And well, this is not cheating as a service. You
can just push up button and you get a proposal response. Right. If
you don't, if you don't have good past performances, if you don't have
good background, if you don't have creative architects who can come up with good
creative solutions to the problem, chances are that what we generate is going to
look very flat to you. So we're not claiming authorship. Those things have
to be in place for an organization to be able to use over tool.
So do you do you ask your customers. When you give a proposal,
do you mind if we use this as and feed it back into the machine,
so as as training data. So cal one important thing to note is
we are not training the models. And this is a question that I get
asked often. We're not training the model at all, because it's not like
your data is going to be trained and then and then use for someone else.
But you do you do use those as? Yes, we use them
that we use them in a rag manner. Where right? And so I
was just about to say this is retrieve all mana generation. This model is
the model, I would say, you still have to ask permission. Yeah,
So, so just to be clear, if somebody acquires our tool,
they go through an onboarding stage where we look at the documents and we tell
them some best practices and then we ingest those documents. Of course, that's
their asset and that data belongs to them. And whenever engine runs, it
dynamically makes a call into this database to look for right pieces of information.
So of course we are using their data, but then we dynamically call into
this engine. If there is no results to be returned because they have not
done this kind of work, then don't have any white papers and that technology,
then it's going to return pretty flat results and we can't infuse that content
back into language Models and c rate the content appropriate. Okay, soa vishwas
what's on your to do list? What's next? Well, the next is
to continue to improve the product. That's what I'm focused on for the foreseeable
future. Take advantage of all of the amazing enhancements that are coming to the
Language Model space. What I love about this is this allows me to tap
into all of the Azure work that I've done in the last ten years.
Because ultimately Language Model is just one service. There are thirty eight other Azure
services that are powering this solution. So a lot of things that we've learned
over the last ten years with the community are coming into play. So continue
to see how far we can take this, how we can support our customers
better. So that's what next for me. Fantastic. Hey, it's always
great to talk to you. We always learned something, we always get a
new perspective on AI and when we talk to you, and thanks, thank
you, all right, and we'll talk to you the next time at dot
NetBox. Dot net Rocks is brought to you by Franklin's Net and produced by
Pop Studios, a full service audio, video and post production facility located physically
in New London, Connecticut, and of course in the cloud online at pwop
dot com. Visit our website at d O T N E T R O
c k S dot com for RSS feeds, downloads, mobile apps, comments,
and access to the full archives going back to show number one, recorded
in September two thousand and two. And make sure you check out our spot
sponsors. They keep us in business. Now, go write some code.
See you next time. You got jad middle vans down there that means hard
than my taxes are