Diagnosed with cancer. I'm Chris Karragio, NBC News Radio, NBC News on
CACAA Lomolinda sponsored by Teamsters Local nineteen thirty two, protecting the Future of working
Families, Teamsters nineteen thirty two. Dot org. The information economy has a
rod. The world is teeming with innovation as new business models reinvent every industry
industry. Inside Analysis is your source of information and insight about how to make
the most of this exciting new era. Learn more at inside analysis dot com,
insideanalysis dot com. And now here's your host, through Eric Kavanaugh.
All Right, ladies and gentlemen, Hello and welcome back once again to the
only coast to coast show all about the information economy. It's time for Inside
Analysis. You're truly Eric Kavanaugh here with an old buddy of mine in the
industry, Todd Mostak. As with Heavy AI these days, they've changed names
a couple of times to be called Omniside. Now it's heavy dot Ai,
which is cool stuff. And they've got a new offering and folks, this
is really some interesting stuff. So it's a database. It's at the GPU
Accelerated database. Of course, you know about all the different database technologies.
There's a real explosion of database technologies really about twelve fifteen years ago is when
it started. And I remember I had a theory at the time I floated
by some of my friends. I said that I think a lot of this
open source database stuff is they're really trying to hack away at Larry Ellison's corner
of the market. Basically, they want to rest control from Oracle, and
hey, let's be honest. You know, Oracle has some heavy handed practices
over the years with lawyers and kind of messing with folks, and it became
painful and it caused a lot of bad customer experience. I would say,
not that the tech didn't work, just that it was difficult to dealing with
the licensure and all this stuff. And now you have this whole I mean
explosion of database types, all these open source thingsdatabases, and one of them
is heavy Heavy AI. It's a database and heavy IQ is a new offering.
So we're going to talk to Todd about what this stuff is and what
it means and how you could use it. Frankly, so Tom want you
to start us off with what is the platform itself, Heavy AI, and
what are you announcing now soome Yeah, well, first off, thanks for
having me Eric, It's great to get back. So yeah. Yeah,
So you know Heavy AI, so Heavy AI as a platform, we are
a big data analytics platform. Our kind of claim to fame is that we
are GPU accelerated, So we leverage the massive parallel processing power of GPUs,
all those thousands of cores and terabytes of memory done with a second of memory
down with that you have on these cards originally designed to render video games obviously
now kind of part and parcel and core to the machine learning and AI revolution.
So we use all that supercomputing bandwidth to good effect to actually process the
EAM much faster. In particular, run SEQL queries of billions of records in
milliseconds without needing to downsample, to pre index, to pre aggregate your data.
We can use kind of the GPU's power to do smart roof force across
massive data sets and allowing you to get kind of interactive real time answers,
you know, not waiting for tens of seconds, minutes, hours, depending
on your database. You get questions, you get answers back immediately. We
can also visualize those SEQL results in line. We have our own front end
called heavy immerse. So heavy dB is the database engine running on GPUs because
one of the killer apps of having that kind of low latency and that kind
of speed interactivity was enabling interactive visualization. So imagine real time data, you
know, millions or billions of call records, social media data, you know,
whether it's Twitter now x or anything else, whether it's you know,
oil and gas, whether it's it's cybersecurity, you know, logs on a
server. We can process all that data, visualize it. We can actually
use the gpu is to render that data. So if you want to display
that data, g you spatially put billions of points on the map. And
so a lot of our customers say, you know, this is the core
of it is fast database front end is very interactive. Some people have called
it Tableau on steroids, but we have people using us both as a database
and kind of a full stack visual analytics platform. Well it's interesting because you
attacked what is a fairly well populated space already but with something new, and
you look at like what a cloud Darra and a Hortant works were doing,
and initially, of course that was all hdfs, which is a filesystem it's
not a database, it's a filesystem, which is why they ran into so
many problems. I think that was one of the the achilles heels, if
you will, of that whole approach, and that whole movement was that HTFS
is a filesystem. Many of the value propositions for that approach kind of dissipated,
especially when cloud storage got so cheap, so there were some flaws in
the architecture that. You know, they've done a really good job to pivot,
so they're doing much more interesting stuff now. Then you have all the
historical players, like your terror data is your verticas then of course Oracle and
IBM and all these guys. But those relational database structures are good in certain
use cases, but they're not good in certain others. And a couple domains
in particular cause them real trouble. One is time series and one is geospatial.
And if you bring in both of those things, you know, architecture
is really important for being able to process that information, for being able to
parallelize it, as you suggest, being able to get some meaningful insight out
of it. And I think that's where your sweet spot is right is because
you have this GPU foundation because you can parallelize a lot of this heavy lifting,
if you will. Maybe that's where the name came from for heavy AI.
You're able to tackle some of these really dense challenges, you know,
think life sciences. Like I say, I think the nexus of geospatial and
time series, you're able to do that much better than some traditional relational database
could do, right, correct, you know, absolutely, you know,
I think relational databases tend to be grab bags of functionality, right, and
and that's you know, SEQL is a very powerful versual language. We also
speak SQL. However, we've really focused on those use cases that we think
need kind of that real time performance uh temporal and geospatial or space show temporal
use cases kind of first and foremost among them, because like you know,
we see that our customers and often you know, they're not working in the
world of like old school tph Let me join my line line item to my
orders table, to my customer's table, right, this is kind of real
time logs they're looking at. You know. For example, one use case
we have with a customer illegal fishing, right like looking at on the federal
side, looking at where you know, ships may be crossing and areas where
they shouldn't be and actually doing real time analysis and figuring out, okay,
these ships turned off their beacons, and so you're not only doing temporal now
where these ships are going, or doing spatial analysis. And to be able
to run that over billions of records with like very fast, high velocity incoming
feeds and looking at historical data, that's just something that even you know,
even a performance a redshift or a big barrier or snowflake, these databases just
don't scale to. And so I think we've found kind of our core differentiator,
and it really is real time data often with space shift, temporal and
just today's databases aren't really built to handle those kind of workloads and scales right
well, you know your snowflake. So the world certainly are designed for structured
data, which I've always believed really means data in relational tables essentially, that's
what they mean by the term structured. And you know, data breaks is
along the same lines, maybe a slightly different approach to getting there, but
there's still similar engines. And what you've done is special because again you've you've
targeted some of these much more complex use cases that would require tremendous compute power
fromditional CPUs, so much so that it's just not really economically feasible, and
it's not even really compute intensive feasible in a sense. Is that about riting?
Yeah, that's that's absolutely correct. And then the thing about CPUs,
right, it's also about compute density in the sense of yes, you could
pound four pound scale up a CPU cluster to have equal compute to a GPU
cluster. By that time you're at hundreds of nodes, right, And then
you got to think about interconnect You got to think about how you're sharding your
data. There's so many inefficiencies of going to a massive scale out cluster versus
scaling up. And we can also get a multi noode, but first and
foremost we're scaling up within even a single machine multigpus each with thousands of course,
that gives us a lot of processing capability, and the bandwidth between these
GPUs, if you're familiar with some of the nvidia's mv link capability can be
measured in you know, many hundreds of gigabytes a second, going into the
terabytes a second range, and so that's a whole different architecture, right,
where you can actually move data around very quickly Versus your two hundred nodes of
impula on some commodity cluster. That gets very difficult to actually get that kind
of performance just because you're spending most of your time shipping data between nodes and
not actually doing kind of the core analytics operations. Yeah, that's pretty interesting.
So the n v M e SSDs is that part of your vision?
Is that part of what makes the magic happen here? Yeah, we're getting
more into that world, right, So you know, we are I call
it in memory light in the sense that we do persist a disc. You
can we can page in and out of disc. Obviously, our systems happiest
when we can at least cash in CPU memory kind of the core data that
we're actively analyzing. But we're actually working with some partners on the storage side,
and there's some innovations called GP directs and GP file system from the Nvidia
side where we can read straight from the MVMME device and without with bypassing effectively
bypassing the CPU and going straight to mv ME over the PCI PCI bus,
and you can get some tremendous performance and in fact you can even have remote
storage these days with you know, these flash blades and be pulling at hundreds
of gigabytes a second, which is kind of unimaginable, you know, from
storage, and so all of a sudden, your storage can start looking at
memory, except you have potentially petabytes on tap. Yeah. Well, the
reason I'm bringing this up is because we have another company we're working with,
Ocient. I don't know if you've come up, Yeah, I notice it,
But they really thought through what these NVM solid state DROGECT and in particularly
the interface, which I think is what you're kind of speaking to now,
the ability to just pull in massive amounts of data and what they call it
is a is a compute adjacent storage is kind of how they're referring to it.
It sounds very similar to what you're talking about, and it is.
It's orders of magnitude greater in terms of possible throughput. So it just changes
the entire game in terms of how you think about what you're going to do
with this data and how you think about setting up the architecture. And that's
where I see you guys doing something very similar in that you know you have
specif typically addressed these heavy duty use cases, which is also what Ocean is
doing, and you're conquering it really with the GPU accelerator, but also just
a different architectural approach to solving the problem, right, Yeah, exactly,
And I think you know, these new problems and new hardware architectures really do
the new thinking, right. Like you know, most common kind of data
warehouses were built in an age where you know, interconnect was was very slow,
where storage was very slow. You know, all these things are in
CPUs, we're kind of getting faster at a very small fixed rate per year.
All of that's been kind of up ended with GPUs with fast you know,
whether fast fabric connect right, whether it's internal, external to machine mv
link, PC I four, pc I five, And so I think it
demands different approaches and how you tackle this with software and in terms of deployment
on prem also in the c what are the options for deployment. Yeah,
So this is I think actually been one of the blessings. You know,
heavy AI we were mat D back in the day. You know, we
came into a world where there was very little cloud CHAPU compute, and so
that forces us to develop first and foremost, foremost on prim options. Obviously,
now almost all our customers run the cloud, except we do have the
ability to run fully hybrid, to run on prem, to run cloud where
Docker rise. We can also run bare metal. We can run fully offline
and air gaps, which is important for some of our customers. And so
we're pretty versatile in that respect versus being pure SaaS Micro service Play, which
kind of hms you in a little bit in terms of your deployment options.
So I'm one of these dorks who sits around and thinks about this stuff all
the time. And you know, I've said for a long time now that
on Prem's demise you know, has been exaggerated, or the rumors of on
prem's demise have been exaggerated. It's going to be here for a while.
I think you're going to see a bit of a movement back for sensitive data
for PII, but also for lms, you know, to be able to
for big organizations to be able to do that stuff very effectively. You know,
they're going to probably want to have a lot of that IP stuff in
house and basically recast their on prem data center to be able to do new
and different interesting things. But I just wonder about you know, the full
cloud environment because as I think about it, all, you know, we've
gone from the monolith to the containers, which is a complete departure. It's
a whole new way of doing things right, and there are trade offs in
going fully containerized. I joked that we sacrificed what is it, we sacrifice
state at the altar of scale with some of this stuff. And yeah,
you can scale out and then scale back down and that's very useful, but
you do have to to manage states some other way, and it is a
bit of a it's a bit of an overkill in certain use cases, in
certain situations. But what do you think about all that? Yeah, you
know, I think the you know, containeration I think is generally a good
thing. Most of our customers do run containerized. But I think the push
into extreme microservices, I mean that can make sense if you're running at Facebook
or metascale, right, I mean obviously there are a lot of advantage of
that, but the amount of complexities, deployment, of development of often overhead
just in terms of performance upgoing, you know, with the micro services approach
of all these services talking with each other, redundantly calculating things, you know,
we've we're not quite a monolith, but by having our core databases one
architecture very close to the metal, we think that's been a big part of
our kind of performance wins versus parceling it up into you know, you know,
tens or even hundreds of different micro services. So I think there's always,
you know, there's always a pendulum, just like there is with you
know, cloud to on prem and then back to hybrid. I think the
market finds equilibrium at some point, and obviously extremes, you know, often
don't make sense. So I think we're at a people are a little bit
more realistic and also realistic about the privacy piece of running on prem and also
sometimes the cost advantages. We've had customers starting the cloud, but then as
they kind of deepen their investment and you know, heavy and big data analytics
in general, you know, they want to run on prem just because the
costs the economics makes sense, right, No, that makes complete sense,
and you know, just real quick before the first break comes up. Here,
there's also this concept of the modern data stack, which I saw spring
up I guess a couple of years ago. Now it kind of feeds into
this that you know, again, you have sort of best of breed components
that you string together in order to have a very robust solution. Well,
one of my best friends in the industry, he passed away last year,
unfortunately, but a super smart guy, and I remember talking to him a
couple of years ago. He's like, yeah, it's modern data stack.
It tends to be strung together with a couple of frail lines of code.
And I'm not a big fan of that because if any point along the way
it collapses, you've got yourself a big problem. Right, And it seems
to me that you've got enough heavy lifting again in the database and in your
environment, and you can visualize in the same environment with the same technology that
you're kind of circumventing some of those challenges that about, right, I would
say, so. I mean, I think that's one of the potential attractions
for customers to our platform, but also something that we have to educate the
market on. You know, everybody's thinking, okay, we need to have
all this piece wise modern data stack, and we go in and say,
hey, actually, you know, we don't want our place necessarily your data
warehouse a record or your data leak, but for your last mile analytics.
We can actually do the sequel, the visualization, even a lot of the
data science work all in one stack. And you know, you're seeing the
data through a single pane of glass, and we can pull from your data
bricks, we can pull from your Partka store. It's people say, it's
much simpler to deploy and think about, and it's less headaches. Obviously if
people are steeped in the notion that you have to you know, parcel out
your data stack into all these different services. Obviously that can be a challenge
selling. But we found that, you know, customers get a lot of
value out of having one kind of a single platform to choke if you will.
But you know, having all their data in one place, with all
the efficiencies both operationally and from a performance and analytics perspective that you gain from
having data consolidation versus many different silos. Yeah. Well, and I'm curious
to know In the next segment, we'll dive deeper into this. You know,
how you populate this thing. How much time does that take? You've
talked about serving as a front end with a data lake or a data warehouse
in the background. That's also pretty interesting because you know, I get that
one environment for doing the processing. If you could parallelize very well, that's
a very compelling storyline because you're not bouncing from system to system. You're in
one environment and it's designed to handle all these different queries that you do and
give you the visualization all that stuff. So it's a very interesting, bold
approach, I would say, but don't touch that, tot. Folks will
be right back. You're listening to Inside Analysis. Welcome back to Inside Analysis.
Here's your host, Eric Tabanaugh. All right, folks, back here
at Inside Analysis talking to Todd Mastak with Heavy dot AI doing the heavy lifting
for you, folks. I always joke about the heavylifting and data warehousing and
analytics and data science of course, and one of the cool things about heavy
AIS they talk about really bridging the gap between those worlds because and I've always
been fascinated by this. I'd like to get your thoughts on this, Todd.
But from what I've seen, data science teams are very separate from data
warehousing teams, and I have no idea why that would remain the case as
data science matures. I would think you'd want these people talking to each other
and working together because the same goals. Why would you have a completely discrete
team for your data warehouse versus your data science team. And it's just organizationally
how things evolved. But I don't think it makes a lot of sense.
So maybe talk about that for a minute and then we'll get into how you
load this thing. Go ahead, Yeah, it is an interesting phenomenon,
and you can explain why it happened. But I do think in many cases,
for many organizations, it's suboptimal to have these you know, silot orgs,
you know, data science versus you know, your your database or your
data engineering or your analytics teams, because they're often working on roughly the same
thing across purposes, and they distrust each other, you know, and have
different systems and different you know, it can be kind of a mess.
Some organizations handle it much better than others. You know. We we're not
claiming to be the panacea here, but certainly we've had you know, one
of our customers, a big hardware manufacturer out in California, has seen themn's
success with actually having both kind of production operational teams hitting the visualizations, running
BASER re COORSE and SQL, and having data science teams working on the same
data, cleaning the same data on the same instance. Since it's you know,
and they say it's a huge it's a huge gain, and it actually
these teams are working more closely together than they ever have before. They're sharing
insights and even though they're very different personas, And some people are sitting there
in a jupinter notebook which is attached to heavy via our Python connector, and
some people are using Immerse and then other people might bull Tableau on top.
So yeah, that's the dream. It doesn't always happen that way, but
you know, we like to think that we're helping the two sides of the
aisle get a little closer together. Yeah. Well, and there's this jen
Ai component too, So your new offering, heavy IQ is allowing your customers
to leverage the Geni's capabilities. And we were talking before the show about RAG
models, retrieval, logmented generation, which I think are going to be a
huge area of focus from here on out. Basically, I think they're going
to be the mechanism by which very serious companies in the space do their data
quality. Their data got even some security is going to get baked in there,
which is very different because in a traditional data warehousing world, the data
governance, the data quality that's baked in in different ways, and you have
different kinds of guardrails. But it's like, if the LM is the interaction
point for all of your questions, well, now I can focus on building
my guardrails inside the RAG model of the LLM. But what do you think
about that change? Because the other thing I look at is the capacity of
these engines, not just the text generative capability, but their pattern recognition capabilities
for example, and the mechanism by which they choose what they choose is pretty
interesting. It's not an exact science yet. And you know a lot of
people like talk to Joke, including people who are very experienced in this industry.
I really don't know exactly how it works. It's like, well,
you know, how's that going to work? But what do you think about
all of that? I mean, are we really looking at upending many of
the traditional ways we've done things over the past twenty thirty years? What do
you think? I think it will upend things, and it's you know,
it's going to be faster and slower than anyone kind of imagines. I mean,
I think faster in the sense of like wow. You know, two
years ago or a year and a half ago, there was no shate GPT,
you know, generative AI was being talked about. That it was people
who were just kind of putting their feet in the water. And today everybody's
thinking about everybody's using it, you know. I think it's the whole high
cycle thing. Right. There's a lot of power you can get immediately,
there's a lot of problems. You know, the non determinism and the fuzziness
is both a blessing and a curse of it, right. You know,
a lot of the work that we've done in heavy IQ is how can we
take these kind of you know, this neveroless power that seems to have fallen
from the sky. It's almost like alien technology, and how can we like
ballow it up into something that's predictable, that's reliable, that gives people the
answers they want, that doesn't hallucinate or almost never hallucinates. And how do
we treat it like an engineering thing. We have unit tests around it,
and what can we put in the pipeline that's deterministic. I think sometimes people
throw everything to the LEM when in fact you want to use LMS for what
they're very good at, like pattern recognition, natural language processing, and you
want to try to still you know, parts of our heavy IQ offering,
which I guess we're kind of jumping ahead of here, but basically allows users
to ask questions and get answers back the system. Heavy IQ's writing sequel for
them. It's writing natural answers, it's picking visualizations. Parts of it,
you know, will actually correct the SQL queries and figure out, oh,
the user got this little wrong. We'll do that through traditional software approaches,
right, we won't necessarily trust the LM to do everything, but use the
LM for the core what it's good for, which is, yeah, to
take the user's natural language utterance and translate that into what they actually need or
what they want well. And I mean, I think that's going to be
a very significant change in the usage of these tech analogies because when you can
just ask questions and it's not new. I mean, I was joking with
someone the other day. I remember progress Easy Ask, which was like I
mean twenty years ago, I think or eighteen years ago, where you would
type in a natural language query and it would build the sequel for you underneath,
so you could see how it's doing that. That's not new, but
it's certainly much more sophisticated today than it was. And to me it's great
because it opens up analysis to a vastly broader market of business users who don't
know SQL, or don't know SQL very well, or don't want to sit
around and write sequel. Right, So that compoundent ended up itself as the
front end is I think, in a revolutionize and greatly expand Do you see
these technologies and the fact that you're doing some of the more traditional deterministic stuff
under the covers, I think puts you in a good category because to your
point, you can ask an LM all kinds of stuff and you'll get sometimes
very good answers and sometimes very wrong answers, and you certainly don't want that,
right, So you I think you have a pretty clever fusion of old
and new technologies in this stack. And of course heavy is the background,
the dB, that's where a lot of the action takes place. You're using
the LM on the front end to facilitate a conversation with that data, but
then you're converting to SQL as part of your offering, right, so you
are bringing together these two worlds to be able to enable very very rapid analysis
of very large data sets. Right, Yeah, that's that's correct, And
I think like having the data engine, a very fast form of data engine
is actually dovetails with the use of the LM. For example, it's great
to have the LM, but part of what we do to make it accurate
is we actually mine often our customers have multi billion record tables, so part
of it is that we're actually querying all that data to figure out what are
the top string values, what's the time ranges for this data, what are
you know, what are possible and so we're feeding all that information to the
l M, you know, deterministically so you can see it. So it's
a great boon to have a very fast engine so we can pull all that
data dynamically, so you can just look at tables that the system hasn't seen
before and instantly start being able to ask questions. Also, like once you
actually generate a query from the LM, you know, it's very nice to
be able to have a fast system where you want this to be conversational analytics,
not sit and wait analytics. So used to ask a question, you
know, it could be on a ten billion record data set. You run
that SQL query, it takes another you know, half a second or something,
and the user has an answer. I think. You know, I've
seen some of these other systems and if they're divorced from the system of record,
if you know, that's you have your l M service hitting one of
could be terr data, could be Redshift, could be snowflake. Yeah,
they're not optimizing for their databases, and so it's it's much less seamless in
terms of being able to get you know, time to answer, time to
insight. Yeah, so that's interesting helps a lot. Well, it brings
me to my next big question was how do you load this thing? So
in order to use this, do you have to essentially etl in vast amounts
of data? How are you doing that? Like, what's your mechanism for
populating the database to be able to get rolling? Yeah, what's cool.
You know what's cool about iav IQ is you load your data into heavy dB
and you can use our web interface heavy immerse if you want, you can
do it, you can script it, you can do cover once it's there,
it's automatically variable via natural language via heavy IQU. You have to do
zero else the system. We have a kind of server running behind the scenes
that once you start asking questions, it will know which metadata to pull.
But all of that happens kind of automatically for the user, and so there's
no setup, no configuration, It just kind of works out of the box.
The other nice thing is we're running on GPUs, right, and these
lms run best when accelerated by GPUs, and so we can use that same
infrastructure that we use to run our database to run the lms themselves, and
so that's kind of cool as well. And then talk about how you would
connect to like a data warehouse for example, a Snowflake or a Redshift or
something like that. You talked about being able to use heavy as the front
end basically, so you would be pulling some data from these warehouses getting it
into this environment. How does that work? And you know, can you
do things like change data capture for when there are updates or different things of
this nature. How do you how do you deal with that sort of threading
of multiple information systems going into heavy. Yeah, that's a great question.
So we have something that we released a few years ago called heavy connect,
which basically is it's almost a foreign data wrapper kind of interface for heavy.
So that means we can sit on top of another store record, whether it's
Snowflake, whether it's Terra Data Redshift or like a Arcade store or something like
data Bricks, and it seamlessly. Any queries that you issue will get pushed
down and data and then a data will get fetched from the underlying store.
We are working on things like change data capture. Right there's some notion of
it's more built for a pin streams right now, it can pick up pin
streams updates. You can do retriggers every night to like fetch the metadata changes.
So we're getting more sophisticated with that. But for most of our customers,
they're not necessarily putting this on top of their transactional store, and they're
putting on top of another analytics store in which the underlying data it may be
appended to rapidly, but it's not changing. You know, you're not seeing
a ton of updates. You may be seeing deletes, but overall it's a
huge gain because you know, these systems like Snowflake is a very good system
the separation of computing storage. It's economical in many ways for users, but
in terms of providing real time analytics. You know, it's less strong and
so using heavy for what it's good at. They interactive visual analytics, they
interactive quarium and then having that push down to snowflake is your store record,
I think is really exciting for people. That's pretty cool. Yeah, and
I on your site you talk about unprecedented context across location and time. That's
kind of where I was going with earlier in the game. The time series
is so important for understanding what's happening in any kind of environment. You want
to understand, you know, what what happened over a period of time,
and what is the key period of time to understand is it a day,
is it an hour, is it a week, Sometimes it's a month.
And being able to bounce around in those different time frames to look at the
difference, look at the impact, or look at what's what's happening. That's
a really powerful feature to enable analysis because if you have to reset, you
know, restructure the query, wait ten minutes or something for it to repopulate
or materialize views or something. Any any friction you put between the mind of
the user and the information system being queried is going to slow you down.
And it's going to cause you know, a hindrance and morale to go down,
et cetera. So I think that's why what you have is so interesting,
because you are greasing the tracks for the analysts to figure out what is
the appropriate time window I should be applying to this problem. What do you
think about that? No, I always say, you know, people talk
about real time analytics, but real time analytics is often not very powerful without
historical contexts. A good example of this is one of our longtime customers,
Charter Charter Wireless, Charter Spectrum, you know, essentially uses us you know,
as they see you know, they might see an issue, or they
may get something on social media or people saying I'm having an outage or whatever.
They can instantly go into our system with a bunch of historical data and
say, okay, what are the piece of network equipment in that area and
we had faults with them historically? What's been the network performance this area?
Is this actually an issue or is this a statistical anomaly? And you know,
I don't think they'd be able to do that, at least in real
time without without heavy We've seen that from a lot of our customers. They
say We used to have an outage or some issue in our system. We
work with a lot of telcos, and we'd be sitting there, sitting on
our hands waiting for an answer back from the data engineering team, Versus they
can actually do that themselves. Now they can dive into the data and often
get to answers in order of magnitude faster, you know, minutes or tens
of minutes and set of hours or you know a day or two days,
which really has helped with kind of faster turnaround times, faster resolution to customer
complaints. And it's all about drilling in on space and time and being able
to look at the real time data as well as the historical contexts. Yeah,
I love it. I mean that's just really really good stuff. So
folks, look at these folks up online. Heavy dot AI just like it
sounds heavy like the weight and heavy IQ is the new but heavy dB is
in the background. Todd Mastak, thank you so much for your time.
Yeah, thanks so much. Or it's been great. Folks. Stand by.
You're listening to Inside Analysis. Welcome back to Inside Analysis. Here's your
host, Eric Tabanac. All right, folks, welcome back to Inside Analysis.
Your host here, Eric Kavanaugh, And next up we have Peter Voss
from a company called the ig Ai that's a I G O dot ai and
they have a conversational chat but a chat out with a brain. That's the
tagline. So Peter, welcome to the show. When you say a chat
out with a brain, what does that exactly mean? And how is your
chatbot different? Yes, good question. So literally ours has a cognitive engine
or a brain, and as far as we know, we are the only
company you're offering chatbot with a brain. So what the brain does is it
has short term memory, long term memory, reasoning, abilowity, and deep
understanding. So it's you know, it's obviously much more effective than a chatbot
that just relies on a float chart, which is pretty much what everybody we
also doing that you go down a bloadshot and you know that that's kind of
a decision tree. So we've developed this over many many years. This is
actually like a third generation of our technology and allows us to really very effectively
replace calls into agents instead of just augmenting or instead of just doing you know,
the very simple task. Now, when you say long term memory and
short term memory, I mean I understand persisting something either to memory or to
disc for example, but what are you really talking about? Is there a
semantic layer to this, and then there's persistent data like concepts and historical context.
Basically go into the details about when you say memory, what does that
exactly mean and how is that memory then called and used to create context?
Yes, absolutely, So the backbone of the system is a dynamic knowledge graph
that basically has the knowledge about the company, the product, but also the
individual that you're dealing with. So their personalization is hyper personalization at the individual
level. To complatize it. You know, one of our customers is one
eight hundred Flowers Harry and David group of companies, and they use this obviously
as a concercial level service to individual customers. By the way, just a
few weeks ago in Valentine's Day, we actually replaced three thousand agents doing that
and you know, very successfully automated service calls. So what it means is
that the system remembers what you said in previous conversations. For example, if
you say I want to buy some chocolate for my niece of birthday, then
that becomes part of the knowledge graph and that knowledge can then be used so
later on in the conversation you established what the niece's name is and you know
her address or whatever. That information is kept in the brain and it's available
for future conversations or to follow up conversations. So the long term memory is
an integral part of the knowledge graph, and that is then used both for
short term memory as appropriate and for context during the conversations. As you say,
it's a semantic level understanding. Well, that's interesting. So basically,
every time there is a customer call of guessing, you're creating a new node
and new edges on that knowledge graph, and it's ear marked as a node
of this conversation I'm having with John Smith, for example, on this date.
Is that correct? So every conversation. Yeah, And actually it's a
little bit more sophisticated than that, because you really want to separate out individual
customers. You don't want to you know, combine information that you have any
risk of, you know, you know, security, security risk. So
what we do is actually there's a three level architecture of the knowledge graph.
The the core level, the inner level is information shared by all of our
customers, so it's just you know, the generic information. The next level
out is the unique information for the enterprise that we're working with, like their
business rules, you know their product ontology and everything you need to know about
the business. And then the third layer is unique for each individual customer.
In the case of one eight hundred Flowers, it's twenty million customers that each
have their own part of the knowledge graph that is loaded. As soon as
we identify with the customer is we load that additional part of the knowledge graph.
And then when the conversation finishes with whatever has been added that is then
you know it away. That's very interesting. So is the is phone number
the key value to identify who's who? Or is it phone and some other
Oh? It really it really depends on what the customer provides and what channel
you're using, because we you know, our system is truly omni channel,
so you can really start a conversation from a website. You can go to
chat, but then you could switch over to phone. So if you're coming
through chat, might well be emailed address, or it might be the order
number that you give to identify the customer, or it could be the phone
number. Of course, there has to be validation to make sure you are
in fact talking to the right person. But the identification really, you know,
can happen through different things. Now, if it's coming in through say
Apple Business Chat, you already have validation of the customer. It's coming from
a web website where the customer is already logged in, then you'll you'll actually
have a customer ID. That's interesting and so hmm, so you're replacing call
centers. Basically, are you able to do voice or is it only chat?
We do we do both voice and chat. Yeah, it's truly omni
channel that you can. You can because it's the same brain whether you're switching
your channel, your communication channel, whether that switches from you know, SMS
to Apple Business Chat to voice, it's the same brain and you'll have the
same context that available. So is it a computerized voice, then it's just
a computer voice that's having it's correct for the lephany as computerized text to speech,
you know those does that become really quite good these days. And I
think one of the important things is we believe it's important that you let the
customer know that it isn't AI they're talking to I. Really you don't like
it when enterprise want to full customers into you know, it's got to sound
just like a human. I don't think that's a good idea at all.
We always give a well subject to the enterprise we're working with, but we
very much want to design the system in a way that the customer can ask
for a live operator at any time. Very simple. There's no you know,
there should be no barrier to talking to a human if you want to.
For whatever reason. You know, either the system doesn't have the knowledge
or somebody to simply simply prefers it. But you know, in the example
I just gave them a Valentine's date, ninety percent of the interactions were handled
entirely through automation by our system, and only you know, ten percent opted
or needed to go to an operator. That's pretty interesting, So you do
inform people. It's funny because that was going to be a question I asked
you, Is that you telling people that this is a computerized chat butt and
you are have you seen have you seen much pushback on that? Or are
most people okay with that? Oh? No, not not at all.
You know, I was recently at a conference and and and somebody said to
me, we're talking about the difference between chat and phone, And he's saying,
you know, surely boomers, you know, they still want to talk
by by phone. They don't want to use chat. And I looked at
him and I said, well, do you prefer talking to somebody? Do
you prefer to use chat? He said, well chat? Well, you're
a boomer on't you. So I think people are increasingly becoming comfortable with with
smartphones. Everybody is, and people actually learning that it's that it's it's easier
to text in many cases. And of course the younger generation doesn't even want
to talk to to to a person. People just want to get stuff done.
That's really what we're finding. As long as your automation, whether it's
voice or text, is moving forward, and you know that people know this
thing is really helping me. You know, I'm getting things done. They
don't want they don't want to talk to humans. I just want to get
things done. That's that's really what we're finding. But you know, as
soon as it becomes frustrating, you know, press one, please listen carefully.
Options have recently changed. You know, all your business is important to
us. I mean, you know, you pull your hair off and you
just press zero zero zero, get me. It's an operator. The society
has to be useful, you know, and it has to get you to
what you want to do quickly. Yeah, you know, we're going to
do the podcast bonus seven here in just a moment, and I think we'll
dive into how you enrich or how you fuel the chatbot. I'm guessing you
use corporate documentation, a history of the conversations that people have had, things
of that nature, because you have to have a starting point, and then
of course over time, you curate and you refine and you find in tune
and you get these things to where they're pretty good at being able to handle
whatever questions the person might ask. And that is the key, right because
customer service is very important. Everyone says customer experience. It's all about the
experience. It's going to be mindful of all that while still leveraging modern technology.
Well, folks were talking to Peter Voss of Igo dot Ai. Be
right back for the podcast bonus segment. Standby, Well, okay, folks,
time for the podcast bonus segment here on Inside Analysis talking to Peter Voss
of Igo dot ai, and we're talking about your chat by your conversational chatbot
with a brain. I love that it's got long term memory, it's got
a knowledge graph. I'm a big fan of knowledge graphs. I think they're
going to be very helpful in the AI age because not just do they store
information, but they store it in a way it's very easy to access in
terms of relevance, because the relevance are on the edges, right, You
have nodes and edges in a graph, nodes of the entities, edges of
the characteristics basically, and so in a knowledge graph you can capture a lot
of information about individuals, their buying habits, the characteristics of who they are.
Those are all the edges. First of all, is that true with
your solution? And then second of all, tell us how you actually populate
the knowledge graph in the first place to get rolling. What sort of information
can you load into this graph? And in what format do you load it
to be able to get somewhere? Yeah, this is exactly right. And
you know, an important benefit of the approach of a knowledge growth based system
is that it's truly scrutable and it's auditable and as opposed to know the current
trend where everybody's trying to do things with statistical models, large language models,
which are you know not they black boxes, they can't be ordered it and
they hallucinate. You don't have that with us. Dappa actually calls this the
third wave of AI, where you have cognitive AI, a cognitive system as
opposed to the statistical system. So the way we populate the knowledge graph is,
you know, as I mentioned in the previous section, there are three
layers. The middle layer, the center layer, is the common knowledge that
is required for every conversation. You know about people, places, time,
and you know, the common knowledge and that knowledge we curated over several years
to act as the core of the intelligence of the conversational engine, and we
keep adding to that. But then the second layer is basically where we import
the customers ontologies for their different products that they have product categories and the various
business rules and so on. So that is then added to the knowledge graph
and you know, just carefully curated to make sure that you have accurate information
in that little lay. You also have connections via APIs to the back end
system to obviously give you like you know, the current status of an order
and to be able to get the basic information from the back end system.
And then the ult layer is the one that is unique to each individual end
user customer, and that is built up dynamically through the conversation by basically expecting
the semantics of the conversation and adding that to the knowledge graph as you go
along. And you know, as you pointed out, the obviously getting all
the business rules and interpretations and the ontologies and the specialized language that people use
or how people might ask for help, all of that definitely requires tuning.
I mean, it doesn't matter how practiced you are at this art, you
never you can never fully anticipate how people will actually use the system, So
the ongoing tuning is definitely important. Also from the point of view that you
know, products and business rules and demographics change all the time in a company,
so you don't want the system to deteriorate as it gets out of sync
with whatever you know. The company and customer dynamic is. I guess that's
my last question to you, which is, you know, the world changes,
habits change, products change, but it's hard for a model to unlearn
something. So in your graph, do you have the capacity to update offerings
for example, like what the chatbot is saying. Obviously it's pulling from the
knowledge graph to some extent and you can curate that and find tune it,
et cetera. But can you actually go in and overwrite. Can you go
in and say, Okay, this old product is not in our service anymore.
Here's a new product. And so how do you do that? In
other words, how do you maintain the repository of information such that it's all
current and you don't have all the information sitting around that is going to throw
people off, right absolutely. And the big difference with a cognitive AI approach
with the knowledge croft approach that we have is you can literally retrain the system
from scratch in minutes, and you know, and it costs next to nothing.
It's not the one hundred million dollar type of model that you know,
large language models, where you have to retrain the whole model and you really
cannot change the information in the model. We don't have that problem at all.
So you can increment the change knowledge, or you can retrain the model
if bigger changes happen from scratch, and literally it will train it in minutes.
That's how effect that this kind of approach is. Because it's not working
with ten trollium bits of information, right yeah, right, that's because it's
a smaller model, right, I mean it's would you call it a small
language model or it's just a knowledge graph basically the knowledge graph, I think
is a better way to describe it. The language capability are also part of
the knowledge part part of the knowledge graph. But you know that part has
been built of a long time. And I mean basic language understanding doesn't doesn't
really change. You may add extra synonyms, you know, or new product
names and so on, that they're part of the ontology. The core language
understanding doesn't really doesn't really change. Gotcha, very very cool stuff. So
I go dot AI. You were conversational AI. So you work with one
hundred flowers. Basically anyone who has a call center, that's a good use
case for you, right, yes, call center replacing call center agents really
is the most obvious use case for us, but can also be used as,
for example, a diabetes coach, you know, a hyper personalized diabetes
coach that'll ask you and learn whether you love broccoli or you know you're vegan,
or what you know, whether you when you want to do exercise,
if you if you want to exercise, and what kind of thing well,
or an assistant for salespeople. So it has many different use cases. The
core engine that we have, the cognitive AI engine. But our right now,
our focus and our strength really is in replacing call center agents very effectively.
Good stuff. Well, look these folks up online, Peter Voss,
Theoss from Igo. That's a I T O dot AI. Thanks for your
time and folks want to be in the show. Sending email in for at
inside analysis dot com. You've been listening to Inside Analysis and now the voices
of GACAA for an exciting announcement. Want to hear NBC and News or KCAA
anywhere you go, Well, now there's an app for that. CACAA is
celebrating twenty five years in our silver anniversary with a brand new app. The
new KCAA app is now available on your smart device, cell phone, in
your car or any place. Just search KCAA on Google Play or in the
Apple Store one touch and you can listen on your car radio, Bluetooth device,
Android Auto or Apple Car Play, catch the KCAA buzz in your earbuds
or on the streets. Celebrating twenty five years of talk news and excellence with
our new KCAA app. Just do it and download it. KCAA celebrating twenty
five year to Heblt Club's original pure powdy Rco super TA helps build red corpuscles
in the blood, which carry oxygen too our organs and cells. Our organs
and cells need oxygen to regenerate themselves. The immune system needs oxygen to develop,
and cancer dies in oxygen. So the T is great for healthy people
because it helps build the immune system, and it can truly be miraculous for
someone fighting a potentially life threatening disease due to an infection, diabetes, or
cancer. The T is also organic and naturally caffeine free. A one pound
package of T is forty nine ninety five, which includes shipping. To order,
please visit to Hebot club dot com. To hebo is spelled T like
tom, a h ee b like boy oh. Then continue with the word
T and then the word club. The complete website is to Hebot Club dot
com or call us at eight one eight sixty one zero eight zero eight eight
Monday through Saturday nine am to five pm California time. That's eight one eight
sixty one zero eight zero eight eight to Ebot club dot com. With sixty
years of fascinating facts. This is the man from Yesterday and back in time
we go to this time in nineteen sixty four, the Beatles have six songs
in the top thirty, including do You Want to Know a Secret. Another
British band is breaking through. They're called the Searchers. That band got its
name from a John Winn movie. Here is Sugar and Spice, Sugar and
Spier gives a se Sugar Spies you know the Licus And from this time in
nineteen seventy four, looks like NBC is adding The Rockford File starring James Garner,
from the world premiere movie shown just a few weeks ago. Also being
added Little House on the Prairie starring Michael Landon from the recent high rated TV
movie This is Jim Rockford. It don't leave your name and message. I'll
get back to you. Hi, Jim, thanks for the dinner invitation.
I'd love to, but does it have to be the taco stand? And
from this time in nineteen fifty seven, NBC TV says it is experimenting with
the new videotape on some of its more important shows, among them The Steve
Allen Show, Caesar's Hour starring Sid Caesar, and The Sunday Night Donna Shore
Chevy Show Chevrolet in behalf of your local authorized Chevrolet Dealer Live from Mollywood.
God Dinah Shore Heavy Show with more at Man from yesterday dot com. It's
that time of year again, No, not the holidays. Medicare open enrollment
and if you have questions about Medicare, you should talk to the local experts.
Paul Barrette and Associates, all of his agents are certified with plans that
are accepted by most of the medical groups in our area. Call nine oh
nine seven nine three oh three eight five. Their services free and after forty
two years of the business, their agents are trained to help you pick the
plan that's right for you. KCAA Radio has openings for one hour talk shows.
If you want to host a radio show, now is the time.
Make KCIA your flangship station. Our rates are affordable and our services are second
to none. We broadcast to a population of five million people plus. We
stream and podcast on all major online audio and video systems. If you've been
thinking about broadcasting a week radio program on real radio plus the internet, contact
our CEO at two eight one five nine nine ninety eight hundred two eight one
five nine nine ninety eight hundred. You could skype your show from your home
to our Redlands, California studio, where our live producers and engineers are ready
to work with you personally. A radio program on KCIA is the perfect work
from home advocation in these stressful times. Just type KCA radio dot com into
your browser to learn more about hosting a show on the best station in the
nation, or call our CEO for details to eight one five nine nine ninety
eight hundred. Ralph Waldo Emerson told of a dinner guest who went on and
on about the virtue of honesty, offering his own life as a model of
perfect rectitude. The louder he talked of his honors, said Emerson, the
faster we counted our spoons. That's my reaction to the cacophony of phony piety
arising from Republican governors and legislators who are trying to enact more than two hundred
and fifty new state laws to stop black, Latino, Asian, American,
Indigenous, and other non Caucasian voters from casting ballots. Yet they proclaim we're
not racists, we're righteous crusaders protecting the sanctity of the vote. Really,
so, why are they specifically targeting people of color with their repressive voting restrictions.
For example, panicky Republican lawmakers in Georgia tried to outlaw any early voting
on Sundays. Odd why it's a flagrantly racist attack on the black church.
For years, a joyous civic tradition called Souls to the Poles has played out
in southern Black churches on Sundays prior to election Day. After the sermon and
prayers, congruentts, ministers, musicians, and others in the church family travel
in a caravan to early voting locations to cast ballots. It turns voting into
a civic, spiritual, and fun experience. What kind of shriveled soul tries
to kill that? Apparently the same shameful souls and the Georgia GOP who want
to stop local groups from providing water and snacks to citizens forced to wait for
hours in line to vote. They're actually trying to make it a crime to
give water to thirsty voters. Hey, Republicans, what would Jesus do?
This is Jim Heitr Saying the goal and duty of every public official ought to
be to maximize voter turnout. After all, the more Americans who vote,
the stronger are democracy. But there's the ugly political truth. Republican officials no
longer support democracy. KILLISTINAKCAA Lowell, Linda at one six point five FM,
K two ninety three, c F Marino Valley, NBC News Radio. I'm
Chris Kragio. The US is rejecting claims by Russian President Vladimir Putin that Ukraine
may have been involved in a deadly attack on a Moscow concert hall. In
an ABC interview on Sunday, Vice President Kamala Harris said there is no evidence
of that whatsoever. He added that we know isis k is actually responsible for
what happened. Former Republican National Committee Chairerona McDaniel says she disagrees with Donald Trump's
promise to free individuals involved in the attack on the US Capitol. If you
attacked our capital and you have been you and you've been convicted, then that
should stay. During an interview on NBC's Meet the Press, McDaniel said the
violence that happened on January sixth was unacceptable. She said those convicted should serve
out their sentence. McDaniel emphasized, however, she doesn't hold the former president
responsible for the riot. She stepped down as chairwoman of the RNC earlier this
month and will join NBC News as a political analyst. The interview took place
prior to NBC News announcing McDaniel's hiring. Former Supreme Court Justice Stephen Bryer is
speaking about the High Court's decision to overturn roeb Wade and what he and two
other justices wrote in their dissent. One of the things we said is what
we fear. They think this will be simpler the majority, because it will
leave it all up to the states. We don't think it will be simpler.
We think that there will be a lot of more cases coming up,
Bryar told NBC's Meet the Press the leak of the decision was unfortunate. He
said he had a theory about who leaked the decision, but declined to share
any names. The former justice added, however, he'd be amazed if it
was a judge. Briar said it's possible the Supreme Court could one day overrule
the decision. La Dodger starshow heey Otani is set to address the media tomorrow
for the first time since allegations of illegal gambling and theft involving his former interpreter's
surface. Major League Baseball says it has started a formal investigation. The interpreter
is accused of receiving more than four and a half million dollars from Otani to
payoff gambling debts made with an illegal bookie. The Dodgers fired the interpreter after
The La Times reported on the case. I'm Chris Caragio, NBC News Radio,
NBC News on CACAA Lomel sponsored by Teamsters Local nineteen thirty two Protecting the
Future of Working Families, Teamsters nineteen thirty two dot org. Was your car
involved in an accident or just need help with dents? All Magic Paint and by