Speaker 1: Welcome to React Roundup, the podcast where we keep you
Speaker 1: updated on all things React related. This show is brought
Speaker 1: to you by Void and top End Devs. Unvoid provides
Speaker 1: high quality design and software development services on a client
Speaker 1: friendly business model. Unlike all other software agencies, Unvoid allows
Speaker 1: clients to only pay after the work is delivered and approved.
Speaker 1: Visit unvoid dot com to learn more and reach out.
Speaker 1: If you know a company that needs more professionals to
Speaker 1: help with design and software development, that's u n void
Speaker 1: dot com and top end Davs helps you stay up
Speaker 1: to date with cutting edge technologies like JavaScript, Ruby, Elixir,
Speaker 1: and AI. Visit topandevs dot com to join their AIDV
Speaker 1: boot camp, weekly community meetups and access expert tutorials. I'm
Speaker 1: Lucas Paganini, founder of Onvoid and host of this podcast.
Speaker 1: Thank you for tuning in. Let's jump into the episode.
Speaker 2: Hey everybody, and welcome to another episode of React Roundup.
Speaker 2: I am your host today TJ Van Tol and with
Speaker 2: me on the panel, I have Paige need you House.
Speaker 2: Hey everyone, and our special guest today is actually a
Speaker 2: React Round of returning Champion. We have even Lovery here. Ian.
Speaker 2: Welcome back to the show.
Speaker 3: Hey, thanks for having me back.
Speaker 2: Yeah, so why don't you start, you know, for people,
Speaker 2: I think it's show we're looking back? Is the show
Speaker 2: is about a year ago. We'll have to look up
Speaker 2: the episode number and toss it in the show notes.
Speaker 2: But it's been a while, So why don't you tell
Speaker 2: people know who you are, what you do in your
Speaker 2: background while you're famous, all those sorts of things.
Speaker 3: Yeah. So I work for a speech recognition company called
Speaker 3: pegle Boys, and we're a developer focused company that tries
Speaker 3: to power developers all over on any platform to have
Speaker 3: to bring voice to their platform. So we have a
Speaker 3: whole variety of different propus that cover speech to text,
Speaker 3: voice activation, wake word, all that, and we just want
Speaker 3: everybody to have a voice on their platform. Besides that,
Speaker 3: I'm a I do like interactive media hard and I
Speaker 3: play bass in a couple of bands.
Speaker 4: That's awesome, not just one band but multiple.
Speaker 3: Yeah, I'm an over at cheap I.
Speaker 2: Guess well cool. So Peka Voice looks interesting. I remember
Speaker 2: us talking about it last time, but maybe you can
Speaker 2: get an overview of like how it works. Like if
Speaker 2: I if I use Peka Voice, what am I? What
Speaker 2: am I getting? Am I getting a service that I
Speaker 2: can send like audio to you, and it comes back
Speaker 2: with the words like what other features maybe you could
Speaker 2: give us, like the rundown of everything. It does everything
Speaker 2: you do.
Speaker 3: Yeah. So the big thing with us is and our
Speaker 3: sort of thing that sets us apart from pretty much
Speaker 3: every other voice service is that we're entirely on device
Speaker 3: and so there is no there is no service. There's
Speaker 3: no cloud API that you're calling to send your audio to,
Speaker 3: which I mean, look look around. That's pretty much every
Speaker 3: single voice thing is just an API. So we're one
Speaker 3: of the only ones out there that is actually giving
Speaker 3: you the ability to hold on to your audio data
Speaker 3: and your user's audio data and process it on the
Speaker 3: device and return Again. We have like a variety of products.
Speaker 3: So we have like wakeword detection, where it's just like hey,
Speaker 3: Siri and okay Google. It's just all it's doing is
Speaker 3: sitting there processing frames of audio, waiting for you to
Speaker 3: say the thing, and then when it wakes up, it
Speaker 3: does the thing that you tell it to do. But
Speaker 3: we also have voice activity detection and which just basically
Speaker 3: peaks when it hears somebody talking. And obviously speech to text.
Speaker 3: Everyone wants speech to text, so auto transcription of voice.
Speaker 2: Yeah, it's very cool. It's also like one of those
Speaker 2: problems that I feel like is it's becoming more commonplace.
Speaker 2: We have smart devices in our house. Our phones can
Speaker 2: listen to wake words and that sort of thing. But
Speaker 2: I still I'm still sort of fascinated by the underlying technology.
Speaker 2: Maybe you could just start give us like the world's
Speaker 2: simplest rundown of like how does how does it actually
Speaker 2: work on the back end? Like do you just have
Speaker 2: a whole bunch of like low level C code that
Speaker 2: looks for patterns in audio data or like, I don't know,
Speaker 2: we don't need it. Sounds like two hours, but I'm.
Speaker 3: Just no, it's a good question. So, I mean, basically,
Speaker 3: it's deep learning, right, It's it's it's machine learning. So
Speaker 3: we teach through machine learning. We teach a machine a
Speaker 3: statistical model of what a word sounds like, or what
Speaker 3: a series of sounds sounds like. So we basically take
Speaker 3: audio in our actual When we're teaching our machine, all
Speaker 3: we're doing is sending it frames of audio that are labeled,
Speaker 3: and we get it to remember them and like form
Speaker 3: a little statistical pattern, and then it for something like wakeword.
Speaker 3: It's just like, hey, remember this pattern of three things.
Speaker 3: Just remember that and say, hey, I think I saw it.
Speaker 3: So it's a lot more complicated when you get into
Speaker 3: speech to text because not only are you teaching it
Speaker 3: every sound in the language, but you're also teaching it
Speaker 3: every word in the language, because then you're dealing with
Speaker 3: audio and writing, which are different things. I think people
Speaker 3: think language is a combination of those things, but really
Speaker 3: they're two entirely separate things. They're like that there's the
Speaker 3: series of sounds you make with your mouth that other
Speaker 3: people understand, and then there's the symbols you write them
Speaker 3: down with and the grammar and punctuation and everything that
Speaker 3: you put into the written form, and they're different, so
Speaker 3: we actually have to treat them differently. But you'll see
Speaker 3: a lot of the big cloud providers out there. The
Speaker 3: reason they got it so right so fast is because
Speaker 3: they had such large machines in the cloud in order
Speaker 3: to do this, so sort of like it outpaced the
Speaker 3: actual progressive voice recognition, and now everything's kind of caught
Speaker 3: up and we can actually do it on Devine, which
Speaker 3: is a big win because, to be honest, we were
Speaker 3: like boiling the ocean for like a while doing speech
Speaker 3: to text, and now we can do it on like
Speaker 3: a micro controller.
Speaker 4: So if you're using something like Peaco Voice, is it
Speaker 4: something that you as a user have to train the
Speaker 4: models or the models already there. It's trained. It knows
Speaker 4: you're speaking English or it knows you're speaking Spanish, and
Speaker 4: it will just it should be smart enough to be
Speaker 4: able to take that audio and translate it into the
Speaker 4: correct written words.
Speaker 3: Right, So, like for speech to text, for instance, we
Speaker 3: basically just have a general language model. You just give it.
Speaker 3: We offer eight different languages, and you just give it
Speaker 3: the language you want and we'll understand that language. But
Speaker 3: we actually use this thing called transfer learning, and we
Speaker 3: have a website Peako Voice Console where you can basically
Speaker 3: we have sort of a general model, but then you
Speaker 3: sort of do train it yourself. Because for something like wakeword,
Speaker 3: we have a model that understands a bunch of sounds
Speaker 3: in whatever language you give it, but then you want
Speaker 3: it to represent a certain series of sounds like okay, Google,
Speaker 3: So you literally type that in to our console and
Speaker 3: hit train, and then it will pop out a model
Speaker 3: that understands that. So that's that's sort of the when
Speaker 3: we say you train it, it's not like, oh, you
Speaker 3: have to go out and gather four thousand recordings of
Speaker 3: this word and you know, submit it to something and
Speaker 3: watch statistics go and decide. No. No, it's just like
Speaker 3: we are. We did the hard work.
Speaker 2: I was gonna say, because by saying that, you're sort
Speaker 2: of implying that you got went out and have four
Speaker 2: thousand recordings of these different words, right or like.
Speaker 3: No, No. So the thing is, it's again we've we've
Speaker 3: trained the general model, so it understands the sounds we
Speaker 3: needed to understand. You just tell us which sounds you
Speaker 3: want us you want to form your wake word, and
Speaker 3: we pop out a model that's that that just waits
Speaker 3: for those series of sounds.
Speaker 2: Interesting because I would have guessed that your building of
Speaker 2: the model was to get a bunch of people to
Speaker 2: say like it almost seems it kind of breaks my
Speaker 2: mind a little bit as possible, right, that you can
Speaker 2: sort of general.
Speaker 3: Us the old style the like. So I worked. I
Speaker 3: worked at a speech recognition company right out of college.
Speaker 3: And what we did we had one of the early
Speaker 3: early wakeword engines, and what we would do is we'd
Speaker 3: it was all b to be the company. We basically
Speaker 3: enter a contract with the company that says, hey, we're
Speaker 3: going to go out and gather four thousand recordings of
Speaker 3: this wake word, and we're going to train it and
Speaker 3: then deliver you the model. And it was very formal,
Speaker 3: and that was basically state of the art at the time.
Speaker 3: But we're actually a bit past that now because we're
Speaker 3: able to use this concept of transfer learning to take
Speaker 3: a general model and just kind of pointed in the
Speaker 3: right direction. So we no longer need to do all
Speaker 3: that all that pounding the pavement asking for people to
Speaker 3: say a wake word, because that was a lot of
Speaker 3: work and it took months, like every time somebody signed
Speaker 3: a contract. And I know because I was running the
Speaker 3: crowdsourcing technology for that company, So I would have to
Speaker 3: post these jobs and these these people would record it
Speaker 3: on their on their like mobile device, and I'd have
Speaker 3: to go through all the recordings and like you know,
Speaker 3: some people would just yeah. Some people would just you know,
Speaker 3: speak their manifesto into the phone, and I'd be like, no, no, no, no.
Speaker 4: So one one thing that I'm curious about is I'm
Speaker 4: assuming that when you would do these these wake word gatherings,
Speaker 4: you would have to take into account accents, because I
Speaker 4: know that that is something that every automated assistant struggles with.
Speaker 4: This English accents, Scottish accents, Caribbean accents, all speaking English,
Speaker 4: but all slightly differently. So is PEKO voice able to
Speaker 4: account for that and be able to interpret, you know,
Speaker 4: a deep Southern accent versus maybe a New York Boston accent.
Speaker 3: Yeah, So I mean that that's still a challenge for us.
Speaker 3: But I think the reason we're a bit more resilient
Speaker 3: to it is because we've trained this general model on
Speaker 3: like g'z like ten hundred thousand hours of speech. It's
Speaker 3: heard all the accents, not not all the accents, but
Speaker 3: it's heard it's heard a lot of variation, so it
Speaker 3: tends to be a bit more resilient. When I was
Speaker 3: doing the old style where we would get people to record,
Speaker 3: that was actually a lot less resilient to it because
Speaker 3: we only had like, you know, three hundred participants recording
Speaker 3: these wake words, and how much variety are you going
Speaker 3: to get between three hundred people? Like? Not enough? But
Speaker 3: when we train these general models, we have like tens
Speaker 3: of thousands of different speakers, maybe more, so we tend
Speaker 3: to be a lot more sensitive to the variations. But
Speaker 3: but it is, it is definitely a challenge because even
Speaker 3: us as humans, if you hear like a really thick
Speaker 3: accent that you're not used to, it can be confusing,
Speaker 3: like like we're we're not perfect either with it. So
Speaker 3: it's it's it's a challenge.
Speaker 2: So I think you so you added multiple language depart
Speaker 2: I believe that's new or at least newish from the
Speaker 2: last time we talk. So does that that like more generalizability,
Speaker 2: make that easier or I imagine there's still all sorts of
Speaker 2: challenges that go into that.
Speaker 3: Yeah, So when you when you actually work with a
Speaker 3: totally different language, that's basically starting over because accents is
Speaker 3: one thing you've already taught it the series of sounds
Speaker 3: in the language, and you're just looking for a combination
Speaker 3: of those sounds and those symbols. But when you move
Speaker 3: into a new language, there's a new set of symbols,
Speaker 3: and there's a new set of sounds. You know, there's
Speaker 3: everybody has an inventory. We call it a phonemic inventory,
Speaker 3: and it's basically a series of sounds that you hear
Speaker 3: in the language, and every language has a different phonemic inventory,
Speaker 3: and we need to train the machine to understand only
Speaker 3: that inventory of sounds and all the symbols that go
Speaker 3: into that. So when we start a new language, we
Speaker 3: have to do it completely from scratch. We have to
Speaker 3: get new data in that language, We need to get
Speaker 3: new text in that language, and we need to do
Speaker 3: our best to even understand the language enough to work
Speaker 3: with it because we need to listen to these recordings.
Speaker 3: We need to normalize the text we get and make
Speaker 3: sure it's not like full of symbols and stuff, but
Speaker 3: understand it enough so that we actually don't confuse the
Speaker 3: machine learning process, and that that could be a real challenge.
Speaker 3: It's a lot of work.
Speaker 2: Actually, so it's fascinating. Does that mean like when you
Speaker 2: kick off anywe language, I feel like you almost need
Speaker 2: to have like a professional linguist on staff for almost
Speaker 2: each of these languages, right, Like, or do you like
Speaker 2: bring on somebody who's, like, you know, a world class
Speaker 2: I don't know, a Spanish linguist to help, or like
Speaker 2: like how much of it are you able, like as
Speaker 2: a software developer to sort of test on your own
Speaker 2: and how much do you have to rely on a
Speaker 2: native speaker as the only person that can actually figure
Speaker 2: some of these things out.
Speaker 3: Yeah, so we do have like basically our like machine
Speaker 3: learning team. They do have to be part linguist, like
Speaker 3: because if you've studied languages, you at least understand the components,
Speaker 3: and basically every language is just a combination of the components.
Speaker 3: So they have a lot of expertise in that field
Speaker 3: to understand when they approach a new language how it works.
Speaker 3: But then that's not enough. So usually what we'll do
Speaker 3: is we'll get somebody, well, we will get a native speaker.
Speaker 3: Usually will basically hire somebody on a contract to work
Speaker 3: with us to help with the language, because you do
Speaker 3: need that expertise. Like, the fact is, even somebody who's
Speaker 3: like a language expert, if they sit down to an
Speaker 3: entirely new language, they're not going to be able to
Speaker 3: understand it enough to do the work that needs to
Speaker 3: be done to actually get it to a production ready state.
Speaker 3: So we often do need to get a native speaker
Speaker 3: in there to provide their input and that will really
Speaker 3: speed the process along. We tried to do it without
Speaker 3: experts a couple times, and it's just like you just
Speaker 3: don't get the performance and you spend a lot more time,
Speaker 3: you waste a lot more time. I should say sure.
Speaker 4: I mean that makes a lot of sense. When you
Speaker 4: think about getting expertise in anything else, it's a lot.
Speaker 4: It will almost undoubtedly go much quicker if you have
Speaker 4: somebody who is proficient in whatever it is that you're
Speaker 4: trying to do.
Speaker 3: Yeah, well, they can recognize mistakes, grammar and stuff, the
Speaker 3: stuff that's really hard to pick up as a non
Speaker 3: native speaker.
Speaker 4: Yes, So what languages do you currently offer Peko voice for?
Speaker 3: So we have I believe last year we announced we
Speaker 3: had Spanish, French, German, English, and then this year we
Speaker 3: added four new languages. We added Japanese, Korean, Portuguese, and Italian.
Speaker 4: These are some tough ones.
Speaker 3: Yeah, well, especially like when you get into the written
Speaker 3: forms of Korean and Japanese, they become very challenging. Like
Speaker 3: you know, we in English we have twenty six characters.
Speaker 3: Japanese has two alphabets of fifty six and then an
Speaker 3: additional alphabet of tens of thousands. So yeah, yeah, so
Speaker 3: that the text representation of that is really difficult. The
Speaker 3: actual spoken version of Japanese is a lot easier than
Speaker 3: English because Japanese has fifty six sounds, and they all
Speaker 3: map to a combination of characters. English mapping a combination
Speaker 3: of characters to the sound is incredibly difficult. Turns out
Speaker 3: we made some mistakes early on and we didn't really
Speaker 3: fix them.
Speaker 4: I mean, just thinking about the amount of spellings that
Speaker 4: we have for the same sounding word based on context,
Speaker 4: I can not even imagine how you would be able
Speaker 4: to figure that out for a transcript.
Speaker 3: And it's all exceptions in English. It's like, oh, yeah,
Speaker 3: it's this unless this or this unless this, and like
Speaker 3: here's three different reasons why this rule is wrong.
Speaker 2: And yeah, you see this when you have like younger
Speaker 2: kids that are starting to write, and you look at
Speaker 2: their writing because they start they don't know the exceptions yet, right,
Speaker 2: but they can speak it because they know. So you
Speaker 2: get like, they it's words you don't even think about too,
Speaker 2: because we internalize them so quickly. Because one of my
Speaker 2: kids spelled because ron and then you're like, oh, bill,
Speaker 2: because it's pretty easy. But then you think about it
Speaker 2: for like half a second and you realize, like, actually,
Speaker 2: the word because makes absolutely no sense, like right.
Speaker 3: Like if you try and explain it, you suddenly find
Speaker 3: yourself going just is what it is. Yes, just memorize it, yep.
Speaker 4: I mean that's really fantastic that you have taken on
Speaker 4: and it sounds like gotten through some very difficult dialects.
Speaker 4: Are what are future future languages that you hope to
Speaker 4: be able to process as well?
Speaker 3: So we're going, yeah, exactly, So we're going to try
Speaker 3: next year. We're going to double our language count again,
Speaker 3: I think, and we're going to do going to do Chinese, Vietnamese,
Speaker 3: what else? Dutch? I believe, Russian, Polish, I think, yeah,
Speaker 3: I can't remember all of them. But you basically need
Speaker 3: to be like a fully inclusive speech recognition company, you
Speaker 3: basically need like a bare minimum of like fifty languages.
Speaker 3: So like we're going to get to like twenty of
Speaker 3: the most popular and hold there for a while. Is
Speaker 3: kind of our plan because that covers a lot of people,
Speaker 3: Like that covers the majority of people, because because even
Speaker 3: in the cases where the people might not speak the language,
Speaker 3: they usually are like, oh but I speak this what
Speaker 3: this more popular language? But to really get up there,
Speaker 3: like I mean, you do need to get to like
Speaker 3: fifty or something. And I mean Google has like one
Speaker 3: hundred and fifty, so you know, it's it's kind of
Speaker 3: a never ending thing for us.
Speaker 4: How about Hindi that's a big one.
Speaker 3: Oh yeah, that's actually one of the other ones we're
Speaker 3: going to do next year.
Speaker 2: Yeah, so I guess I got to ask one last question.
Speaker 2: Are there any languages like you've come to hate, like
Speaker 2: because it was like very difficult or.
Speaker 3: It's funny how much you can hate your own language. No, actually,
Speaker 3: like seriously, English is the only Like I look at
Speaker 3: all other languages we've done, and I'm like, these are
Speaker 3: so much easier, like English is. Actually, it's just it
Speaker 3: came out of a mess of languages. It was a
Speaker 3: lot of combinations that happened over time, and a lot
Speaker 3: of them happened during like you know, a lot of
Speaker 3: English developed during like illiteracy, and so there's like really
Speaker 3: interesting examples you can find of like stuff where it's
Speaker 3: just like, oh, yeah, this was just a mistake that happened,
Speaker 3: you know, two hundred years ago that they kept in
Speaker 3: or actually, I have a fun fact the word dumb.
Speaker 3: So you look at that, you're like, why does it
Speaker 3: have a be at the end that apparently was there
Speaker 3: was a time where the like ruling class of England
Speaker 3: was trying to make it harder to write English so
Speaker 3: that the peasantry could like pick it up. And they
Speaker 3: literally just added some letters to the language here or there,
Speaker 3: and we're like, this is the proper way to write it,
Speaker 3: and then just to confuse people. And we literally still
Speaker 3: have that to this day. So English is so weird.
Speaker 4: So that's why knife has a K in front of it.
Speaker 3: Yeah, yeah, like like stuff like that. I think they
Speaker 3: were just messing with us and now we're just like
Speaker 3: we have to live with that.
Speaker 2: So I want to pivot a little bit and talk
Speaker 2: about the actual web development, like the side where you
Speaker 2: might actually use a service like this, because I remember
Speaker 2: last time we chat it a little bit too about
Speaker 2: common use cases, right, So maybe we could just start
Speaker 2: with a review, like how we have a lot of
Speaker 2: web developers listen to this show. What do you think, like,
Speaker 2: I guess, A, what would using something like this look like?
Speaker 2: Like how do you actually get it in an app?
Speaker 2: And B I guess like, what are some common use
Speaker 2: cases that you see for use on the web as well?
Speaker 3: Right? So one of the big things is obviously on
Speaker 3: the web, people are a lot more comfortable calling like
Speaker 3: an API and that is what they've come to expect
Speaker 3: for speech trek cognition and stuff. But we're actually bringing
Speaker 3: the We're actually kind of bringing back the power of
Speaker 3: the browser itself. So the I mean the browser is
Speaker 3: a virtual environment that can run whatever you want. And
Speaker 3: we actually can run entirely in the browser on the
Speaker 3: client side. And that's that's big because I mean, in
Speaker 3: the these days, we're getting a lot of progressive web apps,
Speaker 3: and the sort of web app is a big thing,
Speaker 3: especially with like SaaS companies and stuff. So if you're
Speaker 3: running like a SaaS company and you you want to
Speaker 3: integrate like voice into your console or something, having it
Speaker 3: on the client side is is I mean, it lowers
Speaker 3: the latency, It gives you a lot more direct control
Speaker 3: of what happens when you get boys, and it means
Speaker 3: you can be robust to connection issues, which like that
Speaker 3: that you know, that's a huge thing. Not everyone has
Speaker 3: amazing Internet and you don't want to have to be
Speaker 3: I can calls out to an API and just hoping
Speaker 3: it comes back for you or feature to work. This
Speaker 3: will just work. And also on top of all that,
Speaker 3: it's it's less expensive because we're not calling an API,
Speaker 3: We're not depending on cloud infrastructure. So you're actually if
Speaker 3: you're a developer and you integrate Peka Voice into your
Speaker 3: web app, your client is going to be using their
Speaker 3: machine to do the processing. So I think it's just
Speaker 3: a win win situation for that. Yeah.
Speaker 2: I feel it's especially important considering it's audio too, So
Speaker 2: like bandwidth is like you you're not just shipping off
Speaker 2: like a couple of things in a query string to
Speaker 2: some service. You're like uploading.
Speaker 3: Audio h mega adio.
Speaker 2: Yeah, so the bandwidth consideration is amplified significantly.
Speaker 3: Yeah. No, and it makes everything that that actually allows
Speaker 3: for like something like real time audio. Real time audio
Speaker 3: is very challenging to do for an API because you
Speaker 3: basically need to stream it to the service and have
Speaker 3: responses being streamed back. That's that's really expensive. That's like
Speaker 3: a constant bandwidth issue. But when you're doing real time
Speaker 3: audio and it's all running in your browser on the
Speaker 3: client side, it's it's snappy and you can do things
Speaker 3: that require timing and.
Speaker 2: Yeah, very cool. And I know I think I remember
Speaker 2: from last time too that because one of the ways
Speaker 2: you keep it snappy is it's not JavaScript code running
Speaker 2: in the browser, right, it's your I don't remember your
Speaker 2: exact tech stack, but I know you have some sort
Speaker 2: of fancy way of doing that. Maybe you could walk
Speaker 2: people through some of the magic and challenges of how
Speaker 2: that works.
Speaker 3: Yeah, so our core code is in like C because
Speaker 3: we we were trying to keep it as efficient and
Speaker 3: snappy as possible. Now C code, and when you think
Speaker 3: of C code next to React, you're like, how does
Speaker 3: this even work? Like can these two ever talk? But
Speaker 3: it turns out they can with WAM, And what we
Speaker 3: do is we compile basically all our core code into
Speaker 3: a WAHSM binary and then we ship that with our
Speaker 3: like MPM package. So when you MPM install Peka Voice,
Speaker 3: part of the part of what's going to be shipped
Speaker 3: with your website is OURASM blob. And basically wasm's really
Speaker 3: cool because it basically just wraps your native code in
Speaker 3: JavaScript and then allows you to basically attached to it
Speaker 3: like any sort of dynamic library, say, here's the functions
Speaker 3: I want to call, here's the data I'm going to
Speaker 3: put into it, and then you just call it like
Speaker 3: you would any any other library. It's a little trickier
Speaker 3: to work with because you're dealing with I mean JavaScript
Speaker 3: obviously one of the things we're pretty aware of, and
Speaker 3: I'm sure the listeners of your show are aware of.
Speaker 3: Is JavaScript is like eh, types whatever, even typescript is
Speaker 3: like is like yeah, types, but like you know, a
Speaker 3: number is a number, right? Well, see is like what
Speaker 3: how many bits is your number? Like he needs to know?
Speaker 3: So you start to need to think about that. When
Speaker 3: you work with WASM, you start to need to think
Speaker 3: about okay, is this a thirty two bit and going
Speaker 3: in here? And you need to start to think of okay,
Speaker 3: you need to start to think of memory, like okay,
Speaker 3: I need to have a pointer. I need to pass
Speaker 3: in a pointer here to get something back and then
Speaker 3: convert that pointer to like a JavaScript object of some sort.
Speaker 3: So it's challenging to work with, but once you get
Speaker 3: it working, it's extremely powerful because then we can ship
Speaker 3: something that's incredibly complex piece of code and just put
Speaker 3: basically a slim interface of JavaScript around it, and then
Speaker 3: any jabscript developer can just call is just talking to
Speaker 3: it like it's JavaScript. They don't need to worry about
Speaker 3: the WAHSM that was our problem. Yeah, So it's challenging
Speaker 3: to work with, but I do if anybody's thinking of
Speaker 3: has a challenging problem that requires the efficiency of c
Speaker 3: Don't be afraid of it. It's not that it's not
Speaker 3: that hard, and it is pretty amazing when you start
Speaker 3: working with it.
Speaker 4: Actually, Okay, so it works or there there is an
Speaker 4: NPM package if you want to use JavaScript with it.
Speaker 4: But what if you are a Python developer or maybe
Speaker 4: you're working with a micro controler like our do we
Speaker 4: know is there are there options for other other languages
Speaker 4: like that?
Speaker 3: Yeah, So, I mean we support since we're a developer
Speaker 3: focused company, we're pretty obsessed with our SDKs. So I
Speaker 3: think for our two most popular products, I think we
Speaker 3: have like twenty SDKs for each one, and it covers
Speaker 3: all the favorites. And we even have you know, we
Speaker 3: have three No, we have four different web SDKs. We
Speaker 3: have Vanilla JavaScript, but we also have React, Angular and view.
Speaker 3: So it allows we basically wanted to be like, use
Speaker 3: it in your favorite environment, like, yeah, use it like
Speaker 3: you use anything else in your in your stack, Like
Speaker 3: we don't want to just disturb that basically.
Speaker 4: Right, that's awesome. So what are some of the use
Speaker 4: cases that you've seen people employing it for recently?
Speaker 3: So we've seen, so we've actually come to some interesting
Speaker 3: ones lately. So auto content moderation is a big one
Speaker 3: right now. So let's say you're Minecraft or something, or
Speaker 3: you're I guess, let's let's go Fortnite, and you have
Speaker 3: open audio streams hundreds of thousands of players, and you're
Speaker 3: trying to moderate all that. That's that's very difficult. And
Speaker 3: it turns out a lot of big companies out there
Speaker 3: are using auto moderation, which basically takes that audio and
Speaker 3: is basically looking for key phrases let's call them, and
Speaker 3: it's just looking to flag them and then and then
Speaker 3: they'll usually have you know, a person go in and
Speaker 3: inspect the actual content of it and decide whether it
Speaker 3: was you know, a mistake or whether it is actually
Speaker 3: like a bannable offense. Yeah, so that that's actually a
Speaker 3: new exciting one. Also, like call centers, it turns out,
Speaker 3: again we've got open phone lines, like like a whole
Speaker 3: building full of them, and we're trying to understand, you know,
Speaker 3: what's being said on all these different calls, and you
Speaker 3: can't have people listening to all that audio. So a
Speaker 3: lot of big call center companies need some sort of
Speaker 3: automated system to take in all the audio from all
Speaker 3: their phones and do something with it. So we're we're
Speaker 3: encountering more use cases like that lately.
Speaker 2: Actually, those are both really fascinating. It's funny. The content
Speaker 2: moderation one really resonated with me because I play I
Speaker 2: don't know. My kids are eleven, so they're right at
Speaker 2: that impressionable age, but they're also right in the age
Speaker 2: where they want to play like games that are the
Speaker 2: sort where they have open audio. So there's a game
Speaker 2: we play that's like five y five, so five people
Speaker 2: on each team, and it has it has a way
Speaker 2: for you to do audio communication. And the very first
Speaker 2: thing I did was make sure to shut that off,
Speaker 2: like disable it, because like I'm a professional Internet user,
Speaker 2: and that's the first thing to learn is I don't
Speaker 2: trust anybody. I wouldn't even want to hear it myself,
Speaker 2: much less my kids, though I know.
Speaker 3: It like brings me back to like like when I
Speaker 3: was like you know, eleven or twelve, Like the Internet
Speaker 3: was like a new exciting thing, and I just would
Speaker 3: like I remember going to like I like was like
Speaker 3: really into like going to like like different video game
Speaker 3: websites and stuff. And then there was these just these
Speaker 3: chat rooms about video games you could go to and
Speaker 3: it was literally just like a room with like everybody
Speaker 3: their microphones are on and you just start talking and
Speaker 3: it was like, when I think of that now, I'm like,
Speaker 3: oh my god, that's frightening. Yeah, but uh yeah, I
Speaker 3: mean the fact is is we can we could keep
Speaker 3: those spaces safe with these sorts of tools, because then
Speaker 3: then air Duell's out there at least get banned when
Speaker 3: they're when they're being inappropriate or whatever.
Speaker 4: Oh god, Well, one thing that you put in the
Speaker 4: show notes today that I would really like to hear
Speaker 4: more about is a new speech to text engine or
Speaker 4: engines cheetah and leopards. So maybe you could tell us
Speaker 4: a little bit more about those.
Speaker 3: Yeah, so I think, yeah, last time we spoke, we
Speaker 3: actually didn't have a publicly available speech to text engine,
Speaker 3: and we were using our speech to intent engine, which
Speaker 3: was called Rhino, which was basically, like you, basically, yeah,
Speaker 3: the founder of the company is pretty obsessed with animals,
Speaker 3: So Rhino basically you teach it a small grammar and
Speaker 3: then it would understand that grammar, which is great for
Speaker 3: stuff like you know, controlling a coffee maker or like
Speaker 3: you know, there's only so many functions that needs to understand,
Speaker 3: but we decided to kind of go that extra mile
Speaker 3: and bring speech to text to devices. And traditionally language
Speaker 3: models are in the gigabyte realm of size, and we
Speaker 3: actually got ours down to twenty megabytes for language and
Speaker 3: that's sort of the big win for this is like
Speaker 3: we can run on anything that can take twenty megabytes
Speaker 3: of memory or of storage. And so Leopard and Cheetah
Speaker 3: are actually two different sides of the same coin. So
Speaker 3: Leopard is a speech to text engine that takes in
Speaker 3: a set amount of audio, so like an audio file
Speaker 3: or something, and gives you a transcript of that, and
Speaker 3: that's a lot that's an easier problem because you can
Speaker 3: basically say, okay, this is all the audio I'm going
Speaker 3: to get, so I'm to look forward, I'm going to
Speaker 3: look back, I'm going to make inferences based on the
Speaker 3: future in the past and give you a response. But
Speaker 3: then Cheetah, of course, because it's the fast one, it
Speaker 3: goes it's real time, so it has zero look ahead,
Speaker 3: which means it will take in every frame of audio
Speaker 3: that you give it and it will return what it
Speaker 3: thinks is being said, so they're both speech to text engines,
Speaker 3: but they just work at different use cases. So I
Speaker 3: mean audio files. The accuracy is much better, but of
Speaker 3: course you sacrifice the sort of real time effect.
Speaker 2: Yeah, so twenty megs is impressive, but is that still
Speaker 2: like small enough for a browser to use, Like does
Speaker 2: a user have to download that to use it in
Speaker 2: their WebP.
Speaker 3: So that was a challenge we recently. So we recently
Speaker 3: did the webstcs for Cheatah and Leopard, and we actually
Speaker 3: had to kind of redesign our whole system of delivering
Speaker 3: language to the the browser to handle this. So, yes,
Speaker 3: twenty megabytes is a lot, but we actually separate the
Speaker 3: language model from the package, so basically we let the
Speaker 3: developer decide how that's delivered to the user, but we
Speaker 3: also made it part of our system that it could
Speaker 3: be either a basicxty four representation that you can bake
Speaker 3: into your website if you just want it to always
Speaker 3: be there, or if you want to be kind of
Speaker 3: smarter about it, what you can do is put it
Speaker 3: in your public folder and have it downloaded to the
Speaker 3: user's browser on first load, and then cashed in local
Speaker 3: storage for the rest of the time, so that the
Speaker 3: next So the very first load, yeah, it'll be a
Speaker 3: twenty megabyte load, but the second load will be instant
Speaker 3: because they already have the language model.
Speaker 2: It's a pretty neat system because I think like it's
Speaker 2: it's the nature of the beast, because I mean it's
Speaker 2: it's in a way, it's kind of more of like
Speaker 2: a native app feature, and native apps are downloading like
Speaker 2: gigs at times of stuff, and so it's like a
Speaker 2: feature that helps the web sort of compete with that.
Speaker 2: So I think it makes sense, and I think, like
Speaker 2: honestly that I think that's the best that this solution
Speaker 2: is kind of clever because that's kind of all you
Speaker 2: can do because you can't you can't magically get it
Speaker 2: to the user ahead of time, like through an app
Speaker 2: store or something.
Speaker 3: So and a developer, if they're if they want to
Speaker 3: be clever about it, they can they can stream it
Speaker 3: from their public folder asynchronously on the first load, so
Speaker 3: that it's just like by the time the user wants
Speaker 3: to activate the voice feature, it's already downloaded. You know.
Speaker 3: It's it's just the sort of thing you need to
Speaker 3: you need to handle these sorts of ways. Because you know,
Speaker 3: we were working with a company recently that they do
Speaker 3: this all the time in their mobile apps. They'll their
Speaker 3: mobile app actually downloads like stuff all the time to
Speaker 3: keep their their app working, and it does it all asynchronously,
Speaker 3: like when you open up the app and you know,
Speaker 3: the user's none the wiser, but behind the scenes there's
Speaker 3: all this stuff. So when when you look up, why
Speaker 3: is this app using three point six gigabytes? When I
Speaker 3: downloaded it, it was only five hundred miniwtes, that's because they
Speaker 3: only delivered like the core code and the rest of
Speaker 3: it was downloaded later on. Yeah, so's it's it's just
Speaker 3: how it's just how you do stuff now is just
Speaker 3: keep keep the package sizes small, but then just deliver
Speaker 3: the features kind of as they're being used.
Speaker 2: Yeah, I know, iOS and Andrew even have like APIs
Speaker 2: built in to help you do that sort of thing
Speaker 2: because it's it's such a common model.
Speaker 3: Yeah, I think all the big companies want that. You know,
Speaker 3: if you're Spotify, you just you got to have the features.
Speaker 3: You don't want people to see three point six gayabytes
Speaker 3: when they go to download your app. There's like a
Speaker 3: sticker shock thing that happens. So it's kind of a
Speaker 3: funny thing because it ends up being that it's it's
Speaker 3: like when you book like an Airbnb and there's all
Speaker 3: these extra expenses that like get reported later, or like
Speaker 3: a or a flight where you get like the info later.
Speaker 3: It's sort of like that. It's like reduce the sticker
Speaker 3: shock and then we will show you the expenses after.
Speaker 2: So you also have an article in here about writing
Speaker 2: a podcast at transcription server to struggling to pronounce for
Speaker 2: some reason, which is a fascinating idea that I think,
Speaker 2: Like I know when we were talking before the show too,
Speaker 2: we've done transcriptions and videos. I'm sure there's other people
Speaker 2: that are call centers is another example, right, the things
Speaker 2: that you want to transcribe. So does that use Peeter
Speaker 2: Leopard or how does that work?
Speaker 3: Yeah, so it used this Leopard because we actually have
Speaker 3: the ability to get a whole file, like an hour
Speaker 3: long podcast and transcribe it from start to finish. And Yeah,
Speaker 3: the reason I kind of came up with that as
Speaker 3: an idea to sort of demo or technology is like,
Speaker 3: I know, I've listened to podcasts for years, and like
Speaker 3: it's so often on a long running show. I'm sure
Speaker 3: on this show you get the Hey have we talked
Speaker 3: about that? Did we talk about this? I feel like
Speaker 3: we've talked about this, and having show notes to go
Speaker 3: back too is probably a really helpful thing. Or like
Speaker 3: I was thinking of doing a next phase of the
Speaker 3: article where I actually make a podcast like searchable. So
Speaker 3: I made it transcribable and basically stored the like text representation.
Speaker 3: But once you have the text representation, you can make
Speaker 3: it searchable and then you can start being like, oh,
Speaker 3: when did I say this? And then it will just
Speaker 3: pop up the episode you set it in. So it
Speaker 3: was just kind of kind of an idea I came
Speaker 3: up with because I see a lot of people using
Speaker 3: Leopard on a server to basically hook into an event
Speaker 3: that's happening somewhere, whether it be on ourrs feed, RSS
Speaker 3: feed that's like updated or you know, yeah, like a
Speaker 3: new audio file or video is uploaded, and it hooks
Speaker 3: into that event, it runs it through Leopard and then
Speaker 3: stores it in a database. I thought that was like
Speaker 3: kind of a universal use case, Like it's just so
Speaker 3: it seems like a fundamental part of the web to
Speaker 3: like have something like that in a server.
Speaker 4: Yeah, I mean, it's it be so useful and it
Speaker 4: would it would help I think everybody from people who
Speaker 4: just want to reread part of a podcast if they're
Speaker 4: looking for something specific instead of having to just kind
Speaker 4: of hop through trying to figure out where it was
Speaker 4: that that useful bit of information was well.
Speaker 3: And and you can think too, like you can deliver
Speaker 3: these like let's say you attached it to your podcast.
Speaker 3: You can like deliver these transcripts along with your podcast
Speaker 3: because if you have the server hook in, transcribe it,
Speaker 3: and then deliver the transcript along with the podcast, suddenly
Speaker 3: you've got a follow along with the transcript podcast. So
Speaker 3: these sorts of things are useful for like auto captioning
Speaker 3: like videos or audio as well for like accessibility.
Speaker 2: It's accessible. It's also like marketing. People like it for
Speaker 2: SEO purposes too, because you know audio, Google can index audio,
Speaker 2: but if you have a transcription, it absolutely can even better.
Speaker 3: Yeah, one hundred percent correct. Yeah, search engines aren't very
Speaker 3: good at indixing audio, so you just have to plaster
Speaker 3: the text somewhere.
Speaker 2: Can you recognize different speakers because that's the other thing
Speaker 2: about a transcript, right is knowing who's talking you do
Speaker 2: you have the ability internally even if you don't know
Speaker 2: names obviously, but can you say, like this is voice one,
Speaker 2: voice two.
Speaker 3: Yeah. So actually we're working right now on a I'll
Speaker 3: give you guys the scoop right now on a speaker
Speaker 3: identification system that will basically be able to tell people
Speaker 3: apart because yeah, when when you think of something like
Speaker 3: doing like like a Zoom meeting, if you want like
Speaker 3: to have meeting notes, Yeah, it would be really useful
Speaker 3: to have like this came from this person, This came
Speaker 3: from this person, This came from this person, and you
Speaker 3: can use different I mean, Zoom obviously has the ability
Speaker 3: to know where the audio is coming from, so it
Speaker 3: can kind of just label it. But if you have
Speaker 3: an anonymous audio stream with a bunch of different voices,
Speaker 3: that's that's challenging because you don't know. You just have
Speaker 3: to base your assumptions on the character of the voice.
Speaker 3: Who's who's different. That's actually a problem we're working out
Speaker 3: right now. And I mean that's that's useful for not
Speaker 3: only speaker labeling, but also a speaker verification. So like
Speaker 3: if we want to voice activate something that only responds
Speaker 3: to your voice, that's also like another use case for it.
Speaker 4: That would be cool.
Speaker 2: Well, this has been a blast. Is there anything that
Speaker 2: you wanted to discuss today that we have not gotten
Speaker 2: to at all?
Speaker 3: No, I think I think we covered a lot here.
Speaker 2: Yeah, yeah, excellent. So that why don't we move into
Speaker 2: our picks, and Paige, do you want to kick us off?
Speaker 4: Sure? So my pick is going to continue the trend
Speaker 4: that I started last week, which was Star Trek. As
Speaker 4: many of you who have been listening for a while,
Speaker 4: I've been on a Star Trek journey through Next Generation
Speaker 4: and forward. So most recently I've begun watching Star Trek
Speaker 4: Lower Decks, which is their animated series, and there's only
Speaker 4: I think there's maybe two, maybe three seasons of it,
Speaker 4: but it is. It is in the stuff of Rick
Speaker 4: and Morty, and it is the funniest Star Trek that
Speaker 4: I've ever seen, to the point where I'm actually laughing,
Speaker 4: which is unusual for anything animated, but it is really good.
Speaker 4: There's a lot of references to other Star Trek franchises,
Speaker 4: so if you are familiar with Next Generation or Voyager
Speaker 4: or Enterprise, they throw in all sorts of little jokes
Speaker 4: that are related to those characters. So I would definitely
Speaker 4: recommend it. It's it's as family friendly as the rest
Speaker 4: of the Star Trek a franchise is, and it's also
Speaker 4: got a much bigger dose of humor than most of
Speaker 4: them do. So if you're looking for something that's quick
Speaker 4: twenty twenty five minute episodes, I would definitely say it's
Speaker 4: a good one, excellent.
Speaker 2: I still have not gotten into the Star Trek world,
Speaker 2: so it's at some point. I've had it recommended several times,
Speaker 2: but I feel like it's such an like you can't
Speaker 2: like casually wade into it, right, like you kind of
Speaker 2: have to. Yeah, so if I pick this week is
Speaker 2: going to be The Great British Bakeoff, which I think
Speaker 2: was the previous of Year's page. I started off watching
Speaker 2: it because I just wanted to know what it was about, right,
Speaker 2: just that sort of thing. And then next thing I
Speaker 2: knew I had watched a few episodes and I didn't
Speaker 2: even really understand why.
Speaker 3: It's so it's so comforting that show, Like there's just
Speaker 3: something so positive and warm about it.
Speaker 2: It's it's strangely compelling, Like I can't even understand why
Speaker 2: I ended up watching it, but it's quite good. I
Speaker 2: think Netflix has like five seasons or so, I mean
Speaker 2: I don't know how much I'm going to watch, but
Speaker 2: I've it's a good thing to just have on you're
Speaker 2: not sure what to do. It's just comforting, nice to
Speaker 2: have on in the background. So I've been sucked into
Speaker 2: that as well.
Speaker 3: That's funny. My wife and I like literally just started
Speaker 3: watching that like a few weeks ago, and for the
Speaker 3: same reason, like why are people talking about this so much?
Speaker 3: And yeah, now we're like shouting out the screen that's
Speaker 3: not a good bake. Look at the problem mature.
Speaker 2: Yeah, excellent, Ian. What picks do you have for us?
Speaker 3: Yeah? So I think last time I brought Mandy a
Speaker 3: really gnarly horror film. So I figure back up and
Speaker 3: maybe do something a little different this time and actually
Speaker 3: go with something tech related. So we've been working with
Speaker 3: Mixed Panel recently, which is an amazing service. It's really
Speaker 3: helped us because we're trying to add some analytics to
Speaker 3: our website and our console, but custom analytics that allow
Speaker 3: us to like basically track like basically when somebody enters
Speaker 3: the website, what they interact with and like how long,
Speaker 3: and like develop our own metrics based on the code
Speaker 3: we actually put in the website and mixed Panel is
Speaker 3: amazing at this. What they basically do is they're just
Speaker 3: all they do is they say, hey, we're just going
Speaker 3: to take events and we're going to represent them in
Speaker 3: a whole bunch of different ways. You can filter, you
Speaker 3: can form funnels, you can show userflow, you can take
Speaker 3: a user and actually watch where they go on the
Speaker 3: website and stuff. It's super super helpful for us. We've
Speaker 3: actually been we've had a total crush on them since
Speaker 3: we started working with their their product because they just
Speaker 3: not only is their their UI like so nice to
Speaker 3: work with, but it's just made our life. We were
Speaker 3: thinking of building this far because basically we know wanted
Speaker 3: data analytics to our website, but Google analytics and that
Speaker 3: the sort of general analytics were just not enough. We
Speaker 3: needed like really specific ones, right, and we were going
Speaker 3: to build it ourselves, and then we stumbled the prompt
Speaker 3: upon Mixed Panel and it was like, oh my god,
Speaker 3: this saved us, Like like they made what we could
Speaker 3: have made. It would have taken us. We would have
Speaker 3: had to start a new company to make what they made,
Speaker 3: and it's just it's so it's so helpful, So definitely
Speaker 3: for any developer out there, that wants to add like
Speaker 3: customer analytics, mixed panels, really really helpful.
Speaker 2: Awesome, excellent, Well this has been amazing. My last question
Speaker 2: for you. If people want to follow you, keep up
Speaker 2: with you, what are the best we're the best places
Speaker 2: to go to do that.
Speaker 3: Yeah, I mean, let's see, I don't I don't have like,
Speaker 3: I don't have like professional socials out there, but I
Speaker 3: do have. I am on medium as Ian Lavery, so
Speaker 3: you can read any articles I put up there. You
Speaker 3: can follow Peak a Voice AI on Twitter, and we
Speaker 3: have a YouTube channel if you want to check out
Speaker 3: my bands. Yeah, fellow kids in Sleep Circle check them out.
Speaker 2: Yeah, no, excellent, that's great. We'll get those in the
Speaker 2: show notes. And yeah, thanks for joining us today. This
Speaker 2: is a great chat.
Speaker 3: Yeah, thanks for having me. This is great.
Speaker 2: Cool, all right, everybody until next week.
Speaker 4: See you then,