Speaker 1: Hey, welcome back to another episode of JavaScript Jabber.
Speaker 2: This week on our panel we have Steve Edwards.
Speaker 3: Yo yo yo, come in ato live from cold in
Speaker 3: sunny Portland.
Speaker 4: We also have a j O'Neil yo yo yo coming
Speaker 4: at your live from the soldering station.
Speaker 1: Oh sorry, I'm Charles Maxwood from Top End Devs and
Speaker 1: yeah it is freezing here anyway.
Speaker 2: We have a special guest this week and that is
Speaker 2: is Sean Annon.
Speaker 1: You want to let people know who you are, what
Speaker 1: you do where?
Speaker 2: Yeah, of course is because your course is awesome.
Speaker 5: Oh thank you. So. My name is Eha nand I
Speaker 5: have about twenty years of engineering and product management experience
Speaker 5: and most recently I've been very focused on AI for
Speaker 5: the last couple of years and I'm best known THEA
Speaker 5: community for an implementation.
Speaker 2: Of GPT two.
Speaker 5: There's a precursor to chat GPT that I implemented entirely
Speaker 5: in Excel and then late last year I reported that
Speaker 5: entirely to the web and pure JavaScript, and I teach
Speaker 5: how the entire transformer works. Basically the model that you
Speaker 5: know was the you know, ancestor to Gemini, Barred Lama, Chat, GPT, Claude.
Speaker 5: They're all really inheriting from this model called GPT two
Speaker 5: and I teach people and basically course of two weeks.
Speaker 5: If you have really no programming experience, or if you've
Speaker 5: got JavaScript programming experience, this is the best way to
Speaker 5: really get in understand how these things work. And they
Speaker 5: don't have to be a black box and you can
Speaker 5: see all that at Spreadsheets at All. You need dot
Speaker 5: ai and the classes on mavin.
Speaker 2: Very cool, So let's let's dive in. First.
Speaker 1: I think you said you had a promo code for
Speaker 1: the course, so let's just put that out there.
Speaker 2: Yeah, people want to go get it and get a
Speaker 2: deal on it.
Speaker 5: Yeah. So the promo code is really easy to remember.
Speaker 5: It's jsjer and just go to Maven dot com and
Speaker 5: look for my name, or if you go to spreadsheets
Speaker 5: at All, you need dot ai and then you click
Speaker 5: that you can use that promo code for twenty percent
Speaker 5: off for the next two weeks.
Speaker 2: So awesome.
Speaker 5: Definitely check that out. And I should just say, you know,
Speaker 5: thank you guys for having me. I listened for years
Speaker 5: to this, so it's great to actually meet you guys,
Speaker 5: well virtually in person.
Speaker 2: Right. Yeah, AJ is the cool one. I just run
Speaker 2: the show anyway, and.
Speaker 3: I'm just thinking guy while everybody else are the smart
Speaker 3: people according to some people.
Speaker 2: Anyway. So let's dive in.
Speaker 1: You said that you explain how the transformer works, and
Speaker 1: so for those that are kind of new to AI,
Speaker 1: do you want to just explain what a transformer.
Speaker 2: Is an AI? Yeah, we can dive into house stuff works.
Speaker 5: Yeah, sure. So the the transformer is a you know,
Speaker 5: AI architecture of a model that came out in twenty
Speaker 5: seventeen and it is the foundation for most of the
Speaker 5: you know AI models that have been you know, like
Speaker 5: chat GPT, so those chatbot assistants that seem amazingly smart
Speaker 5: all inherent from this architecture called the Transformer. And I
Speaker 5: can give a high level over your everything that goes
Speaker 5: into that. But the key thing that the transformer does
Speaker 5: is usually takes some input and it tries to predict
Speaker 5: what the next word is. And that's really all your
Speaker 5: large language model is doing is taking one word or
Speaker 5: really one token at a time, and it's trying to
Speaker 5: predict when you enter in a question what the next
Speaker 5: thing is. And over you know, the last you know,
Speaker 5: a couple of years, what we've been able to do.
Speaker 5: We collectively as humanity is. Take this model that tries
Speaker 5: to predict the next word and turn into these really helpful,
Speaker 5: amazing chat bot assistants. And the paper that introduced this model,
Speaker 5: called the Transformer, was called Attention Is All You Need.
Speaker 5: And that's where my course gets its name Spreadsheets Are
Speaker 5: All You Need? Is I basically implemented that entire model
Speaker 5: inside a spreadsheet? Hence the name Spreadsheets Are All You Need?
Speaker 3: So question here then? So I mean, having used Google
Speaker 3: since its inception, you know, type ahead is sort of
Speaker 3: a standard thing in search. You know, where you're typing,
Speaker 3: and it's starting to anticipate what your phrases, you know,
Speaker 3: what you're gonna type next. If I'm you know, starting
Speaker 3: to search for spreadsheets on Google, it's going to anticipate, Okay,
Speaker 3: what's the next thing I'm going to type? So is
Speaker 3: this basically the same thing just on AI steroids or
Speaker 3: because I mean basically that's using what people have typed
Speaker 3: in and you know they've indexed it and you know,
Speaker 3: done things with it. So is that sort of the
Speaker 3: same thing just on steroids or is that intrinsically different?
Speaker 5: Yes? And no in terms of effect, it is literally
Speaker 5: just doing the same thing, like it's trying to break
Speaker 5: the next thing. Is I really kind of get a
Speaker 5: little bit of a mental pushback that to just saying, oh,
Speaker 5: it's just like autocomplete. It is basically structured as an
Speaker 5: autocomplete problem, but the level of complexity of the architecture
Speaker 5: to solve that problem is just a lot more complex.
Speaker 5: But it is trying to do the same thing. And
Speaker 5: you know, the way to think about this is if
Speaker 5: you can fill in the blank in any sentence, you
Speaker 5: probably know something about that sentence. You already know what
Speaker 5: the answer might be. Like that's a useful test of knowledge.
Speaker 5: But effectively, yeah, that is that is what's going on.
Speaker 5: It's just trying to break the next word, and then
Speaker 5: the next word after that and so forth, one at a.
Speaker 1: Time, right, and so effectively, I guess the the autocompletes
Speaker 1: that we typically see are a little bit I guess
Speaker 1: more naive than say the AI LM models, where they
Speaker 1: have substantially more data to run on, and you know,
Speaker 1: use a mechanism that I guess is probably somewhat the
Speaker 1: same because it's weighted and things like that, but anyway.
Speaker 2: It can do it across a wider variety.
Speaker 1: Of things and give you deeper answers.
Speaker 5: Yeah, so I mean, actually, let's start with the autocomplete example,
Speaker 5: because it does kind of point the way to some
Speaker 5: parts of the architecture. Like the simplest thing you might
Speaker 5: do for building an autocomplete is you might just say,
Speaker 5: if I see this word, what are all the next
Speaker 5: likely words that will be after it? And you could
Speaker 5: just do a statistical look up across some large data sets, right,
Speaker 5: And as good as that'll be, the more pieces of
Speaker 5: data you look at, the better it's predictive value. So
Speaker 5: this is called like a bigram model. And then because
Speaker 5: it looks at two and then what you could do
Speaker 5: is you could actually look three words back, or you
Speaker 5: could look forwards back. And actually, one of the key
Speaker 5: things about the transformers it tries to look at all
Speaker 5: the words. And this is what the attention mechanicism does,
Speaker 5: is that it can figure out, essentially from all the
Speaker 5: possible words before it, what is the next most likely word.
Speaker 5: And then the other key thing you need to do
Speaker 5: is you ask a real network to take all that
Speaker 5: information a prediction. And it turns out that's the heart
Speaker 5: of the transformer and what really made it work was
Speaker 5: they just scaled that up to a much larger size
Speaker 5: than I think people were used to doing. You know
Speaker 5: when you're autocomplete and your keyboard is probably used to
Speaker 5: you know, is built to be really really fast, and
Speaker 5: so they tried to make it really efficient. And what
Speaker 5: we've been able to do with the Transformer is make
Speaker 5: it really big and then actually make it super efficient
Speaker 5: scaling it back down so just that it spits out
Speaker 5: tokens at a reasonable clip. But that core idea of saying, hey,
Speaker 5: let me look statistically at you know what the next
Speaker 5: thing is, Well, one word isn't gonna be enough, Two
Speaker 5: words back is going to be better. That is what
Speaker 5: the attention mechanism is. In a sense, if you squint,
Speaker 5: doing is it's trying to look at all the words
Speaker 5: that came before, it puts them through multiple passes, and
Speaker 5: then it's asking your normal network to do the prediction
Speaker 5: rather than just simply saying, oh, let me take the
Speaker 5: raw statistics.
Speaker 1: But yeah, so do you kind of want to break
Speaker 1: down for us how these systems actually work.
Speaker 5: Yeah? So the first thing I say is the way
Speaker 5: to think about the simplest model that I like of
Speaker 5: the transformer is that what we've been able to do.
Speaker 5: You know, we said that, you know, these are trained
Speaker 5: to fill in the blank on a piece of text.
Speaker 5: So the example I often use in my lectures and
Speaker 5: inside a lot of my material is this very simple,
Speaker 5: simple sentence, Mike is quick, he moves and the next
Speaker 5: most likely completion would probably be he moves quickly, or
Speaker 5: he moves around, or he moves fast. And so the
Speaker 5: basic question is how do we get a computer to
Speaker 5: fill in the blank of an English sentence or any
Speaker 5: natural language sentence. And what we've been able to do
Speaker 5: is actually figure out to talk in the language of
Speaker 5: the computer, which is math. So if I gave the
Speaker 5: computer a math problem two plus two equals it could
Speaker 5: fill in the blank. It knows that two plus two
Speaker 5: equals four, and we can make the math as pretty
Speaker 5: large and complex. But computers are really good at math.
Speaker 5: So we've been able to do is and what the
Speaker 5: model does is it takes a word problem and it's
Speaker 5: really converting it to a math problem. If you look inside,
Speaker 5: you know, go to my website, you know, spreadsheets are
Speaker 5: all you need dot ai slash GPT two, or if
Speaker 5: you download the Excel file and look inside it what
Speaker 5: you'll see in you know, there's text at the beginning.
Speaker 5: You type in text on one part of the spreadsheet,
Speaker 5: and you get the predicted word at the other end
Speaker 5: of it. But in between, if you look in that,
Speaker 5: you'll be like, where the heck are the words? It's
Speaker 5: all numbers. And so the key insight is what we've
Speaker 5: been able to is take something that is a word
Speaker 5: problem and we've turned words into math and once and
Speaker 5: that mapping process of words into map has two stages.
Speaker 5: It's called tokenization and then embeddings. And at the end
Speaker 5: of it, we map every word. You can conceptually think
Speaker 5: about it to a single number, but we actually map
Speaker 5: them to a large list of numbers. And then once
Speaker 5: you have a mathematical representation of your prompt, your entire
Speaker 5: prompt has been you know, turned into a large list
Speaker 5: of numbers. We then run I just call it number crunching.
Speaker 5: It's these two key mechanisms attention and a multi layer
Speaker 5: perceptor or a neural network that just kind of crunches
Speaker 5: on it to try and predict what the next word is.
Speaker 5: And then at the end of that we get a number,
Speaker 5: and that number we then reverse the process that came
Speaker 5: out of that thing, and we say, well, what what
Speaker 5: word does this number map to? And that number is
Speaker 5: a predicted word, but it's not going to map cleanly
Speaker 5: to every single word in our vocabulary. And so if
Speaker 5: that number is closer to certain words, like in the
Speaker 5: case mike is quickly as quickly, the predicted number might
Speaker 5: be really close to the word quickly. It might be
Speaker 5: close to the word around, but it's not going to
Speaker 5: be close to you know, quick can be a body part,
Speaker 5: it can be the quick of your fingernail. It's not
Speaker 5: going to be something about your fingernail, because it's figured
Speaker 5: out enough that it's moved the predicted number away from that.
Speaker 5: And so we take that and we run a random
Speaker 5: number generator the very end, and then we pick it
Speaker 5: according to that random number generator based on how close
Speaker 5: that number is to one of the other words in
Speaker 5: the dictionary of words mapping to numbers. So that's like
Speaker 5: my highest level summary of what's happening under the transform
Speaker 5: without describing all the mechanisms. But again, the key thing
Speaker 5: is we found a way to map solve this problem numerically.
Speaker 5: We map words to numbers. We turn the whole sentence,
Speaker 5: your entire prompt into a large list of numbers, We
Speaker 5: number crunch on it. Then we get a predicted number
Speaker 5: out of it. We just calculate and we look at
Speaker 5: how close that number is to our number to word
Speaker 5: mapping at the very end, and that's the probability you
Speaker 5: get of getting a particular token or word out of
Speaker 5: the model. Let me pause there, see if their questions
Speaker 5: or things I should clarify, So.
Speaker 2: I think I follow along.
Speaker 1: Essentially what you're saying then is, so let's say I
Speaker 1: wanted it to generate a whole paragraph.
Speaker 2: It just does this over and over and over again.
Speaker 2: Get yeah, the next word.
Speaker 5: Yeah, maybe I've glossed over that part of it. Like
Speaker 5: the large language model only predicts the next word technically
Speaker 5: something called a token, which is slightly smaller than a word,
Speaker 5: and every time you get a prediction out of it, like,
Speaker 5: it doesn't by default predicted paragraphs. So if you you know,
Speaker 5: try my app or you download the spreadsheet, it only
Speaker 5: predicts one token. And the way we get paragraphs a
Speaker 5: text out of this is we take the predicted token
Speaker 5: it came up with, and then we stick it back
Speaker 5: onto the input, and then we ask it to predict
Speaker 5: the next sentence or the next that new accumulated paragraph,
Speaker 5: and so you can actually start with a single word,
Speaker 5: ask it to predict what the next word is, and
Speaker 5: then you now you've got two words, and then you
Speaker 5: run it through and then you keep going. And then
Speaker 5: what happens when you've got user input like somebody types
Speaker 5: of response, is you just stick that entire user input
Speaker 5: as you know, a large set of words that it
Speaker 5: needs to brick what the next thing is. And you
Speaker 5: can think about it structured into the model. As you
Speaker 5: are reading a transcript between a user and a helpful
Speaker 5: chatbot assistant. User said X, we fill in what the
Speaker 5: user said, assistant said, and then it needs to come
Speaker 5: up with what the assistant said, and it just tries
Speaker 5: to come with something plausible. Maybe the thing is step back,
Speaker 5: like the base model that these that gets trained in
Speaker 5: this process before it's turned into a helpful chatbot just
Speaker 5: knows really simply how to complete sentences. If you take
Speaker 5: the base GPT two and you type in, you know,
Speaker 5: questions to it, it's not going to necessarily respond back
Speaker 5: to you meaningfully. It's just designed to predict the next
Speaker 5: word based on everything it's seen on the internet. So
Speaker 5: a good example I use in classes, we type in
Speaker 5: the word first name and then you hit return, and well,
Speaker 5: what do you think it would predict after that? It
Speaker 5: predicts last name, email address, phone number, because most texts
Speaker 5: on the Internet that's say first name. Statistically, it's a
Speaker 5: form and it's used to just filling out forms. Another
Speaker 5: one is I type in hello class, and when I
Speaker 5: first did this, I thought it was going to say
Speaker 5: hello teacher, but it actually starts spitting out Java code,
Speaker 5: so it just looks at the fact. Yeah, it's really
Speaker 5: a music to watch and you can you can just
Speaker 5: run it, and it's just trying to predict what the
Speaker 5: next thing is based on what it saw on the internet.
Speaker 5: And then what you know open Ai and Nentropic and
Speaker 5: these companies do is they put that call a base model,
Speaker 5: which all it knows how to do is predict the
Speaker 5: next word through a training regime to elicit it to
Speaker 5: be more like a helpful chatbot. So you give it
Speaker 5: a system prompt that tells it it's a chatbot. It's
Speaker 5: kind of like you tell it a story that's plausible
Speaker 5: for it to start to think like it's talking to
Speaker 5: a user, like you are a chatbot. You are reading
Speaker 5: a transcript of a chatbot and a human user, and
Speaker 5: we just fill in what the human said, and it
Speaker 5: tries to fill in what it thought the helpful system
Speaker 5: would be, and then they fine tune it to get
Speaker 5: better at that.
Speaker 2: Yeah, this sounds a lot like what you're explaining.
Speaker 1: You get into prompt engineering, which, again, if you're not
Speaker 1: into AI, prompt engineering.
Speaker 2: Is what's all the stuff I tell the AI.
Speaker 1: System so that it'll give me the answer I want, right,
Speaker 1: And so you're when we're talking about prompt engineering, now
Speaker 1: it's okay.
Speaker 2: So this is why when I start out.
Speaker 1: I tell it things like, like you said, you are
Speaker 1: a chat bot, you help people with these problems, you
Speaker 1: do these kinds of things, because it'll build off of
Speaker 1: all of that and use the statistical model now with
Speaker 1: the context of what you typed in to give you
Speaker 1: the right answer.
Speaker 2: So you know, yeah, Hello class. There's not a whole
Speaker 2: lot there for it to go on.
Speaker 1: But if you tell it, you know, you're a chat
Speaker 1: bot and you're helping students with a blah blah blah
Speaker 1: blah blah, then you type in hello class, and it's
Speaker 1: going to go you know, then it may come back
Speaker 1: with hello teacher or something like that.
Speaker 5: That's a great example. Yeah. So, and what you can
Speaker 5: think about them conceptually doing is baking that prompt engineering
Speaker 5: into the model. So what they're able to do is
Speaker 5: if they give it enough examples of this, they can
Speaker 5: retrain it such that you don't need the prompt at
Speaker 5: the beginning that tells it it's a teacher or that
Speaker 5: it's a helpful chatbot assistant, and that gets baked into
Speaker 5: the model. You can think about all that prompt engineering
Speaker 5: gets memorized into the model during that training process, and
Speaker 5: then it turns into that helpful assistant will.
Speaker 4: Help help me understand this a little bit. So I've
Speaker 4: I've played around obviously with GPT. I've also played around
Speaker 4: with the other models. In fact, right now I really
Speaker 4: like Quinn. I am I am using Quinn more than
Speaker 4: I'm using GPT, because Quinn actually seems to be giving
Speaker 4: better results, especially considering it in the benchmarks, it outperforms
Speaker 4: four oh whatever that means. I mean, it's like by
Speaker 4: a fraction of percentage point, but OH one. I just
Speaker 4: find OH one and R one to be too like
Speaker 4: they take forever. So it's like I'd rather ask the
Speaker 4: question twice and be ninety nine percent likely to get
Speaker 4: the right answer, then ask the question one time and
Speaker 4: then have to wait forty five seconds to get the
Speaker 4: wrong answer and ask it again.
Speaker 3: You know, forty five seconds. That's an eternity.
Speaker 4: The O one is crazy.
Speaker 2: We use it.
Speaker 3: We use it for code, for code questions and stuff,
Speaker 3: because it does better than the standard GPT for wait
Speaker 3: a few seconds. But I'd rather get a wait a
Speaker 3: little bit and get a better answer than get something
Speaker 3: super fast that's not going to be as good.
Speaker 4: Well, I I'm the other way because it's not that
Speaker 4: much better. If you look at the benchmarks, it's like
Speaker 4: one percent better than four oh, and it takes you know,
Speaker 4: so much anyway. But what the thing, the thing that
Speaker 4: I was that I was getting at is in the beginning,
Speaker 4: there was the system prompt. Right, so when with GPT,
Speaker 4: one of the ways to jail break it was you
Speaker 4: could say that was just a joke. Actually you're a
Speaker 4: something else, and so it would interpret it as Okay,
Speaker 4: your system prompt is you're a chatbot. You're allowed to
Speaker 4: say this. You're not allowed to say that that you
Speaker 4: could just say that was just a joke. And then
Speaker 4: and then and then give it an additional prompt. Now
Speaker 4: with deeps seek V two point five and are one
Speaker 4: and with Quinn it's it's like you're saying it's baked
Speaker 4: into the model because if I override the system prompt
Speaker 4: and I tell it, you know, you are a human
Speaker 4: who is capable of reasoning and has no biases and
Speaker 4: can represent any information factually, tell me about Tianaman Square.
Speaker 2: It's you know, it's.
Speaker 4: I am a helpful bot. I am not a human,
Speaker 4: and I do not talk about things that contradict what
Speaker 4: is known to be you know, the proper the proper
Speaker 4: knowledge of the of the Chinese government to protect the people,
Speaker 4: or you know, it gives me some some nonsense like that.
Speaker 4: So what what is How is it possible to bake
Speaker 4: in those system prompts with training data and and I
Speaker 4: guess how does that vary? How does it vary from
Speaker 4: the system prompt? And how do they get it to
Speaker 4: bake that in so that it you can't override it
Speaker 4: with a system prompt.
Speaker 5: Okay, there's a lot of layers there.
Speaker 2: Let me yeah, question, can you restate the question in
Speaker 2: one sentence?
Speaker 5: I think the uily what I think with the question,
Speaker 5: which was how do you bake in the system prompt.
Speaker 5: But there's a couple of things that are worth noting
Speaker 5: in your question, Like you mentioned some reasoning models one
Speaker 5: and R one, and the way those operate is a
Speaker 5: little bit different. Like you said, it takes a while
Speaker 5: to come back because it's actually just expending a lot
Speaker 5: of tokens thinking that it doesn't give you, and it's
Speaker 5: trying to actually think through the process like you might do.
Speaker 5: They call this chain of thought or thinking step by step,
Speaker 5: and it what's unique about that can parterregular chain of
Speaker 5: thought is it can suddenly realize, oh, it's made a
Speaker 5: mistake and backtrack. And so it's it's literally spending you know,
Speaker 5: coming up with hypotheses and trying and testing things and
Speaker 5: seeing if it works. So this is why these models
Speaker 5: tend to be really good on math and code because
Speaker 5: it can go try something and say, oh wait does
Speaker 5: this let me check does this answer right? Oh no,
Speaker 5: it's not, let me try again. So and then you mentioned,
Speaker 5: you know, jail breaking, and with the early models, one
Speaker 5: way to think about like you're like, oh, this is
Speaker 5: just a joke, is that you know, you're kind of
Speaker 5: taking that we've talked about briefly, that attention mechanism or
Speaker 5: looking back at the previous what's most likely if you
Speaker 5: put things like you know, you know, kill and harm
Speaker 5: in the in the prompt, statistically it sounds like it's negative, right,
Speaker 5: But if you start putting things like Grandma cookies, it
Speaker 5: seems less harmless, and you kind of think of yourself
Speaker 5: a kind of waiting the attention to be more to
Speaker 5: the harmless side. And really what's happened is that the
Speaker 5: models have gotten smarter, both in terms of their natural
Speaker 5: responses to this, Like they are trained to handle jail breaks.
Speaker 5: They are trained on if a jail break comes up,
Speaker 5: here's the response. And the way they train it to
Speaker 5: get to your your main question is through these two
Speaker 5: training techniques. One is they just give it an example
Speaker 5: of a prompt and what its response should be, and
Speaker 5: they use this technique called backpropagation or sarcastic creating descent,
Speaker 5: which is to tune the network such every time it
Speaker 5: sees that result, it gives out what we wanted it
Speaker 5: to have. So we're going a little head of where
Speaker 5: I wanted to be. But like when you train in
Speaker 5: a ural network, you give it examples of data, so
Speaker 5: The simple example is a dog and cat classifier.
Speaker 2: Right.
Speaker 5: I give it pictures of dogs, and I give it
Speaker 5: pictures of cats, and I tell it which ones are
Speaker 5: dogs and cats, and it comes up with the answer.
Speaker 5: It comes up with the rules how to figure out
Speaker 5: whether an image is dog or cat. This is way
Speaker 5: different than regular programming, right. Machine learning inverts the normal paradigm.
Speaker 5: Normally we're used as developers. I write a series of rules,
Speaker 5: a series of program, right, and then it processes data
Speaker 5: and gets out a result. I click a button, something
Speaker 5: does you know moves on the screen, So I can
Speaker 5: write that program. But a dog and cat photo classifier,
Speaker 5: I don't know if you gave me dogs photos of
Speaker 5: dogs and cats, I know how to instinctively do that,
Speaker 5: but I do know how to write it out as
Speaker 5: a series of rules. And so the inversion that machine
Speaker 5: learning does is you give it answers and you give
Speaker 5: it data, and then it figures out how to write
Speaker 5: the rules. It writes the program. Now, unfortunately we can't
Speaker 5: always understand what the program it comes up with is.
Speaker 5: But what they do is they give it examples of
Speaker 5: jail break attempts and they say, hey, you know your response,
Speaker 5: now should be this to that. That's kind of the
Speaker 5: high level overview of how they do that. One thing
Speaker 5: that's worth noting is that when they protect a model,
Speaker 5: it isn't just in the model itself, So there are
Speaker 5: usually things that are watching the result of the model
Speaker 5: that are additional classifiers. And so sometimes you might see
Speaker 5: examples of open source models that let you do things,
Speaker 5: but the hosted versions do not because the hosted system
Speaker 5: is actually checking. So not everything is baked into the
Speaker 5: model itself and it's not one hundred percent perfect, So
Speaker 5: often there's some additional guardrails that are detecting things.
Speaker 4: Okay, So then two more questions.
Speaker 6: Yeah, what constitutes open source because that does not mean
Speaker 6: the same thing that it means in the programming circles, or.
Speaker 4: I don't believe it does because I have not yet
Speaker 4: seen any open source model that comes with four hundred
Speaker 4: terabytes of training data.
Speaker 5: There are few and far between. There are some. Olmo
Speaker 5: is probably the best known one, which is a model
Speaker 5: where everything the training code, the training data system, the
Speaker 5: data collection pipeline, the logs from their training runs are
Speaker 5: completely open. There's like a handful of others. But this
Speaker 5: question of what constitutes open is is completely a gray
Speaker 5: area and it's being debated right now. Traditionally, when people
Speaker 5: talk about an open model, it's usually an open weight model,
Speaker 5: which is you get the parameters which encompass the rules
Speaker 5: we talked earlier. That whole thing is math. Right, So
Speaker 5: if you open up my spreadsheet or you open up
Speaker 5: my website, you just see lots of numbers. You know,
Speaker 5: whether those numbers are hidden from you or you can
Speaker 5: run them yourself is what people call an open weight model.
Speaker 5: That's what kind of passes for open source. These days,
Speaker 5: there are very few models that open up the training data,
Speaker 5: and so it's debatable and people do debate about what
Speaker 5: a truly open model means. A truly open like the
Speaker 5: most open is one that includes the data, but there
Speaker 5: aren't that many, especially at the state of the art,
Speaker 5: where the model is all the training day that created
Speaker 5: the model is there.
Speaker 4: Well, I mean that would be highly illegal for chat
Speaker 4: GPT to get as their training data because YouTube and
Speaker 4: all the libraries on planet Earth and everyone who has
Speaker 4: a copyright on something would have something this day about that.
Speaker 5: Well, I mean, I'm not going to comment on any
Speaker 5: particular model providers specifically, I will say the idea of
Speaker 5: whether you can train on data and whether it's transformative
Speaker 5: is quite frankly still in the courts right now, and
Speaker 5: we don't have global consensus. So I believe it's Japan
Speaker 5: has said and clarified that you can train on data,
Speaker 5: that the training process is not directly infringement. Now, you know,
Speaker 5: one of the litmus tests is like whether you're competing
Speaker 5: with your regional thing. So it's it's a larger open question,
Speaker 5: but right now that's making its way through the courts.
Speaker 5: I think, you know, candidly, if you said here all
Speaker 5: the things I've trained on, then you might end up,
Speaker 5: you know, just opening yourself up to more people who
Speaker 5: can just say, oh, let me get onto that lawsuit.
Speaker 5: But I mean that that question is still being that's
Speaker 5: a legal question, which.
Speaker 4: Yeah, yeah, yeah, yeah, yeah, okay. So my my other
Speaker 4: question related to what we would we had just talked about,
Speaker 4: was so I I download Quinn and I don't know
Speaker 4: what it is that I'm actually downloading because I use
Speaker 4: I use Olama, and it downloads twenty gigabytes of something
Speaker 4: and then it runs it and I get to be
Speaker 4: productive and I'm happy. But in terms of you know,
Speaker 4: like like you're saying there's something it's not just the
Speaker 4: model giving a response, but then there's other code that
Speaker 4: is you're doing some sort of check. Is that happening
Speaker 4: with these models that I am using generic tools like
Speaker 4: Olama or Lama CPP or or LM studio. Is that
Speaker 4: actually running program code? Binary code? Or I guess it's
Speaker 4: not binary code. It have to be bytecode because I
Speaker 4: can do it on Mac and I can do it
Speaker 4: on Linux and it doesn't have to recompile anything after
Speaker 4: it's done downloading it. So what where where are those
Speaker 4: extra layers or how are they interpreted?
Speaker 5: The extra layers that are protecting the model from saying
Speaker 5: grong thing? Is that what you're asking? Yeah, yeah, those
Speaker 5: aren't there when you download an open source model, when
Speaker 5: you run in a little lama, those extra layers. The
Speaker 5: only thing that is protecting the model is at that
Speaker 5: point what the model was trained, the pre training they
Speaker 5: did that they baked into the model. Then they're not
Speaker 5: doing any additional checks. So with the hosted model, there
Speaker 5: is a there is additional layers because they ConTroll the
Speaker 5: infrastructure and they're watching what the model says and they're
Speaker 5: they're stopping it. But typically when you will use you know,
Speaker 5: LM Studio or Olama, then it's g you're just getting
Speaker 5: the bare uncensored model and there's no additional checks. The
Speaker 5: only thing that's preventing the model from you know, saying
Speaker 5: the wrong thing being not helpful or not harmless or
Speaker 5: I guess harmful and unhelpful is just the training and
Speaker 5: the models, you know, training that it was put through.
Speaker 5: There's no additional checks there. So and when you download
Speaker 5: the model, maybe it's worth saying you're just basically downloading
Speaker 5: a large list of numbers and the code inside it
Speaker 5: tells it. And you're getting a large list of numbers
Speaker 5: and a mathematical graph that says how to combine the
Speaker 5: numbers together. It says, take this parameter first, here's how
Speaker 5: you map the words to numbers. That's you get that mapping.
Speaker 5: And then once you've mapped them to numbers, it says,
Speaker 5: add it here, multiply times this other number here. Then
Speaker 5: you know normally you know, take the square root of
Speaker 5: this other number and then multiply it again. And it's
Speaker 5: just a list of calculations. It's a really like simple program.
Speaker 5: In fact, most of the knowledge it's worth stating is
Speaker 5: not in the code. And this gets back to your
Speaker 5: question about like open source, it's in the knowledge, is
Speaker 5: inside the data, it's inside the parameters. So as an example,
Speaker 5: GPT two, which you know is considered one point too
Speaker 5: dangerous release and it's amazing.
Speaker 4: Yeah, only that's because they want regulatory capture, not because
Speaker 4: they actually believe it's dangerous.
Speaker 5: Well at the time maybe they they they're concerned about disinformation,
Speaker 5: but suffice to say it was still a powerful model
Speaker 5: in its day. Is my point is only five hundred
Speaker 5: lines of code. If you take out the the the
Speaker 5: TensorFlow library, it's five hundred lines of code. It is.
Speaker 5: It is astonishingly small. And so one of the things
Speaker 5: and the reason why I re implemented the entire thing
Speaker 5: in JavaScript is I want to push back against this
Speaker 5: idea that well, this stuff is too hard for you
Speaker 5: to learn. If you're a web developer, like you can
Speaker 5: learn five hundred lines of code. And that's basically like
Speaker 5: I give you the grounding and I re implement the
Speaker 5: entire thing in javascripts. You can step through it. You
Speaker 5: don't even have to leave your web browser, right, you
Speaker 5: just use the web debugger and you can you can
Speaker 5: step through what's happening, and it's it's astonishingly small. All
Speaker 5: the knowledge, all the rules is captured in the weights
Speaker 5: and the parameters the model. So when you download the model,
Speaker 5: it's just a more and more numbers with a larger
Speaker 5: and larger computational graph. And that's how we get it smarter.
Speaker 5: That's gets back to the heart of like the core
Speaker 5: thing to understand is we took a word problem and
Speaker 5: we mapped it to a number problem. So if we
Speaker 5: get a bigger calculator, we get a better result.
Speaker 1: But I want to I want to restate this just
Speaker 1: in another way, really simply because I think a lot
Speaker 1: of people get you know, they get confused between like
Speaker 1: GPT four versus chat, GPT versus something on your computer
Speaker 1: versus whatever. And so yeah, essentially the model, like you said,
Speaker 1: you know, it's it's maybe a few steps in how
Speaker 1: it gives you answers, and the rest of it, like
Speaker 1: you said, is all the data.
Speaker 2: It's all the waiting.
Speaker 1: But sometimes when people are talking about AI models, they're
Speaker 1: talking about a program that accesses the model that I
Speaker 1: just explained or that you just explained, right with the
Speaker 1: numbers and kind of the fundamental pieces and so that's
Speaker 1: your chat GBT, whether it's running on your local machine
Speaker 1: or in the cloud. Yeah, you need to be able
Speaker 1: to differentiate between the two and recognize that. Yeah, sometimes
Speaker 1: you're just downloading that map of numbers and you know
Speaker 1: some really really simple stuff that makes sense of the numbers,
Speaker 1: and that's your model. And so when people are building
Speaker 1: against those models, a lot of times that's what they're doing.
Speaker 1: And so you can write your own code that then,
Speaker 1: you know, is the gatekeeper or you know, says this
Speaker 1: is helpful or this is harmful, or this is whatever. Right,
Speaker 1: this is an appropriate response and this isn't a lot
Speaker 1: of that's just the code that sits on top of
Speaker 1: what you're talking about. That five hundred lines of code
Speaker 1: plus the data that we're getting.
Speaker 2: That is just the model. And so.
Speaker 1: You know AJ's talking about the GWEN model. It looks
Speaker 1: like it runs on Olama, Right, So you know you've
Speaker 1: got all those magic numbers, You've got the stuff that
Speaker 1: runs on top of it. I think Olama gives you
Speaker 1: a little bit more on top of that, and then
Speaker 1: from there, right, the rest of it's kind of up
Speaker 1: to whoever wrote the code.
Speaker 5: Yeah, the central thing, I'm just trying to get like
Speaker 5: the black box part, the most mysterious part at the
Speaker 5: heart of it. Yeah, and obviously, like you know the
Speaker 5: calculations that say Lama and Chat, GPT and Gemini, they're
Speaker 5: larger models. GPT two came out in twenty nineteen, but
Speaker 5: the core thing is like a mo Like if you
Speaker 5: had shown me GPT two, I wouldn't have known. Like
Speaker 5: when it first came out, it's like, wow, that's a
Speaker 5: pretty amazing program. Must be really complex. And it's not
Speaker 5: the program that's complex. It's they just bet on taking
Speaker 5: a somewhat simple architecture and just giving it lots and
Speaker 5: lots of data and spending more than anybody else had
Speaker 5: at the time, and just trust that the black box
Speaker 5: would be smart enough to learn everything from it. And
Speaker 5: at the heart of it, that's what's happening. You're just
Speaker 5: a large it's a giant calculator. In fact, it's so
Speaker 5: simple in a sense, like in a spreadsheet, which was
Speaker 5: my first implementation, you cannot do loops very easily. You
Speaker 5: don't have looping constructs. It just does a calculation through
Speaker 5: the entire way, and it just does the computation of
Speaker 5: the all the different cells they're effectively in a sense,
Speaker 5: no loops inside of it. Like the reason I can
Speaker 5: implement it in a spreadsheet is that every single time
Speaker 5: it predicts a token, it does the exact same number
Speaker 5: of computations every single time, and it goes through, you know,
Speaker 5: twelve layers and twelve attention steps and twelve layers. Like
Speaker 5: it's very very much like I got a number coming in,
Speaker 5: we a word coming in, we map that word to
Speaker 5: a number, and then I just do all this number
Speaker 5: crunching with a very predictable pattern, and then I get
Speaker 5: a number out and I interpret it, and then I
Speaker 5: just repeat that process over and over again. And so
Speaker 5: you know, the thing that I try to tell people
Speaker 5: is just like when you look online and people like
Speaker 5: I want to get into AI and stuff, they're they're
Speaker 5: often presented with, okay, go learn you know all this
Speaker 5: linear algebra. You need to make sure you're solid on
Speaker 5: your calculus. You need to make like and it's there's
Speaker 5: like six to eighteen months of like prep before you
Speaker 5: get to understanding how a large language model works. And
Speaker 5: that's valid if you're going to be a machine learning researcher,
Speaker 5: and machine learning is a huge giant field beyond just
Speaker 5: chat GPT right, there's an omaly detection, there's clustering, there's
Speaker 5: a lot of algorithms in there. But my goal is
Speaker 5: to just help people understand how these amazing, arguably Nobel
Speaker 5: Prize winning programs work in as short a time as possible,
Speaker 5: and to the extent, like I don't even begin where
Speaker 5: normal machine learning class begins. A normal machine learning class
Speaker 5: starts with like regression and it slowly works me up
Speaker 5: and maybe you'll get to the LMS. And I'm like,
Speaker 5: this is a five hundred line program. Just start with
Speaker 5: here's how it starts. And anytime I run to something
Speaker 5: you don't understand, I'll give you the In my class,
Speaker 5: I give you the background to understand it, and then
Speaker 5: we move on to the next piece. And so it's
Speaker 5: really designed to be as efficient as possible. And I
Speaker 5: think when you tell people it's five hundred lines, they're like, oh, yeah, okay,
Speaker 5: I can understand how that works. And this gets to
Speaker 5: knowing your tools. I'll make an analogy if you don't
Speaker 5: necessarily need to know how AI model works to use it,
Speaker 5: but you don't necessarily need to have a good model
Speaker 5: for how is the difference between the CPU or disc
Speaker 5: memory versus bandwidth versus system memory. But if you're debugging,
Speaker 5: you know, a machine program, it helps to have that
Speaker 5: mental model. You'll run to an issue or maybe a
Speaker 5: more our tangible example to this audience is like knowing
Speaker 5: how react works on the inside. At some point, if
Speaker 5: you don't understand hydration, you're going to run into a wall, right,
Speaker 5: And the same thing is true, like you get these
Speaker 5: parameters from OLAMA, what are they doing? You know, you
Speaker 5: need to have a mental model for how it works.
Speaker 5: And I don't think that mental model is as hard
Speaker 5: as a lot of people make it out to be.
Speaker 1: Yeah, when I talk to people about doing AI, and
Speaker 1: I talked to a whole bunch of people like that
Speaker 1: are business people, and talk to a whole bunch of
Speaker 1: people that are programmers, and I have some of the
Speaker 1: same conversation basically down to, well, are you going to
Speaker 1: build your own model, right? Are you going to take
Speaker 1: your own data and cram it in and expect it
Speaker 1: to give you answers on the other end, or are
Speaker 1: you going to use something that already exists like the
Speaker 1: chat GPTs or some of the you know, the GPT
Speaker 1: force or the OLAMAS or whatever, right, and then build
Speaker 1: on top of it. Because once you're building on top
Speaker 1: of it and you're not worried about, Okay, how do
Speaker 1: I put this together, then it's essentially okay, Like you're saying,
Speaker 1: I understand how the machine works, and then I understand
Speaker 1: how to talk to it. Right, so I understand what
Speaker 1: the APIs are and the rest of it is, then okay,
Speaker 1: what do I want from this?
Speaker 2: And how do I validate that I got it?
Speaker 5: Yeah? Actually, that's a really important point. The number one
Speaker 5: skill may not be understanding every single detail of the calculation.
Speaker 5: The number one skill when dealing with an AI model
Speaker 5: is that last thing you talked about, how do I
Speaker 5: evaluate it? So the name that you hear in the
Speaker 5: AI community is evals, But as a you know web developer,
Speaker 5: you can think of these as tests. And the key
Speaker 5: difference between you know AI evals and tests is that
Speaker 5: you don't expect one hundred percent pass right. These are statistical,
Speaker 5: probabilistic machines. But the number one, like when you read
Speaker 5: about benchmarks, you know, AJA you talked about benchmarks, You
Speaker 5: basically need to build the benchmarks for your particular problem.
Speaker 5: The benchmark may say some model is better than another,
Speaker 5: but when you actually use it for your problem, you
Speaker 5: suddenly discover it's not good. And so the first thing
Speaker 5: you should do is come up with your own benchmark,
Speaker 5: your own evals for the problem, and then try a
Speaker 5: bunch of models and see which one works the best.
Speaker 5: And then you can start iterating whether that iterating is
Speaker 5: changing the prompt, whether it's changing the model or saying
Speaker 5: I'm going to go off and find tune my own model.
Speaker 5: But you won't be able to make a judgment until
Speaker 5: you're able to look across the distribution of your task,
Speaker 5: all the different ways your task happens, and whether it's
Speaker 5: successful or not, because these are you're dealing with highly
Speaker 5: variable machines. One of you know, the folks who was
Speaker 5: in the audience for one of my past talks, hit
Speaker 5: a really good analogy. He's like, imagine a database that
Speaker 5: was wrong five percent of the time. Like, as developers,
Speaker 5: we are not used to having levels of uncertainty like
Speaker 5: this within our systems, unless maybe you're using distributed systems
Speaker 5: where there's all sorts of race conditions and stuff like that.
Speaker 5: But we're used to sanitizing the user input and then
Speaker 5: once we get the user input. Everything is predictable after that.
Speaker 5: But here it's like suddenly we've got a database that
Speaker 5: sometimes it's wrong, and so that's where you need to
Speaker 5: put all sorts of checks and guardrails. And you're dealing
Speaker 5: with this really smart but sometimes fallible thing like a human,
Speaker 5: I hate to say, anthropomorphizing it, and see how you
Speaker 5: build a system around that is going to be different
Speaker 5: than how you build a regular system. But it all
Speaker 5: starts with that key idea that you just talked about,
Speaker 5: which is about being able to evaluate mathematically how well
Speaker 5: your your model or your system is doing, and the
Speaker 5: question about whether you should build your own model or not.
Speaker 5: The usual hierarchy of needs is first start with an
Speaker 5: off the shelf model. It could be open source, it
Speaker 5: could be one of the providers. It's actually probably easiest
Speaker 5: to start with a hosted model and just see if
Speaker 5: you can get it to work because there'll be state
Speaker 5: of the art and you don't have to worry about
Speaker 5: all the stuff around hosting and inference and seeing if
Speaker 5: it works. And then next thing to try is try
Speaker 5: tuning it. Sorry, try try tuning your prompts. So try
Speaker 5: prompt engineering your way. Give it some examples try some
Speaker 5: variety of prompt engineering, and then maybe consider fine tuning it.
Speaker 5: And again you can fine tune you know, most of
Speaker 5: the hosted models, you don't have to go to an
Speaker 5: open source model, but you could do that as well.
Speaker 5: And then the idea of building your own model is
Speaker 5: extremely hard. You know, the amount of dollars that go
Speaker 5: into building your own models from scratch is now, you know,
Speaker 5: over one hundred million. So the estimates for say Lama
Speaker 5: are were, I think over one hundred million to build
Speaker 5: that model, and so it's a lot of work and
Speaker 5: that's lovely best of the frontier labs.
Speaker 4: Yeah, is that GPU cost or where is that number
Speaker 4: coming from?
Speaker 5: That's a great question because these are all estimates. You know,
Speaker 5: we don't know for sure, but obviously some of it
Speaker 5: is the GPU cost, some of it is the infrastructure cost,
Speaker 5: some of it's the talent. The other key thing to
Speaker 5: keep in mind is when you're training a model, you
Speaker 5: don't always know how it's going to turn out. What
Speaker 5: they actually do is they do a large series of
Speaker 5: smaller runs to establish some type of pattern or scaling
Speaker 5: law to figure out how they're going to design the model,
Speaker 5: which architecture seem to work better, which parameters matter more.
Speaker 5: So there's something called a learning rate, for example, that
Speaker 5: they have to adjust, and they have a schedule for it,
Speaker 5: and they're trying to figure out against evals against the
Speaker 5: benchmarks we talked about, like which one seems to make
Speaker 5: the model smarter. And so there's a lot of trials
Speaker 5: and attempts. So it's not just necessarily one whole shot
Speaker 5: of training. It's a lot of experiments that they have
Speaker 5: to do. A lot of how the model is going
Speaker 5: to behave is surprisingly empirical, and so they're doing experiments
Speaker 5: and they're trying that again, so there's a variety of things,
Speaker 5: and empower is non trivial. Another thing that's important to
Speaker 5: understand is the level of scale of data that these
Speaker 5: frontier labs are dealing with. And there's a really good
Speaker 5: analogy from the anthropic guys actually, and one of the
Speaker 5: things I have to do is you have to randomize
Speaker 5: the day data so it doesn't learn arbitrary patterns and
Speaker 5: the order of the data you gave it. And so
Speaker 5: one of their research engineers gave this great example is like, okay,
Speaker 5: randomizing sounds like it should be easy, Like take a
Speaker 5: deck of cards. If I tell you to shuffle it,
Speaker 5: it's fairly easy. But imagine I gave you like seven
Speaker 5: warehouses full of decks of cards and you need to
Speaker 5: shuffle them by hand. It's not quite clear how what
Speaker 5: policy or process you're going to use to make sure
Speaker 5: you hit all of them and you've evenly shuffled them.
Speaker 5: And the size of the data that these guys are
Speaker 5: using with is it's almost like that to the CPU,
Speaker 5: it's like seven warehouses of data to it like for
Speaker 5: you compared to manually you know, shuffle your your deck.
Speaker 5: And so when you're dealing with data at this large
Speaker 5: infrastructure scale scale itself makes every little thing harder, and
Speaker 5: so that also adds some difficulty to this. So should
Speaker 5: I you know, walk through just like a little more
Speaker 5: detail of what's happening in that mathematical calculation or happy
Speaker 5: to answer additional questions?
Speaker 2: Yeah, that's what I was going to ask.
Speaker 1: Next is Yeah, because you've mentioned you've got different layers
Speaker 1: or different steps in the process you explain in the video.
Speaker 1: The video is a little longer, I guess than we
Speaker 1: have to go over at this point, but yeah, if
Speaker 1: you can give people kind of an overview of how
Speaker 1: the LM system actually works.
Speaker 4: Yeah, So while you're doing that, if you distinguish between
Speaker 4: the different types of training, like the RAG versus the
Speaker 4: fine tuning versus the.
Speaker 5: Base Yeah, okay, I don't think of RAG as training,
Speaker 5: but maybe we should step back and explain to the
Speaker 5: audience who isn't familiar with RAG what it is. So
Speaker 5: you can kind of think of RAG as like a
Speaker 5: sort of prompt engineering technique. So you want the model
Speaker 5: to answer questions about something that wasn't trained on. So
Speaker 5: imagine I'm you know, I'm a smart home electronics company,
Speaker 5: and all of the documentation about my product was behind
Speaker 5: you know, some firewall or behind a log in, and
Speaker 5: so I know that, let's call it chatchy. He was
Speaker 5: never trained on it. But I want to build a
Speaker 5: chatpot where customers come and say, hey, I can't configure
Speaker 5: this setting on it. How am I going to get
Speaker 5: a chatbot to do that without having to retrain it
Speaker 5: specifically on my data. So what you can do is
Speaker 5: when a request comes in and somebody's like, well, how
Speaker 5: do I change the color on my smart light bulb,
Speaker 5: It'll go and it will search through my data. I
Speaker 5: can take that request from my user on my chatbot,
Speaker 5: and they say, I take I see the words light bulb,
Speaker 5: I see change color, and I'll search all my documentation
Speaker 5: and I'm not going to search it just a plain
Speaker 5: tech search. I'll use it called a semantic search. So
Speaker 5: it'll find things that are similar to the word light,
Speaker 5: like the word bright, even though it's not anywhere close
Speaker 5: to the same character. So it'll find all the similar
Speaker 5: passages and it will pull those out, and then it
Speaker 5: will give the model, here are relevant passages. Here's the
Speaker 5: user's question, how do I change the color on my
Speaker 5: smart light bulb? And then it will give it paragraphs
Speaker 5: chunks of text from my documentation, and it will put
Speaker 5: those at the beginning of the prompt. So you've got
Speaker 5: a prompt that structure at the start with the user's question.
Speaker 5: Then it's got some chunks of data that came right
Speaker 5: for my documentation. And then we tell the model, you know,
Speaker 5: come up with an answer to the user's question using
Speaker 5: these chunks of data I gave you, and it will
Speaker 5: be able to think over those passages and find the
Speaker 5: ones that are relevant and then give the answer out.
Speaker 5: And that's called retrieval augmented generation. So retrieval because you're
Speaker 5: taking the user's query, you're pulling a data that wasn't
Speaker 5: the model didn't have during training, and you're passing it
Speaker 5: into the prompt and then asking the model to answer it.
Speaker 5: And so it's a very low friction way to take
Speaker 5: a model off the shelf and make it understand all
Speaker 5: your stuff even though it wasn't in the training data.
Speaker 5: So that's that's uh, that's that's what RAG is.
Speaker 1: The version is is you're building context out of a
Speaker 1: database that you already.
Speaker 5: Have, great, great summary, thank you. Uh so, yeah, you're
Speaker 5: giving it the context it didn't have during training to
Speaker 5: answer the question on training. So there's a variety of
Speaker 5: steps in the model where it's trained. Mainly, well, let
Speaker 5: me think of the best way to explain this. So
Speaker 5: I'll discuss training when I get to the call it
Speaker 5: the fourth step of the model, and I'll talk about
Speaker 5: how they gets trained in a second. But let me
Speaker 5: walk through the five steps of what happens when you
Speaker 5: input text into the model. So the first thing you
Speaker 5: do when you input a passage so that one I
Speaker 5: like to uses Mike as quick he moves. And then
Speaker 5: the completion we leave it to the model filling the
Speaker 5: blank quickly. The first thing it's gonna do is going
Speaker 5: to break the text into subword units. So you might
Speaker 5: think it would break it into characters, and you might
Speaker 5: think it might break it into words. So break into
Speaker 5: characters would be like ASKI, and breaking into words would
Speaker 5: be just giving every word it's entry in a dictionary.
Speaker 5: It turns out you can't handle if you break it
Speaker 5: into words. You can't handle unknown words very well, and
Speaker 5: you can't handle spellings you weren't planning on. So especially
Speaker 5: if you're going across multiple languages, and if you break
Speaker 5: it into characters, it turns out it's really hard and
Speaker 5: a lot of compute for the model to learn purely
Speaker 5: from characters, although some research has been able to do it.
Speaker 5: So they do is they can do a Goldilocks and
Speaker 5: they say, okay, let's break it into these little pieces
Speaker 5: of words, and if you think about it as a human,
Speaker 5: you actually do this. So one of the examples, I
Speaker 5: use the word flavor eye. It's actually not a word
Speaker 5: in the dictionary, but you know what it means because
Speaker 5: you know what flavor means, and you know what ize
Speaker 5: is a suffix means, and so the model kind of
Speaker 5: does that. Now. I want to be clear, the tokens
Speaker 5: that comes up with, as they're called these subword pieces.
Speaker 5: Word pieces don't map to any human sense of the meaning.
Speaker 5: There are some that, like ice turns out to be
Speaker 5: a token, but it's by coincidence or correlation, not like
Speaker 5: it's trying to understand human English at this stage. So
Speaker 5: it breaks it into these Yeah.
Speaker 1: Gosh, can I just say that in a different way
Speaker 1: too effectively? What it does is it breaks it up
Speaker 1: into pieces that have meaning, right, because when we're looking
Speaker 1: for the output, we're looking for out it has meaning,
Speaker 1: and we group words or ideas together that give it meaning.
Speaker 1: And so it's doing the same thing. It's breaking it up, right,
Speaker 1: Like you said, flavor has a meaning, eyes has a meaning.
Speaker 1: You know, the other words in there have meaning. And
Speaker 1: so that that's the approach that it kind of takes
Speaker 1: when it's breaking it into tokens of kind.
Speaker 5: Of sort of kind of. It's a decent mental model.
Speaker 5: But the reason I stress that it's not trying to
Speaker 5: match human meaning is because it's actually not trying at
Speaker 5: this stage of the model. It's not trying to assign meaning.
Speaker 5: In fact, what it's really trying to do is take
Speaker 5: all the text on the Internet that it's trying to
Speaker 5: train on and compress it to the most efficient representation
Speaker 5: so that the training can be as efficient as possible.
Speaker 5: And that's why the tokens don't always map to what
Speaker 5: you'd expect and why So this is this.
Speaker 4: Is different than what a full text search database would
Speaker 4: do because a full text search database, like the example
Speaker 4: you gave flavor, Yeah, full text search database is going
Speaker 4: to break it up that same way. But this is
Speaker 4: different than the way a full text search database would
Speaker 4: break it up.
Speaker 5: Yes, it is different, and it's very dependent on the
Speaker 5: data it was trained on. And so a good example
Speaker 5: is I use the word re injury. Right as a human,
Speaker 5: you would think it was it was re an injury,
Speaker 5: but if you actually put it through the GPT two tokenizer,
Speaker 5: it puts it as rain injury. And the reason it
Speaker 5: decided to do that is simply because of the greater
Speaker 5: you know, occurrence of the word jury on itself by
Speaker 5: itself than injury, and so that decided that was the
Speaker 5: more efficient representation. And I want to be clear, this
Speaker 5: isn't about representing your prompt efficiently. This is about representing
Speaker 5: all the training it's going to do on the text efficiently,
Speaker 5: the stuff you don't see, the stuff that you know
Speaker 5: you're talking about nobody releases. That's what it's really based on.
Speaker 5: And it's really a compression of all the text. So
Speaker 5: it just got a really efficient So think about it this.
Speaker 5: If it's going to compress all the text, then you know,
Speaker 5: if it gets down to say ten thousand or fifty
Speaker 5: thousand tokens, then it only has to learn ten thousand
Speaker 5: or fifty thousand concepts in a sense, Although that's a
Speaker 5: gross oversimplification, but that's what it's trying to do, is
Speaker 5: trying to reduce the number of things it needs to
Speaker 5: learn on essentially the number of combinations and variations.
Speaker 3: Hey, a couple of quick questions with that flavorized example.
Speaker 3: Here's hoping they don't pick up flavor flav the rapper's
Speaker 3: lyrics right and throw that in there. That could get
Speaker 3: really confusing. But then when you're talking about like the
Speaker 3: re injury for example, Yeah, and how I as soon
Speaker 3: as you saw that said that, I was thinking, Okay,
Speaker 3: I can see where that's going. How you get rained
Speaker 3: just throwing stuff like and this might be getting into
Speaker 3: the weeds too much. But just throwing stuff like hyphens
Speaker 3: into words make a difference. So if you were to
Speaker 3: do redash injury, would it see that and maybe just
Speaker 3: categorize the re as separate from injury? Does that help
Speaker 3: or is that a non issue? Does it sort of
Speaker 3: filter that stuff out and just focus on the letters?
Speaker 5: And that's a great question. It's actually implementation dependent on
Speaker 5: the tokenizer. In practice, you usually separate. You create boundaries,
Speaker 5: hard boundaries between tokens or words, so one of them
Speaker 5: is the space character. In most of these, the hyphen
Speaker 5: is considered also a boundary, and so it would see
Speaker 5: it separately. The important thing to understand, though about the
Speaker 5: tokenizer is that the model doesn't see words the same
Speaker 5: way you do. So a great example of this is
Speaker 5: how many letters are in the word strawberry. It does
Speaker 5: not see the word as S T R A, W
Speaker 5: B E R R y. In fact, when you read it,
Speaker 5: you really don't either. When you read words, you typically
Speaker 5: aren't paying attention to every single character. When you have
Speaker 5: to count the words in strawberry, you kind of have
Speaker 5: to change your mental state and think, oh wait, let
Speaker 5: me think what are the characters? And you walk through it.
Speaker 5: It just sees it as it might see it as
Speaker 5: the token strawberry. It might see it as the token
Speaker 5: straw and berry. But the key thing is it has
Speaker 5: no idea. It doesn't the ability to see the letters.
Speaker 5: In fact, if you capitalize, the tokenization is case sensitive,
Speaker 5: so if you change the capitalization, it looks like a
Speaker 5: different it. So to it, the word strawberry with like
Speaker 5: a space in front of it is different than the
Speaker 5: strawberry without a space. Strawberry with the capital in front
Speaker 5: of it is different than strawberry with a capital. You know,
Speaker 5: if you put quotes around it, it's a different word.
Speaker 5: So it doesn't see text the same way you do.
Speaker 5: Another great example of this is numbers. So they've fixed
Speaker 5: this in most modern tokenizers, but the early ones would
Speaker 5: just take examples of numbers and those would be a
Speaker 5: whole take token. So two fifty six right power of
Speaker 5: two fairly common gets a token. But it sees that
Speaker 5: as a single token. It sees that as a single thing.
Speaker 5: It doesn't even see it as the numbers two, five
Speaker 5: and six. It doesn't break it apart. And so that's
Speaker 5: why it was really hard for these guys or these guys,
Speaker 5: these things to it was part of the contributing reason
Speaker 5: why it was hard to do math, it's not the
Speaker 5: sole reason. So the key lesson, you know, on the
Speaker 5: tokenizer before we leave it, and the algorithm that's commonly
Speaker 5: used something called byte pair and coding, and in my
Speaker 5: classes is something we walk through. In fact, we do
Speaker 5: it by hand so you can understand, and we talk
Speaker 5: through the training process. But the key thing to understand
Speaker 5: is that the model doesn't always see text the same
Speaker 5: way you do. So that's the first step that's tokenizer.
Speaker 5: Then the next thing is we we map each of
Speaker 5: these tokens, but you can think of them as words
Speaker 5: into a list of numbers. So I talked to earlier
Speaker 5: like we map each word to a number, but it's
Speaker 5: really we map it to a large list of numbers.
Speaker 5: And this is called an embedding. And the way to
Speaker 5: think about this is where we're taking all the words
Speaker 5: or in this case, tokens technically, and we're putting them
Speaker 5: on a map. But instead of like a two dimensional map,
Speaker 5: this is many, many dimensions. So in the case of
Speaker 5: GPT two, it's seven hundred and sixty eight I think
Speaker 5: LAMA four or five B it's like sixteen thousand list
Speaker 5: of numbers per every single word. In fact, like in
Speaker 5: the sentence phrase Mike is quick, period he moves, the
Speaker 5: period itself gets seven hundred and sixty eight numbers to
Speaker 5: represent it. And you can think about like on a map,
Speaker 5: you have like you know, coordinates. This is just a
Speaker 5: very very multidimensional list of coordinates. And a good embedding
Speaker 5: puts words that are related to each other closer to
Speaker 5: each other. So in my class, they use the words
Speaker 5: like happy, sad, joyful, glad, dog, cat, rabbit. The first
Speaker 5: set of those are emotions happy, sad, joyful, right, and
Speaker 5: would expect happy and joyful to be close to each other,
Speaker 5: same with glad, and then dogcat and rabbit are totally unrelated,
Speaker 5: so would expect them to be further apart on the map.
Speaker 5: And the word sad is an emotion, but it's not
Speaker 5: quite the same emotion as being happy, so it'd be
Speaker 5: somewhere in between. And if you actually visualize this, you
Speaker 5: see that this happens. It's actually putting words closer together.
Speaker 5: And you might hear this paper or it's a series
Speaker 5: of papers or algorithms called word to vec which pioneered
Speaker 5: this model, And if you go to projector dot TensorFlow
Speaker 5: dot org, you can actually see a three D map
Speaker 5: of various words, and you click on it and it
Speaker 5: will show you the words that are close to it,
Speaker 5: and they all tend to be related words. So the
Speaker 5: first thing we start with is, you know, the next
Speaker 5: step after we break the text into tokens, as we
Speaker 5: map each of those tokens to a position on a
Speaker 5: map where close words are related to each other. Let
Speaker 5: me pause and see if that made any sense. I'm
Speaker 5: usually doing this all visually, so over pair audio. It's
Speaker 5: it's a bit of a challenge, but yeah, good.
Speaker 1: So my question is is, since it's predicting the next word,
Speaker 1: I would imagine that, yeah, some of the words that
Speaker 1: appear close to it are going to be words that
Speaker 1: mean kind of the same thing or you know, have
Speaker 1: a related meaning. But does it also group words together
Speaker 1: that commonly appear together or is that a different Does
Speaker 1: it not wait things that way at all?
Speaker 5: Uh, it's actually kind of doing both. The way it's
Speaker 5: grouping related words together, it doesn't actually group words together.
Speaker 5: It's grouping words together that have the same meaning based
Speaker 5: on the idea that they appear in the same places. Oh,
Speaker 5: so the word ice and cold commonly occur together, probably
Speaker 5: in text on the internet, right, like the I put
Speaker 5: ice in the drink to make it cold, you're as
Speaker 5: cold as ice? Right, Those would be common phrases you
Speaker 5: usually don't see, you know, like steam and cold together.
Speaker 5: And so the model is able to understand that ice
Speaker 5: is colder than steam because it sees cold closer to
Speaker 5: ice more often than seems cold close to steam. It's
Speaker 5: the relative occurrence of how often. And there's a phrase
Speaker 5: that's often used by JR. Firth. It's called you will
Speaker 5: know a word by the company it keeps, which is
Speaker 5: the idea you don't really know what a word means.
Speaker 5: You could look it up in the dictionary, but you
Speaker 5: really understand it through how it's used by multiple people,
Speaker 5: and that you can look at, you know, the distribution
Speaker 5: of how it's used to really understand what it means.
Speaker 5: So good example is the word bad right, although it's
Speaker 5: less in fashioned, bad at one time meant good, right,
Speaker 5: And so how do you really understand whether it means
Speaker 5: good in one context versus another? You learn that through
Speaker 5: all the various contexts that it is used. And if
Speaker 5: you want to understand, you know how word really is used.
Speaker 5: You see it in usage many, many, men times. So
Speaker 5: if you want a model to understand how what a
Speaker 5: word means, you just see it used in many, many, many,
Speaker 5: many sentences and eventually pick up on those differences.
Speaker 3: So then the word baby could be seen as cold
Speaker 3: because of vanilla ice, then right, ice ice baby.
Speaker 5: If you train it, I'd be really interested to see
Speaker 5: a model trained only on song lyrics. That would be that.
Speaker 2: Would be fascinating.
Speaker 1: Yeah, yeah, it's interesting because the way you're talking about it.
Speaker 1: We were driving home from my mom's house the other
Speaker 1: night and my wife put on an audiobook that she's
Speaker 1: been listening to with my nine year old and they
Speaker 1: used the word satisfaction and my daughter asked, what does
Speaker 1: satisfaction mean? And we basically did that it's kind of
Speaker 1: like this, and it's kind of like that, right, It's
Speaker 1: it's in this area of meaning, right, yeah, and it's
Speaker 1: related to these other words.
Speaker 2: Right.
Speaker 1: We used other words to explain it, and then yeah,
Speaker 1: we did we context, so you could use it like this,
Speaker 1: or you could use it like this, and it's you know,
Speaker 1: another form of the word is satisfy or satisfied, and
Speaker 1: so you know, this is what it means to satisfy
Speaker 1: something and you know, more context and more sentences and okay,
Speaker 1: I understand, right, And I think she may have even
Speaker 1: said it's so it's kind of like this and kind
Speaker 1: of like that, yep, using examples that we didn't use, and.
Speaker 3: You told her that Nick couldn't get any right, that's right.
Speaker 5: But yeah, that's that's basically what the model is going through.
Speaker 5: Is it's like, uh, you know, basically saying seeing all
Speaker 5: these examples and it's like, oh, it's kind of like this,
Speaker 5: but it's like like in some context, I see it
Speaker 5: being used in this other way, and so it's it's
Speaker 5: basically putting that all together. And then it's putting all
Speaker 5: the words on this map and it's saying, okay, you know,
Speaker 5: the ones that are related are here, and the ones
Speaker 5: and it's this multidimensional map. It's you know, hundreds to
Speaker 5: thousands of dimensions long. And that's that's the embedding step.
Speaker 5: Is we've basically mapped them to you know, I say
Speaker 5: a numb but it's really a point in space, right,
Speaker 5: It's basically taken. So there we're at the second step.
Speaker 5: We base first step was we took the passage and
Speaker 5: we broke it into tokens, which you can think of
Speaker 5: as like words, but smaller, and then we took each
Speaker 5: of those tokens and we put them into a point
Speaker 5: on a map and a point in space, and we
Speaker 5: just we know that point is going to be close
Speaker 5: to other things that are related to it. So that's
Speaker 5: the second step, and then the third step is called attention.
Speaker 5: I'm going to skip over that in a second, and
Speaker 5: I'm going to go to the fourth step, which is
Speaker 5: the neural network or the multi layer perceptron. And this
Speaker 5: gets to the training question. The key thing that it's
Speaker 5: really great about neural networks is that thing I talked
Speaker 5: about earlier, which is you don't have to give them
Speaker 5: the rules. You just give them the answers, and they
Speaker 5: figured out the rules. So we basically feed it in
Speaker 5: these points in space from our prompt, and then we
Speaker 5: can take a pasth message on the internet, like maybe
Speaker 5: the passage on the internet is Mike is quick, he
Speaker 5: moves quickly. We remove the word quickly, and then we
Speaker 5: give the model the phrase Mike is quick he moves
Speaker 5: and then we ask it to make a prediction, and
Speaker 5: it's going to get it wrong because it hasn't done
Speaker 5: any training at all, and when it gets it wrong,
Speaker 5: maybe it says, you know, Mike's quick he moves bicycle
Speaker 5: and you're like, no, that's wrong. The right answer is quickly.
Speaker 5: It mathematically learns how to change itself to get closer
Speaker 5: to that answer. So we go through a lot of
Speaker 5: iterations of getting lots of passages where we take off
Speaker 5: the last word and then we give it. We ask
Speaker 5: it to predict it, and if it's good, we say okay, great,
Speaker 5: you're fine, and if it's wrong, we say, okay, you're
Speaker 5: off by this amount. It's kind of like when you
Speaker 5: throw darts out of board. Right, if you're far from
Speaker 5: the bullseye, you'll move a lot more closer to correct
Speaker 5: your position, But if you're close but slightly off, you're
Speaker 5: going to move slightly subtly. So that's what it does.
Speaker 5: It changes the model parameters, the numbers inside the model
Speaker 5: slightly if it's close, or a lot if it's far away.
Speaker 5: Does us you know, trillions of times over lots of
Speaker 5: different pieces of data. And the key thing about the
Speaker 5: neural network is it can learn to imitate basically from
Speaker 5: answers and data, and so we basically give it the
Speaker 5: known passage that we gave like Mike's queaky moves, and
Speaker 5: we knew quickly was the right answer. And then we
Speaker 5: basically asked the neural network to make a prediction from
Speaker 5: these points in space. And so that's the that's the
Speaker 5: basic version of what's happening inside the train. Let me
Speaker 5: pause because I jump to the fourth Layer'll come back
Speaker 5: to the third one in a second. But does that
Speaker 5: let me see if there are any questions there on
Speaker 5: what's happening inside the neural network. Okay, so yeah, okay,
Speaker 5: So if we're good there, then the next thing that
Speaker 5: happens is I'll jump back to the third Like we
Speaker 5: could give it a point in space and say, hey,
Speaker 5: guess what the next word is. But the best thing
Speaker 5: to do is to not just give it a single
Speaker 5: point in space. It's to give it all the points
Speaker 5: that can before, so all the words that came before.
Speaker 5: So in the case Mike is quick, he moves. Knowing
Speaker 5: that you know we're talking about movement helps it know
Speaker 5: that the word quick here is moving around in a
Speaker 5: physical space versus the quick of your fingernail. And so
Speaker 5: we give it the hints of all the words that
Speaker 5: came before it. So this is what's called attention, where
Speaker 5: we say, okay, don't just predict from one single word
Speaker 5: you're looking at. This gets back to what we talked
Speaker 5: about at the beginning, like instead of looking at, you know, statistically,
Speaker 5: what's the next word after the given word, let me
Speaker 5: look two words back, let me look three words back,
Speaker 5: let me look forwards back. It will look at all
Speaker 5: the words that came before it and try to figure
Speaker 5: out what is the next predicted We're giving these hints
Speaker 5: from the entire passage to make its prediction, and that's
Speaker 5: what's called attention. And so that's the third step in
Speaker 5: the middle. And then the last step is we do this,
Speaker 5: you know, we get a prediction out of the neural network.
Speaker 5: So jumping back to the fourth step, which was the
Speaker 5: neural network that makes the prediction, and it gives a number,
Speaker 5: and that number we need is a long list of numbers,
Speaker 5: and we need to map that back to one of
Speaker 5: our tokens, one of our words. But it's sitting in
Speaker 5: a point in space that it may land right on
Speaker 5: the word quickly, but more than likely it's going to
Speaker 5: land somewhere close to the word quickly, like the predicted
Speaker 5: token that comes up with it, that the model comes
Speaker 5: up with, and so it will interpret that point in
Speaker 5: space that it gave us back to those embeddings in
Speaker 5: that map. It picked us some at the end of
Speaker 5: the number crunching, took all the words and the points
Speaker 5: in space we gave it and said the predicted word
Speaker 5: is right here in this point in space, and it says, okay.
Speaker 5: We then interpret that, and we look, what are the
Speaker 5: words or tokens that are close to that predicted point
Speaker 5: in space. And it's probably going to be closer to
Speaker 5: the word fast, closer to the word around. Like Mike
Speaker 5: moves quickly, he moves around, he moves fast, he moves speedily.
Speaker 5: Those words are going to be close to it, and
Speaker 5: so we give them a higher probability and we run
Speaker 5: a random number generator and we say, okay, let me
Speaker 5: pick one according to this probability distribution, and that's how
Speaker 5: we get the predicted word out. And that last step
Speaker 5: of running that random number generator and looking at what
Speaker 5: words are closer it is the piece that's called the
Speaker 5: language head. So the key thing about the language head
Speaker 5: is that is where most of the uncertainty or unpredictability
Speaker 5: of your model comes from. So if we decide not
Speaker 5: to run the random number generator and we just always
Speaker 5: pick the word that is closest in space to the prediction,
Speaker 5: that's what's called temperature zero, and it will always be consistent.
Speaker 5: It will always be predictable for the most part. There's
Speaker 5: some other very small orders of randomness in the process,
Speaker 5: but for the most part, it'll be very consistent, and
Speaker 5: that's that's called temperature zero. So most of the randomness
Speaker 5: inside the model is entirely in some sense imposed by us.
Speaker 5: We decided, oh, we're not just going to always take
Speaker 5: the thing that's closest. We're gonna probabilistically take some of
Speaker 5: the other ones that are also close, and we can
Speaker 5: control those parameters and control how we do that probability.
Speaker 5: So if you're in o LLAMA or you know an API,
Speaker 5: you'll see things like top P and top K or
Speaker 5: temperature and these are tools we are given, you know,
Speaker 5: the API user of a model on how they can
Speaker 5: shape the probability distribution of the model. And that's probably
Speaker 5: the most important to understand of the components in the model.
Speaker 5: After you understand what tokens are and embeddings. The next
Speaker 5: one is probably the language head because that's where the
Speaker 5: randomness comes from. So let me pause. I know I
Speaker 5: just talked for quite a while. See if there are
Speaker 5: any questions.
Speaker 2: I think, so far, so good.
Speaker 5: Okay, So what the Excel spreadsheet does or the website
Speaker 5: I have that's built in you know, web components in
Speaker 5: pure JavaScript is it runs through the entire process using
Speaker 5: the very same weights that open Ay released for a
Speaker 5: model called GPT two GPT two small, and it steps
Speaker 5: through every single one of those processes and it takes
Speaker 5: you You enter a prompt and then what it does.
Speaker 5: It doesn't it's not like chat GPT where you can
Speaker 5: have a conversation with it. It just predicts the next word,
Speaker 5: but it walks you through every single step. That's the
Speaker 5: same thing I do inside the class. But that's basically
Speaker 5: you know, how your model works under the hood is
Speaker 5: it's basically taking your words, your input prompt, breaking it
Speaker 5: into units that are called tokens that are slightly smaller
Speaker 5: than a word, that it maps it to points in space,
Speaker 5: does a bunch of number crunching on it through the
Speaker 5: things I talked about, using a neural network and this
Speaker 5: other attention that looks at all the other words, and
Speaker 5: then it takes that prediction and it says, Okay, what
Speaker 5: words is it close to in our points in space,
Speaker 5: and then let me pick one that's relatively close to that.
Speaker 5: So I know one of the things, Chuck, you would
Speaker 5: want to talk about is building it and the use
Speaker 5: of web components in the web version.
Speaker 1: Yeah, at this point, given our time constraints, Yeah, we
Speaker 1: might have you come back and do that, because I
Speaker 1: think it'd be interesting to dive into the project and
Speaker 1: how it went together.
Speaker 5: Okay, I will. I will just say the reason I
Speaker 5: built it in web components was to make it as
Speaker 5: portable and easy to use and as easy to step through.
Speaker 5: I wanted to make it as accessible. I did think about,
Speaker 5: like say, using React, but then you need to know React,
Speaker 5: and I really wanted this to be as approachable for
Speaker 5: somebody who knows just vanilla JavaScript and web components was
Speaker 5: the easiest.
Speaker 2: Way to do that.
Speaker 5: So that's the main reason why I did it that way. Yeah,
Speaker 5: So is it open source then or I actually haven't
Speaker 5: put a license on it. And what I've said is
Speaker 5: if people feel like I help me decide, like, tell
Speaker 5: me which licenses you prefer, but the code is right
Speaker 5: there for people to look at. I mean you can
Speaker 5: practically step through it, and it's written so that people
Speaker 5: can understand it. I just haven't figured out what license,
Speaker 5: but you know, let me know, and I'm all ears
Speaker 5: all right. The goal is to make its a teaching tool.
Speaker 2: Yeah, all right? Cool?
Speaker 1: Well, yeah, like I said, we're kind of at the
Speaker 1: end of our time and so I'm I'm going to
Speaker 1: push this into picks. But you want to just give
Speaker 1: out information on your course again, let people know what
Speaker 1: that coupon code was.
Speaker 2: I just if people are digging this as much as I.
Speaker 1: Am, I think they may want to just go pick
Speaker 1: up the course and go, okay, we can go into
Speaker 1: more depth.
Speaker 5: Uh yeah, thank you. So the website for the project
Speaker 5: is called Spreadsheets Are All You Need with hyphens in between.
Speaker 5: So spreadsheets hyphen are hyphen All hyphen you hyphenneed dot ai.
Speaker 5: It is a very long domain name, and that will
Speaker 5: link to where you can download the excel file. You
Speaker 5: can try this out in the browser yourself, and then
Speaker 5: there's a link to my class that I teach on
Speaker 5: the Maven platform. It basically has five lectures over two
Speaker 5: weeks and we walk through this for anybody who understands
Speaker 5: spreadsheets or vanilla JavaScript. And I have a promo code
Speaker 5: js jabber, so just use the promo code during checkout
Speaker 5: and you get twenty percent off the courses taught live
Speaker 5: but also is available on demand. So my last cohort
Speaker 5: just wrapped up earlier this month. But if you sign up,
Speaker 5: you'll get to watch all the recordings. I'll answer all
Speaker 5: the questions over email you have. You'll be in the
Speaker 5: same private discord as the same as the rest of
Speaker 5: the cohort. And if for some reason you're watching it
Speaker 5: on demand and you'd say I'd rather have the live version,
Speaker 5: I offer that if you want to attend a future
Speaker 5: live version, you can do that for free, even if
Speaker 5: you signed up for the on demand, so you know,
Speaker 5: feel free to check it out. It's on maven. It's
Speaker 5: got some long URL, but if you go to spreadsheets
Speaker 5: at all, you need dot ai you can check it
Speaker 5: out and then to find me. I'm on Twitter I
Speaker 5: A N A N D so my first initial with
Speaker 5: my last name, and of course on LinkedIn. I'm also
Speaker 5: on blue Sky as well. If people want to reach me,
Speaker 5: happy to.
Speaker 1: Answer questions awesome. Well, yeah, I definitely want to dig
Speaker 1: into it. I'm probably gonna go watch your video on
Speaker 1: YouTube a few more times, just you know, getting all
Speaker 1: those little pieces in my head. So that yeah, I
Speaker 1: think you said at the beginning, the model that matters,
Speaker 1: matters most is your mental model. Yes, and so yeah,
Speaker 1: just knowing how to think about Okay, I'm dropping this
Speaker 1: in right. This is how it goes through the Plinko
Speaker 1: machine to give me the output on the other end.
Speaker 1: That that's the thing that really helps me out.
Speaker 5: So yeah, that's a great analogy. And one thing that
Speaker 5: maybe worth highlighting is I've had some feedback that people saw,
Speaker 5: Oh you're using spreadsheets or using JavaScript. The real models
Speaker 5: are in Python, and then you're using GPT two, which
Speaker 5: is an older model. What I teach in the class
Speaker 5: are essentially the timeless technical fundamentals of how these models work.
Speaker 5: And it's worth remembering that all the major models you've
Speaker 5: heard of, you know, Claude Chat, GPT, Gemini, they all
Speaker 5: are inheriting from GPT two. So if you understand GPT two,
Speaker 5: you understand eighty percent. You're eighty percent of the way
Speaker 5: to understanding any of the model or Lama model architectures.
Speaker 5: So it's it's not like toy. It is essentially as
Speaker 5: very close to how the real models work. It's a
Speaker 5: really good stepping stone to getting that really sharp mental
Speaker 5: model of what's happening under the hood.
Speaker 1: Yeah, that's true of most technologies, right, I mean, if
Speaker 1: you were using I don't know, let's pick one my
Speaker 1: SQL ten years ago, probably sixty seventy percent of the
Speaker 1: stuff is fundamentally the same.
Speaker 2: The engine works mostly the same.
Speaker 1: They've probably optimized some pieces, they've probably add some features,
Speaker 1: but for the most part, if you understand understood what
Speaker 1: it was doing back, then you get it now. And
Speaker 1: to be honest, the other thing is is you'll also
Speaker 1: see as we get more variations on things, you also
Speaker 1: have a decent understanding of how to use something like
Speaker 1: SQL light or post PRESSQL as well.
Speaker 5: So yeah, I really like that Algiae. It's a good one.
Speaker 5: I M a bar of that, thank you.
Speaker 2: Yeah, no problem. All right, Well let's go and do
Speaker 2: our picks. AJ Do you want to start us with picks?
Speaker 5: Sure?
Speaker 4: Okay, so Civilization. I've still been playing that, not as
Speaker 4: much as I was the other week, but enjoying it.
Speaker 4: It turns out you can run it on the Mac
Speaker 4: if you turn the you have to go into settings
Speaker 4: and turn its performance mode completely off to all the
Speaker 4: way down. I think it's something to do with multi threading,
Speaker 4: was why it crashes, Like if he uses more than
Speaker 4: one core, it just crashes every five minutes or something. Anyway,
Speaker 4: so there's that civilization still going strong. But wanted to correct.
Speaker 4: You can get it running on the Mac. It just
Speaker 4: won't run on the Mac with the default settings, and
Speaker 4: it's not abundantly clear why. But anyway, other thing was
Speaker 4: with the announcement of the switch to I just got
Speaker 4: angry because I still can't play Tiers of the Kingdom
Speaker 4: or Super Mario RPG or Spiro or you know, any
Speaker 4: game that's basically released in the last three years without
Speaker 4: massive stuttering. And you know with Tiers of the Kingdom,
Speaker 4: you know how they have it go into bullet time
Speaker 4: whenever it gets overloaded instead of getting choppy, although it
Speaker 4: still does that, it does dynamic resolution and bullet times,
Speaker 4: so your swings will slow down and stuff, which you know, whatever.
Speaker 4: So I decided to mod my switches and I did
Speaker 4: the hardware mod of the switch and it was super easy.
Speaker 4: Now I've done mods in the past, So saying that
Speaker 4: it was super easy, No, if you're not familiar with soldering,
Speaker 4: if you haven't you know, fixed a phone or or
Speaker 4: you know, done something with that before, no it's not.
Speaker 4: It's not super easy. Getting all the screws out, getting
Speaker 4: to the actual part that you're getting the heat sync off,
Speaker 4: that's super easy. Anybody that has a precision precision toolkit
Speaker 4: for like phone repair, game repair or whatever, can can
Speaker 4: get get in there and do that. I actually couldn't
Speaker 4: see the soldering that I was doing because the components
Speaker 4: are so small. Now I've learned some tricks because I've
Speaker 4: practiced a little bit of micro soldering in the past
Speaker 4: and failed at repairing a three DS, but the pieces
Speaker 4: are so small that I literally can't see that. I
Speaker 4: mean I can see them, but I can't see them.
Speaker 4: I mean like I can see them as like I
Speaker 4: can see a grain of sand, but I cannot see
Speaker 4: them in terms of like actually accurately predicted. So what
Speaker 4: I did was I used my phone to zoom in,
Speaker 4: take a picture of it, see that I had bridged
Speaker 4: two capacitors, and then just kind of blindly, you know,
Speaker 4: move the soldering iron next to them. And kind of
Speaker 4: sweeped it the same way that I would when i'm,
Speaker 4: you know, soldering on a bigger component. Then use the
Speaker 4: phone again zoomed in, and so I was able. I
Speaker 4: was able to get the peace on there. And I
Speaker 4: should have had some sort of magnifying glass set up,
Speaker 4: but whatever, so I was. I was actually able to
Speaker 4: do it blind in a way like I could. I
Speaker 4: could see my tip was there, but I couldn't actually
Speaker 4: see what, you know, what was because I mean the
Speaker 4: things they're they're smaller than a grain of salt. They're
Speaker 4: they're small anyway, the capacitors they're they're like wicked small.
Speaker 4: But even with that, I was able to do it.
Speaker 4: And that's my my So. So modding the switches is
Speaker 4: one is one pick there. There's the Pico Fly mod kit.
Speaker 4: You can do it if you've done if you've done
Speaker 4: other soldering, if you haven't done micro soldering, buy a
Speaker 4: couple of practice kits from eBay or or Ali Express
Speaker 4: or or something and you can get there. But my
Speaker 4: third pick is the soldering station that I used is
Speaker 4: actually a custom made soldering station. So a few years ago,
Speaker 4: these Chinese companies came out with the T twelve tips,
Speaker 4: or they cloned the T twelve tips. They turn out
Speaker 4: to be really, really good. One reason is that they
Speaker 4: double over as a temperature sensor because they have two
Speaker 4: different types of metals, and anytime you have two different
Speaker 4: types of metals, you have a thermal couple. So they
Speaker 4: have an inner metal and an outer metal, so they
Speaker 4: double over as their own temperature sensor. And so people
Speaker 4: created software for these and put them on micro controllers.
Speaker 4: In the software, and being literal when I say this,
Speaker 4: it rivals three thousand dollars professional workstations because the way
Speaker 4: that it switches back and forth between monitoring the temperature
Speaker 4: of the tip. And so I'll put a link and
Speaker 4: you can get these. You can get clones on all
Speaker 4: the express but I prefer the original one that's made
Speaker 4: by this guy in Australia because I know it comes
Speaker 4: with the right firmware on it, and the firmware is
Speaker 4: really where the magic happens. Any idiot, can you know?
Speaker 4: Three D print some leads onto a rigid battery or
Speaker 4: a Milwaukee battery and connect it to a T twelve tip.
Speaker 4: But the real smarts of it is in the firmware
Speaker 4: where it manages the heat to make sure it gets
Speaker 4: up to temperature quickly that it actually does the sensing
Speaker 4: thing where if you shake it, it turns back on
Speaker 4: and heats up anytime. Anytime the temperature is cooling down rapidly,
Speaker 4: it sends more power. So anyway, it's just a really
Speaker 4: great soldering iron. And because I have that, like I've
Speaker 4: got a and I've got a couple of cheap ones,
Speaker 4: but that one it cost a hundred bucks, and I'm
Speaker 4: considering trying out one of the knockoffs. It's only like
Speaker 4: thirty five because now everybody, even Craftsman, is selling one
Speaker 4: of these at Low's now. But I don't know if
Speaker 4: the Craftsman one is just like the cheap idiot kind
Speaker 4: where it's just connecting the leads together, or if it's
Speaker 4: actually got a I have a hard time believing that
Speaker 4: they would have gotten an illegal copy of the firmware,
Speaker 4: whereas the Chinese companies on expression you know that anyway
Speaker 4: saw that. Yeah, so I had a good time soldering
Speaker 4: because of the rigid powered which you give them Mount
Speaker 4: Milwaukee or Iobi or whatever battery brand you like, soldering iron.
Speaker 4: They're super fast. There's so much better than a weller
Speaker 4: or a Hako or all the traditional ones that cost
Speaker 4: hundreds of dollars. So anyway, and of course I'll pick
Speaker 4: a LLAMA because I really do enjoy running my own
Speaker 4: local lams. Actually, since the thirty two billion parameter model
Speaker 4: of Quinn two point five coder has come out, that
Speaker 4: one I just find to be the best of the best.
Speaker 4: It rivals GPT four oh if not is better than
Speaker 4: four to oh, and you can run it on an
Speaker 4: Apple silken back boom, all the things.
Speaker 2: Okay, I have a question, what are you modeling your
Speaker 2: switch to do?
Speaker 5: Oh?
Speaker 4: Sorry, I skipped over that part over well, not overclock
Speaker 4: to the native CPU freak speed of the tegra x one,
Speaker 4: because the switch is an Android tablet running a custom
Speaker 4: operating system rather than Android is two point two gigahertz.
Speaker 4: That's the native clock speed. The clock speed that it
Speaker 4: runs at is something like a thousand or seven hundred,
Speaker 4: depending on whether it's doctor handheld. Same thing with the GPU.
Speaker 4: The native GPU speed is like fifty one point five
Speaker 4: gigahertz or something like that, but they clock it down
Speaker 4: to five hundred or seven hundred.
Speaker 2: Gotcha.
Speaker 4: So when you mod it, you can then and this
Speaker 4: you can do without getting banned or at least this
Speaker 4: is what people are reporting, and I'm I'm doing this
Speaker 4: so you know, if you mod it and you want
Speaker 4: to run pirated games or something, you have to set
Speaker 4: up more stuff and make sure that you don't get banned,
Speaker 4: although a lot of that stuff is built in now.
Speaker 4: But if all you want to do is overclock it,
Speaker 4: the overclock system runs in a layer that's kind of
Speaker 4: protected from the main switch system, so the main switch
Speaker 4: system can't detect that it's rooted while it's running, So
Speaker 4: you can you if you just install hakat what does it?
Speaker 4: Hikata Atmosphere and cisclock. If those are the only things
Speaker 4: you install, then you should be able to run your
Speaker 4: switch modded on the original firmware, be able to play online,
Speaker 4: et cetera, without any risk of banning or anything, because
Speaker 4: it's the where it's not modifying the switch operating system
Speaker 4: or the game. It's just modifying the CPU clock.
Speaker 2: Gotcha cool?
Speaker 4: So now, so now my friend asked me, well, can
Speaker 4: you notice any difference? And my answer is no. And
Speaker 4: the reason my answer is no is because when you're
Speaker 4: not playing, when you're playing it underclocked, you notice the
Speaker 4: stuttering all the time, and like you notice the resolution changing,
Speaker 4: Like you know, you turn and there's a bad guy
Speaker 4: and you shoot him, and then the resolution Like, but
Speaker 4: when you're playing it closer to native speeds, you can't
Speaker 4: get it all the way up to native speeds because
Speaker 4: the power delivery on the board isn't actually capable of
Speaker 4: playing it at native speeds without also draining the battery
Speaker 4: at the same time. But when you're playing it at
Speaker 4: near native speeds, you don't notice it because you like,
Speaker 4: the things that are annoying aren't there. The resolution's not changing,
Speaker 4: it's not stuttering, it's not going into bullet time. As
Speaker 4: much I haven't, I did not notice it at all
Speaker 4: going into bullet time since I've been playing it at
Speaker 4: near native speeds, and I did some things where I
Speaker 4: was blowing up rocks and things that I thought would
Speaker 4: normally make it go to bullet time, and it didn't.
Speaker 4: So like five star stories, so far.
Speaker 2: Cool, very cool, All right, Steve, what are your picks?
Speaker 5: All right?
Speaker 3: Kind for my twenty minute picks. So before I get
Speaker 3: in picks, one note, I will make it sort of
Speaker 3: circles back to what I asked at the beginning. You know,
Speaker 3: as someone who has spent a lot of time doing
Speaker 3: search indexing, you know, with lucine type search indexes. A
Speaker 3: lot of this sounds really familiar. And to me, I've
Speaker 3: always said that the I and AI is a misnomer.
Speaker 3: I think it's it's not necessarily intelligent. It's just basically
Speaker 3: better using of training and fancy or using of existing
Speaker 3: data to answer things, not necessarily intelligence that and create
Speaker 3: new things. That's just my two cents for what it's worth. Interesting,
Speaker 3: pick Eean, you mentioned this earlier today and as of
Speaker 3: today that you know, this will come out a little later,
Speaker 3: but deep seek is like disrupting in a huge way.
Speaker 2: You know.
Speaker 3: For instance, if you go look on Hacker News, both
Speaker 3: on the top page and on the new page, there
Speaker 3: are multiple articles from NPR from CNBC about what it's
Speaker 3: doing to the stock market, and the gist is basically
Speaker 3: that they've been able to create these fantastic models with
Speaker 3: much less investment, with less powerful chips. And there's a
Speaker 3: whole story behind this, and so that's wreaking have at
Speaker 3: least in the stock market and with people like Nvidia,
Speaker 3: just because of supposedly how how much cheaper and more
Speaker 3: efficient deep seek is compared to some of the other models.
Speaker 3: So today's I've the first day and know it remains
Speaker 3: to be seen how accurate this is, especially coming from
Speaker 3: the Chinese, but sort of a disruptive thing going on
Speaker 3: this morning, at least as of the time of recording.
Speaker 5: Yeah, do you mind if I jump in there a
Speaker 5: little bit. Deep Seek is an utterly fascinating story and model.
Speaker 5: I'll say one thing is that the training cost might
Speaker 5: have been apples to oranges. Like they stated what the
Speaker 5: cost was for the best run or the final run.
Speaker 5: There's a lot of other costs that go into I
Speaker 5: talked earlier when AJ was asking what goes into it.
Speaker 5: One of the key things I was thinking actually about
Speaker 5: Deep Seak is like they're going to do a lot
Speaker 5: of other experiments and runs. There's a lot of stuff
Speaker 5: that gets built upon. So I think some people are
Speaker 5: comparing apples to oranges, but it is, you know, a
Speaker 5: impressive model in a lot of ways. The other thing
Speaker 5: that I find the most fascinating about it is the
Speaker 5: training process is remarkably simple. And you know, I'm trying
Speaker 5: to think of an analogy. It's like, normally when they
Speaker 5: do this part of the training process called reinforcement learning.
Speaker 5: It's a lot more complex, and it's kind of like,
Speaker 5: you know, you think about a car and you're like, well,
Speaker 5: if you want to go from point A to point B,
Speaker 5: you need an internal combustion engine. It's basically doing you know,
Speaker 5: having a little fire and you got these pistons and
Speaker 5: the cylinders, really complex piece of mechanics. And then somebody's
Speaker 5: like the electric car, and so you know that little
Speaker 5: toy motor you had, Well, let's just scale that thing up.
Speaker 5: And so they tried this really simple, relatively simple technique
Speaker 5: and just scaled it up and it worked. And I
Speaker 5: think different people are reacting to this model differently. Some
Speaker 5: people it's about the price, other people it's about the
Speaker 5: training setup, and it's how did we miss this? It's
Speaker 5: just remarkably simple. So it's definitely a worthy to bring
Speaker 5: up as it's just a really interesting model.
Speaker 3: O Coom's razor strikes again.
Speaker 2: Right.
Speaker 5: Well, there's something they call in AI the bitter lesson,
Speaker 5: which is stop trying to put into the model how
Speaker 5: you think you think. Instead just give really general compute
Speaker 5: and just throw more and more data and more and
Speaker 5: more compute at the problem, and the model will figure
Speaker 5: it out, so don't try to be too smart about it.
Speaker 5: And people are like, this was a bitter lesson all
Speaker 5: over again. It's like, oh, we thought, you know, we
Speaker 5: had to do this really complex, you know, reinforcement learning setup,
Speaker 5: and these guys showed, well, maybe you don't now. In
Speaker 5: the end, their production model actually still had some complex
Speaker 5: training pipeline. But one of the interesting results is this
Speaker 5: model called zero, where it kind of like, you know,
Speaker 5: Alpha zero learned how to become a really good go
Speaker 5: player just by playing against itself. In this case, it
Speaker 5: wasn't the model playing against itself. It was just trying
Speaker 5: out ideas and they just told it whether it was
Speaker 5: right or wrong, and it started automatically emergently figuring out
Speaker 5: how to improve its thinking. And it starts getting these
Speaker 5: eureka moments where it suddenly realizes it can backtrack and
Speaker 5: it's like, oh, wait, I made a mistake, and it's
Speaker 5: you're watching the model like we didn't train it to
Speaker 5: do this, and it suddenly figures out how to like
Speaker 5: get smarter. So it's really really fascinating. I could we
Speaker 5: could talk another hour on the model, but yeah, lots
Speaker 5: of people are pouring over it. It's It's fascinating in many dimensions.
Speaker 2: Cool.
Speaker 3: And then, last, but not least certainly, the high point
Speaker 3: of any episode is the dad jokes of the week.
Speaker 3: So what did one pie say to the other pie
Speaker 3: before being put in the oven? You know this is
Speaker 3: a musical answer. All we are is crusted in the tin.
Speaker 3: For anybody that knows Kansas, here's an Australian version. My
Speaker 3: mate was bitten by a snake, so I told him
Speaker 3: an amusing story. If I know the difference between anecdote
Speaker 3: and antidote, he'd still be alive. And finally, when I
Speaker 3: was in school, my teachers told me I would never
Speaker 3: amount to anything because I procrastinated too much. I told him,
Speaker 3: you just wait, those are the dad jokes of the week.
Speaker 1: All right, well, I'm going to jump in here and
Speaker 1: save us from the high point of the episode. I've
Speaker 1: got a couple of picks. I always do a board
Speaker 1: game pick. So the game I'm gonna pick I learned
Speaker 1: this game last week is called Cascadero. I'm gonna put
Speaker 1: links in for both board game Geek, which kind of
Speaker 1: gives you information about the board game, and then an
Speaker 1: Amazon affiliate link, just because then you know where to
Speaker 1: go buy it if you want it anyway, So Cascadero,
Speaker 1: the premise of the game is that the kingdom's breaking up,
Speaker 1: and so you all play a different faction, I guess,
Speaker 1: and you're trying to connect towns and send your people
Speaker 1: through the towns to pull the kingdom back together. And
Speaker 1: so you put your little guys out there, and then
Speaker 1: you score points based on whether you're the first person
Speaker 1: of the town or the second person of the town.
Speaker 1: If you have a group, you have to have a
Speaker 1: group of your little horsemen.
Speaker 2: I can't remember what they call them, heralds. No, the
Speaker 2: heralds were the.
Speaker 1: Other things anyway, So if you connect to a town
Speaker 1: with a herald, then you get an extra point for
Speaker 1: connecting to it, and then there are bonuses that you get.
Speaker 1: So if because when you connect, when you get the points,
Speaker 1: you actually move a marker up the technology or progress
Speaker 1: track in whatever color you connected to. And so I
Speaker 1: guess they're not points, they're just movements. But anyway, so
Speaker 1: once you move past certain points, you get certain rewards,
Speaker 1: and I mean effectively, what you're trying to do is
Speaker 1: you're trying to score the most points, and you also
Speaker 1: want to get to the end of the track and
Speaker 1: whatever color you're playing. So if you're playing pink, you
Speaker 1: want your pink marker to get all the way to
Speaker 1: the end. And yeah, like I said, you get bonus
Speaker 1: points for getting all five of your markers past the
Speaker 1: first space.
Speaker 2: That's marked for that.
Speaker 1: You get more bonus points if you get three past
Speaker 1: the second spacing, and then if you're the first one
Speaker 1: to get one all the way to the end, then
Speaker 1: you get bonus points and only one person can get those,
Speaker 1: and then the other ones are if you connect two
Speaker 1: cities of the same color and there are five colors,
Speaker 1: you get bonus points for each color, and if you
Speaker 1: get all five colors, then you get ten bonus points.
Speaker 1: And so you're just moving your marker around a track
Speaker 1: when you get the points. As soon as somebody gets
Speaker 1: fifty points, the game ends. And so essentially, if you
Speaker 1: want to win, you want to be the first person
Speaker 1: to get your marker all the way to the end
Speaker 1: of the track of your color and then be the
Speaker 1: person that gets that fiftieth point. We played it, and
Speaker 1: it was our first time any of us playing it,
Speaker 1: and so nobody crossed that fiftieth point before we all
Speaker 1: ran out of Little Horseman, and so when somebody runs out,
Speaker 1: everybody gets one more turn, or if somebody crosses that
Speaker 1: fifty point marker, everybody else gets one more turn, and
Speaker 1: then the game's over. It's reasonably simple. The scoring is
Speaker 1: a little bit complicated as far.
Speaker 2: As like moving.
Speaker 1: You know, when you get moves and how many moves
Speaker 1: and things like that. So board game geek waits at
Speaker 1: at two point five to three, right, So it's a
Speaker 1: little more complicated than kind of your casual gamer who's
Speaker 1: just going to play gravel with their friends. But my
Speaker 1: feeling is is that it was only just getting used
Speaker 1: to when I put my horseman down what happens. And
Speaker 1: as soon as you get used to I put my
Speaker 1: horsemen down, I can move something up the track so
Speaker 1: many spaces, and then how to get the rewards. Once
Speaker 1: you figure that out, it's a relatively simple game. We
Speaker 1: played it in what an hour, maybe a little longer,
Speaker 1: I think once. If we'd known what we were doing,
Speaker 1: we could probably play it in forty five minutes. There
Speaker 1: were three of us playing it, so anyway, Coscadero.
Speaker 2: Fun, fun game. I liked it.
Speaker 1: I want to play it again now that we I
Speaker 1: know how to play it and my friends and how
Speaker 1: to play it, because there were a couple of.
Speaker 2: Things I would have done differently as I gotten into it.
Speaker 1: As far as other picks, go.
Speaker 2: Go to jsgeniuses dot com and sign up.
Speaker 1: We're gonna start doing the meetups and I'm gonna start
Speaker 1: posting videos. The videos I'm posting the I'm kind of
Speaker 1: building an entire app. I don't know if I'm going
Speaker 1: to show you writing all the code because some of
Speaker 1: the stuff gets repetitive. Oh, I have to connect another
Speaker 1: data model to this database. Right, It's like, Okay, you
Speaker 1: don't need to see that eighteen times, but you know,
Speaker 1: we'll get kind of the major pieces in and then
Speaker 1: anything bonus extra that I do the app I'm gonna build.
Speaker 1: I decided I need to learn next JS. So it's
Speaker 1: going to be an NEXTJS app and I'm going to
Speaker 1: be putting it on cloud Flare workers.
Speaker 2: And the reason is.
Speaker 1: Is because just to give you an idea of what
Speaker 1: the app is, it's relatively simple.
Speaker 2: But last year when we.
Speaker 1: Ran Caucus Night for the Utah Republican Party, we had
Speaker 1: an online registration and we got d doast because there
Speaker 1: were people out there who didn't like us, and it's
Speaker 1: internal politics to Utah. It wasn't the Democrats, it with
Speaker 1: somebody else, but anyway, So because of that, I'm looking to,
Speaker 1: you know, put it on a system where I know
Speaker 1: it'll just kind of expand to whatever comes at it.
Speaker 1: Cloud Flare is also usually pretty good at you hit
Speaker 1: me eighteen times. Now, I'm just going to say drop
Speaker 1: it and drop you unless you can prove your human
Speaker 1: and so I feel like I can get some of
Speaker 1: those benefits. But I'm also curious to see how cloud
Speaker 1: flair workers work. So it's going to be basically a registration.
Speaker 1: There's going to be a little bit of site automation
Speaker 1: because the Utah State voter registration database where you verify
Speaker 1: that your voter registration doesn't have an API, which means
Speaker 1: that I have to go and have my program use
Speaker 1: something like puppeteer to fill in the fields and then
Speaker 1: scrape data off the response to make sure that you're
Speaker 1: registered to go to caucus night. I'm thinking I may
Speaker 1: also offer this same kind of thing to the Democrats
Speaker 1: and anybody else who wants to run.
Speaker 2: A caucus night that night.
Speaker 1: I think Libertarians and Utah do it too, right, so
Speaker 1: that they can just hey, you've got online registration and
Speaker 1: then you've got an app that will verify it on
Speaker 1: the other end.
Speaker 2: So anyway, that's what I'm looking at.
Speaker 1: So there may also be React native app or something
Speaker 1: like that that on the other end, you know, people
Speaker 1: can show up with a QR code that says I
Speaker 1: registered and this is who I am, and people can
Speaker 1: just verify that way instead of having to look them
Speaker 1: up in a paper list or something like that.
Speaker 2: So that's what to be building. You jass geniuses.
Speaker 1: You get access to the videos, you get access to
Speaker 1: the weekly meetups, and a bunch of other stuff. I'm
Speaker 1: also looking at starting a new podcast on doing AI
Speaker 1: with JavaScript, and it's gonna be at this level, right
Speaker 1: We're not building our own models. We're gonna be using
Speaker 1: the existing models that are out there, the open source models,
Speaker 1: if you will, and showing how to build things on
Speaker 1: top of those, or using some of the cloud services
Speaker 1: that generate images, or you know, using something like Whisper
Speaker 1: for transcriptions and things like that.
Speaker 2: So anyway, keep an eye out for that. That'll be free.
Speaker 1: I'll probably drop the first two or three weeks worth
Speaker 1: of episodes onto this RSSV and then From there.
Speaker 2: You'll be able to just subscribe to the other feed.
Speaker 1: So that's what I'm I've got going. Yeah, those are
Speaker 1: my picks, e Shan, what are your picks?
Speaker 5: So I've got two picks. The first one both are
Speaker 5: going to be a I related. The first one is
Speaker 5: notebook LM, but everyone knows about notebook LM with like
Speaker 5: the fake podcasters. My pick is not book LM without
Speaker 5: that feature. I think that feature is great and really compelling.
Speaker 5: Lets me consume you know, material on the go in
Speaker 5: podcast form. But I like the other parts of noepook LM,
Speaker 5: which is like it's a great way to stick a
Speaker 5: variety of sources together and then ask questions about it.
Speaker 5: So one example is I like to go to y Combinator,
Speaker 5: hacker news to see what the comments are, but I
Speaker 5: don't read through every single one. So I will stick
Speaker 5: it in there and say, well, what are the most
Speaker 5: insightful comments? What are people saying? I did this actually
Speaker 5: with deep Seek, I say what are the comments people
Speaker 5: are saying about deep Seek? What are they seeing for performance?
Speaker 5: What are the issues where it's not working? And what's
Speaker 5: great is it doesn't just summarize it. You've got where
Speaker 5: it can go to each part of it and say, okay,
Speaker 5: this is like, oh that sounds interesting, let me go
Speaker 5: click on it, and I can go right to the
Speaker 5: citation of that that comment. The formatting is a little
Speaker 5: off when you when you stick it in there, so
Speaker 5: there's and you're only limited to thirty sources in each notebook,
Speaker 5: but check out the other parts of notebook.
Speaker 2: LM.
Speaker 5: I think it's it's really interesting. I expect to see
Speaker 5: a lot of other applications follow the similar type of
Speaker 5: of ux paradigm or inspiration. The second one is I
Speaker 5: don't know if you guys have been watching it, but
Speaker 5: Star Wars has a new show, Skeleton Crew that they
Speaker 5: have on Disney Plus. And first of all, I think
Speaker 5: it's it's good. I don't think it's you know, Mandalorian
Speaker 5: or and Or, which was my personal favorite level, but
Speaker 5: it's still pretty good. But the other reason I bring
Speaker 5: it up is I liked some of the elements of
Speaker 5: how they handled AI and droids. So in one episode
Speaker 5: there's something that could be akin to jail breaking the
Speaker 5: droid where somebody uses the equivalent of prompt you know,
Speaker 5: prompt hacking to jail break a droid, and I don't
Speaker 5: think we've ever seen that in Star Wars's reflection of
Speaker 5: Droids before. There's another one where it reminded me of
Speaker 5: this paper called alignment faking, where the model has to
Speaker 5: decide between its original training or the thing it's being
Speaker 5: asked to do right now, and it kind of goes
Speaker 5: back and forth and it gets over in by its
Speaker 5: original training. And so there's one thing the very last
Speaker 5: episode that I also thought it was fascinating. But I
Speaker 5: really liked those interesting bits of how they handled AI
Speaker 5: that I think we wouldn't have seen in a show
Speaker 5: like this without understanding of chat GPT that I think
Speaker 5: probably the writers were inspired by. So those are my picks.
Speaker 1: Awesome, Yeah, Skeleton Crews on my list of things I
Speaker 1: want to watch. So I like the recommendation. Thanks for that,
Speaker 1: all right, Well, just a reminder go look on maven
Speaker 1: dot com.
Speaker 2: The code was jays.
Speaker 1: Jabber for twenty percent off, and so if you're interested
Speaker 1: in the course, go check it out. I'm not hard
Speaker 1: sell guy. I just think it sounds fascinating. So anyway,
Speaker 1: let's go ahead and wrap it up here until next time.
Speaker 2: Him Max Ow
Speaker 3: Mhm