Speaker 1: How'd you like to listen to dot NetRocks with no ads?
Speaker 2: Easy?
Speaker 1: Become a patron for just five dollars a month. You
Speaker 1: get access to a private RSS feed where all the
Speaker 1: shows have no ads. Twenty dollars a month, we'll get
Speaker 1: you that and a special dot NetRocks patron mug. Sign
Speaker 1: up now at Patreon dot dot NetRocks dot com. Hey
Speaker 1: guess what, it's dot net rocks. I'm Carl Franklin at
Speaker 1: amiraterid Cabell. We're here again for the nineteen hundredth and
Speaker 1: sixty first time.
Speaker 2: Well you.
Speaker 1: Okay, you're here for the eighteen hundred and sixtieth time.
Speaker 2: I think something like that. Second.
Speaker 1: I'm the new guy, remember new guy new the new guy. Yeah,
Speaker 1: we are the OG podcast certainly when it comes to development,
Speaker 1: and one of the first five that's still existence since
Speaker 1: podcast hit the ground running. We've been going since August
Speaker 1: two thousand and two, A long, long, long, long time.
Speaker 2: It's been a while. Yeah, how are you, Richard? I'm good.
Speaker 2: It's summertime, and summertime it's good. Sometimes it's beautiful on
Speaker 2: the coast.
Speaker 1: It's a little hot where I am, but it's not bad.
Speaker 2: It's not bad. It could be worse. Well, and she
Speaker 2: who must be obeyed is turning sixty, so I've well,
Speaker 2: at this point we'll have thrown that party.
Speaker 1: So wow, it's always congratulations Stace. All right, let's roll
Speaker 1: the crazy music because I've got something for Hannas for
Speaker 1: better no framework awesome?
Speaker 2: All right, man, what do you got?
Speaker 1: Since this is episode nineteen sixty one, you can go
Speaker 1: to nineteen sixty one dot dot me and that brings
Speaker 1: you to this article from Menga, which is just somebody's
Speaker 1: blog I think. I don't even know the name of
Speaker 1: the person, doesn't even say all right, but anyway, it's
Speaker 1: called computerized guitars and whether they're worth buying or not.
Speaker 1: I know, Hanness plays guitar and makes guitars and yeah.
Speaker 2: Builds guitars.
Speaker 1: Yes, he's a guitar guy, right, Yeah, So this article
Speaker 1: basically says nobody likes these electronic you know, computerized guitars. Really,
Speaker 1: nobody likes them. Nobody wants one.
Speaker 3: There's some weird experiments out there, like there are Gibson
Speaker 3: experiment with de robotic tuners and volunteery that that's great,
Speaker 3: and nobody seems to want that on their guitar.
Speaker 2: That's just it. You know.
Speaker 1: The sales say that they're cool, but nobody wants them,
Speaker 1: like the Gibson Firebird X, which is I think the one,
Speaker 1: or Firebird ten, which is I think what you're talking about,
Speaker 1: and the Les Paul Robot guitar was another one. But
Speaker 1: this is without question the most computerized electric guitar in
Speaker 1: the market right now. It has everything on it. The
Speaker 1: only it doesn't have is sales. Nice nobody bought it,
Speaker 1: and Gibson actually ended up crushing, like with a bulldozer,
Speaker 1: about a hundred of these things because they were a
Speaker 1: liability to have on the books, because they got had
Speaker 1: to get rid of their inventory, and because nobody was
Speaker 1: buying them.
Speaker 3: Weirdly, Yeah, like every every guitar manufacturer that does strangely
Speaker 3: innovative stuff has been has been having trouble selling them.
Speaker 3: Like my favorite guitar that I own is a park
Speaker 3: Or Fly, which was way ahead of its time, and
Speaker 3: they struggled to sell them. Yeah, although they were selling
Speaker 3: guitars that were actually work twice what they were selling
Speaker 3: them for, but nobody bought them because they were so unconventional.
Speaker 1: Yeah, and so let me just point out the other
Speaker 1: ones here the Fender Veg stratocaster, and I actually have
Speaker 1: one of these, but it was before that it was
Speaker 1: the Fender Veg. It's a Fender strat, but it also
Speaker 1: has the role In logo on it because you can
Speaker 1: connect it to the rollin VG system and it's got
Speaker 1: the right pick up in there and all. But as
Speaker 1: a strat, it's just kind of bleuh right, I have
Speaker 1: a strat plus that I love. I mean, it's hard
Speaker 1: to keep in tune and all that stuff, like a
Speaker 1: regular strat is, but at least it's a good sounding strat.
Speaker 2: The isn't the question here why people play guitar, because
Speaker 2: most people who play guitar aren't making money playing guitar. No,
Speaker 2: that's true, sadly no.
Speaker 1: They also mentioned the epiphone Les Paul Ultra, which has
Speaker 1: a USB port where you can connect the guitar to
Speaker 1: you know, your computer. The problem with that is that
Speaker 1: he says here too, machines and computers just don't mix
Speaker 1: when it comes to guitars. An electric guitar is a
Speaker 1: machine that by nature is a very physical instrument. This
Speaker 1: isn't like a synthesizer where it just sits there and
Speaker 1: you play it with a guitar, you sit down way that,
Speaker 1: you stand up with that you move around playing and
Speaker 1: so on. But most players consider these computerized guitars not
Speaker 1: real instruments because, let's face it, the sound of playing
Speaker 1: a synth through a guitar is just no, it's just bad.
Speaker 1: So they're out there. I think if I was going
Speaker 1: to get one, I'd just get an automatic tuning one
Speaker 1: and leave it at that.
Speaker 3: Actually that system works, it's not that yeah, yeah, it's
Speaker 3: just good. Well, in this blog post is thirteen years old.
Speaker 3: You think the technology has probably moved on a little
Speaker 3: from there.
Speaker 1: Yeah, a little bit, but I mean those are still
Speaker 1: not that much though. Those are still the models that
Speaker 1: you know that people like, except for the Fiber ten,
Speaker 1: which is not available anywhere.
Speaker 3: Actually, the musician that has done the weirdest stuff to
Speaker 3: some of their guitars that a lot of people might
Speaker 3: know about is Matt Bellamy from muse Be built in
Speaker 3: like x Y controllers that control synthesizers that were like
Speaker 3: controlled from his guitar and built in effects into the
Speaker 3: body of the guitar. Like that had more electronics in
Speaker 3: it than a lot of other instruments out there.
Speaker 2: Right.
Speaker 1: Well, and then you got guys like Brian May who
Speaker 1: just made a very unique sounding guitar working with phase
Speaker 1: cancelation and phasing and all that stuff. But you know
Speaker 1: he's just just built by himself, Yeah, and his father, Yeah, out.
Speaker 3: Of very unconventional woods in the guitar world. He used
Speaker 3: things like oh, which nobody he uses.
Speaker 1: Yeah. So anyway, that's what I got today. And I
Speaker 1: thought we would have a nice conversation about guitars, and
Speaker 1: we did so.
Speaker 2: Yeah. Richard, who's talking to us today, grabbed a comment
Speaker 2: off a show nineteen thirty nine, which is the one
Speaker 2: we did with Jeremy Miller talking about vertical slice architecture,
Speaker 2: and I know we got into events, sourcing and things
Speaker 2: with him at one point or another. Two. This comment
Speaker 2: comes from cash bon Fields, who said, you mentioned the
Speaker 2: buzz factor, which is how many people could be hit
Speaker 2: by a by while the project still survives. I have
Speaker 2: been recently considering the opposite, how many people should be
Speaker 2: hit by a bus for a project to be effective.
Speaker 2: The bus should most often be used for selecting managers,
Speaker 2: though I think there's a little anger in their cash.
Speaker 2: You want to talk to someone, don't don't hurt anybody. No,
Speaker 2: I get you. You know, there's definitely that effect of this.
Speaker 2: You know, how many cooks before you spoil the soup
Speaker 2: kind of thing, right, And we were six months behind
Speaker 2: on the project, so they had a developer. Now we're
Speaker 2: nine months behind on the project. People don't understand how
Speaker 2: to fix things. Hey, I got you the I got
Speaker 2: your book Mythical Man month, but actually got you two
Speaker 2: of them so you can read it twice as fast. Boom, yeah, Cash,
Speaker 2: Thank you so much for your comment, and a copy
Speaker 2: of music Coba is on its way to you. And
Speaker 2: if you'd like a copy of music Cobe. I read
Speaker 2: a comment on the website at dot and rocks dot
Speaker 2: com on the facebooks. We publish every show there, and
Speaker 2: if you comment there and I read it on the show,
Speaker 2: we'll send you a copy of music. Go buy Cobey
Speaker 2: is still going strong.
Speaker 1: Twenty two chapters if you call them or episodes or tracks,
Speaker 1: I guess I.
Speaker 2: Kind of like chapters.
Speaker 1: Twenty two tracks, yeah, whatever you want to call them.
Speaker 1: There's twenty two musical tracks and they're all twenty five
Speaker 1: minutes long, perfect for your Pomadoro technique. Music took code
Speaker 1: by dot net all right, before we bring on Haness
Speaker 1: what happened in nineteen sixty one besides John F. Kennedy's inauguration,
Speaker 1: after which he immediately invaded.
Speaker 2: Cuba by pigs thigs. Yeah, I don't know, just parallel,
Speaker 2: maybe kind of a hairy year, no two ways about it,
Speaker 2: and without a dough.
Speaker 1: Yeah, the establishment of the Peace Corps and Juriga Garin
Speaker 1: became their first official human to travel into space. Additionally,
Speaker 1: the and directly into orbit too, you know, yeah, she
Speaker 1: orbited as opposed to you know, the US first effort
Speaker 1: with Alan Shepard in the sub ardable flight. Yeah, yeah,
Speaker 1: so I know, there's all the Beatles came in, you
Speaker 1: know around there, I said, the Peace Corps, some nuclear
Speaker 1: testing was going on in the Soviet Union, and the
Speaker 1: Academy Awards thirty third Academy Awards took place, with the
Speaker 1: Apartment winning Best Picture. You know, just some random trivia.
Speaker 1: But you've got the reallest Richard, what really happened in
Speaker 1: nineteen sixty one?
Speaker 2: I mean I always go off to the invention side,
Speaker 2: just because you know, so many cool things are made.
Speaker 2: Sixty one is when the first industrial robot is deployed
Speaker 2: into a factory. It's called Unimate. It had been developed
Speaker 2: in the fifties by George Deval and General Motors puts
Speaker 2: it into a automotive factory to remove metal parts from
Speaker 2: a die caster. So a die caster is basically a
Speaker 2: big hydraulic ram. You put sheets of metal into it
Speaker 2: and it slams them into the shape of like a
Speaker 2: door panel. But that force that molds that that quickly
Speaker 2: makes the metal really HoTT. So not when a person
Speaker 2: has handled at any clubs and things, is very easy
Speaker 2: get burned like it's dangerous work. And so having the
Speaker 2: robot deal with that we made a lot of sense. Made sense. Yeah,
Speaker 2: And over the next decade or so, automation would come
Speaker 2: throughout the factory process. So it begins in sixty one
Speaker 2: with the Unimate. A little corollary to that, as they
Speaker 2: were celebrating the Unimate success, it got an appearance on
Speaker 2: Johnny Carson and poured a cup of coffee.
Speaker 1: I think I do remember that. Yeah, yeah, it was
Speaker 1: ninety sixty one. You were not born yet, no, but
Speaker 1: I saw the rerun that was a famous famous clips.
Speaker 2: Could be famous episode. Nineteen sixty one is also the
Speaker 2: first integrated circuit in production. Now I step into this
Speaker 2: very carefully because it is a hotly contested topic. In fact,
Speaker 2: it's got a Wikipedia entry just on the invention of
Speaker 2: the integrated circuit, because there's lots of arguments here. I mean,
Speaker 2: for I mean meaning who did it first? Yeah? And
Speaker 2: what represents an integrated circuit? Mean, we'd been messed around
Speaker 2: with semi conductors for a while, We've made a bunch
Speaker 2: of different kinds of transistors. You've got Back at fifty
Speaker 2: twosh radio engineer Jeffrey Dummer was making semiconductor circuits. There's
Speaker 2: also Harwick Johnson, Sydney Darlington, Yasatarui. But it really comes
Speaker 2: down to two people for the most I think most
Speaker 2: people agree. Now it's Jack Kilby of Texas Instruments with
Speaker 2: a thing he called the hybrid ice, so some discrete
Speaker 2: components and some integrated in a containerized package with pins,
Speaker 2: and of Robert Nois of Fairschald Semiconductor, who's also the
Speaker 2: guy who later goes on to form Intel with his
Speaker 2: monolithic ices.
Speaker 1: So the integrated circuit means everything is on one ship
Speaker 1: with the pins and the transistors are mounted inside that.
Speaker 2: That's the idea, and there's a bunch but if you're
Speaker 2: looking at just the packaging, So I have a package
Speaker 2: with pins on it, and it has a bunch of
Speaker 2: stuff in it, and it does a particular task that's
Speaker 2: pretty integrated. But then you get into what's actually inside
Speaker 2: that package, and you know, Noise is designed with what
Speaker 2: they call the monolithic icee is actually a layers of
Speaker 2: ustrate silicon crystal that are doped into different potentials so
Speaker 2: that they can be etched together to make transistors and
Speaker 2: resistors and other components. I get it. Yeah, Killey's approach
Speaker 2: was more discreete.
Speaker 1: So ICs continued to evolve and get more powerful. But
Speaker 1: you're calling that one the first because it was the
Speaker 1: first contained integrated.
Speaker 2: Well, I called the first really going in ninety sixty
Speaker 2: one is when it was first made into a product,
Speaker 2: at least briefly, and it was you know, the nexus
Speaker 2: of this is in two thousand, Jack Kilby was given
Speaker 2: the Nobel Prize in Physics for his quote unquote contributions
Speaker 2: integrated circuit and created a huge fuss around all that
Speaker 2: because Kilby's design that the hybrid icee doesn't last. Ultimately,
Speaker 2: Noises design is what makes integrated circuits to this day.
Speaker 2: Although limittedly. The technology has evolved a lot, but in
Speaker 2: nineteen sixty one, Texas Instruments started manufacturing a product called
Speaker 2: the Multivibrator five h two. Inside of that package, some
Speaker 2: of it some silicon substrates, some discrete were two transistors,
Speaker 2: four diode, six resistors, and two capacitors to be used
Speaker 2: as a waveform generator, which was part of a miniaturized
Speaker 2: radio receiver for the US Air Force. Wow. So they
Speaker 2: made a few hundred of these and built the test articles,
Speaker 2: and it ultimately failed, like in the end, it wasn't
Speaker 2: sturdy enough, but it set the big path for integrated
Speaker 2: circuits in that kind of ministriization at that time called
Speaker 2: molecular electronics because they were so small even though they
Speaker 2: were not molecule size in aerospace, and the Air Force
Speaker 2: would lead on this for quite a while. It leads
Speaker 2: to the ICBM and a bunch of other technology. But
Speaker 2: the first, you know, you can kind of hang your
Speaker 2: head on, Texas Instruments produce this first thing they called
Speaker 2: the integrated circuit, even though it was a design that
Speaker 2: ultimately fell into obsolescence in the model ethic ic would
Speaker 2: go on from there. Wow big one though, like I said,
Speaker 2: it's not simple. Yeah, it is.
Speaker 3: Something simpler that we may all remember. Nineteen sixty one,
Speaker 3: Phillips introduced the audio cassette, the Compactus that we would
Speaker 3: all remember from compact cassette player from our Walkman's Wow
Speaker 3: throughout the eighties.
Speaker 2: The Walkman's Walkman's is still twenty years away at that point.
Speaker 2: But yeah, I know, I know, but wow, you know
Speaker 2: what I you know, I pulled up a graphic on
Speaker 2: the other day on on the Nakamichi autocassett player because
Speaker 2: it was the one that would flip the tape. I
Speaker 2: have one.
Speaker 1: It's right there. Actually, yeah, I have one. It doesn't
Speaker 1: work it, Yeah, flips the tape.
Speaker 2: Over, pops the thing would pop out, turn the tape over,
Speaker 2: and go back because they didn't want to try and
Speaker 2: adjust the head alignments to play both both sides of
Speaker 2: the tape. You know.
Speaker 1: Yeah, classics. Okay, all right, cool stuff. So I guess
Speaker 1: we can introduce Hannis officially. You've heard him talk there.
Speaker 1: But Hannis Lauette started his dot net career during college
Speaker 1: with Framework one point one. He continued this trend in
Speaker 1: his first jobs and has been using dot net ever
Speaker 1: since to deliver projects. For many different clients. He's always
Speaker 1: has had a passion for back end development in his projects.
Speaker 1: He's always gravitated to problems that involves scaling, architecture and complexity.
Speaker 1: Because of that, he developed a love for event driven
Speaker 1: architectures that fits with what we're going to talk about.
Speaker 1: Since twenty eleven, surprise, surprise, Yeah, yeah, Since twenty eleven,
Speaker 1: he's been working for ACES in Belgium.
Speaker 2: Is that how you say it?
Speaker 1: Yes, ax Xes currently as principal consultant. Aside from his
Speaker 1: technical role, Hannas has always been driven by helping other
Speaker 1: developers grow. He actively coaches people, runs workshops, gives conference talks,
Speaker 1: and is a course author on dome Train is free time.
Speaker 1: You can find him on stage playing guitar for line Breakers,
Speaker 1: The line Breakers, I guess that's the line Breaker Dylan's band.
Speaker 2: Right.
Speaker 1: When he's not doing things with his three children, he
Speaker 1: builds guitars, plays chess badly in parentheses, I didn't write this,
Speaker 1: and loves tasting whiskey and who doesn't to keep moving.
Speaker 1: He loves to mountain, bike or swim. All right, Hannas,
Speaker 1: The floor is All of those things are true. Wow, yeah,
Speaker 1: And the floor is yours. I'm sorry, I'm pretty sure
Speaker 1: you wrote this.
Speaker 3: I'm just yeah, I kind I kind of sent this
Speaker 3: to Krawl in events. But hey, thanks for having me,
Speaker 3: Thanks for having show. I'm embarrassed almost that we haven't
Speaker 3: had you before at this point. Well, this is your
Speaker 3: first appearance on dot Ney Rocks, even though we've hung
Speaker 3: out at conferences and seen each other talk. Yeah many times.
Speaker 3: So welcome, Thank you, Thanks for having.
Speaker 1: The floor is yours? You're going to talk about event sourcing.
Speaker 1: I guess maybe other things, maybe other shoes. Yeah, let's
Speaker 1: see where the conversation goes. Should we start with the
Speaker 1: elevator pitch? What is event sourcing? If you've been hiding
Speaker 1: under a rock for the last fifteen.
Speaker 3: Years, okay, elevator pitch. We have been using systems like
Speaker 3: the default when we develop software systems seems to still
Speaker 3: be to use normalized databases. And basically, when you or
Speaker 3: state in a normalized database, what you're basically doing is
Speaker 3: storing the state of the system right now, and you
Speaker 3: lose all of the history that came before it. And
Speaker 3: with event sourcing, what we actually try to do is
Speaker 3: the opposite. We store all of the events that lead
Speaker 3: up to the current state, which means that if we
Speaker 3: replay the events, we can recalculate the state. If we
Speaker 3: have the state, we cannot recalculate the history because that's gone. Basically,
Speaker 3: if you switch your philosophy from storing state to storing events,
Speaker 3: you enable a whole lot of other things in your
Speaker 3: architectures as well. But the essence of event sourcing is
Speaker 3: basically that you're going to store events instead of the
Speaker 3: projected safe.
Speaker 1: You neveric going to update, You're going to add a
Speaker 1: new record with the changes. Yes, some people have asked
Speaker 1: this by adding a couple of fields to every table.
Speaker 1: You know, update user and update date. You know, insert date,
Speaker 1: insert users.
Speaker 3: Like auditing fields and stuff like.
Speaker 1: Yeah, like auditing fields, and that's only good for the
Speaker 1: last change that was made.
Speaker 2: Yeah.
Speaker 3: The whole idea for me that makes this interesting is
Speaker 3: if we look at where database normalization comes from. A
Speaker 3: lot of that was conceived when the first Squel standards
Speaker 3: were developed in the nineteen eighties. And what's relevant for
Speaker 3: that time is that your average hard drive cost a
Speaker 3: lot more than the computer that it was attached to, Right,
Speaker 3: So normalizing had a couple of benefits. First of all,
Speaker 3: it made your data consistent and you try to avoid
Speaker 3: any update conflicts or any data manipulation concurrency issues that
Speaker 3: you might have. You would also store every bit of
Speaker 3: information only once, which conserved.
Speaker 2: Disk space back back when that was important.
Speaker 3: Right back when that was important. But nowadays on your
Speaker 3: cloud build, disk space is not typically the thing that
Speaker 3: most like CPU cycles and and memory usage and bandwidth
Speaker 3: like that's that's all going to be way more expensive
Speaker 3: than than your disk space.
Speaker 2: Yeah.
Speaker 3: But what we have, I think in our industry maybe
Speaker 3: ingrained in our brains a little bit, is all the
Speaker 3: tradeoffs that we've made to normalize our databases, because it
Speaker 3: comes with a whole bunch of tradeoffs, but we have
Speaker 3: just learned to work work around, right, all of those tradeoffs.
Speaker 1: Sure, So what are those trade offs with normalized relational
Speaker 1: database that we might have forgotten about?
Speaker 3: Well, the first thing that comes to mind is you
Speaker 3: are sacrificing CPU cycles to be able to store your
Speaker 3: data in a normalized shape, because whenever you have to
Speaker 3: fetch a meaningful amount of data to do work on,
Speaker 3: you're often hitting multiple tables, which means that you're doing joints,
Speaker 3: and joints are expensive. Yeah, and the problem with doing
Speaker 3: joints in a typical relational database is you also have
Speaker 3: no way of easily scaling that compute because it's going
Speaker 3: to happen in your database engine and you cannot just
Speaker 3: like put a cluster of nodes in front of it
Speaker 3: that are going to do the joints for you. So
Speaker 3: that's that's the first thing. You're sacrificing a bunch of
Speaker 3: compute to make that happen, and you introduce locking problems
Speaker 3: because whenever you touch your record, you have to make
Speaker 3: sure that there's no race conditions and no threading problems, so.
Speaker 1: You have to use transactions in case there are problems,
Speaker 1: and then you roll it back.
Speaker 3: Yeah, so either you're going to deal with that with
Speaker 3: optimistic concurrency or with pessimistic concurrency, but at some point
Speaker 3: you are going to deal with some kind of because
Speaker 3: concurrency issues that you're going to solve in your database
Speaker 3: as well, because you're only storing that one bit of state,
Speaker 3: and everybody that touches that is going to have to
Speaker 3: touch the same record and update it or deleted or whatever.
Speaker 2: That we want to do with it.
Speaker 3: And we've taken all of that for granted, because if
Speaker 3: we look at even the modern versions of dot Net.
Speaker 3: The first stack that they brought to Core included entity framework,
Speaker 3: and I'm a huge enity framework fan. But part of
Speaker 3: why we have or rems is because we try to
Speaker 3: solve some of the issues that you have with this
Speaker 3: way of thinking, because almost never are you in a
Speaker 3: complex application are you touching just a single table. You
Speaker 3: usually need a whole graph of different related tables, a
Speaker 3: parent entity with some with some related entities that you're
Speaker 3: all fetching and then updating and then writing back and
Speaker 3: all that sort of stuff. And that's why you have
Speaker 3: ORMs to make that a lot easier to do in code.
Speaker 3: But all of that is still working around the idea
Speaker 3: that we're only storing the truth at this very moment
Speaker 3: in our relational database.
Speaker 2: And part of storing the truth is actually creating data
Speaker 2: structures that keep the truth. You know, the classic one
Speaker 2: is yes, only one address per customer, and then they move, Yeah.
Speaker 3: Like you lose the history, Like what was the state
Speaker 3: of this entity last week, last year, or whatever. We've
Speaker 3: actually lost that. And that's where the whole idea of
Speaker 3: event sourcing comes from, is that if instead of storing
Speaker 3: the resulting state, we instead store all the business events
Speaker 3: that lead up to this state becoming what it is.
Speaker 3: We can always recreate the state because the state is
Speaker 3: just very cheaply replaying the events.
Speaker 2: Right.
Speaker 3: And in event sourcing, we assume that every event that
Speaker 3: we have written to our event stream what's true at
Speaker 3: the time that it was created, so we don't have
Speaker 3: to doubt it, right, this was the truth, which means
Speaker 3: that your replay mechanism is actually going to be relatively
Speaker 3: cheap because what you usually have is you have your
Speaker 3: root entity with all of its dependencies, and you're replaying
Speaker 3: all the events on that and you don't have to
Speaker 3: doubt the event. So a lot of these methods are
Speaker 3: just going to be setting a couple of fields, a
Speaker 3: couple of properties that you're going to fill out or
Speaker 3: modify or whatever, and you can replay thousands of events
Speaker 3: in just a couple of milliseconds. But the cool thing
Speaker 3: about that is that replaying is not happening in your
Speaker 3: database engine, is right. It's happening somewhere outside of it,
Speaker 3: in your net code, which is easier to distribute.
Speaker 1: So the light bulb just went off because I was
Speaker 1: asking myself, when you're talking about the problems that it
Speaker 1: solves in this whole you know, multiple joints and everything. Well,
Speaker 1: now you're really kind of talking about flat document, right
Speaker 1: that instead of having all these joints. So is event
Speaker 1: sourcing something that you can use alongside a relational database,
Speaker 1: like your events are separate from the relational database? Or
Speaker 1: do you remove the hierarchy of a relational database when
Speaker 1: you're using events sourcing?
Speaker 3: Well, typically what an event stream looks like is an
Speaker 3: append only store. You're going to append new events to
Speaker 3: the store, and all of these events happen usually in
Speaker 3: the scope of a certain stream ID or a certain
Speaker 3: aggregate I D or whatever you want to call it,
Speaker 3: which typically like what orders, it kind of points to
Speaker 3: the root and dy that this event happened for. Right,
Speaker 3: So if you have a certain order number, like customer
Speaker 3: Richard is placing an order on my website to order
Speaker 3: a new guitar, and he's going through the whole process
Speaker 3: of specking said.
Speaker 2: Guitar, it'll be a tuba though for Richard, totally.
Speaker 3: Totally hypothetical, and we could we could basically model Richard's
Speaker 3: configuration process of the guitar in multiple events. Now, he
Speaker 3: selected the color and he did this and he did
Speaker 3: that and whatever, and every event is going to point
Speaker 3: to Richard's order, which means that if you Carl at
Speaker 3: the same time are configuring a guitar as well, those
Speaker 3: two aggregates will have nothing that has that could potentially
Speaker 3: have a side effect on one another, because those will
Speaker 3: be two separate aggregates or two separate streams, if you will.
Speaker 3: And when we append everything to the store, we have
Speaker 3: a couple of things that play into the key. One
Speaker 3: is the aggregate that event happened for, but also the
Speaker 3: sequence number. And the sequence number is what's going to
Speaker 3: help us with concurrency, because at some point you're gonna
Speaker 3: have in an event source system, you're gonna want to
Speaker 3: scale out, you want to go You're gonna want to
Speaker 3: have a way to deal with multiple commands coming in
Speaker 3: on the same aggregate because they're all gonna spawn events
Speaker 3: on the same aggregate and you don't want them to
Speaker 3: be written to the database without knowing about the other
Speaker 3: events that might have happened in the mean one. So
Speaker 3: the the the stream idea or the aggregate idea, and
Speaker 3: the sequence number that's going to be the important bit
Speaker 3: of storing new events and events. The payload is just
Speaker 3: an object something that we serialize, and it could be
Speaker 3: several it could be as deep as we want, it
Speaker 3: could be a whole object graph. It's just a single
Speaker 3: object that we have to be able to de serialize again.
Speaker 3: And that's that's basically the essence of what you need.
Speaker 3: Simplest event stream you could build.
Speaker 1: So let's say the first person to log on after midnight. Right,
Speaker 1: Let's say you're going to get all the data from
Speaker 1: your relational database, et cetera and make an event that's like, hey,
Speaker 1: this is the first event today.
Speaker 2: Is that?
Speaker 1: Is that the idea, and then all the modifications that
Speaker 1: happen to it just work with that those event streams
Speaker 1: and when you when you're doing updates and everything.
Speaker 3: No, the idea is that the idea is that if
Speaker 3: you need that root entity, instead of fetching that from
Speaker 3: the database, you were actually going to fetch all of
Speaker 3: the events that have happened on that entity and then
Speaker 3: you're going to replay them in code. Okay, and that
Speaker 3: will bring you up to speed to the point where
Speaker 3: we are. But where do we start? I mean we
Speaker 3: start with a database, right that has on a data
Speaker 3: in it. Yeah, well, you could know your database could
Speaker 3: just be a single table with all of your events
Speaker 3: in it.
Speaker 2: Oh all right, so this is what I was asking before.
Speaker 1: You get rid of your relational database in but you
Speaker 1: still have your objects in memory, right. You define your
Speaker 1: memory that you're going to turn into Jason yes in
Speaker 1: story in your event database.
Speaker 3: Whenever the live cycle of how, let's say, an API
Speaker 3: call comes in. I think that's the easiest way to
Speaker 3: reason about. It's like an API call comes in typically
Speaker 3: a command something you want your system to do, and
Speaker 3: you're going to want to execute that. Usually your commands
Speaker 3: are going to touch a certain root entity. If I'm
Speaker 3: going to update the address for Carl, the root entity
Speaker 3: is going to be Carl, right, So I'm going to
Speaker 3: have to fetch your date details, and then your current
Speaker 3: state is going to dictate what the result of the
Speaker 3: command is going to be, because what your current address
Speaker 3: is might affect what happens with this address update. So
Speaker 3: in order to get your current state, I'm going to
Speaker 3: fetch all of the events that have happened on entity Carl,
Speaker 3: replay them, get your current state, then run my command,
Speaker 3: and then append the new events to the stream.
Speaker 2: Again. Yep, I get that. I get that.
Speaker 1: I guess what I didn't get was that we're completely
Speaker 1: replacing our relational database well in this with a document
Speaker 1: centric database.
Speaker 3: For the events sourced stuff. Yes, yes, And what does
Speaker 3: happen a lot is that you will still have relational
Speaker 3: tables with some reference data and so on. There's no
Speaker 3: point in making every single entity in your entire system
Speaker 3: events sourced. I mean, you don't want to hurt yourself
Speaker 3: too much. And so if you have a table that
Speaker 3: just has a list of countries or whatever, you could
Speaker 3: make that event sourced if that is part of your
Speaker 3: core domain. Usually that kind of stuff is reference data
Speaker 3: and you can just keep that in relational tables. But
Speaker 3: when it comes to the entities where you're doing you're
Speaker 3: running your actual domain logic, you're going to basically want
Speaker 3: to get your truth from the events. That's the whole idea.
Speaker 1: Yeah, that's great, and this is a great place to
Speaker 1: pause for a break, So we'll be right back after
Speaker 1: these very important messages don't go away. Did you know
Speaker 1: you can easily migrate asp net web apps to Windows
Speaker 1: containers on AWS. Use the app to Container tool to
Speaker 1: container eye is your iis websites and deployed to AWS
Speaker 1: managed container services with or without Kubernetes. Find out more
Speaker 1: about app to container at aws dot Amazon dot Com, slash,
Speaker 1: dot net, slash modernize, and we're back. It's dot net Rocks.
Speaker 1: I'm Carl Franklin, an amateur Campbell, and we're talking to
Speaker 1: Hanness Lowett about event sourcing.
Speaker 2: You know, I'm reminded that the relational Databases developer reporting
Speaker 2: and it got sold on transactional velocity because that was
Speaker 2: easier to sell. But there is a there's a case
Speaker 2: to be made for after an event is logged, if
Speaker 2: you think it's reportable or it's a you know, falls
Speaker 2: in a certain class of event, then you do decompose
Speaker 2: it asynchronously, not making the customer way into relational data
Speaker 2: tables for reporting purposes, because.
Speaker 3: That's what we call projections. Yeah, and so what you
Speaker 3: basically do if you think about your c qres architecture,
Speaker 3: Let's say that you have a comment side of your
Speaker 3: system where all of the business logics is triggered when
Speaker 3: other systems or users Q commands, and you have your
Speaker 3: red side of the system where you're querrying data. It
Speaker 3: makes sense to prepare some data for querying because if
Speaker 3: you regularly need to pull a list of all of
Speaker 3: your customers with their current addresses. That's going to be
Speaker 3: very expensive to run if you have to project it
Speaker 3: from the source events, because what you're basically going to
Speaker 3: have to do is get all of your users and
Speaker 3: project them all one by one until you have the
Speaker 3: list of addresses. As way too expensive. So basically, whenever
Speaker 3: we queue, whenever we save an address changed event, that's
Speaker 3: going to update that table. So we have little bit
Speaker 3: of bits of business logic that are going to be
Speaker 3: running and listening to certain events and then updating projected tables. Now,
Speaker 3: the big advantage of doing that is that also that
Speaker 3: is relatively inexpensive. You can look at the contents of
Speaker 3: your event and you know, okay, it's this entity. This
Speaker 3: is talking about Carl, and Carl has new address. So
Speaker 3: all I have to do is without really thinking, because
Speaker 3: I don't have to check the validity of the event
Speaker 3: that has happened when we executed command, so I don't
Speaker 3: have to recheck that. I can just take that and
Speaker 3: then update the view or at the table that we
Speaker 3: are going to querry on the query side of our system,
Speaker 3: and we can keep that in sync. And the cool
Speaker 3: thing about these projections is if we want, we can
Speaker 3: rebuild them so whenever the logic to go from source
Speaker 3: events to whatever projected data we want to have. Whenever
Speaker 3: that changes, we just throw away the table and repopulate it.
Speaker 3: We replay the stream from the beginning of time and like,
Speaker 3: if you have a system with millions and millions of events, like,
Speaker 3: some of these projections might take a while to rebuild,
Speaker 3: so there are techniques to do that at a synchroncy
Speaker 3: before you replace the table, that sort of stuff. But
Speaker 3: you can basically get an updated view of your query
Speaker 3: data by just rerunning the events and replaying them because
Speaker 3: you still have everything that has happened in the system
Speaker 3: ever since.
Speaker 2: Yeah, so it means you could always regenerate any aggregate
Speaker 2: resulting data set one or the other at least validate
Speaker 2: whether it's correct or not. But my concern is as
Speaker 2: your number of events starts to build up, finding out
Speaker 2: how many orders did we have last month can get costly.
Speaker 3: Well, not if you have a projection that updates that
Speaker 3: view on the floor right right. If your projection is
Speaker 3: caught up and you're not changing the way that that
Speaker 3: projection works. Theory, you should not have to rebuild it.
Speaker 2: So it should always be correct.
Speaker 3: It should always be correct, So you can you can
Speaker 3: flip this around. You could have a projection for every
Speaker 3: query endpoint on your querry API that hits a single
Speaker 3: table on an index, and you can have that table
Speaker 3: populated from your event stream with a dedicated projection just
Speaker 3: for that table.
Speaker 2: But I appreciate that we're both saying should because things
Speaker 2: do go wrong, but can also be fixed.
Speaker 3: Of course, it can be fixed, and that's that's the
Speaker 3: big advantage. Because you have the whole history, you you
Speaker 3: can rebuild anything. And in theory, like one of the
Speaker 3: examples that's often used is okay, you can you can
Speaker 3: generate new insights from past events. And in theory that's
Speaker 3: true if you have saved all of the events, you
Speaker 3: can basically generate new insights, new features, even based on
Speaker 3: events that have happened in the past, and you can
Speaker 3: repopulate the view by just letting that projection run until
Speaker 3: it's caughta In practice, like often when you're extending a
Speaker 3: feature set of an application, you're also introducing new events
Speaker 3: and you will not taken those into account in the past,
Speaker 3: so it that that claim has a limited validity. In practice,
Speaker 3: but it's a cool selling point, right.
Speaker 1: Yeah. So you've been talking about some of the problems
Speaker 1: that have event sourcing solves. Are there any new challenges
Speaker 1: with event sourcing that come into plug.
Speaker 3: Oh, definitely, some of some of the challenges that, weirdly,
Speaker 3: you don't have with a normalized database because if you
Speaker 3: normalize your database properly, every bit of information is one
Speaker 3: linked to the primary key of the table that it's in,
Speaker 3: and it's only stored in one bit. So if you
Speaker 3: do it right, you should end up with a pretty
Speaker 3: deterministic view of what your data looks like and the
Speaker 3: way that you query it and update it on whatever.
Speaker 2: Like.
Speaker 3: All of that is coding that happens afterward, so you
Speaker 3: don't have to think about that too much. As soon
Speaker 3: as you start applying event sourcing, partitioning your system based
Speaker 3: on the root entity that the commands are going to
Speaker 3: happen in basically determining the boundaries of your aggregates, that
Speaker 3: becomes very important. But it's a problem that you also
Speaker 3: see when you use a document database. For instance, if
Speaker 3: you switched over to something like the dB or AVNDB
Speaker 3: or whatever, you're going to see the same problem because
Speaker 3: you have to really think about which bit of my
Speaker 3: data belongs together and is a sensible scope to process commands.
Speaker 3: Because as soon as you have to start working with
Speaker 3: multiple documents and multiple streams at the same time, that's
Speaker 3: when things get tricky, right, But.
Speaker 1: Generally you know what those things are, right, I mean,
Speaker 1: you generally have to find the tip of the iceberg
Speaker 1: and then you know all the related data goes in
Speaker 1: there as well. So isn't that's something that you would
Speaker 1: probably know if you're familiar with the data shape.
Speaker 3: If you're familiar with the data shape, yes, But fromans,
Speaker 3: I feel like most developers that come from the crut
Speaker 3: normalized database world they don't think that way and they've
Speaker 3: never actually wondered like, which are my root entities?
Speaker 2: Where does the stuff happen?
Speaker 3: And as soon as you start mapping out business processes
Speaker 3: and you start thinking about what is happening here in
Speaker 3: my business? When does a user or an external system
Speaker 3: q a command? What is happening when that command? Which
Speaker 3: entity does that command hit? That's when all of that
Speaker 3: becomes very clear. So if you practice things like domain
Speaker 3: driven design and you start using things like even storming
Speaker 3: to map out your business processes. With post its on
Speaker 3: a wall, you end up with not only a very
Speaker 3: clear view of how you're going to partition that system,
Speaker 3: but you end up with post its that you can
Speaker 3: one on one basically convert into class in code, because
Speaker 3: those will become your commands and your events and all
Speaker 3: of that. And of course there's there's work to do,
Speaker 3: but at.
Speaker 2: Least the the.
Speaker 3: Amount of information that can get lost when you try
Speaker 3: to translate that from that wideboard full of post its
Speaker 3: into code is a lot closer than it would be
Speaker 3: if you're using a relational database. It also will have
Speaker 3: way less side effects.
Speaker 1: So if you have if you create your what you
Speaker 1: think is your hierarchy and your your your shape and
Speaker 1: your root unit down and you start saving some events
Speaker 1: and all that stuff, and then you realize, oh, I
Speaker 1: need this entity in here as well. Does that change
Speaker 1: anything or do those past events become invalid or in
Speaker 1: the new event you've just added this entity.
Speaker 3: And not really like there are definitely scenarios where you
Speaker 3: are going to touch multiple multiple streams as a result
Speaker 3: of a single command. Think about your shopping car, for instance,
Speaker 3: at some point you're also going to touch the stock system,
Speaker 3: so that might be another stream that lives in the
Speaker 3: same application. That might be an external system that needs
Speaker 3: to get called, right, so.
Speaker 1: You're not trying to make one stream that rules them all.
Speaker 1: This is what you're talking about boundaries having where do
Speaker 1: you separate those things exactly?
Speaker 3: And you have to figure out where the boundaries are
Speaker 3: and cross them with a very conscious state of mind.
Speaker 3: I have to worry about where am I leaving, like
Speaker 3: the thing that I can do on a single stream,
Speaker 3: and there has to be a very good reason for that. Like,
Speaker 3: if all of your commands are doing that, then you
Speaker 3: model something really wrong.
Speaker 2: Yeah, I think single stream for everything is bad too,
Speaker 2: but I'm sure you end up with arguments about too
Speaker 2: many streams.
Speaker 3: So the like, there's a couple of patterns that you
Speaker 3: can use. For instance, like the saga pattern is very
Speaker 3: well known. If you have a command that comes into
Speaker 3: your system that's going to touch multiple streams or even
Speaker 3: multiple systems if you have to make external calls, what
Speaker 3: you typically have is a quite long running transaction that
Speaker 3: is going to be happening as soon as that command
Speaker 3: comes in, right, So you're going to save some intermediary
Speaker 3: states and let the whole thing play out because with
Speaker 3: external systems, it might be an outgoing call and then
Speaker 3: a callback coming in and whatever you're going to have
Speaker 3: to be able to pick up the threat. That's a
Speaker 3: perfect way to model that, and it's not that dissimilar
Speaker 3: from how you model more complex transactions across multiple streams
Speaker 3: and inside the system. Now if you use frameworks, because
Speaker 3: a lot of it depends on what is backing your
Speaker 3: what is backing your event sourcing framework, Like you can
Speaker 3: roll your own. A lot of it is not that
Speaker 3: complex and you needed probably about a day to to
Speaker 3: roll something that can do a lot of the things
Speaker 3: if you want to have something battle tested and let
Speaker 3: somebody else think about hard work. You've had Jeremy on
Speaker 3: the show and we talked about Martin his Martin framework
Speaker 3: is well, it's not is it a framework or a
Speaker 3: document database? It's an event store right right? What Martin
Speaker 3: calls it a web framework. Yeah, well that's that's if
Speaker 3: you involve Wolverine and and and all the other things
Speaker 3: of the Critter, Critter Stack, the creator. But Martin Martin
Speaker 3: is basically I don't know if that's my story to tell.
Speaker 3: I'm sure Jeremy told it on the show.
Speaker 2: As well, Like he's been on a few times.
Speaker 3: They they ran into trouble with with raven dB, if
Speaker 3: I remember correctly, and they needed something capable of handling
Speaker 3: the load, and they looked at postcrests. It's like, cool,
Speaker 3: Postcrest can do a lot of stuff that we need
Speaker 3: when it comes to the document stuff, but the projection
Speaker 3: stuff that's in raven like not so much. And that's
Speaker 3: why they ended up building a lot of what Martin
Speaker 3: eventually became. And because that had documents in there and
Speaker 3: had projections in there like, it became a very nice
Speaker 3: candidate for building an event source system as well, because
Speaker 3: if you have a stream with all your documents that
Speaker 3: have all of your events that have happened in the past,
Speaker 3: and you can run projections on that, then you're already
Speaker 3: like eighty percent of the way there to have an
Speaker 3: event source system where you have your event stream and
Speaker 3: you have your projections that project into other documents or
Speaker 3: other tables or whatever, which is why Martin now has
Speaker 3: a very explicit support for event sourcing, and you get
Speaker 3: an out of the box event stream that you can
Speaker 3: pentax to and it supports all the things that you
Speaker 3: will run run into at first when you start growing
Speaker 3: one of these systems, because one of the typical problems
Speaker 3: is you have a certain aggregate that becomes very long
Speaker 3: lived and it starts accumulating not thousands, but millions of
Speaker 3: events that becomes very expensive to replay. So at some
Speaker 3: point you're going to want to start snapshotting that. Right,
Speaker 3: We're going to take a snapshot at events one million,
Speaker 3: two hundred and sixty four so that we only have
Speaker 3: to replay the last fifty, right, and all of that
Speaker 3: functionality is already built in. Somebody has thought about that way,
Speaker 3: about those things way harder than most of us ever will,
Speaker 3: And I think that's a very valid argument to start
Speaker 3: using one of those ready made frameworks to do that
Speaker 3: kind of stuff. But you do have to understand what
Speaker 3: goes on behind the scenes, because if you register a
Speaker 3: projection in there, that's going to spin off a background
Speaker 3: worker that's actually going to be querrying your database and
Speaker 3: running code and writing back to the database. So none
Speaker 3: of that's it happens magically, but it's not.
Speaker 2: Free, right. Yeah. I was just going back and look
Speaker 2: at the Jerry Miller shows and realized the entire thing
Speaker 2: that you've just told we've done as a series of
Speaker 2: shows with him about Martin on Postgrass, then playing with
Speaker 2: events sourcing, and then Wolverine, and it's literally working through
Speaker 2: the problems you're describing as this works up to this
Speaker 2: point and then you need to go over here and
Speaker 2: like it's just it's.
Speaker 3: Yeah, Wolverine is very nice SAGA support, right, So if
Speaker 3: you pair that with Martin, that works really well.
Speaker 2: Now we're back in the critter stack exactly. There's a
Speaker 2: reason that we have all these different critters.
Speaker 3: Yeah, But like I think at this moment in the
Speaker 3: dot net space, Martin is probably one of the one
Speaker 3: of the best ways to get started with events sourcing
Speaker 3: because you get a lot of stuff out of the
Speaker 3: box and the complexity of the setup for you as
Speaker 3: the person developing the system, the complexity is relative fully
Speaker 3: low because what you need is a dot net process
Speaker 3: that connects to a postcress database, right, and if you
Speaker 3: have multiple nodes doing that, they will actually communicate through
Speaker 3: postcress locks, so you don't need anything out of band
Speaker 3: for your notes to make sure that they're not interfering
Speaker 3: with one another.
Speaker 2: What makes postcrests special in this situation, Like I I
Speaker 2: already have SQL server why wouldn't I use it basin
Speaker 2: binary chasin binary Jason.
Speaker 1: Okay, no, well SQL server has adjacent table field type now, yeah,
Speaker 1: but it's more expensive.
Speaker 3: Yeah, but for a long time they didn't. And postcress
Speaker 3: is is still a lot more powerful than what the
Speaker 3: server is trying.
Speaker 2: I also like the licensing on postgrass, which is to
Speaker 2: say free. Yeah, but.
Speaker 3: Postcress is ridiculously powerful and it does a lot of
Speaker 3: things very well. And Basin is basically a way that
Speaker 3: they figured out to store chasing documents in a binary
Speaker 3: format in a way that they could still index on
Speaker 3: fields that are pretty deep in the document's tree, which
Speaker 3: is cool. Added benefit is this works in an acid
Speaker 3: transactional database, which Mongo. In Mongo, for instance, you have
Speaker 3: transactions on document level, but not across multiple documents. And
Speaker 3: with postgress, that's a thing that's there by default because
Speaker 3: by default it's an ASTE compliant relational database. And because
Speaker 3: basin is so powerful, like in a lot of scenarios,
Speaker 3: it will outperform a lot of other document databases while
Speaker 3: basically being a relational database. And I get the irony, like,
Speaker 3: now we're going to use a relational database to store
Speaker 3: our events stream and our documents.
Speaker 1: But you know, but at the same time you get
Speaker 1: the relational stuff you need that too, So.
Speaker 3: Yes, and and that's one of the cool things. If
Speaker 3: you want to project your events into actual sequel tables,
Speaker 3: like you can do that in the same transactions. You
Speaker 3: will be able to run if you want. You can
Speaker 3: have some of your projections transactionally consistent, meaning that they
Speaker 3: will project when you're saving the event. Like that's the
Speaker 3: hard stuff, like figuring figuring all of that out in
Speaker 3: a framework that you're going to roll yourself is going
Speaker 3: to cost so much time. And that's when Martin, for instance,
Speaker 3: shines because it has.
Speaker 2: All that building.
Speaker 3: But Basin is a lot of the reason why that
Speaker 3: works with postgress.
Speaker 1: Do you think agentic coding with LLLMS is going to
Speaker 1: help or hurt the efforts of events sourcing?
Speaker 3: I think, But of course i'm comfort in this. I
Speaker 3: might be prejudiced because a lot of the patterns that
Speaker 3: come from using event sourcing they're relatively side effect free.
Speaker 3: Like your projection is going to touch the table that
Speaker 3: it's updating, but it's not going to touch anything else.
Speaker 3: It reads from the statement writes to a single table,
Speaker 3: and the same with your commands, like you're fetching a
Speaker 3: bunch of events and then you're appending to the stream.
Speaker 3: You're not updating records, you're not walking records. Like a
Speaker 3: lot of that is actually easy to performance tune down
Speaker 3: the line because that code is relatively side effect free.
Speaker 3: The patterns that are used in event sourcing would actually
Speaker 3: be very very adequate for generating code, because if you're
Speaker 3: trying to explain to an agent like, hey, this is
Speaker 3: what my business process looks like, these are the commands,
Speaker 3: and then this happens. Basically the way that we think
Speaker 3: out the world, right, if we figure out our drink
Speaker 3: order at the bar, it's a conversation, it goes back
Speaker 3: and forth, and it's the way our world works, is
Speaker 3: the way that software systems work, and especially event source systems.
Speaker 3: And I think that because there is less of a
Speaker 3: translation happening between the business problem and the actual code,
Speaker 3: I think this will actually lend itself very well to
Speaker 3: getting generated systems from from agents or whatever. Of course,
Speaker 3: we don't have to train them on code basis that
Speaker 3: deal with normalize databases, but that's not a hard problem
Speaker 3: to solve, right if you're training LLLMS.
Speaker 2: I haven't done any of this testing yet, but it
Speaker 2: makes a lot of sense to see could we point
Speaker 2: an LLM at an existing SQRES implementation, say, is this
Speaker 2: a well implemented event source pattern application? You know, because
Speaker 2: the first thing I want to do is be able
Speaker 2: to test. Is they already doing this right before I
Speaker 2: even ask you to generate any code? Sure? Yeah, I've
Speaker 2: not tried, no, but this is an interesting idea to say,
Speaker 2: could these tools be really effective at this pattern based
Speaker 2: development approach? Yeah?
Speaker 3: Curious est part of the reason that this is gaining
Speaker 3: a lot of traction in the DDD world is because
Speaker 3: of the lack of translation from the business concepts to
Speaker 3: the actual classes that you're going to see for your events.
Speaker 2: I like the attitude, and I thought I've said this
Speaker 2: before on the show of Just Store the truth. Order
Speaker 2: comes in store the order. What you do with it
Speaker 2: after that is secondary the point. But the fact that
Speaker 2: you have a record of what happened, a complete as
Speaker 2: it was at the moment. I think it's really powerful.
Speaker 2: And the only price there is disk space, and this
Speaker 2: space is trivial.
Speaker 3: It's cheap, especially if it's structured data like chasing documents.
Speaker 1: Yeah, it's very easy, and I imagine the base on
Speaker 1: data takes up way less dispace. Then oh yeah, ASKI
Speaker 1: chasonski chasing.
Speaker 3: But even asky chasin isn't that bad.
Speaker 2: It's not yah, and this base is not the issue.
Speaker 2: I mean parsing all that after the fact, you can
Speaker 2: you can discuss from there, But first and foremost, it's
Speaker 2: just a good idea to keep a record of the truth.
Speaker 3: You know, you can don't be mistaken like you're not
Speaker 3: getting auditing information like out of the box, because that's
Speaker 3: a common misconception about events sourcing. If you want to
Speaker 3: have your full audit trail of what happened for legal
Speaker 3: reasons or whatever, you're gonna need to enrich that data
Speaker 3: a little bit because who is important, Like who made
Speaker 3: the command that came in? What was the command that
Speaker 3: came in, what events resulted from that, what changes it
Speaker 3: a like due to the entity? Like the last part
Speaker 3: is what you're getting for free with your event stream,
Speaker 3: but you need to also lock your commands and add
Speaker 3: some metadata like who did this and when did they do?
Speaker 2: Is the only thing you're getting for sure is the results,
Speaker 2: But to keep the detail of how you got to
Speaker 2: this result, that's up to you exactly where does this struggle?
Speaker 2: Like where do where do you get to? Where does
Speaker 2: this get into trouble?
Speaker 3: Oh like GDPR becomes tricky?
Speaker 2: Oh yeah, oh interesting, mostly protecting privably identifiable information.
Speaker 3: Yeah, because you you in a way, you want to
Speaker 3: have everything that happened in your system in your in
Speaker 3: your streams. And it's very easy, like if Carl asks
Speaker 3: to have his information deleted, like we're is going to
Speaker 3: delete his streams from our events stream, but that might
Speaker 3: affect the next time that I'm running.
Speaker 2: Check the stuff out. Yeah, well, and he's not. He's
Speaker 2: not asking to delete the sales record. The sales record
Speaker 2: is the truth. It's the identifiable information. They identifiable information.
Speaker 2: And then you come into situations where there's certain information
Speaker 2: like passwords for instance, you're not gonna want in your
Speaker 2: events stream, but they still need to go somewhere, right, Yeah,
Speaker 2: so you're you might end up masking some data in
Speaker 2: the event stream that you're gonna store somewhere else or
Speaker 2: deal with elsewhere. Right. I don't know if it's compliant,
Speaker 2: but my instinct there is I never modify the stream,
Speaker 2: but I do flag we don't have access to this
Speaker 2: person's information.
Speaker 1: Anymore, so you never show it. Well, but is that
Speaker 1: the same as deleting it in in in.
Speaker 3: Like I think the GDPR legislation forces you can actually
Speaker 3: delete ith right, you can no longer have it on file.
Speaker 2: That's what I thought too. Yeah, so do you have
Speaker 2: to go back through all of your historical records that
Speaker 2: were burned a DVD and delete those two? That's I
Speaker 2: love compliance. Compliance is the best, but that's that's.
Speaker 3: One of those problems where events sourcing gets tricky.
Speaker 2: For instance, Yeah, this idea of keeping a record of
Speaker 2: the truth.
Speaker 3: Now to be honest, like any any system here, if
Speaker 3: you start thinking about.
Speaker 2: That, Yeah, and my niceing innolational database is I can
Speaker 2: this customer wants to be forgotten and so we erase
Speaker 2: their identifal inform infusiation from the record? I think I
Speaker 2: still keep the pointer record, like just the ID, but
Speaker 2: none of their information. Isn't that ID anymore? And that
Speaker 2: that also.
Speaker 3: Begs the question like, okay, we still want to track
Speaker 3: Carl as one of the people that registered for our website,
Speaker 3: like how many users sign ups did we have? Right?
Speaker 3: And if we drop his stream like that event might
Speaker 3: be gone. So then you have to start thinking about Okay,
Speaker 3: if we want to model that particular bit of information
Speaker 3: like new users signed up, we're going to have to
Speaker 3: have a user signed up event in another aggregate that
Speaker 3: does not get deleted when Carl's stream is removed from
Speaker 3: the system, so that that information is still still accurate.
Speaker 3: Like those things becomes more intentional.
Speaker 1: Do you leave the name and erase everything else or
Speaker 1: replace it with not available or scrubbed or something like
Speaker 1: redacted or something like that.
Speaker 3: There's there's different different approaches that you can take. Yeah,
Speaker 3: but very very most Margin for instance, allows you to
Speaker 3: archive certain streams, which means that like this aggregate is
Speaker 3: never going to get fetched back again.
Speaker 2: And then.
Speaker 3: In theory, you could just delete that from the database.
Speaker 3: You could go through you could execute a simple command
Speaker 3: saying like every every event where we have the archived
Speaker 3: flag set to TROW, we can delete that. But then
Speaker 3: you have to really worry about like, okay, which projections
Speaker 3: might that affect and is that a bad thing? Is
Speaker 3: that a problem?
Speaker 2: Okay?
Speaker 3: If this, and that's also a reason why you might
Speaker 3: want to rebuild stuff, because I might delete the data
Speaker 3: from the stream, I might delete defense, but the data
Speaker 3: might still be in the projected in the projection tables.
Speaker 3: So that's one of the reasons to periodically rerun or update,
Speaker 3: like rebuild certain protections.
Speaker 2: And you got to get I'm so low to the
Speaker 2: leade data just because you know what you are, you
Speaker 2: sure you're deleting the right things, Like I can also
Speaker 2: get in trouble with the government for not reporting sales
Speaker 2: because I had to forget this person and forgot their sales.
Speaker 1: Yeah, well that's true. Got to keep it around at
Speaker 1: least until you pay the sales.
Speaker 2: Tax or whatever that. Yeah, and probably have to keep
Speaker 2: it around forever you can. You know, you get audited,
Speaker 2: you still have to make sure your sales are correct. Yeah,
Speaker 2: there's a in the United States, there's like a ten
Speaker 2: year window or something after which you can delete anything
Speaker 2: you want. Yeah, but even then you I mean, it's
Speaker 2: smart to keep track of all of your sales, who
Speaker 2: you sold it to. That's identifiable information. You can delete that,
Speaker 2: yeah yeah, yeah, yeah. I just suddenly have this vision
Speaker 2: of you have these streams marked GDPR redacted user yeah
Speaker 2: number you know eleven, that kind of thing.
Speaker 3: If certain events, for instance, contain identifiable information, you can
Speaker 3: mask that. You can say, like this field and that
Speaker 3: type of event. I don't want it to be safe
Speaker 3: to the stream.
Speaker 1: And then you just but if the federalies say you
Speaker 1: have to delete it, you have to delete it.
Speaker 2: Yeah, answers the question of what are you deleting, right
Speaker 2: because at the same time I have another federale saying
Speaker 2: you can't delete sales records. Yea, I will send you
Speaker 2: to prison for hiding income.
Speaker 3: Another thing that becomes tricky over time is versioning of events.
Speaker 3: Events may evolve over time and become.
Speaker 2: Sure new version of the sotom more elaborate or more
Speaker 2: complex or simpler.
Speaker 3: A certain command might certainly spawn two events instead of one.
Speaker 3: Suddenly you have as long as your objects are backwards
Speaker 3: compatible to whatever is saved in or data store, like,
Speaker 3: that's not a problem. But at some time you're going
Speaker 3: to break that.
Speaker 2: Yeah, you a replayable state somehow. And the easiest thing
Speaker 2: is is to.
Speaker 3: To just have the two versions in your code. Just
Speaker 3: have two versions of that event in two different namespaces
Speaker 3: or with two different class names or whatever. And that
Speaker 3: works fine.
Speaker 2: But that because how sweet do you think we're only
Speaker 2: going to have two?
Speaker 3: That becomes a mess over time, And then you start,
Speaker 3: Then you start practicing things like upcasting or but that's
Speaker 3: that's something that I tried to steer away from as
Speaker 3: long as I can, and ideally never do, is start
Speaker 3: updating the table and upcasting the events and resaving them
Speaker 3: too ideally new streams. But that's a tricky mechanism as well,
Speaker 3: because your new stream will have new eyelins to it.
Speaker 3: It's now incorrect, and it escalates, it escalates.
Speaker 1: I think when I hear you're saying, is higher harness
Speaker 1: if you want to tackle these problems?
Speaker 3: Yeah, well yeah, And there's there's a bunch of other
Speaker 3: people out there that do great work in this space
Speaker 3: as well. Like if you think about Oscar do for instance,
Speaker 3: you've met Oscar right. Oscar does some great stuff with
Speaker 3: events sourcing, and the whole d d D community seems
Speaker 3: to have adopted this as their pattern of choice. So
Speaker 3: there's many many people out there. But like think before
Speaker 3: you actually do this. And this works very well if
Speaker 3: you have complex domains that need to be modeled in
Speaker 3: code where you have high concurrency, for instance, because it's
Speaker 3: very good at dealing with that very explicit business logic.
Speaker 3: You don't want to have too many side effects. That's
Speaker 3: a very good reason to choose for this, But in general,
Speaker 3: as soon as you become proficient at this, you start
Speaker 3: using it for a lot more things because it's.
Speaker 2: Just so easy. Yeah, it's definitely. It sounds like a
Speaker 2: pattern that once you get the groove, you see places
Speaker 2: to use it. But it's not everything.
Speaker 3: It's just you have to flip the switch in your
Speaker 3: brain and for me, that live cycle of command comes in.
Speaker 3: We replay events, then we execute our business logic and
Speaker 3: that spawns new events which we now append to the
Speaker 3: stream again. As soon as that clicks in your head
Speaker 3: like it's no, it's not harder or easier than working
Speaker 3: with normal entities that you're pulling through entity framework from
Speaker 3: your r M and then calling safe changes. I mean,
Speaker 3: it's it's it's a trivial, a trivial amount of thinking
Speaker 3: to fit that in.
Speaker 2: Well, honess what's next for you? What are you doing?
Speaker 2: What's in your inbox?
Speaker 3: I'm leaving on holiday tomorrow or where you're thinking about
Speaker 3: what's next for me professionally? Either you know where you're
Speaker 3: going so European, I'm going I'm going to the to
Speaker 3: the Dutch coast with my family. We're going to go
Speaker 3: to a little house that's actually on the beach, much
Speaker 3: like the way that Richard lives, but not as fans.
Speaker 2: Wow, no fancy, my place has been Okay, that's pretty we.
Speaker 3: We will have neighbors for instance, that we can see.
Speaker 2: The family.
Speaker 3: The family is going to to the ghost and and
Speaker 3: I'm gonna recuperate for a little bit because I've had
Speaker 3: a rough year, like renovating a house and and becoming
Speaker 3: an independent contractor for half of my time and making
Speaker 3: courses and all that sort of stuff. And what's next
Speaker 3: professionally what I'm hoping to do in the fall, because
Speaker 3: I've just released two courses on events sourcing on the
Speaker 3: Home Train and the second one. There's like two more
Speaker 3: chapters that I really want to make that aren't there yet,
Speaker 3: So that might be a thing that I'm that I'm
Speaker 3: going to do in the fall. Great. Yeah, while we're
Speaker 3: looking and it's looking like conference season is going to
Speaker 3: be busy from November to beginning December, so I'll probably
Speaker 3: run into you guys a couple of times somewhere somewhere.
Speaker 2: All right, Well, that man, this is great stuff. Thanks Hanas. Yeah,
Speaker 2: great talking to you.
Speaker 3: I was happy to be on the show and thanks
Speaker 3: for letting me rand a little bit about this thing
Speaker 3: that I'm really passionate about.
Speaker 2: Absolutely crazy explanation. Thanks so much, very clear, all right,
Speaker 2: and we'll talk to you next time on dot net rocks.
Speaker 4: Dot net Rocks is brought to you by Franklin's Net
Speaker 4: and produced by Pop Studios, a full service audio, video
Speaker 4: and post production facility located physically in New London, Connecticut,
Speaker 4: and of course in the cloud online at pwop dot com.
Speaker 4: Visit our website at d O T N E t
Speaker 4: R O c k S dot com for RSS feeds, downloads,
Speaker 4: mobile apps, comments, and access to the full archives going
Speaker 4: back to show number one, recorded in September two thousand
Speaker 4: and two.
Speaker 2: And make sure you.
Speaker 4: Check out our sponsors. They keep us in business. Now
Speaker 4: go write some code, See you next time.
Speaker 3: You got jam Vans
Speaker 2: And