Speaker 1: How'd you like to listen to. NET Rocks with no ads? Easy.
Speaker 1: Become a patron. For just $ 5 a month, you get
Speaker 1: access to a private RSS feed where all the shows
Speaker 1: have no ads. $ 20 a month will get you that
Speaker 1: and a special. NET Rocks patron mug. Sign up now
Speaker 1: at patreon.dotnetrocks.com. Hey, and welcome back to. NET Rocks. I'm
Speaker 1: Carl Franklin.
Speaker 2: And I'm Richard Cavill.
Speaker 1: We have been doing this stupid show for 2019 episodes.
Speaker 1: What is wrong with us?
Speaker 2: I don't know. It's a pattern. And this, you know, yeah,
Speaker 2: can kind of anticipate that sometime next week might be
Speaker 2: another one.
Speaker 1: You know, I say it's a stupid show. I got
Speaker 1: to admit, it got a lot smarter after episode 100.
Speaker 2: I don't know why, but I don't know why. It
Speaker 2: was really stupid. Before that. There was a period. I
Speaker 2: remember drawing you out a Venn diagram. It's like, here
Speaker 2: are the people who like a technical interview. Here are
Speaker 2: the people that like music and comedy and so forth.
Speaker 2: And the intersection between the two is small. Very small.
Speaker 3: Yeah.
Speaker 2: Okay. So, we made two shows.
Speaker 1: Yeah. Spoiler alert. We are fixing to make a new Mondays.
Speaker 2: Oh, don't give it away, man. I just did. It
Speaker 2: might even be out by the time this thing publishes,
Speaker 2: but who knows?
Speaker 4: Maybe.
Speaker 2: Maybe.
Speaker 4: Who knows?
Speaker 2: All right.
Speaker 1: Well, this being episode 2019, we are going to talk
Speaker 1: about what happened in 2019. And by the way, Jeffrey
Speaker 1: Sachs is with us, and I just want to invite
Speaker 1: Jeffrey to jump in if he wants to during this whole.
Speaker 2: He probably remembers 2019, too. It was not that long ago.
Speaker 2: It was not that long ago.
Speaker 1: All right. So, believe it or not, I have a
Speaker 1: few good news stories. For 2019. Nice.
Speaker 2: You get tired of being the harbinger of doom, do you?
Speaker 2: Not your style at all.
Speaker 1: I am. Yeah, especially last 2018 was just dark.
Speaker 2: Just depressing. Yeah.
Speaker 1: So I will have some bad news too, but for
Speaker 1: some good. The world's first malaria vaccine began rolling out
Speaker 1: in Africa.
Speaker 2: Exciting.
Speaker 1: Malawi, Ghana, and Kenya began vaccinating children with RTSS, the
Speaker 1: first malaria vaccine used in a large-scale public health program.
Speaker 1: Our friend Bill Gates and Melinda Gates were behind that.
Speaker 1: I don't know if they're actually our friends, but we've
Speaker 1: seen them. Yeah, I met them. About 360,000 children per
Speaker 1: year were expected to receive it during the pilot. So cool.
Speaker 1: Ethiopia and Eritrea's peace efforts received the Nobel Peace Prize.
Speaker 1: Very good.
Speaker 2: Those two have been at loggerheads since their formations.
Speaker 3: Yeah.
Speaker 1: And see, it is possible. Nearly 100 captive whales and
Speaker 1: orcas were returned to the wild. Russian authorities gradually released
Speaker 1: animals from the notorious whale jail in the country's far east.
Speaker 1: By November, the remaining 50 beluga whales had been freed,
Speaker 1: completing the operation.
Speaker 2: Wow.
Speaker 1: Reminds me of Star Trek IV. Algeria and Argentina eliminated malaria.
Speaker 2: Eliminated. How cool is that? Well, there's a bunch of
Speaker 2: ways to tackle that, right? Like you can do it
Speaker 2: by just controlling mosquitoes. Yeah. But also, you know, handing
Speaker 2: out nets. Like there's all kinds of tools to try.
Speaker 2: And it all works together. Ethiopia launched a massive tree
Speaker 2: planting campaign. Millions participated in the country's green legacy reforestation effort.
Speaker 2: The Ethiopian government reported that roughly 350 million seedlings were
Speaker 2: planted in a single day in July, part of a
Speaker 2: campaign aimed at planting billions of trees. The exact single-day
Speaker 2: figure wasn't independently verified, but the enormous public participation was real. So,
Speaker 2: now let's get to the depressing stuff. Well, maybe not depressing,
Speaker 2: but you know. Donald Trump is impeached. For the first time.
Speaker 1: Depressing? It's up to you, I think. Hong Kong pro-democracy protests.
Speaker 2: Do not go well.
Speaker 1: What began as... What's that?
Speaker 2: Do not go well.
Speaker 3: No.
Speaker 1: What began as opposition to an extradition bill grew into
Speaker 1: enormous demonstrations demanding democratic reforms and resistance to Beijing's increasing influence.
Speaker 1: The protests continued for months and frequently resulted in violent
Speaker 1: clashes with the police. Brexit crisis and Boris Johnson's rise. Richard,
Speaker 1: do you remember we were sitting in a restaurant somewhere
Speaker 1: and you looked over and you said, hey, that's Boris Johnson.
Speaker 1: Do you remember that? It was in London.
Speaker 3: Yeah.
Speaker 1: We were at NDC London.
Speaker 5: Yeah.
Speaker 1: And you're like, that's Boris Johnson. And I said.
Speaker 2: He was in the restaurant.
Speaker 1: I said, who? Because I didn't know.
Speaker 2: He was the mayor of London one point. And he
Speaker 2: was kind of funny when he was the mayor of London.
Speaker 2: The quirky guy.
Speaker 1: I think this was before his further rise, so I
Speaker 1: really didn't know who he was.
Speaker 2: Before he was prime minister. Yeah.
Speaker 1: But, you know, we did see him. Theresa May repeatedly
Speaker 1: failed to get her Brexit agreement through parliament and resigned.
Speaker 1: Boris Johnson became prime minister, negotiated a revised deal, and
Speaker 1: then won a huge conservative majority in December. Christchurch mosque massacres.
Speaker 1: Bad news. This was in New Zealand, your home country.
Speaker 1: A white supremacist attacked two mosques in Christchurch on March 15th,
Speaker 1: killing 51 people. But the massacre led New Zealand to
Speaker 1: rapidly tighten its gun laws.
Speaker 2: Yeah, what a concept.
Speaker 1: Notre Dame Cathedral set ablaze on April 15th.
Speaker 2: It was an accident.
Speaker 4: Yeah.
Speaker 1: On April 15th, the world watched as the 850-year-old cathedral
Speaker 1: in Paris burned. Its famous spire collapsed and much of
Speaker 1: the medieval roof was destroyed, although firefighters managed to save
Speaker 1: the main structure. And many important artworks and relics. And
Speaker 1: they rebuilt it. It is rebuilt now, yeah. I haven't
Speaker 1: seen the new one, but it's on my list. And
Speaker 1: you can still see the guy in the tower.
Speaker 2: Going, yes, master.
Speaker 1: Boeing 737 MAX crashes in worldwide grounding. Was this the
Speaker 1: beginning of the end for Boeing?
Speaker 2: Oh, no. This was towards the end of the end.
Speaker 1: Towards the end of the end.
Speaker 2: When you get to the point of actually building an
Speaker 2: unsafe aircraft and ignoring the fact that your engineers are
Speaker 2: telling you it's unsafe until it starts killing people, you
Speaker 2: are very lost.
Speaker 1: Yeah. Just a couple more. The U.S.-China trade war escalates.
Speaker 1: ISIS leader Abu Bakr al-Baghdadi killed. Mass shootings in El
Speaker 1: Paso and Dayton, Ohio. And worldwide protest movements erupt. Beyond
Speaker 1: Hong Kong, 2019 saw enormous anti-government demonstrations in Chile, Lebanon, Iraq, Algeria, Sudan, Iran, Bolivia,
Speaker 1: and Haiti, among others. So, people not happy.
Speaker 3: No. Yeah.
Speaker 2: You done? Yeah, I guess I'm done. Let me give
Speaker 2: you one more grim one.
Speaker 1: Okay.
Speaker 2: Right at the end of December 2019, authorities in Wuhan,
Speaker 2: China report an unusual pneumonia.
Speaker 1: Interesting.
Speaker 2: Yeah.
Speaker 1: That might have been COVID.
Speaker 2: Might have been. Yeah.
Speaker 1: Yeah, because that's when.
Speaker 2: It really started. All right. You ready for some space stuff? Sure.
Speaker 2: Go for it. January, the New Horizons spacecraft, the one
Speaker 2: that had gone to Pluto, still going, had its next
Speaker 2: close flyby to a Kuiper belt object. Today, we call
Speaker 2: that Kuiper belt object Arrokoth, although at the time they
Speaker 2: called it Ultima Thule. So that's over a billion kilometers
Speaker 2: beyond Pluto. That's how fast that thing was moving. And
Speaker 2: was basically a primordial solar system object. It was literally
Speaker 2: two blobs of rock and ice stuck together in a
Speaker 2: very low gravitational space. But, you know, just an extraordinary
Speaker 2: scene back to the early history of the solar system
Speaker 2: that way. That's 2019. It's been seven more years. New
Speaker 2: Horizons is still whizzing out at very high speed. Also
Speaker 2: in January, China's Chang'e 4 lander lands on the far
Speaker 2: side of the moon. First time ever in the Von
Speaker 2: Karman crater. It has the U-2-2 rover on board. I
Speaker 2: think that means little rabbit. It will start its exploration
Speaker 2: in the previous year. China had also put up their
Speaker 2: relay satellite so they could communicate to the far side
Speaker 2: of the moon. In February, back on Mars, the Opportunity
Speaker 2: Rover finally ends its mission after 15 years. Designed about
Speaker 2: the size of a golf cart, designed for operations for
Speaker 2: 90 days, ran for 15 years. Now, in 2019, it
Speaker 2: was when it declared its mission over. Its last communication
Speaker 2: was actually in 2018. There had been a major dust
Speaker 2: storm in 2018 on Mars. And that was always the
Speaker 2: challenge with these rovers is that there was spirit and opportunities.
Speaker 2: They are all solar-based, and so the dust covers up
Speaker 2: the solar panels. But the atmosphere of Mars occasionally would
Speaker 2: create dust levels that would clean those solar panels off. Classically,
Speaker 2: at that time, Opportunity had its own Twitter account, and
Speaker 2: so the operators would take the telemetry data and turn
Speaker 2: it into text as a tweet from Opportunity. And so
Speaker 2: Opportunity's last tweet was, my battery's getting low, and it's
Speaker 2: getting dark. Of course, the reality for a device like
Speaker 2: that is that if it doesn't have enough power to
Speaker 2: keep its electronics warm overnight, those will freeze and it
Speaker 2: will not power back up. And that's what happened to Opportunity. Bummer.
Speaker 2: In March, the Crew Dragon by SpaceX does its first
Speaker 2: flight to the International Space Station. It's completely autonomous. It
Speaker 2: actually does its own docking. There's nobody on board. It
Speaker 2: works perfectly. It then leaves the station and de-orbits and
Speaker 2: lands in the ocean and is recovered successfully. All good news.
Speaker 2: Here's the bad news. The following month, that same capsule
Speaker 2: is now having an abortion. system test. This was an
Speaker 2: unmanned test tethered to the ground firing the Super Dracos
Speaker 2: and something goes terribly wrong and the vehicle is destroyed
Speaker 2: in the explosion. Bummer. There was a leak in the
Speaker 2: helium system for the Super Dracos that created a mass detonation.
Speaker 2: NASA actually considered it lucky. It was a huge gift.
Speaker 2: You know, you'd hate to have that occur during an
Speaker 2: actual flight. It'll delay things for the rest of the year.
Speaker 2: They won't flag it until 2020, although that will actually
Speaker 2: deliver crew successfully. They'll do a new abort system, use tests,
Speaker 2: including an in-flight unmanned abort. Also in April, also on Mars,
Speaker 2: the InSight lander will get its first telemetry by measuring
Speaker 2: a Mars quake to start mapping the interior of Mars.
Speaker 2: So opportunity's done, but the InSight lander's doing its thing. Uh,
Speaker 2: also in April, the Israeli nonprofit, uh, company space IL
Speaker 2: launched the bear sheet lander, the first commercial privately funded
Speaker 2: lunar lander ever flies on a Falcon nine. It originally
Speaker 2: been part of the lunar X prize, which started back
Speaker 2: in 2017 and 2007 and had ended in 2018. So
Speaker 2: it was actually over. It did make it to the moon,
Speaker 2: but failed on its landing, which is not that unusual, uh,
Speaker 2: because it's really hard to land on the moon. In fact,
Speaker 2: Later in the year, in July, India will launch their
Speaker 2: Chandran-2 to the moon, and their orbital will work perfectly,
Speaker 2: but their Vikram lander will crash on its descent towards
Speaker 2: the South Pole. Also in April, the Event Horizon Telescope
Speaker 2: publishes the first image of a black hole in the
Speaker 2: center of the galaxy, Messier 87. I remember. So this
Speaker 2: is using a technology called very long baseline infrometry, where
Speaker 2: they actually harness data from a dozen or so different
Speaker 2: radio telescopes around the world. to create a sort of
Speaker 2: super radio telescope. And then they composite the image into
Speaker 2: what they, um, black hole look like. And it turned
Speaker 2: out looked exactly like the one in interstellar, uh, which, which, uh,
Speaker 2: there's a good reason for that. Uh, Christopher Nolan hired
Speaker 2: a Caltech physicist named Kim Thorne, who actually figured out
Speaker 2: what it should look like. And, uh, the, EHT actually
Speaker 2: validated that.
Speaker 1: And it was very low resolution, if I remember correctly.
Speaker 2: Of course. But, you know, you're literally talking billions of
Speaker 2: light years away. So it's a miracle it works at all.
Speaker 2: But it's just, you know, we're getting new superpowers. In May,
Speaker 2: SpaceX launches the very first batch of Starlink satellites, an
Speaker 2: array of 60. Yeah. In August, SpaceX does their first
Speaker 2: flight of Starhopper. This was a funny little looking can
Speaker 2: of a thing with some legs on it. Flew just
Speaker 2: a short distance and landed again. It's the beginning of Starship,
Speaker 2: but the really important thing is this is their new
Speaker 2: Raptor engine, a methane liquid oxygen engine. And it's also
Speaker 2: full flow combustion. So that was the first time that
Speaker 2: a full flow combustion stage combustion engine had ever flown.
Speaker 2: And it'll get bigger for that. And one little piece
Speaker 2: of additional Boeing news alongside killing people in their dangerous 737,
Speaker 2: which admittedly is now fixed.
Speaker 3: Yeah.
Speaker 2: And I have flown on it. Uh, Boeing flies their
Speaker 2: Starliner to the International Space Station for the first time,
Speaker 2: except for that part where it never got to the
Speaker 2: space station.
Speaker 1: Right.
Speaker 2: Software bugs in the system cause the vehicle to decide
Speaker 2: that it's actually farther along the mission than it actually is.
Speaker 2: And it tries to correct its orientation inappropriately. And before
Speaker 2: they can send commands to get it to stop doing that,
Speaker 2: it burns up so much of its maneuvering fuel. That
Speaker 2: it's unable to make it to the space station. They
Speaker 2: do get control of it, fix the problem with the
Speaker 2: software and successfully de-orbit and recover the vehicle. But that
Speaker 2: will only be the beginning of a whole series of
Speaker 2: terrible problems for Starliner that are not resolved to this day.
Speaker 1: Hey, when you mentioned Starlink, you were using Starlink, right?
Speaker 1: Are you using it now?
Speaker 2: I wasn't at that point. It would be later. I
Speaker 2: would apply for the beta and get into the beta
Speaker 2: in a couple of years after that. But are you
Speaker 2: using it right now? I'm not right now, no. It's
Speaker 2: my backup. I have fiber now to this house in
Speaker 2: the middle of nowhere, which is ridiculous. It is ridiculous.
Speaker 2: Starlink is the backup. Okay, onto the computing side of things.
Speaker 2: A whole bunch of stories around OpenAI. So February of
Speaker 2: 2019 is the release of GPT-2. And remember that the
Speaker 2: mantra of OpenAI had been to do AI out in
Speaker 2: the open, do it also open source, right? That's why
Speaker 2: it's called OpenAI. But they come to the conclusion that
Speaker 2: their largest model in GPT-2, which was a 1.5 billion
Speaker 2: parameter model, oh, how quaint, is too powerful and that
Speaker 2: it shouldn't be shared publicly. So they only release a
Speaker 2: very tiny version of it just to show its potential.
Speaker 2: And they are starting to reconsider what the company's all about.
Speaker 2: Part of this is they're working on a paper, which
Speaker 2: they'll publish in 2020, about... the scale necessary to build
Speaker 2: these language models and that being a not-for-profit is just
Speaker 2: not feasible. It's not the way they're going to be successful.
Speaker 2: And so in March, they'll restructure OpenAI to become what
Speaker 2: they call a capped profit company, which is funny because
Speaker 2: this is all around the time that likely they're talking
Speaker 2: to Microsoft because in June, Kevin Scott wrote an email
Speaker 2: to Bill Gates and Satya Nadella called Thoughts on OpenAI.
Speaker 2: where he talked about the fact that OpenAI and DeepMind
Speaker 2: and Google Brain were well ahead of Microsoft and that
Speaker 2: there was an opportunity to invest in OpenAI and put
Speaker 2: them over onto Azure and maybe have a relationship. That,
Speaker 2: of course, will happen in July. Sam Altman and Nadella
Speaker 2: will be on stage together. We'll announce an exclusive cloud partnership.
Speaker 1: I do remember that.
Speaker 2: And finally, in December, Microsoft will put a billion dollars
Speaker 2: into OpenAI, although how much of that they'll spend buying Azure,
Speaker 2: we don't really know. Related to one of your stories
Speaker 2: early on about China, in March, the UK publishes a
Speaker 2: report about Huawei, concerns about Chinese security with kill switches
Speaker 2: on 5G networking and things like that. The UK has
Speaker 2: a group that's actually getting into the software that drives
Speaker 2: all of those things, but they consider the software such
Speaker 2: poor quality that they can't even verify its reliability or
Speaker 2: security and consider it unsafe to use. That will result
Speaker 2: in May, the Trump administration will literally say, you can't
Speaker 2: use Huawei stuff, full stop. And it'll spread around all
Speaker 2: of the Western world, really. That's around the time the
Speaker 2: first 5G networks actually get deployed in the US, in April.
Speaker 2: Also in April, but on April 2nd, not April 1st,
Speaker 2: probably deliberately, Google shuts down, Google +. Can't do that
Speaker 2: on April 2nd, it's not funny. Okay, September, Visual Studio 2019.
Speaker 2: with Framework 4.8, but also. NET Core 3.0, which is
Speaker 2: really the point where you kind of hit parity, so
Speaker 2: the features, In the old framework that we wanted in
Speaker 2: the open source cloud platform version, they were all there
Speaker 2: now and more or less complete.
Speaker 1: That's where I started using it.
Speaker 4: Yeah.
Speaker 1: I started using it at three.
Speaker 2: Yeah, and it was the point where it's like, hey,
Speaker 2: this is the end of the road. There won't be a.
Speaker 2: NET Core 4. There won't be two versions of. NET
Speaker 2: after this. There'll just be five, although you know the
Speaker 2: old framework, 4.8 will never go anymore. And in December,
Speaker 2: they'll put out 3.1, fixing some of the problems. Two
Speaker 2: more stories. October. Google's Sycamore processor. So the first real
Speaker 2: public demonstration of what we would call quantum supremacy. This
Speaker 2: was one of those chandeliers soaked in liquid helium style
Speaker 2: quantum computers, very fragile. It had 54 qubits, only 53
Speaker 2: of them were functioning, did a 200 second run doing
Speaker 2: a random number sampling task. Google argued that this would
Speaker 2: take 10,000 years with a traditional computer and they finished
Speaker 2: it in 200 seconds. IBM, of course, challenged the numbers
Speaker 2: and then demonstrated various strata on quantum strategies to get
Speaker 2: pretty good results. I think they're fastest when they got
Speaker 2: it down to a couple of days, but whatever. But
Speaker 2: it is in a wave of advancement in quantum computing.
Speaker 2: And last but not least, because I do think it
Speaker 2: impacts a lot of the conversations we're having today in
Speaker 2: December at Amazon's reInvent conference, They talk about Graviton2, which
Speaker 2: was custom ARM processors for workloads in EC2. They're promising
Speaker 2: 40% better price performance. Up until this point, for the
Speaker 2: most part, the cloud had been regular machines bought with
Speaker 2: Intel and AMD chips and so forth, bought by the
Speaker 2: big vendors, already pre-racked and so forth. But now the
Speaker 2: cloud companies, Amazon being one, but also Microsoft and Google
Speaker 2: as well. Google was already going into their TensorFlows and
Speaker 2: things like that. Are starting to build their own hardware
Speaker 2: to optimize for cloud workloads. And that's going to change
Speaker 2: things going forward.
Speaker 4: Yeah.
Speaker 2: Those are my stories, friend.
Speaker 1: All right. Well, that brings us to better know a framework.
Speaker 2: Awesome.
Speaker 1: Or something. Better Know Something, maybe we should call it.
Speaker 1: Better Know Something.
Speaker 2: That would have been a great title.
Speaker 1: All right, man, what do you got? Okay, so for
Speaker 1: those of you paying attention to my other shows, Rocky
Speaker 1: Latka and I started with him as co-host on Code
Speaker 1: It With AI, which is presented by DevExpress.
Speaker 3: And we....
Speaker 1: Just put Quen 3.8 local to the test. Quen 3.8
Speaker 1: is a model, and I ran it on Ollama. And
Speaker 1: we had done on episodes 29 and 30 with Jeff Fritz,
Speaker 1: we had done comparisons between local models running in this
Speaker 1: hardware that I have, right, in Ollama.
Speaker 4: Yeah.
Speaker 1: And Quen 3.6 was the winner by far. So I
Speaker 1: was thinking Quen 3.8, going to be great.
Speaker 2: Yeah.
Speaker 1: Not so much. Interesting. because Alama allocates context depending on
Speaker 1: the amount of VRAM you have.
Speaker 2: So I got 30, 35 or 32, 32 K based
Speaker 2: on the amount of VRAM that I had, but I
Speaker 2: don't think it was enough because it was, it was
Speaker 2: doing the confusing things that the other models had done earlier,
Speaker 2: which is telling you that it, you know, couldn't reach
Speaker 2: the API, giving you some weird nil error, and stopping.
Speaker 2: So Rocky tells me about llama.cpp. And llama.cpp is an
Speaker 2: alternative to Ollama that it allows you to host models.
Speaker 1: But the cool thing about it is that it can
Speaker 1: take the compute heavy stuff and run that in VRAM.
Speaker 1: And then it can use your system RAM for the
Speaker 1: rest of it. Now, I don't know if it swaps
Speaker 1: things in and out, system RAM, VRAM, or it just
Speaker 1: runs some things that don't require such a heavy load
Speaker 1: in system RAM, but that's it. So then there are
Speaker 1: only certain models that can run, but there's a QEN
Speaker 1: model that runs that's bigger that I'm going to try
Speaker 1: for next week's Code with AI. Well, I say next week.
Speaker 1: We're recording this on August 21st. or 20th, rather.
Speaker 2: I'm sorry.
Speaker 1: We're recording this on August 21st, and it won't come
Speaker 1: out until September 10th. So the episode would be 43,
Speaker 1: and that would be on last week, September 2nd. So
Speaker 1: if you want to see what that does, I'm going
Speaker 1: to have a live demo of it. I'm going to
Speaker 1: try it, but llama.cpp. Know it, learn it, love it.
Speaker 1: Who's talking to us today, Richard?
Speaker 2: I had to go back. We're doing mathematics today, so
Speaker 2: I had to go back to a mathematics show. So,
Speaker 2: I went way back. 1252, which is from 2016. Our
Speaker 2: friend Seth Juarez talking about his library, NUML. Because secretly,
Speaker 2: Seth Juarez, as charming and entertaining as he was on
Speaker 2: Channel 9, actually a maths guy. Got a real, you know,
Speaker 2: got a master's degree. Super smart, you know. Great at
Speaker 2: explaining the whole.
Speaker 1: Oh, yeah.
Speaker 2: LLM wave too. We've done some cool shows with it.
Speaker 2: So this is a comment from, uh, Kirian O'Neill, who
Speaker 2: says another scheduling library along the lines of courts.net. We
Speaker 2: were talking about that library as well. Worth mentioning is
Speaker 2: hang fire. The advantage I've gained from using hang fires
Speaker 2: that it runs in process. It doesn't require a separate
Speaker 2: window service. And depending on your use cases can be
Speaker 2: a big advantage. And it's one less thing to think
Speaker 2: about when deploying because, yeah, pushing out stuff to count
Speaker 2: on Windows services and stuff is ugly if you're going
Speaker 2: to try and make things work properly for you. But,
Speaker 2: you know, this was a whole conversation we had with
Speaker 2: Seth about how you do these long compute workloads when
Speaker 2: you're working out things like machine learning models and so
Speaker 2: forth where they may run for a couple of days.
Speaker 2: And you don't want to just tie up the whole machine.
Speaker 2: You want to run them in the background. Yep. So, Kirian,
Speaker 2: thank you so much for your comment. And a copy
Speaker 2: of Music to Code By is on its way to you.
Speaker 2: And if you'd like a copy of Music to Code By,
Speaker 2: write a comment on the website at. netrocks. com or on
Speaker 2: the Facebooks. We publish every show there. And if you
Speaker 2: comment there and I read it in the show, we'll
Speaker 2: send you a copy of Music to Code By.
Speaker 1: And if you want to just go get Music to
Speaker 1: Code By, it's at musictocodeby.net.
Speaker 4: All right.
Speaker 1: We've taken up so much time with this intro stuff.
Speaker 1: I think we ought to take our break now so
Speaker 1: we can have an uninterrupted talk with Jeffrey Sachs. So
Speaker 1: we'll be right back after these very important messages. And
Speaker 1: we're back. It's. NET Rocks. I'm Carl Franklin. That's Richard Campbell.
Speaker 2: Hey.
Speaker 1: And also here with us is Jeffrey Sachs.
Speaker 3: Hi. Hi.
Speaker 1: Jeffrey Sachs is the founder and sole developer of Numerics.net,
Speaker 1: a set of numerical computing libraries for.
Speaker 4: NET.
Speaker 1: He has been building the library since 2005 when the
Speaker 1: first version was released for. NET 1.1. Over the years, Numerics.net
Speaker 1: has been used for applications ranging from calculating radiation doses
Speaker 1: for cancer treatment and developing truck braking systems to building
Speaker 1: support software for atomic force microscopes. Jeffrey's work on numerical
Speaker 1: software predates.
Speaker 2: NET.
Speaker 1: Jeffrey, welcome.
Speaker 4: Hi.
Speaker 5: Thank you. Thanks for having me.
Speaker 2: You're welcome.
Speaker 1: Sorry about the long intro. If you're okay with it,
Speaker 1: we can go a little longer today. But my first
Speaker 1: question is, what is numerical computing?
Speaker 5: I would say numerical computing is the task of computing
Speaker 5: values to a requested accuracy with limited precision. And do
Speaker 5: that as fast as possible.
Speaker 1: So it's basically just math equations in code. Essentially, like
Speaker 1: a MATLAB kind of thing?
Speaker 4: Yeah, that's what it comes down to.
Speaker 1: Simulink, yeah.
Speaker 2: So this is outside the spectrum of what the built-in
Speaker 2: mathematical functions in. NET could do?
Speaker 4: Yes, way beyond, actually.
Speaker 2: Right.
Speaker 5: Vectors, matrices, probability, and statistics, all kinds of fun stuff.
Speaker 1: So it's all based on the fundamental stuff that's in
Speaker 1: the framework, but then you do these... abstractions on top
Speaker 1: of it that give you the higher level.
Speaker 4: That's right.
Speaker 1: Yeah, constructs.
Speaker 2: All right, let's dust off our math chop, see if
Speaker 2: we remember any of this stuff from school. Holy man. So,
Speaker 2: I mean, just in your comment alone, you were talking
Speaker 2: about things like radiation doses. So, can you talk a
Speaker 2: bit about what kind of math we need there? It's
Speaker 2: a great thing to figure out.
Speaker 4: Actually, I'm not sure.
Speaker 1: I just.
Speaker 5: hear a lot of stories from customers.
Speaker 4: Oh, right.
Speaker 2: Um, these are just your customers doing cool things with
Speaker 2: your software.
Speaker 4: Yeah.
Speaker 5: So that's the most fun part of what I do.
Speaker 5: I hear all sorts of things.
Speaker 2: Yeah. Yeah.
Speaker 4: Um, and you heard some variety of what I hear.
Speaker 2: Your background is in mathematics then?
Speaker 4: Yes, it is. I actually, right.
Speaker 2: So you just take him what you've learned and said like,
Speaker 2: how could I do these things in.net?
Speaker 5: Yes. I, I've been doing this pretty much my whole
Speaker 5: life ever since I got my hands on the computer.
Speaker 4: Um, And then went to university.
Speaker 5: Got a degree in computing science, so specializing in the
Speaker 5: numerical stuff.
Speaker 4: And then.
Speaker 5: After some time of building things privately, like just for
Speaker 5: my own enjoyment, I thought, well, why not turn this
Speaker 5: into my job and do it professionally?
Speaker 3: Wow.
Speaker 5: And that's how Numerics.net got started.
Speaker 2: Wow.
Speaker 1: You remind me of a product that when I was
Speaker 1: at Crescent Software, we sold. I didn't write it. I
Speaker 1: just did tech support. But it was called QuickPack Scientific.
Speaker 1: And so in that was a bunch of assembly language things,
Speaker 1: calls that you could call from basic, from QuickBasic, Basic
Speaker 1: 7 PDS, and even Visual Basic, that did all these
Speaker 1: kinds of things. And I wondered even back then, why
Speaker 1: do you need all this if you have the fundamentals?
Speaker 1: And it's kind of like, you know, why do you
Speaker 1: need a database if you can open and read and
Speaker 1: write files?
Speaker 2: Right?
Speaker 1: These are layers of abstraction that help you as a non-programmer,
Speaker 1: maybe a scientist that dabbles in programming, do stuff easily.
Speaker 5: Well, it's not just about the layers of abstraction, although
Speaker 5: that can be useful as well. It's also about the
Speaker 5: limitations that you have doing math on a computer and
Speaker 5: still doing things correctly.
Speaker 1: Yeah.
Speaker 2: So I said, it's, we got to dig into this
Speaker 2: because I think it's fascinating. What, what are the, we never,
Speaker 2: most of the time we don't bump into these limitations.
Speaker 2: What are the limitations?
Speaker 5: Well, the primary limitation is you can't represent every number
Speaker 5: from real life in a computer system.
Speaker 4: Right.
Speaker 1: Right.
Speaker 5: So we have, integers, and then we have what we
Speaker 5: call real numbers. But those are really the double precision
Speaker 5: floating point numbers is the official name. Right.
Speaker 2: Yeah. And I think the floating point's a better term
Speaker 2: just because it lets you know. This is only so accurate.
Speaker 4: Yeah.
Speaker 5: So the limitation is these numbers have a very specific format.
Speaker 5: They're an integer times a power of two.
Speaker 3: Right.
Speaker 5: And you only have a certain range to work with.
Speaker 5: So every calculation you do, you basically are able to
Speaker 5: use only those numbers. Yeah.
Speaker 4: And that's what makes it hard.
Speaker 1: Right, right. So it's the precision. And you mentioned precision
Speaker 1: in your definition of numerical computing, that you may want
Speaker 1: less or more. But I get more. Why less? Yeah.
Speaker 5: Well, in modern-day computing, there's a lot of talk about–
Speaker 5: reducing the number of bits required for LLMs and so
Speaker 5: forth to do their calculations and still have good results.
Speaker 5: So they'll sample down from the original 16 or 32
Speaker 5: bits down to sometimes as little as four or two
Speaker 5: bits even.
Speaker 2: Wow.
Speaker 5: So that's why you would use less. In general, for calculations,
Speaker 5: you would use the standard four or eight number formats.
Speaker 5: because the resources aren't as tight and the computation isn't
Speaker 5: as intense.
Speaker 1: Yeah, okay.
Speaker 2: And so the point being, when you use a higher precision,
Speaker 2: it's just going to take longer for no value.
Speaker 5: Yeah, but if you use too little precision, things can
Speaker 5: go terribly long as well.
Speaker 2: Right.
Speaker 5: There's a famous example of um an arianne 5 rocket
Speaker 5: in 96 i think.
Speaker 2: It was oh yeah they lost the first one they.
Speaker 5: Lost the first one because some value was um stored
Speaker 5: as a 16-bit value for the arianne 4 rocket the
Speaker 5: predecessor right and they converted down from 64 bits to
Speaker 5: 16 got an overflow which sent their navigation system into
Speaker 5: some kind of error state and they had to terminate
Speaker 5: the launch.
Speaker 2: Yeah, that's a very expensive boo-boo.
Speaker 4: Yes, it is.
Speaker 2: Like billion-dollar boo-boo.
Speaker 5: Yeah, and so one of the things that makes it
Speaker 5: hard is because you have that limited set of numbers,
Speaker 5: whenever you do a calculation, you have to start by
Speaker 5: approximating the number you get from real life by a
Speaker 5: number that's in your set that you can work with.
Speaker 5: So you have to round it off and that already
Speaker 5: right off, right out of the gate, you start with
Speaker 5: an error and then errors propagate. Like every calculation you do,
Speaker 5: when you add two numbers, the sum of those two
Speaker 5: numbers may not be a floating point number again. So
Speaker 5: then you get additional errors And sometimes it can explode and, well,
Speaker 5: lead to explosions.
Speaker 2: Right. Quite literally.
Speaker 4: Yeah.
Speaker 2: Yeah, I remember watching that, actually. And it's like suddenly
Speaker 2: the engines were trying to move the rocket in ways
Speaker 2: it really wouldn't want to move. And then it just
Speaker 2: sort of broke into pieces.
Speaker 4: Yeah, that's right.
Speaker 1: I had a friend. He's gone now. But he was
Speaker 1: a professor of mathematics. And he called himself a philosopher
Speaker 1: of math.
Speaker 4: And I.
Speaker 1: Was, I guess, into things that were so abstract that
Speaker 1: nobody could understand them, and you had to just think
Speaker 1: about them completely differently. One of those things that I
Speaker 1: think of in that vein are imaginary numbers. What the
Speaker 1: hell is an imaginary number?
Speaker 4: Right?
Speaker 5: Well, an imaginary number is a trick, basically, to.
Speaker 4: Make... the math work. So there's.
Speaker 5: A theory where when you compute the roots of a
Speaker 5: quadratic equation, you always have two roots. If you have
Speaker 5: real numbers as roots, that's not always the case. But
Speaker 5: if you allow this extension by using the square root
Speaker 5: of minus one as another dimension, then you can always
Speaker 5: get two roots. And so there's all kinds of lovely,
Speaker 5: beautiful math that just falls into place perfectly when you
Speaker 5: do that.
Speaker 1: I still don't get it.
Speaker 2: I'm remembering enough of this stuff to realize, as soon
Speaker 2: as you said imaginary numbers, like square root of negative two,
Speaker 2: that shouldn't be possible, but there's a fix for it.
Speaker 5: Yeah. So it's not purely mathematical. It's actually some, like
Speaker 5: most of quantum physics is also expressed in quantum, in, uh,
Speaker 5: in terms of complex numbers. So, how real that is
Speaker 5: is also questionable, but that's a different discussion.
Speaker 2: Well, you know, God doesn't play dice with the universe.
Speaker 1: Would you consider yourself a philosopher of math?
Speaker 4: A little bit, but not too much.
Speaker 1: Not the philosopher king, just a philosopher. No.
Speaker 5: The pure math and the computer math are actually very different.
Speaker 5: It's a very different story.
Speaker 2: Oh, sure, yeah.
Speaker 5: You don't get into extreme abstractions that only a handful
Speaker 5: of people understand.
Speaker 2: But as I understand it, mathematics and computing in general
Speaker 2: has been a series of compromises based on the limitations
Speaker 2: of the hardware available at the time.
Speaker 4: Yes, that's right.
Speaker 1: And the language.
Speaker 5: Language, too. There's actually funny stories about Fortran, the first
Speaker 5: programming language, which has... Originally, it didn't have if statements.
Speaker 5: It had a conditional jump. It didn't have.
Speaker 5: The logical operators and the comparison operators are written out
Speaker 5: with dots. Dot, L, T, dot is less than. The
Speaker 5: reason they did that was because at the time... computers
Speaker 5: didn't have less than signs.
Speaker 2: Right. The ASCII set hadn't been done yet.
Speaker 4: That's right.
Speaker 3: Wow.
Speaker 1: Was it EPSIDIC?
Speaker 2: Even earlier.
Speaker 5: Another interesting thing about Fortran, white space doesn't matter at all.
Speaker 2: Right.
Speaker 5: You can put spaces anywhere in the middle of a name,
Speaker 5: of a variable, anywhere. The reason they did that is
Speaker 5: because back in the day, people wrote their programs on
Speaker 5: punch cards and there were punch card operators who translated
Speaker 5: or like who entered the text in punch cards, did
Speaker 5: the punching to then feed into the computers. And what
Speaker 5: they found was that they were very sloppy with spaces
Speaker 5: in the transcriptions. And so there were lots of errors
Speaker 5: because spaces were at the time significant. And so they
Speaker 5: did away with that so that their programs could actually run.
Speaker 2: It's a workaround.
Speaker 5: Yeah.
Speaker 1: And speaking of languages, how does JavaScript hit you for
Speaker 1: a math language?
Speaker 4: JavaScript is kind of, it's not.
Speaker 1: You can say it.
Speaker 2: It's okay.
Speaker 5: It's not the best language for computing.
Speaker 2: Yeah, right.
Speaker 5: The nice thing about it is that you have numbers
Speaker 5: and they're all the same. But that's when you do
Speaker 5: numerical stuff, that's also the, The problem.
Speaker 2: Right. Yeah.
Speaker 1: As we found out.
Speaker 2: We've talked a little bit about complex numbers so far
Speaker 2: in the library here. I don't know that I exactly
Speaker 2: understand everything when we talk about space of complex numbers. Because,
Speaker 2: I mean, I've always thought that not a number, you know,
Speaker 2: speaking of JavaScript, was such a computer thing. But is
Speaker 2: that something relevant in complex numbers as well?
Speaker 5: It's relevant in all of numerical computing.
Speaker 2: Oh, okay.
Speaker 5: So a complex number just has a real part and
Speaker 5: an imaginary part.
Speaker 3: Right.
Speaker 5: And once you work with those two, it's just real
Speaker 5: math again. There's nothing complex about it. So the same
Speaker 5: issues that you encounter in real numbers, you will encounter
Speaker 5: in complex numbers again. Right.
Speaker 2: And so problems like infinity and not a number, these
Speaker 2: are just things you have to contend with.
Speaker 4: Yeah.
Speaker 5: And that's one of the weird things. Not a number
Speaker 5: is a special thing. encoding in the binary floating point format.
Speaker 5: And they compare false. So any operation, any logical operation
Speaker 5: with not a number returns false, including itself.
Speaker 2: So not a number is also not a number.
Speaker 5: Well, yeah. So you have to special case all kinds
Speaker 5: of code to work with that to trap it. One
Speaker 5: of the Places where you can get not a number
Speaker 5: is if you try to compute the square root of
Speaker 5: a negative number.
Speaker 4: Right.
Speaker 2: Does that make an imaginary number?
Speaker 5: That would make an imaginary number, but it's outside of
Speaker 5: the number format. Right.
Speaker 2: I'm starting to feel like in this context, not a
Speaker 2: number is problematically like nulls are problematic, where you have
Speaker 2: to test for them, and they're hard to test against themselves.
Speaker 2: So they just have to build a set of exception
Speaker 2: cases around it.
Speaker 5: That's true, and it's more of an annoyance than anything.
Speaker 5: If you're careful, you won't encounter them.
Speaker 2: Right.
Speaker 5: But the other challenge is of maintaining precision. If you
Speaker 5: can do that, you're good. If you can't, then you
Speaker 5: can get things like dividing zero by zero, which is
Speaker 5: also not a number.
Speaker 2: Not a number.
Speaker 1: Yeah.
Speaker 2: That's fair. So, I mean, on one hand, I'm losing
Speaker 2: this library because I have a particular use case where
Speaker 2: I have a particular requirement for precision that perhaps doesn't
Speaker 2: fall into what's built into.
Speaker 3: NET.
Speaker 2: Does that make sense? Is that a reason to go
Speaker 2: into using the library?
Speaker 4: That's one possibility. We have.
Speaker 5: Big floats that can Do enormous ranges of numbers to
Speaker 5: whatever precision you want, basically.
Speaker 2: How big is a big float there, Jeff? A Google?
Speaker 5: I think the exponent is about 10 billion.
Speaker 3: Oh.
Speaker 5: And you can have like up to a couple of
Speaker 5: billion digits, something like that.
Speaker 2: So you could calculate with Googles.
Speaker 5: Yeah.
Speaker 2: Wow.
Speaker 5: A Google actually fits in a normal double floating boy number.
Speaker 2: Really?
Speaker 3: Wow.
Speaker 4: If it's 10 to the power 100. Right.
Speaker 5: It's not exact. Because we don't have enough bits for that. But, yeah,
Speaker 5: it is within the range.
Speaker 2: Nice. Okay. But then now when I think about, you know,
Speaker 2: why would I want a library like this, I immediately
Speaker 2: head towards calculus. Because that's hard to do without a library.
Speaker 1: That's right.
Speaker 5: Well, more generally, the reason you would go to a
Speaker 5: library like this, is that avoiding the loss of precision
Speaker 5: is a challenge. Right? So we talked about every operation
Speaker 5: introduces potentially another additional small error. Right. There's also something
Speaker 5: called catastrophic cancellations. which is you have two numbers that
Speaker 5: almost have the same magnitude, and you subtract them. So
Speaker 5: the result is very small relative to those two numbers.
Speaker 1: Oh, okay.
Speaker 5: Well, your precision will plummet.
Speaker 2: Yeah, and turn it to a zero when it shouldn't be.
Speaker 5: Yeah.
Speaker 2: Right, okay.
Speaker 5: And then you get into other things like... One of
Speaker 5: the biggest challenges for me that I have to be
Speaker 5: always aware of is... Computer math is not quite like
Speaker 5: regular math. So, for example, the order of operations matters.
Speaker 5: In normal math, when you add three numbers, it doesn't
Speaker 5: really matter which order you do it in.
Speaker 4: No.
Speaker 5: But say you have 1 plus 10 to the minus
Speaker 5: 20 plus minus 1.
Speaker 1: Oh, I see the problem already.
Speaker 5: So, 1 plus 10 to the minus 20, you don't
Speaker 5: have the precision to register that little thing you're adding,
Speaker 5: so that remains one. And then if you subtract again,
Speaker 5: it's zero. Whereas if you do one minus one and
Speaker 5: then add, it's zero plus your small number and you
Speaker 5: have a different result.
Speaker 1: Because the first item in the list is what defines
Speaker 1: the type that it's cast to, right?
Speaker 4: That's right.
Speaker 5: So the programming language defines the order and you have
Speaker 5: to make sure that you do things in the right order.
Speaker 5: And that can sneak up on you like A simple expression,
Speaker 5: a plus k minus 1, where a is a real number,
Speaker 5: k is an integer. That looks innocent, right?
Speaker 3: Yeah.
Speaker 5: Can't really go wrong with that. Well, what happens if
Speaker 5: a is a small number and k equals 1? Then
Speaker 5: you're going to have a plus 1. You're going to
Speaker 5: lose tons of precision there. And then subtract 1 again.
Speaker 5: So you've just lost lots of precision in that simple,
Speaker 5: innocent-looking operation.
Speaker 2: Yeah, you ended up at zero and it shouldn't have been.
Speaker 3: Yeah, or.
Speaker 5: Something close, but much less accurate than what you started
Speaker 5: out with.
Speaker 2: Right.
Speaker 1: Is there such a thing as a specialized computer and
Speaker 1: language that brings computer math closer to real math?
Speaker 5: You could say that there are symbolic computer systems, but
Speaker 5: that's really kind of cheating.
Speaker 2: Uh-huh.
Speaker 5: So I wouldn't say for practical use, not really.
Speaker 2: Okay. I mean, wasn't that what Fortran was about, was
Speaker 2: trying to get rid of these precision problems back in
Speaker 2: the day?
Speaker 5: Fortran was to make it easy to write out equations.
Speaker 5: It's literally derived from formula translation.
Speaker 3: Right.
Speaker 5: So eliminating the floating point traps and so on is
Speaker 5: really not practical to.
Speaker 4: Do in a In a language, you have to sort
Speaker 4: of choose your algorithm.
Speaker 3: Right.
Speaker 5: And so you have to, sometimes you get a formula
Speaker 5: and there's something that can blow up or where you
Speaker 5: can lose precision. And then you have to find a
Speaker 5: different path of obtaining the same result that doesn't have
Speaker 5: that problem. And that's the biggest challenge.
Speaker 2: Yeah, and so you're no longer just working on the
Speaker 2: math problem. You're working in the constraints of the computer.
Speaker 4: That's right.
Speaker 2: You have to adapt your math problem to what the
Speaker 2: computer can do for you.
Speaker 4: That's right. And that's why it's hard. That's why so
Speaker 4: much time is spent on getting it right.
Speaker 2: And the problem is if you're not aware of the problems,
Speaker 2: you just write an expression. And in certain test cases,
Speaker 2: it works fine. And in other cases, it ruins everything.
Speaker 5: Yeah, and... Most cases you'll test are like mid-range, normal,
Speaker 5: everyday things. And sure, it'll work fine with that. Like, yeah,
Speaker 5: you might lose a digit or two, but nothing relevant.
Speaker 5: But then extreme cases happen, and then, yeah, you might
Speaker 5: run into trouble.
Speaker 2: Disasters. Should we go down some of the paths of these?
Speaker 2: different math sets you've done here, I'm just trying to
Speaker 2: figure out what a Gauss Cronrod numerical integrator is. You're
Speaker 2: making that up, Richard. I wish I was. I'm just
Speaker 2: reading through the spec here and going, oh boy. If
Speaker 2: you don't know what that is, you probably don't need one.
Speaker 5: Yeah, Gauss is the famous mathematician who pioneered numerical integration,
Speaker 5: so computing the area under a curve.
Speaker 4: Using approximations.
Speaker 1: You said Gauss, right?
Speaker 4: And Gauss, yeah.
Speaker 1: Is that what the Gaussian blur is based on?
Speaker 4: That's also him.
Speaker 5: Wow. Like anything Gauss, like the Gaussian distribution, Gaussian blur,
Speaker 5: all those things are all the same person.
Speaker 2: Wow. Right. Clever guy.
Speaker 1: Yeah.
Speaker 5: So one way to compute errors is you compute the
Speaker 5: same result in two different ways, one more accurate than.
Speaker 3: The other one.
Speaker 4: And then you compare results.
Speaker 3: Right.
Speaker 5: And Gauss-Krohn wrote is a way of doing that with
Speaker 5: reusing some of the calculations you've done so that you've,
Speaker 5: so to minimize extra work. So that's what that is.
Speaker 2: No, and I appreciate that. So now I'm thinking the
Speaker 2: software engineer in me is like, okay. We measure this
Speaker 2: particular problem space in two different ways and then use
Speaker 2: an integrator to get what we think is the most
Speaker 2: accurate possible answer.
Speaker 4: That's right.
Speaker 2: Yeah, smart. And it's good to have a tool. You
Speaker 2: could try and code your way around that. You'd probably
Speaker 2: screw it up. Use the tool.
Speaker 4: That's right.
Speaker 5: And so that particular one is used for what's called
Speaker 5: adaptive integration, which means if you have a curve that's
Speaker 5: really oscillating a lot, like changes very quickly, if you
Speaker 5: try to approximate it with a few points, you're going
Speaker 5: to be way off. And so by computing the error,
Speaker 5: you can say, okay, we're good in this part of
Speaker 5: the function, but here we need to dig deeper and
Speaker 5: be more precise. So you do a recalculation there. What's
Speaker 5: the error now? And you keep doing that until your
Speaker 5: result is accurate enough.
Speaker 2: Accurate enough.
Speaker 1: I imagine FFTs, Fast Fourier transforms. I only know a
Speaker 1: lot of this stuff because, you know, the Gaussian blur
Speaker 1: you use in Photoshop and other things like that. And
Speaker 1: the FFTs we use in noise reduction algorithms in Adobe Audition.
Speaker 1: And I've been using them ever since I've been recording
Speaker 1: with digital audio. But from what I understand about noise reduction,
Speaker 1: it uses a fast Fourier transform and you select the
Speaker 1: number of points. And what it does in the audio
Speaker 1: world is it acts as, let's say it's 8,000 points.
Speaker 1: It's an 8,000 band compressor, right? So you have a
Speaker 1: compressor with three bands, high, low, and mid, or high, mid,
Speaker 1: and low, whatever. But if you break that up into
Speaker 1: the frequency spectrum into 8,000 bands, then it can compress
Speaker 1: each band. band individually using a fast Fourier transform I
Speaker 1: don't know how but I know it works.
Speaker 5: Fourier transform basically turns a signal into its frequencies and
Speaker 5: so then you can filter out high frequencies or mid
Speaker 5: or low or like apply equalizers and then transform back
Speaker 5: to get your new signal with the high frequencies filtered
Speaker 5: out Right.
Speaker 1: Makes a lot of sense, especially we use an 8K
Speaker 1: FFT size for noise reduction. Always have. And it seemed
Speaker 1: to be the sweet spot for audio, for speaking anyway.
Speaker 2: Just one of those things.
Speaker 5: Yeah.
Speaker 2: Wow. Should we, some other areas to explore here, Jeff?
Speaker 2: What do you want to, what would you like to
Speaker 2: talk about? Maybe some Jebyshev polynomials?
Speaker 4: Jebyshev is one of the luckiest people in the world.
Speaker 2: This is a mathematician, I presume.
Speaker 5: He's a mathematician. I'm not even sure which century he's from.
Speaker 2: Right.
Speaker 5: But he played with a certain type of polynomials that
Speaker 5: looked interesting. And so they ended up being named after him.
Speaker 5: And these days they're used all over the place. Right.
Speaker 5: Because they're defining property. What makes them useful is that
Speaker 5: They're very even. They oscillate, yes, but they oscillate within
Speaker 5: a narrow band, and it's the narrowest possible band. Oh, interesting.
Speaker 5: JPL uses that for their planetary models. They will give
Speaker 5: out positions of planets and asteroids and everything as a
Speaker 5: series of Cheverchev polynomials. because you can very accurately model
Speaker 5: basically any curve using them. And what I was saying
Speaker 5: about controlling the growth of your errors, they're extremely favorable
Speaker 5: that way. So you will get minimal error propagation, which is, again, why.
Speaker 2: They're so common. I looked him up. He was a
Speaker 2: Russian from the 1800s.
Speaker 1: Yeah.
Speaker 2: Sounds right. That's a lot of mathematicians, actually, I think.
Speaker 2: And he was at a time of defining all this stuff. So, again,
Speaker 2: you've come back to this a few times, Jeff. It's
Speaker 2: like you're trying to work through a set of data
Speaker 2: and get an aggregate or some kind of valuation from
Speaker 2: it without introducing a lot of error.
Speaker 4: Yeah.
Speaker 2: And these are the kinds of tools you use to
Speaker 2: try and keep the errors.
Speaker 5: And the problem is, too– As I said earlier, as
Speaker 5: soon as you bring a real world number into your
Speaker 5: computing system, you have some deviation from reality. And there's
Speaker 5: another famous.
Speaker 2: Because the computer does an approximation on it.
Speaker 4: Yeah.
Speaker 5: There's another famous and rather tragic example from the Gulf
Speaker 5: War in 1991. There was a Scud missile heading towards
Speaker 5: a U.S. base in Tehran in Saudi Arabia. And they
Speaker 5: had a Patriot system. for air defense, but it failed,
Speaker 5: it missed. And the reason it missed was round off error.
Speaker 5: They were counting time by incrementing a counter every tenth
Speaker 5: of a second. So after about four days, the counter
Speaker 5: was at something like three and a half million. And
Speaker 5: then they computed the current time by multiplying that counter
Speaker 5: by by 0.1. 0.1 is not a floating point number,
Speaker 5: not a binary floating point number. So there's this very
Speaker 5: tiny error there of about 95 nanoseconds. But when you
Speaker 5: multiply 9 to 5 nanoseconds by 3.5 million, you get
Speaker 5: about three-tenths of a second. And missiles fly very fast
Speaker 5: and cover a lot of ground in that time. And
Speaker 5: so the interceptor missed And unfortunately, I think 28 people
Speaker 5: died in that attack. So, it can have very serious consequences.
Speaker 2: Yeah, no kidding.
Speaker 1: We were talking about in the news stories, Boeing and
Speaker 1: the problems that they had. And I wonder if, I mean,
Speaker 1: it was obvious incompetence in a culture of, you know,
Speaker 1: not reporting the truth and trying to appease the higher-ups
Speaker 1: and all that stuff and sycophancy, but I wonder if
Speaker 1: any of that stemmed from having bad algorithms or bad math.
Speaker 2: Do you know?
Speaker 5: I have no idea. There have been mistakes with the
Speaker 5: Ariane rocket and the example you gave too earlier.
Speaker 2: Sure, which I think is a really interesting one just
Speaker 2: because they– changed the precision on the number and then
Speaker 2: introduced it into a different rocket that behaved differently and
Speaker 2: that number suddenly was a huge problem. And more importantly,
Speaker 2: the software doesn't get an overflow and the software just
Speaker 2: treats it as a number still, not acknowledging that, no,
Speaker 2: that's not a valid number and you should stop what
Speaker 2: you're doing. Whatever you do from here is going to
Speaker 2: be worse, not better. The MCAS story, which was part
Speaker 2: of that 737 MAX problem, is a complicated one and
Speaker 2: probably none Appropriate here, but yeah, it was the decision
Speaker 2: on hardware and software and how you communicated to the
Speaker 2: pilots combined that creates that situation.
Speaker 5: Speaking of radiation doses, there's also a story about a
Speaker 5: system to administer the radiation where some people got bad
Speaker 5: burns and it turned out They looked into it, maybe
Speaker 5: as the operator not experienced. And no, the operator is
Speaker 5: very experienced. And actually that turned out to be the
Speaker 5: problem because they entered the information so quickly that the
Speaker 5: system couldn't keep up and skipped an input. And that
Speaker 5: led to overdoses. So there's all kinds of like the
Speaker 5: numerical part is one thing, but there's also the good
Speaker 5: software practices of making allowances for exceptions and treating exceptional
Speaker 5: conditions adequately and correctly.
Speaker 2: Yeah. Yeah. It's interesting to try and write software that
Speaker 2: contends with that kind of thing because the consequence was
Speaker 2: so serious. You know, they're just sort of recognizing that
Speaker 2: the data doesn't make sense. What's reasonable. before it actually runs.
Speaker 2: The Therac-25 issue, that was the radiation treatment where if
Speaker 2: you entered the data too quickly, software went into a
Speaker 2: race condition.
Speaker 5: And one thing they did, I believe, in that instance
Speaker 5: is the previous version of that equipment had hardware guards. Right.
Speaker 5: So if the radiation was too high or voltage or
Speaker 5: whichever way they measured it, it would refuse to do that.
Speaker 5: But they took those off and they replaced them with
Speaker 5: software tests. That obviously didn't work.
Speaker 2: Yeah, obviously created other consequences, no question.
Speaker 1: So your point is well taken. Math is kind of important.
Speaker 2: To get right. Well, yeah, precision. I don't know that
Speaker 2: we even now actually understand fully the compromises that were
Speaker 2: made to store numbers in computers efficiently.
Speaker 5: They did a pretty good job, actually, with the standard
Speaker 5: IEEE 754 format. In some ways, it's quite ingenious. The numerical,
Speaker 5: if you compare the bit value, if you look at
Speaker 5: it as a 64-bit integer, it is in the correct
Speaker 5: sort order. So you can compare the numbers as integers
Speaker 5: and and have the correct sort order, except for.
Speaker 4: The not a number cases, which are an annoyance.
Speaker 5: And there's another problem that's not obvious at first sight.
Speaker 5: Because of the format, you always have an integer times
Speaker 5: a power of two. And so there are as many
Speaker 5: floating point numbers between one and two as there are
Speaker 5: between two and four, or four and eight. in 16,
Speaker 5: so they've become scarcer and sparser, the more, the larger
Speaker 5: your numbers get, and they get closer and closer together.
Speaker 4: The smaller your numbers get.
Speaker 5: And then the question is, well, what do you do
Speaker 5: close to zero? And the solution they found was, well,
Speaker 5: let's just switch from a floating point format into two
Speaker 5: times a power of two. to a fixed point format.
Speaker 5: And so all the numbers with exponent zero are called
Speaker 5: subnormal numbers and they are a fixed point format. So
Speaker 5: they are that number interpreted as an integer times, I
Speaker 5: think it's two to.
Speaker 4: The power minus 1,075.
Speaker 5: So that when you get to zero, You have what
Speaker 5: they call gradual underflow. You don't just cut to zero
Speaker 5: straight away. You have like a little bit of breathing
Speaker 5: room to get there.
Speaker 2: Yeah, this is way more complicated than you think about.
Speaker 2: Most of the time we just use numbers and don't
Speaker 2: worry about it. Sometimes you want integers and sometimes you
Speaker 2: want floating point and you're fine. But then this is here.
Speaker 2: I've just looked at this huge roster of capabilities that
Speaker 2: are in all these exception cases. All right. How do
Speaker 2: people figure out they need your tool, Jeff?
Speaker 5: Well, most people just need some kind of calculation where
Speaker 5: they need the level of extraction that our library provides.
Speaker 5: So if you have to solve a set of equations, okay,
Speaker 5: you can do it all yourself and try to figure
Speaker 5: out how to do it efficiently.
Speaker 2: Mm-hmm.
Speaker 4: Or you can use my library and just say, okay,
Speaker 4: these are a number of delegates.
Speaker 2: And to be clear, your library is not free.
Speaker 4: No, it's not.
Speaker 2: Yeah. And not that I have a problem with that.
Speaker 2: It makes sense. It sounds like you've done a ton
Speaker 2: of work. And it's worth it for the folks. If
Speaker 2: you need it, you need it. And you should pay
Speaker 2: for it.
Speaker 1: If you need it, you need it. That's right.
Speaker 4: That's right.
Speaker 2: This is literally a tool to save lives and stop
Speaker 2: spacecraft from exploding. Yeah.
Speaker 1: Yeah. talk about performance a little bit. Uh, some of the,
Speaker 1: I mean, the.net framework has gotten more and more performant
Speaker 1: with every release. Was there ever a time when you considered,
Speaker 1: you know, dropping to assembler to do stuff because it
Speaker 1: wasn't fast enough?
Speaker 5: Well, we've been doing this for 20 years and, um,
Speaker 5: we've seen a few performance, uh, concepts come and go.
Speaker 4: Uh, when, uh.
Speaker 5: The uh, task parallel library was first introduced around 2010,
Speaker 5: that seemed to be the thing to speed things up. Uh,
Speaker 5: turned out there was so much overhead that it was
Speaker 5: limited in what we could do with it. Um, so
Speaker 5: one example of where I did, um, role my own,
Speaker 5: so to speak, is big integer. The initial implementation that
Speaker 5: shift with. NET 4.0 in 2010 was adequate for small sizes,
Speaker 5: but as soon as your numbers got large, it got
Speaker 5: slower and slower. And so we did some work there
Speaker 5: to make a faster version of that, But I really
Speaker 5: have to give it to the.
Speaker 4: NET guys.
Speaker 5: In recent years, they've done so much to bring their
Speaker 5: numerical side of things forward into what's actually now to
Speaker 5: the point where we're considering giving up our big integer
Speaker 5: and just using the.
Speaker 4: NET standard version.
Speaker 1: Do you utilize span of t? I know that's a
Speaker 1: huge win.
Speaker 4: We use it all the time.
Speaker 1: Yeah.
Speaker 2: Oh, wow.
Speaker 5: All the time because, well, one of the advantages, too,
Speaker 5: is a span can be over managed memory, but it
Speaker 5: can also be over unmanaged memory, like anything you get
Speaker 5: from a third-party library or application. And so using span
Speaker 5: allows us to use the same code for– Managed memory,
Speaker 5: so vectors stored in managed memory or in native memory.
Speaker 5: And then the performance, too, it's very compact. It lets you,
Speaker 5: and it's very much optimized more and more. It's actually
Speaker 5: interesting when we look at our test suite. You can
Speaker 5: see we still support.
Speaker 4: NET 4.6.2 and. NET standards.
Speaker 5: And you can see in our execution times of the
Speaker 5: test suite how long it takes for each edition.
Speaker 4: To run.
Speaker 5: And without even changing any code, you can see a
Speaker 5: progression by like every edition is like 10% faster overall.
Speaker 5: And it's just a runtime that does that.
Speaker 1: Yep. And overhead, memory overhead has also gone way down.
Speaker 4: Yeah.
Speaker 1: Just wonderful. Well, it's good to see that you picked
Speaker 1: the right horse.
Speaker 2: Yeah, 20-plus year run.
Speaker 1: Yeah.
Speaker 5: I'm very impressed with the work they've been doing. And
Speaker 5: also in the library itself, when, again, Complex Numbers first.
Speaker 4: Came out, also in.
Speaker 5: NET 4, it was really like you're– textbook or school
Speaker 5: book implementation. It was just the formulas without really understanding
Speaker 5: the intricacies of.
Speaker 4: How to get precise results.
Speaker 5: Even in there, when you computed the square root of
Speaker 5: minus one, which should be purely imaginary I, because that's
Speaker 5: the definition of the imaginary unit, it was off, it
Speaker 5: wasn't correct. But in the last couple of years, and
Speaker 5: now with. NET 11, they've done a ton of work
Speaker 5: to clean that up. And we're not quite ready to
Speaker 5: give up our complex number implementation, but we're seriously considering it. So, again,
Speaker 5: great job,. NET team.
Speaker 2: We should pass it along to Steve and Todd. Those
Speaker 2: guys are freaking brilliant.
Speaker 1: They are, yeah. Well, Jeffrey, thanks for hanging with us.
Speaker 1: We went a little bit over, but that's only because
Speaker 1: we had such a long intro. But it was great
Speaker 1: just going through your interview without an interruption. And this
Speaker 1: is awesome stuff. And I'm actually a little impressed with
Speaker 1: both Richard and I for keeping up with you.
Speaker 2: I think Jeff is a great explainer.
Speaker 1: You are, yeah.
Speaker 2: Which I really appreciate.
Speaker 4: Thank you.
Speaker 1: And good luck in the future. And thanks very much.
Speaker 5: Thank you.
Speaker 1: And we will talk to you dear listener next time on.
Speaker 1: NET Rocks.
Speaker 3: NET Rocks is brought to you by Franklin's Net and
Speaker 3: produced by Plop Studios. a full-service audio, video, and post-production
Speaker 3: facility located physically in New London, Connecticut, and, of course,
Speaker 3: in the cloud, online at pwop.com. Visit our website at dotnetrocks.com
Speaker 3: for RSS feeds, downloads, mobile apps, comments, and access to
Speaker 3: the full archives going back to show number one, recorded
Speaker 3: in September 2002. And make sure you check out our sponsors.
Speaker 3: They keep us in business. Now go write some code.
Speaker 1: See you next time.