Speaker 1: How'd you like to listen to dot net rocks with
Speaker 1: no ads?
Speaker 2: Easy?
Speaker 1: Become a patron for just five dollars a month. You
Speaker 1: get access to a private RSS feed where all the
Speaker 1: shows have no ads. Twenty dollars a month, we'll get
Speaker 1: you that and a special dot net Rocks patron mug.
Speaker 1: Sign up now at patreon dot dot NetRocks dot com. Hey,
Speaker 1: welcome back to dot net rocks. I'm Carl Franklin, an
Speaker 1: amateur Campbell and this is episode in nineteen forty six.
Speaker 2: Richard Yay, end of the war.
Speaker 1: Yeah, we're inching closer, aren't we. Let's talk about what
Speaker 1: happened in nineteen forty six. The of course, the post
Speaker 1: World War II world saw the establishment of the United
Speaker 1: Nations after the dissolution of the League of Nations, the
Speaker 1: Philippines gained independence, Italy abolished its monarchy, the US Atomic
Speaker 1: Energy Commission was established, the Soviet Union rejected a US
Speaker 1: proposal for an international agency to control nuclear energy production
Speaker 1: and research, and the ENIAC, the first electronic digital computer,
Speaker 1: was dedicated. You got anything else to add to that list?
Speaker 2: Well, I mean INIAC was electronic. It had eighteen thousand
Speaker 2: vacuum tubes. Yeah, so you know, you can you can
Speaker 2: debate how electronic it necessarily was. I was trying to
Speaker 2: figure out what the processing rate of it was, because
Speaker 2: it could do something like five thousand additions per second
Speaker 2: and it did have one hundred kilohertz clock. Wow that
Speaker 2: there was one table I found this at It runs
Speaker 2: at about five hundred flops or floating floating boat operations
Speaker 2: per second.
Speaker 1: So I just remember Uniblab from the Jetsons. Do you
Speaker 1: remember that?
Speaker 2: Yeah, a year in a blab And it was of
Speaker 2: course originally built to do artillery table calculations, right, that
Speaker 2: was three or four years earlier. But by the time
Speaker 2: they actually get it built, most protect stuff solved, and
Speaker 2: they've shrunk down artillery table calculators enough that they're now
Speaker 2: being put inside of battleships. The very first program that
Speaker 2: it ran was a Monti Carlo simulation for calculating neutron
Speaker 2: propagation in thermonuclear weapons.
Speaker 1: How does Monty Carlo and thermonuclear weapons go together? In
Speaker 1: the same sentence, I'm glad you asked.
Speaker 2: A Monty Carlo simulation is an idea that we don't
Speaker 2: know all of the variables. We just know some behavior
Speaker 2: of some elements. In this particular case, you're talking about
Speaker 2: neutron propagations of how neutrons move through certain materials, and
Speaker 2: so you're able to run multiple simultaneous calculations. It's too
Speaker 2: hard to do by hand. You needed a fast computer,
Speaker 2: and at that time, ENIAC was the fast computer that
Speaker 2: sort of proved that a Monte Carlo simulation is even possible.
Speaker 1: So they call it a Monti Carlos simulation because it's
Speaker 1: related to gambling of some kind.
Speaker 2: Yeah, that was partly at one of the scientists that
Speaker 2: have a degenerate uncle that was prone to gambling. They
Speaker 2: needed a code name, so it started out as kind
Speaker 2: of the code name. Okay, but it is a a
Speaker 2: probabilistic heroism calculator, which is kind of the way a
Speaker 2: lot of gambling numbers work anyway, So it all kinds
Speaker 2: of goes together. There's a randomness factor to this, But
Speaker 2: I would point out it did demonstrate that beryllium was
Speaker 2: an effective reflector of neutrons at a certain rate, so
Speaker 2: they knew how thick to make the linings to be
Speaker 2: able to do the second stage explosions like it worked
Speaker 2: at bend the path that would take several years for
Speaker 2: them to actually build a working thermonucular device and go
Speaker 2: from kilotons to megatons. But this was the beginning of it.
Speaker 2: An ediac was key to it. Wow, it's good to
Speaker 2: know history, kids.
Speaker 1: Yeah, just listen to Uncle Richard and we do geek
Speaker 1: outs on these things occasionally once a year around the
Speaker 1: end of the year. So if you just search geek
Speaker 1: out on our air rocks dot com set, you can
Speaker 1: find a whole host of topics. All right, well, let's
Speaker 1: get started with better no framework roll the music. So
Speaker 1: what I have is a little tool that I wrote
Speaker 1: for Blazer. It's a Blazer component called AVN Audio AVN
Speaker 1: for app v next that records audio and provides buffers
Speaker 1: in real time. So, knowing the topic that we're going
Speaker 1: to talk about with l S today, I pulled this up.
Speaker 1: This is not audio and video in dot net. This
Speaker 1: is Blazer. So it's web based audio. We're actually using
Speaker 1: a web based tool right now to record audio and
Speaker 1: video in real time. And the reason that it works
Speaker 1: is because the browsers now support these standards that make
Speaker 1: this stuff possible and finally obtainable for your average JavaScript developer.
Speaker 1: And it's really remarkable. Like we're looking at our video
Speaker 1: in real time, we're listening to each other in almost
Speaker 1: real time. You know, there's a little delay, but you
Speaker 1: don't notice it.
Speaker 2: It's close enough, right, that's the key.
Speaker 1: It's not perfect, right, And it's uploading our audio in
Speaker 1: the background as we're recording it. And then by the
Speaker 1: time we're done, you know, maybe five ten seconds later,
Speaker 1: it's there. It's all there in the cloud. So the
Speaker 1: technology has really gotten good. And I use this opportunity
Speaker 1: to create a component that we'll start recording audio and
Speaker 1: then call you back in you know in Blazer and
Speaker 1: see sharp were the buffer and you can do whatever
Speaker 1: you want with that buffer. You can compress it and
Speaker 1: stream it, you can write it to disc all that
Speaker 1: kind of stuff. So I thought that would be a
Speaker 1: fun little way to start just and it uses well,
Speaker 1: it used to use ffm peg, but there's one demo
Speaker 1: that does use it. So ffm peg is a command
Speaker 1: line tool for doing all kinds of things with audio
Speaker 1: and video and it works great and c sharp if
Speaker 1: you're using standard in Standard out. Just a great little tool.
Speaker 1: Use it all the time anyway, That's what I got Richard,
Speaker 1: and who's talking to us today.
Speaker 2: We don't do audio video too often, so I jumped
Speaker 2: into the wayback machine. I went back to twenty thirteenth
Speaker 2: episode eight point fifty four, when Mark Heath talked about
Speaker 2: audio and Windows. Yeah, specifically the project he was working on,
Speaker 2: which was an audio in CodePlex code plex. I love
Speaker 2: in audio and got it. We got a ton of
Speaker 2: comments on the show. I just grabbed one give mely
Speaker 2: twelve years ago. This from Jph says, Hey, Carl, what
Speaker 2: a great geek out show. I'm sure some of the
Speaker 2: discussion took a lot of the listeners, including myself, back
Speaker 2: to the SoundBlaster days. Oh yeah, I remember getting my
Speaker 2: sound blaster back in the early nineties. I was thinking
Speaker 2: how magical it was to record and play wave files
Speaker 2: a computer, producing real time sounds. Magic. Yeah, And obviously
Speaker 2: we've come a long way since then, and things usually
Speaker 2: work pretty well on Windows when you just need a
Speaker 2: standard audio playback. There's one thing though, when we all
Speaker 2: went from XP or Vista to Win seven and most
Speaker 2: people with built in sound cards lost the capability record
Speaker 2: exactly what you hear as standard out of the output back,
Speaker 2: which it can awesome be useful. There's also an option
Speaker 2: to you in the Windows Sound Mixer, but for some
Speaker 2: reason we've lost that and it was you know, it
Speaker 2: was in there somewhere, but they were messing like how
Speaker 2: many different flipping control panels do you need? Right, It's crazy.
Speaker 2: Discussion about doing real time audio processing and showing wave
Speaker 2: forms of meters during recording reminded me of something that
Speaker 2: has in fact only a tangent but appealed to me
Speaker 2: as a dev when I was tickering with MIDI controllers. Hey,
Speaker 2: and it may be helpful people who don't want to
Speaker 2: shallow a lot of cash for MIDI hardware. Touch osc
Speaker 2: is an application for touch devices, iOS and Android admittedly
Speaker 2: is in twenty thirteen. Right I moved on that lets
Speaker 2: you define a rich MIDI control surface with buttons, rotary knobs,
Speaker 2: faders and even xypads and other controls, then shows up
Speaker 2: as a regular MIDI controller on computer via Wi Fi.
Speaker 2: Had great fun hooking up different controls to minifunctions using
Speaker 2: your form designer and using it on an iPad as
Speaker 2: a control ser geeky audio stuff and computers. I love it.
Speaker 1: And you know, I have a little history with a
Speaker 1: sound blaster because at the time the MIDI pack for
Speaker 1: the sound Blaster came out company. I worked for Voyetra Technologies.
Speaker 1: They had the deal with the sound Blaster to provide
Speaker 1: that hardware and the sequencer sequencer plus software, and I
Speaker 1: was tasked with making some sample MIDI files that could
Speaker 1: be shipped with it. And so when if you got
Speaker 1: the sound Blaster MIDI add on you look at those
Speaker 1: mini files. Yeah, I wrote those.
Speaker 2: Nice.
Speaker 1: Also, I remember going out to dinner with mister sim
Speaker 1: who is the inventor of the sound Blaster, and seeing
Speaker 1: actually him at PCXPO I think it was at the
Speaker 1: time the Javit Center, and he wanted me to record
Speaker 1: re record the files for the Talking Parrot. You remember
Speaker 1: the talking parrot Elias right, yeah, yeah, And I never did.
Speaker 2: I was such a jerk.
Speaker 1: I could have done that too, because he wanted, you know,
Speaker 1: funnier things than he had responded with the parrot. Basically,
Speaker 1: you talk to the parrot and it gives you these crazy, wacky,
Speaker 1: high pitched responses.
Speaker 3: Don't touch me.
Speaker 1: Yeah yeah, die q, I think that was one of mine. Anyway,
Speaker 1: didn't do it, but it was fun, fun times, fun times.
Speaker 2: So thanks JP for your comment and a copy of
Speaker 2: music co buy is on it's way to you, and
Speaker 2: if you'd like copy music, go buy. I write a
Speaker 2: comment on the website at dot at rocks dot com
Speaker 2: or on the facebooks. We publish every show there and
Speaker 2: if you comment there, and every ready on the show
Speaker 2: was at your copy music, go by.
Speaker 1: And music to code by going strong. We've got track
Speaker 1: twenty two now and you can go to music toocode
Speaker 1: by dot net, which is a URL rerouter. So HDTPS
Speaker 1: isn't going to work. But if you just say music
Speaker 1: tocode by dot net, that'll bring you there and you
Speaker 1: can get just track twenty two if you want, or
Speaker 1: the entire collection in MP three wave or flack format.
Speaker 1: So with that, let's bring back to dot net Rocks.
Speaker 1: Elias Purinin He is an early innovator in digital event
Speaker 1: experiences and the founder of Tractus Events. As both a
Speaker 1: programmer and producer, he leverages av over, IP and NDI technology,
Speaker 1: which I also love to deliver broadcast quality conferences, trade shows,
Speaker 1: and workshops on any budget. Elias personally develops much of
Speaker 1: the software his team uses, ensuring every aspect of production
Speaker 1: is tightly controlled. His high standards for quality have led
Speaker 1: to award winning events, including a collaboration with University of
Speaker 1: Waterloost Speed that garnered over fourteen thousand views and earned
Speaker 1: a Case Circle of Excellence Award. He is also the
Speaker 1: author of Memorable, Profitable Virtual How to run virtual Events
Speaker 1: that create meaning, value and lasting connections. Finally, for the
Speaker 1: nerd crads. Elias first started with VB dot net one
Speaker 1: point one back in two thousand and five, but has
Speaker 1: since crossed over the dark side and writes almost everything
Speaker 1: in C sharp these days. He used C to write
Speaker 1: games for the Game Boy Advance. He programmed a game
Speaker 1: for the Atari twenty six hundred in six five h
Speaker 1: two assembly in twenty four hours, and loves writing a
Speaker 1: great select Star from a sequel server database.
Speaker 2: How to make your DBAs angry. That's right, select Star.
Speaker 1: Welcome back, Elias, Thank you gentlemen.
Speaker 3: It's awesome to be back. I cannot wait to talk
Speaker 3: P and voke and NDI and.
Speaker 1: Every day day over ip P and voke Man, we
Speaker 1: were talking about one of the earliest components or ds
Speaker 1: that I wrote for dot net was in vbnet and
Speaker 1: I used P and voke to do to simplify access to.
Speaker 3: MIDI right and so like back then, we didn't have
Speaker 3: any audio, we didn't have any of those other tools,
Speaker 3: Like you had to go into what was it the MMAPI, right, MULTIMEDIAPI.
Speaker 1: To do that MULTIMEDIAPI.
Speaker 3: Yeah, yeah, that was. But what's stunning is how little
Speaker 3: the p and voke APIs have changed in dot Net,
Speaker 3: Like how you actually write that code. Yeah, and how
Speaker 3: relevant it is like twenty plus years later.
Speaker 1: And just to back up, p invoke stands for platform invoke.
Speaker 1: So this is a way that you can use whatever
Speaker 1: managed language that you want to make API calls to
Speaker 1: the unmanaged platform APIs that you're running on.
Speaker 3: Right, So imagine you've got a unmanaged DLL of some
Speaker 3: sort and it does something for you, and you want
Speaker 3: to use the result of a function call from that
Speaker 3: DLL in you want to as a result of that
Speaker 3: in C sharp or just dot net in general. You
Speaker 3: could do that with VBNT two. So you would write
Speaker 3: a special function wrapper, you know, especially defined function like
Speaker 3: public extern void, whatever, the signature name is, put in
Speaker 3: the right parameters, decorate it with the correct attribute, and maam,
Speaker 3: now you can call into As long as that DLL
Speaker 3: can be found by your process, you can call into
Speaker 3: that DLL.
Speaker 1: And your parameter types are absolutely accurate. If they're not
Speaker 1: all things, all sorts of weird things will happen.
Speaker 3: Oh boy, how do you well? Lots of weird things happen.
Speaker 1: Hey, your compiler won't necessarily know about it ahead of time,
Speaker 1: will it?
Speaker 3: No, not necessarily. And sometimes the runtime errors that you'll get,
Speaker 3: you know, you'll just crash out. You won't even get
Speaker 3: an exception, and things will just fall over that said
Speaker 3: the compiler, even though the compiler may not catch it.
Speaker 3: There have been instances, and we'll talk about that. We'll
Speaker 3: talk about specifics later, especially around union types where at
Speaker 3: runtime you'll get very specific pacific errors to say, hey,
Speaker 3: your structs aren't defined properly. Yeah, but that's we'll talk
Speaker 3: about kind of the nuts and bolts there later.
Speaker 2: Can we talk about the wisdom of using a garbage
Speaker 2: collected language in a real time operation like an event?
Speaker 3: Yeah? For sure.
Speaker 2: Shouldn't this all be written in C plus plus? Like?
Speaker 2: Why are you in the p and voke land? Why
Speaker 2: aren't you staying closer to the metal?
Speaker 3: So I thought about this. I thought about this a
Speaker 3: lot because the background behind what I've been doing for
Speaker 3: the last probably two years has been real time audio
Speaker 3: and video, specifically using a technology called NDI. NDI short
Speaker 3: for Network Device Interface. Long and short of it is,
Speaker 3: it's a way to send real time audio and video
Speaker 3: over a LAMB. And when I'm talking real time, we're
Speaker 3: talking latency of a frame or less.
Speaker 1: In some cases it can be Yeah, you have to
Speaker 1: have a very fast network. But there's oh boy, are
Speaker 1: there a lot of variables on that one. So it's
Speaker 1: the network, The knicks have to be configured, good drivers,
Speaker 1: you know, there's a lot that goes into it.
Speaker 3: And I've done a lot of research on it. But
Speaker 3: the upshot is the lowest theoretical latency that I've read
Speaker 3: is down in the number of scan lines. Wow, so
Speaker 3: less than a frame of latency you get down to
Speaker 3: scan lines.
Speaker 2: That's great. Well, especially when you're talking about four K
Speaker 2: where you've got what twenty one sixty scan lines per frame.
Speaker 3: Yeah, so it's I mean there's a lot of asterisks
Speaker 3: to achieve that low latency, right, But I mean, if.
Speaker 1: You're doing four K over a one hundred megabit network
Speaker 1: is not going to get You're not do net ain't
Speaker 1: going to work.
Speaker 2: Ivan gig is humming at the limit at that point.
Speaker 3: Yeah, yeah, depending on your encoder, depending on the device.
Speaker 3: Like you can cram about four full bandwidth NDI sources
Speaker 3: onto a gig connection. Probably don't want to do that,
Speaker 3: but you can.
Speaker 2: You can. Yeah, yeah, because network storms are still a thing.
Speaker 2: We usually don't hit them because we have enough bandwidth
Speaker 2: we don't think about it. But once you get upwards
Speaker 2: at eighty percent or so and start getting collisions, it
Speaker 2: gets way where it falls off a clip fast.
Speaker 3: Oh yeah.
Speaker 1: So, and is a spec that is created by a company, right,
Speaker 1: and they give you the NDI tools that you can use, right,
Speaker 1: Can you talk a little bit about that, yeah?
Speaker 3: Sure. So NDI was originally created by Newtech who created
Speaker 3: the tricasters and going even back even further the Amiga
Speaker 3: video toasters right, right, And how NDI evolved was they
Speaker 3: wanted to find a way to bring in remote camera
Speaker 3: sources or remote screen share sources as video sources over
Speaker 3: a network. I think the original term for it was
Speaker 3: ivgay and then it's kind of grown from there. Like
Speaker 3: behind me, I've got a whole bunch of cameras. They're
Speaker 3: all NDII cameras. There's the NDI tools, which is just
Speaker 3: software tools that you can use to have a virtual
Speaker 3: webcam to send out test patterns to preview your NDI
Speaker 3: sources on your network. And then there's the actual hardware itself,
Speaker 3: right like cameras, there's some audio gear that's coming out,
Speaker 3: converters that will take an NDII signal and turn it
Speaker 3: into SDI or HDMI many three and four letter acronyms
Speaker 3: around this world. So the idea behind that is instead
Speaker 3: of running physical copper cables, SDI or HDMI cables that
Speaker 3: carry audio and video, and then you have another cable
Speaker 3: for doing control and then maybe another cable for power.
Speaker 3: If you've say, got an NDI camera and it's power
Speaker 3: over Ethernet, I'm now sending audio, video, control, and power
Speaker 3: over one cable.
Speaker 2: Yeah right, and it's an Ethernet cable.
Speaker 3: It's an Ethernet cable. So like if I destroy a
Speaker 3: fifty foot Ethernet cable, okay, so what I go to
Speaker 3: Amazon and buy a new one twenty bucks?
Speaker 2: Yeah yeah, yeah, if.
Speaker 3: I destroy a fifty foot SDI cable, someone's getting in trouble.
Speaker 2: Everybody's sad. Yeah, but I mean tricasting the day was
Speaker 2: the device. The market's blown wide open now with atems
Speaker 2: roads stuff and so forth. Although I think the real
Speaker 2: thing that I noticed was one day everyone said, you know,
Speaker 2: maybe it's okay to drop a frame now again, Like
Speaker 2: SDI was so strict you needed those really good and
Speaker 2: expensive cables and they never dropped a frame Like that
Speaker 2: was the whole claim to fame, and HDMI was just
Speaker 2: nowhere near that rigorous. And I remember when the first
Speaker 2: ATM mini pros came along in the minis and you're like, yeah, no,
Speaker 2: occasionally we'll throw a frameway to maintain SINC and live
Speaker 2: with it. It's like, but we're going over the internet.
Speaker 2: It's not that reliable anyway, what do we care?
Speaker 1: The TriCaster still is like the Mac daddy of all things, right,
Speaker 1: it's essentially a Windows PC.
Speaker 2: It's a with a with a console on it.
Speaker 1: With the TriCaster software yeah, with the console yeah, and
Speaker 1: all sorts of consoles that you can control it with.
Speaker 1: And it's expensive. Yeah, you know, let's face it.
Speaker 3: Yeah, it's a it's an amazing product. It's expensive. The
Speaker 3: same compan so the company that made it, New Tech,
Speaker 3: they eventually got acquired that's now vis RT. VISRT split
Speaker 3: this NDI project off into its own entity. So NDI
Speaker 3: exists as a company as well, and so you can
Speaker 3: you as a developer, can go out and get the
Speaker 3: basic SDK, integrate that into your application, and then you
Speaker 3: can also get the Advanced SDK, which gives you a
Speaker 3: lot lower level control over how the whole how your
Speaker 3: NDI application behaved.
Speaker 1: Now there's a lower cost alternative to the TriCaster that
Speaker 1: I know you use and because we talked about it
Speaker 1: before we started, and I use it as well that
Speaker 1: I want to talk about. And that's v mix, Yes,
Speaker 1: vmix dot com. And here's what I know about VMX.
Speaker 1: It's a WPF app written in vb net. Yes, how
Speaker 1: cool is that? Yeah, it's it's super cool.
Speaker 3: Like to be clear, they've got unmanaged stuff going on
Speaker 3: as well, like obviously just to interact, but it is
Speaker 3: written in vb net. It is super super cool.
Speaker 1: Yeah, and rock solid. Like these guys are in Australia
Speaker 1: and you can have I don't know they have these
Speaker 1: callers right, and you can buy it based on how
Speaker 1: many input callers you can have remotely and the caller
Speaker 1: API think that max is out at eight?
Speaker 3: I think right, yeah, v mix call I think it's eight, yeah, v.
Speaker 1: Mix Call and so you can give out these you know,
Speaker 1: URLs to everybody who you want to call, and then
Speaker 1: you have this dashboard where you can you know, select
Speaker 1: the It's very much kind of it's kind of like
Speaker 1: if you've used you know, what's the OBS, right, OBS studio,
Speaker 1: but on steroids. That's the way I look at it.
Speaker 3: It's kind of like OBS went to college. Yeah, nice, Yeah,
Speaker 3: it's and like it's for us. We also use v
Speaker 3: mix to do like when we're doing our hybrid shows,
Speaker 3: like we'll have we'll have one single v mix rig
Speaker 3: that's pushing out the local feed, like what's going to
Speaker 3: the local projectors. We'll have one that's pushing out the
Speaker 3: virtual feed to English listeners. We'll have one that's pushing
Speaker 3: out to French listeners, and like all from one application,
Speaker 3: right yeah, And this was the sort of stuff that
Speaker 3: you used to have to get you know, the big
Speaker 3: broadcast consoles to do, right sure, And it's like a
Speaker 3: fifty dollars license and you can do the same thing
Speaker 3: that you know, cost hundreds of thousands of dollars back
Speaker 3: in the day, and it works.
Speaker 1: And as a developer, it's even cooler because they've exposed
Speaker 1: an API that you can access just through a low URL.
Speaker 1: So if you can write a c sharp app that
Speaker 1: you know uses an HDP client with a local URL,
Speaker 1: you form that URL with the commands and data that
Speaker 1: you want, and you can pretty much do everything that
Speaker 1: the app does programmatically from your own interface.
Speaker 3: Yeah, about eighty percent of it, but yeah, it's pretty
Speaker 3: pretty close. We use it. We use it a fair
Speaker 3: We use a fair amount.
Speaker 2: The trick, of course, is getting all the data into
Speaker 2: your machine. But if you're going all Ethernet, if it's
Speaker 2: all POE and stuff, then you don't need a lot
Speaker 2: of hardware on your machine to make this work. Yeah,
Speaker 2: avast of computer.
Speaker 1: I've done multiple events with v mix and I've done
Speaker 1: I did one from my home, and I did one
Speaker 1: down in Brooklyn at a at an event down there
Speaker 1: that was just amazing. And that was the probably the
Speaker 1: hardest event I ever had to do because there was
Speaker 1: so many cues and so many buttons and so many
Speaker 1: things to think about. It can get overwhelming. And I
Speaker 1: think that's why if you get to that level, people
Speaker 1: I know would move up to the TriCaster.
Speaker 3: Or you're loading your desk up with a whole whack
Speaker 3: of stream decks and using bit Focus Companion.
Speaker 2: There you go.
Speaker 1: That's talk about that.
Speaker 3: So I've got on my desk, I've got three stream
Speaker 3: Deck XLS plus along with A plus and those are
Speaker 3: all being used to control you know, which presenter goes
Speaker 3: into which box on screen, which layout we've got, you know,
Speaker 3: transitioning doing takes muting music on muting like it does
Speaker 3: our whole show automation.
Speaker 1: So stream Deck is a hardware device with a bunch
Speaker 1: of buttons on it that are actually you can load
Speaker 1: up with images so in text, so you customize it
Speaker 1: for what you want to do on your particular event
Speaker 1: or stream or whatever. And as Elia said, you just
Speaker 1: pushed the buttons at the right time.
Speaker 2: This is the Elgado hardware. I Gato.
Speaker 3: This is the Algato one, but you can pair it
Speaker 3: with a piece of software called Companion. It's made by
Speaker 3: Bitfocus and it's like if you thought the Algato software
Speaker 3: was awesome. The bit Focused Companion also allows you to
Speaker 3: do stuff like I'm going to dynamically update the text
Speaker 3: on a bunch of my buttons depending on if I'm recording.
Speaker 3: Most common use case I have for that is if
Speaker 3: vmex is not recording, I've got a button that is
Speaker 3: blinking red at me, not recording, not recording, not recording MM.
Speaker 3: That has saved me more than on so many.
Speaker 2: Times, because, yeah, the companion also is basically just a
Speaker 2: macro programmer. Whatever you want that button to do in
Speaker 2: virtually any other app, it'll do it.
Speaker 3: Yeah exactly.
Speaker 2: Yeah.
Speaker 1: You know, the definition of a genius in the twenty
Speaker 1: twenties is, right, someone who can push the right button
Speaker 1: at the right time. Oh, yes, to think about it
Speaker 1: all comes down to that.
Speaker 3: It's doing your thinking ahead of time.
Speaker 2: Yeah, exactly.
Speaker 1: So when we're talking about audio and video and programming
Speaker 1: that with dot net, are you talking about NDI with
Speaker 1: C sharp? Is there an NDIAPI for c sharp?
Speaker 3: So originally there was just the CAPI for NDI, but
Speaker 3: with the magic in vogue again, if you get the
Speaker 3: signatures right, you can basically take your header, your cheader,
Speaker 3: translate that into you know, your public extern, void calls
Speaker 3: or public extern whatever your struck name is right, and
Speaker 3: call into that c API. And as long as you
Speaker 3: do everything, as long as you decorate everything correctly, it
Speaker 3: all just works. And a lot of times even if
Speaker 3: you don't, the marshaler will be there to kind of
Speaker 3: save your bake and do things correctly for you. Strings
Speaker 3: especially is where things get really weird.
Speaker 1: So you know a questions coming next, Yes, have you
Speaker 1: ALISP created a dot Net rapper for the NDI API.
Speaker 3: So I created a dot Net rapper for the Advanced SDK,
Speaker 3: and I improved on the rapper that NDI themselves had
Speaker 3: originally released to make that portable too. They originally released
Speaker 3: it for just the dot net framework, and I ported
Speaker 3: that to dot net at the time dot net core
Speaker 3: and now just dot Net seven plus. Yeah.
Speaker 2: Wow cool.
Speaker 3: But here's the magic of that rapper though, And this
Speaker 3: is where p and voke really starts get powerful in
Speaker 3: the modern era of dot net, because when you think
Speaker 3: about it, dot Net originally was Windows only, right, And then,
Speaker 3: you know, Richard, I would be remiss if I didn't
Speaker 3: mention that the only reason that we have the portable
Speaker 3: CLR today, and we have a lot of what we
Speaker 3: have today is because of all the effort they made
Speaker 3: to make dot Net run in SQL server, right, right,
Speaker 3: there was that huge effort. What was that like two
Speaker 3: thousand and five?
Speaker 2: Yeah, yeah, well delivered in two thousand and five, but
Speaker 2: it had been working on since two thousand and three
Speaker 2: Net two.
Speaker 3: Right, that was dot Net two O and all the
Speaker 3: magic that we got during dot Net two O. So
Speaker 3: that work. So now that we have this cross platform
Speaker 3: dot Net library, you think about we have all of
Speaker 3: these native libraries across all these different platforms. NDI itself
Speaker 3: it has an SDK available for Windows, for Mac, for Linux,
Speaker 3: and then on top of that, for Linux we have
Speaker 3: X eighty six sixty four and we have ARMS sixty four,
Speaker 3: and then on Mac we also have eighty six to
Speaker 3: sixty four and ARM sixty four as well. Dot Net
Speaker 3: also targets.
Speaker 1: All those platforms, right, That's right.
Speaker 3: So I can take my dot Net app, I can
Speaker 3: compile it on all these platforms and it's going to
Speaker 3: run and not just that, but it's going to run
Speaker 3: more or less the same way across all the platforms
Speaker 3: because the same source code.
Speaker 2: I think you just made your argument for why I'm
Speaker 2: writing this is cea Sharp and said at C plus plus.
Speaker 3: Well, so philosophically, I was thinking about this before we
Speaker 3: got on. I was like, Okay, well, why are we
Speaker 3: if we're going to be calling into native libraries anyway,
Speaker 3: why are we even thinking about C sharp? Why is
Speaker 3: that even entering our thought process. When you think about it,
Speaker 3: there's only a small percentage of the operations that I'm
Speaker 3: doing that I really really need that low level raw speed.
Speaker 3: And I would even argue that whatever the dot net
Speaker 3: compiler does in the background, especially for talking about ahead
Speaker 3: of time compilation, is probably going to write faster, more optimized,
Speaker 3: and better code than you can by hand, unless you're
Speaker 3: really talking about assembly. In that case, that's a whole
Speaker 3: other special world.
Speaker 1: Who's going to do that?
Speaker 2: Then you got chromium twit tweezers out and you're stacking
Speaker 2: your own electrons and that's it.
Speaker 3: Oh yeah, exactly that. I mean you start a button. Yeah,
Speaker 3: yeah exactly, So like you can you can go do
Speaker 3: so we could write the whole application and say CE
Speaker 3: or C plus plus. But it's like I don't need
Speaker 3: that low level control all the time.
Speaker 2: And you don't want to pay the price of using
Speaker 2: that low level language all the time.
Speaker 3: Yeah, and there's a big price involved. Yeah, you know,
Speaker 3: I don't get automatic garbage collection. I don't get you know,
Speaker 3: I have to think about reference counting. I have to
Speaker 3: think about okay this because like, yes, there's type checking,
Speaker 3: but there is nothing stopping me from casting one pointer
Speaker 3: type to another in de referencing garbage.
Speaker 2: Sure, you know. At the same time, it's like, does
Speaker 2: a GC bonk here client? Like, I haven't thought about
Speaker 2: a bonked client by our garbage collection in a decade, easy, right,
Speaker 2: Like because things have been so fast and it just
Speaker 2: stopped being an issue, like noticing the garbage that the
Speaker 2: guar coelector was running is a two thousand and five thing,
Speaker 2: not a twenty twenty five thing.
Speaker 3: Sure, and there's ways to mitigate that too, if you're
Speaker 3: smart about memory management, right, you know you don't You
Speaker 3: don't have to worry so much about that these days.
Speaker 2: Yeah, rule number one have more memory.
Speaker 1: Yes, So I was thinking about this. You know, Let's
Speaker 1: say we have written our p and voke stuff for
Speaker 1: Windows and we want to run it on the Mac.
Speaker 1: If you're doing a what is it, you know, a
Speaker 1: MAUI app that can run on the Mac. Right, Those
Speaker 1: P and voke C sharp extern method declarations from what
Speaker 1: I remember, include the name of the library that you
Speaker 1: need to call into, like by name Windows there dot DLL.
Speaker 1: But what are they on the Mac? Do you have
Speaker 1: to change those on each platform?
Speaker 3: So if the marsh the p and VOKE libraries. They
Speaker 3: provide a mechanism that instructions dot net how to look
Speaker 3: up that DLL or how to look out that dynamic library.
Speaker 3: So on Windows, Mac and Linux, you can just generically
Speaker 3: say we're dealing with that dynamic link library. The file
Speaker 3: name is different, but the concept is the same. This
Speaker 3: is a library. It contains some sort of unmanaged code,
Speaker 3: and I want to call into it.
Speaker 1: Great, So you don't you have one codebase and it
Speaker 1: literally goes from platform to platform without modification. You don't
Speaker 1: have any pragmant statements for the platform that you're on.
Speaker 3: See that's and that's the other thing is like if
Speaker 3: you think, I think about doing the code portability stuff
Speaker 3: back in the WPF silver Light days. So like back
Speaker 3: when I was in university, one of my one of
Speaker 3: my co op placements was to write was with a
Speaker 3: broadcaster and I had to write a timeline control for
Speaker 3: a non linear editor like you would see in Adobe
Speaker 3: Premiere or after Effects or something like that. Right, But
Speaker 3: the use case was this needed to be cross platform
Speaker 3: for both WPF and silver Light and had to be
Speaker 3: the same code base.
Speaker 2: And so to make the portable class libraries at the time.
Speaker 3: Yeah, this was Yeah, that was PCL, right, so which
Speaker 3: they moved away from, right, like, oh yeahcls have been retired. Yeah, PCL.
Speaker 3: Because we did PCL, that gave us a baby step
Speaker 3: to dot nets standard once we kind of split, and
Speaker 3: then dot net Standard was kind of the way for
Speaker 3: a little while and then we realized, okay, let's just
Speaker 3: open source dot net and we're going to go, and
Speaker 3: that's good his future.
Speaker 2: Well, and because the PCL approachment that every time you
Speaker 2: added a platform and it added complexy to everyone else,
Speaker 2: like it was an N over N minus one problem
Speaker 2: and it just got bad. Yeah as the count went up.
Speaker 3: Yeah, and it's and you also ran into hey, this
Speaker 3: is supported on this platform. So you still needed to
Speaker 3: do some checks, right, Yeah, And to a degree, you
Speaker 3: still need to do certain checks depending on what you're
Speaker 3: doing and how you're using p and voke and whatnot,
Speaker 3: Like there are certain educating conditions that do matter that
Speaker 3: you need to check. But back in those days when
Speaker 3: I was doing WPF and silver Light, I was doing
Speaker 3: a lot of if deaf and then checking for certain
Speaker 3: compiler flags right right when I'm doing my you know,
Speaker 3: where should I find the NDI library. What's it called
Speaker 3: on this platform? That is a native if statement in
Speaker 3: c sharp. Nice, So that is compiled into each version
Speaker 3: and that check is being done at runtime.
Speaker 2: Okay, all right, which he also gives you room for
Speaker 2: new versions and new configurations. It should just work.
Speaker 1: Yeah, right, all right, this is a good time. We're
Speaker 1: going to take a break, I guess and about halfway
Speaker 1: through the show stick around after these very important messages,
Speaker 1: we'll be right back with more from Elias Purinin. You know,
Speaker 1: dot net six has officially reached the end of support,
Speaker 1: and now is the time to upgrade. Dot Net eight
Speaker 1: is well supported on AWS. Learn more at aws dot
Speaker 1: Amazon dot com, slash dot net. And we're back. It's
Speaker 1: dot net Rocks. I'm Carl Franklin, that's Richard Campbell, hey,
Speaker 1: and that's Elias Purinin. We're talking p and voke and
Speaker 1: audio and video stuff that you can do in dot
Speaker 1: net and c sharp. Now you've written a rapper for NDI,
Speaker 1: but you've also written rappers for other native you know,
Speaker 1: libraries and things. What's what's another cool one you want
Speaker 1: to talk about?
Speaker 3: So for my NDI multiview, which kind of which is
Speaker 3: where all this p and voke. Goodness came from for
Speaker 3: me and where I saw the light of pnvoke and
Speaker 3: dot net in general. I as part of that, I
Speaker 3: needed to write against the NDI Advanced SDK because I
Speaker 3: wanted to be able to send out multiview outputs in
Speaker 3: a technology called NDI HX. So NDI has two major flavors.
Speaker 3: There's HX, which is compressed either H two sixty four
Speaker 3: H two sixty five or the full bandwidth version, which
Speaker 3: is used as something called speed HQ.
Speaker 2: Because I want to make my Ethernet cables glow yes white.
Speaker 1: Speed HQ maybe a misnomer, let's see.
Speaker 2: Yeah, there's a lot of data in this stuff HQ.
Speaker 3: They're big frames, but yet still compressed. So that's that's
Speaker 3: the thing. One thing a lossless compression, A lossless compression exactly.
Speaker 2: Yeah, where two sixty four five are both loss e compressions.
Speaker 3: Yeah. So one of the but one of the reasons
Speaker 3: I want to do the HX output is because there's
Speaker 3: two decode H two sixty four two sixty five. Pretty
Speaker 3: much anything on the planet can do that real time now,
Speaker 3: and there's a lot of devices coming out now that
Speaker 3: if they do NDID code they only do HX, so
Speaker 3: for so for me, I wanted to be able to
Speaker 3: send out an NDI feed as HX so that we
Speaker 3: could send it to those devices. Opens up a couple
Speaker 3: other categories two and so I went and took a look,
Speaker 3: and there's there were a bunch of N Video. There
Speaker 3: were some nvideo wrappers out there already for c SHARP
Speaker 3: because what every if you've got an N video graphics card,
Speaker 3: you have access to something called the end video encoder
Speaker 3: or en VANC. And so this is the hardware on
Speaker 3: your card based H two sixty four, H two sixty
Speaker 3: five and depending on new your card is av one.
Speaker 2: Isn't this Kuda under the hood.
Speaker 3: So it can use Kuda under the hood. There's also
Speaker 3: ways to access it via DirectX and with OpenGL context
Speaker 3: I believe, which gives you a little bit of more platform.
Speaker 2: See look at this using video cards for video not AI.
Speaker 2: I'm so excited.
Speaker 3: It's a cool X it's your foot, isn't it cool?
Speaker 3: We can actually use GPUs for video again, Yeah, what
Speaker 3: a concept. It's amazing. To give you an idea of
Speaker 3: how fast it encodes though, I can put a ten
Speaker 3: EIGHTP sixty or I can put a ten ADP nineteen
Speaker 3: twenty by ten eighty video frame into endvanc and get
Speaker 3: an encoded frame out in about two to three milliseconds.
Speaker 2: Wow, Jayvers Miles and marsh Miles. And you're not talking
Speaker 2: at fifty ninety here, you're talking about like a ten eighty.
Speaker 3: Yeah, a ten eighty. Like you know, I've got a
Speaker 3: machine here that's got like a forty sixty, forty seventy.
Speaker 3: It is like two three milliseconds. It's nothing, it's nothing.
Speaker 2: Yeah, now you'll be able to keep up. You could
Speaker 2: do a bunch of those, like, yeah, sixty frames per second?
Speaker 2: How many can you do? Five off that card real time?
Speaker 3: So again depends on the card. That card I think
Speaker 3: is just hardware is limited by the drivers to five.
Speaker 3: But like if you go with the RTXA four thousands
Speaker 3: or like the really professional grade.
Speaker 2: Cards, which is what they're bill for. Yeah, the work
Speaker 2: those workstation class ones are designed for multiple parallel threads,
Speaker 2: which is not great for gaming, but for this application,
Speaker 2: that's brilliant.
Speaker 3: Oh, they are absolutely beefy cards. They're super cool.
Speaker 2: Yeah.
Speaker 1: Anybody who knows what I do knows that I create
Speaker 1: videos and things, and I'm a big fan of Adobe Premiere.
Speaker 1: But I remember in the days when Adobe Premiere Pro
Speaker 1: didn't have access to the GPU for rendering, and it
Speaker 1: was just using the CPU and it was slow, and
Speaker 1: it's like, oh yeah, you got to go turn on
Speaker 1: this thing. And then it was it was faster, it
Speaker 1: was much faster. Now now I can take an our
Speaker 1: video and render it on my I nine, you know,
Speaker 1: with my Nvidia card in maybe five minutes.
Speaker 2: Yeah.
Speaker 3: Yeah, it's that's that's really cool. That's that's the beauty
Speaker 3: of the modern GPU.
Speaker 2: Yeah.
Speaker 3: So, and because I've got access to that, you know,
Speaker 3: I've got access to those two GPU n code and decode.
Speaker 3: Like all of a sudden, I can sort of get
Speaker 3: around licensing around h two sixty four and two sixty
Speaker 3: five because that's a whole weird legal beast that I'm
Speaker 3: not even going to touch.
Speaker 1: So you created wrap around the end Video encoery encoder,
Speaker 1: so INVEC.
Speaker 3: Yeah, So ENVEC gives you if you go on end
Speaker 3: video site, you can download the CSDK for ENDVEC and
Speaker 3: I call it end bank.
Speaker 2: I don't know why, but I think you're right.
Speaker 3: Yeah, that's that's how I'll refer to it. I'm used
Speaker 3: to that one. So you download that, you get a
Speaker 3: c API. But what's interesting in the if you look
Speaker 3: at the and we've provided links. I've provided a bunch
Speaker 3: of show links for this. You can find it if
Speaker 3: you look uptractice dot encoders on GitHub.
Speaker 1: We'll put them on the main page too.
Speaker 3: I have yeah, so we'll be on the main page.
Speaker 2: Two.
Speaker 3: So if you look up, if you look up that library,
Speaker 3: you'll notice that I actually only have one direct P
Speaker 3: and voke function call into envank, and that is to
Speaker 3: initialize the library and get back astruct. Now what does
Speaker 3: that struct contain. It contains a bunch of unmanaged function
Speaker 3: pointers to whatever version of the API I'm referencing, so
Speaker 3: for backwards and forwards compatibility, like if i'm I can
Speaker 3: say I'm on version twelve, I've compiled against version twelve
Speaker 3: of the nvank SDK, So that gives me access to
Speaker 3: anybody that's running n Video Drivers five twenty and above,
Speaker 3: which is striking a nice balance between supporting some legacy
Speaker 3: stuff like that goes back I think to about twenty nineteen,
Speaker 3: twenty twenty, but not being so new that like you
Speaker 3: need a fifty ninety and all the latest drivers to
Speaker 3: run it right.
Speaker 2: Yeah, and well, let's face it, the fifty ninety is
Speaker 2: crazy overcow for virtually everything pretty much. Yeah, except we're
Speaker 2: just just taking money off you. You know, I'm realizing
Speaker 2: there's Envy Inc and Envy Deck, so I can't I
Speaker 2: don't think he can say n bank because you can't
Speaker 2: say env deck. Ah.
Speaker 1: True, that would be the decoder version.
Speaker 2: Yeah, there's an encoder and a decoder. This is legit,
Speaker 2: you know, we get this. Yeah, so I think if
Speaker 2: I'm going to go for a naming consisting, it's going
Speaker 2: to be Envy inc, an Envy Deck, envanc en v deck.
Speaker 2: I like that. They're still awkward, just bank deck like
Speaker 2: modem because n bank is easier to say no toys
Speaker 2: about them. But there's nothing nothing you do about the decoder.
Speaker 2: It's glunky.
Speaker 3: Yeah. Yeah, at least it's not WPF.
Speaker 2: They'll save me.
Speaker 3: Who was who was it that was ranting about that
Speaker 3: way back in the day we can't have actros w No,
Speaker 3: it was it.
Speaker 2: Was Brad Abrams who said, listen, if we give the
Speaker 2: project a good code name, it gets a tear product name. Yeah.
Speaker 2: So that's why the code name for silver Light was
Speaker 2: WPF slash E because that's the worst code name ever.
Speaker 2: So the product name was awesome.
Speaker 1: I was just waiting for Windows Transaction Framework, which would
Speaker 1: be WTF.
Speaker 2: He said, where did my data go?
Speaker 3: So, but getting back to the getting back to the
Speaker 3: n bank wrapper library. So there's only one direct p
Speaker 3: in voke function call in, and that gives you a
Speaker 3: struck back and that struct has nothing but function pointers, right,
Speaker 3: and that's how you actually interact with the API. That's
Speaker 3: how they're maintaining that backwards compatibility. So even if I'm
Speaker 3: running the.
Speaker 2: This is really not that horrible, right. Once you've got connected,
Speaker 2: it's really smart. Then you're just feeding it data. It's
Speaker 2: just just dream.
Speaker 3: And then throughout that library, I then have a bunch
Speaker 3: of Marshal delegate to function pointer. I forget the exact
Speaker 3: get delegate for function pointer calls, so it's Marshal dot
Speaker 3: debt get delegate for function pointer. I pass it in
Speaker 3: the end pointer, you know, native end pointer for each
Speaker 3: of those function each of those function pointers. And I
Speaker 3: now have a first class c sharp delegate that I
Speaker 3: can call. Great, so now I can take my whatever
Speaker 3: memory I've got allocated, that's got my raw video frame.
Speaker 3: I can now say, okay, here's a point of that
Speaker 3: memory end bank. Here you go, give me a frame back,
Speaker 3: and it all just works. And I'm doing all of
Speaker 3: these calls in c sharp.
Speaker 1: And if it changes, you don't care because maybe the
Speaker 1: memory changes, maybe the pointer changes, but that's.
Speaker 3: Fine, exactly. So I'm I'm shielded from all that stuff.
Speaker 3: I get the advantages of native, of the speed of
Speaker 3: the native code, but I'm also able to and i
Speaker 3: can access all these native libraries. But I've also got
Speaker 3: a lot of the protections that c sharp gives me
Speaker 3: out of the box.
Speaker 1: Yep, You've got another layer of abstraction there, which is
Speaker 1: always a good.
Speaker 3: Thing, exactly. And as long as I've got all of
Speaker 3: my strucks set up the right way, and I've got
Speaker 3: you know, and I've been very careful about how I've
Speaker 3: instructed dot net about how these structs are laid out,
Speaker 3: everything just works. That's a big.
Speaker 1: Asterisk, though, I jw technology.
Speaker 3: It just works. Technology it works works.
Speaker 2: Yeah, what's the asterisk?
Speaker 3: So in some cases you do have to be very
Speaker 3: specific about what the unmanaged code is expecting in terms
Speaker 3: of how is this struct laid out? How should this
Speaker 3: If I'm p invoking how should this function be called?
Speaker 3: Because there's the there's standard call. In the unmanaged world,
Speaker 3: we have standard call, which is the way that we
Speaker 3: usually call into like a WIN thirty two API for example,
Speaker 3: and standard call one of the versions it's the caller
Speaker 3: is responsible for cleaning up the stack once the function returns,
Speaker 3: and in the other way, the call e is responsible
Speaker 3: for cleaning up the stack. It's the collie that's responsible
Speaker 3: for cleaning up the stack in a standard call call,
Speaker 3: whereas with a with a C decal call, the caller
Speaker 3: is responsible for cleaning up the stack. Which is why
Speaker 3: and C you're able to do like a print aff
Speaker 3: with your string, your format string, and then you can
Speaker 3: put as many parameters after that as you want because
Speaker 3: it's an also for the standard call. That's why if
Speaker 3: you ever crack open you know, a C plus plus
Speaker 3: library in a tool like GEDRA, reverse engineering tool like GEDRA,
Speaker 3: you'll see these really awful mangled names like you know,
Speaker 3: get function pointer at three and that at three tells
Speaker 3: you this version this overload is expecting three parameters, no more,
Speaker 3: no less, okay, and if you call it with more
Speaker 3: things could happen.
Speaker 1: Self documenting sort kind of sort of sort of you
Speaker 1: don't know what they are, but that's why you have
Speaker 1: to see sure, you.
Speaker 3: Know, you could just you know, like anything else, like
Speaker 3: sometimes you just got to poke the function calls and
Speaker 3: see what happens. Just like let's call it with five.
Speaker 3: Let's see if it works.
Speaker 1: Now, you had a specific need to create this wrapper,
Speaker 1: but what might other people need to use it for?
Speaker 3: So for Endvank, it's part of my philosophy of creating
Speaker 3: some of these rappers was like was to was to
Speaker 3: show that, look, if you want to get this high
Speaker 3: performance real time audio and video, you can totally use
Speaker 3: a higher level language like c sharp because when you
Speaker 3: need to drop down and get that crazy speed, you can.
Speaker 3: M h. So let's talk about crazy speed. Let's talk
Speaker 3: about unsafe code, and let's talk pointers.
Speaker 1: Yeah, so love unsafe code reminds me of my teenage
Speaker 1: days driving a car at one hundred miles an hour
Speaker 1: through the town. Chickens flying everywhere, mothers pulling their children
Speaker 1: off the street.
Speaker 3: Go ahead, that sounds like an interesting hobby. I just tried.
Speaker 1: I had to pick up the feathers afterwards, though, I had.
Speaker 3: Oh man MESSI my biggest adventure of driving a car
Speaker 3: was knocking over construction fences.
Speaker 1: There you go, all right, well that could be unsafe.
Speaker 3: Yeah. So if you compile your C sharp code with
Speaker 3: you know this, with slash unsafe for you marketing your
Speaker 3: project file that you want unsafe code, then you can
Speaker 3: use what's called Then you can use the unsafe keyword.
Speaker 3: Now what does unsafe do? This allows you to do
Speaker 3: stuff like pin objects in memory and get raw function
Speaker 3: point and get raw pointers. So you would think, okay,
Speaker 3: So a common misconception is that C sharp doesn't have pointers.
Speaker 3: Oh yes it does. Let me tell you about them.
Speaker 3: So there is if you're working in an unsafe context,
Speaker 3: there is nothing stopping you from calling Marshall dot alec
Speaker 3: h Global and saying I want a big chunk of
Speaker 3: memory and it'll go cool. Here's a pointer that memory.
Speaker 3: You can then cast that to whatever struck type you
Speaker 3: want and bam, all of a sudden, I have pointer
Speaker 3: de referencing arrows in my C sharp code.
Speaker 1: But you said the magic word though, which is pinning. Yeah,
Speaker 1: if you don't pin it, it could move around on
Speaker 1: you and you go to access it and you're writing
Speaker 1: over something and security.
Speaker 2: We have a.
Speaker 1: Problem over here.
Speaker 2: Bad things happen.
Speaker 3: Oh yeah, you could. You could get into big problems
Speaker 3: real fast.
Speaker 1: So but pinning is the key. It basically says, don't
Speaker 1: move from this spot right here.
Speaker 2: Yeah.
Speaker 3: Now, from my experience, I when I've been working in
Speaker 3: modern dot net, I found that I've had to do
Speaker 3: pinning a lot less than I used to. Yeah, Like,
Speaker 3: it seems like there's been a lot of work going
Speaker 3: behind the scenes. And it feels like that is because
Speaker 3: of dot net running on mobile platforms and running on
Speaker 3: mac os. Like that work seems to have you know,
Speaker 3: the native interrupt story is way different from like the
Speaker 3: dot net one one two oh days. But you can,
Speaker 3: but you can totally allocate just a block of memory,
Speaker 3: and I actually do this, like for my advanced SDK
Speaker 3: rapper and for some of the encoding that we're doing
Speaker 3: behind the scenes. What NDI expects when I send a
Speaker 3: compressed frame is that it wants in memory physically laid out.
Speaker 3: It wants the struct header that contains information about resolution,
Speaker 3: frame rate, data length, and then after that in memory
Speaker 3: should follow the h two sixty four bitstream. So the
Speaker 3: actual stuff that comes out of the encoder, right, that's
Speaker 3: a little bit hard to do in see you can
Speaker 3: do that. You could just allocate a bunch of memory
Speaker 3: for whatever memory you want for your frame size, plus
Speaker 3: you know, say forty four bites for your for the
Speaker 3: struct header, and you new memcopy in C it's a
Speaker 3: in C sharp, it's a little bit more difficult to
Speaker 3: do that, but you can still do that because you
Speaker 3: can allocate memory marshall allocate global that's equivalent to your
Speaker 3: to your mallek and sea land. You can then cast that.
Speaker 3: You then cast that to a pointer to whatever struct
Speaker 3: type you're dealing with, and in your struct declaration you
Speaker 3: can add attributes that say hey, this struct layout is sequential,
Speaker 3: so don't move stuff. So dot net compiler, don't try
Speaker 3: to get clever with moving my different fields around to
Speaker 3: do memory alignment or you know, save space or do
Speaker 3: anything clever. Just like how I've laid this out, Like
Speaker 3: if I if my first field is an ind to,
Speaker 3: my second field is a long, and my third field
Speaker 3: is a short, like I want it laid out exactly
Speaker 3: like that in.
Speaker 1: Memory Yeah, sequentially exact.
Speaker 2: Yeah.
Speaker 1: So if you do that, you have your own safe code,
Speaker 1: your global maloc and you pin it is it really unsafe?
Speaker 3: So you can still dereference, you know, something outside of
Speaker 3: the bounds of the memory that you've allocated. So in
Speaker 3: terms of unsafe, it's kind of like saying, you know what,
Speaker 3: if you want to dip into doing you know, memory
Speaker 3: management and pointers and all that stuff yourself, you can
Speaker 3: totally do that. It's your foot, It's exactly c Sharp's
Speaker 3: like a big Swiss army knife, right, It's got all
Speaker 3: these different tools and abilities, and then you pull out
Speaker 3: the final one and it's a big friggin' you know,
Speaker 3: precision rifle, perfect for your foot.
Speaker 2: Nice.
Speaker 1: It's got everything, so it's not gonna it's not going
Speaker 1: to move on you, which is one of the problems
Speaker 1: and one of the reasons we have managed you know,
Speaker 1: frameworks in the first place, like that net. But you
Speaker 1: can still overcopy, under copy, like you said, de reference
Speaker 1: to the wrong thing. You can still do all those
Speaker 1: stupid things, and the compiler is probably not going to
Speaker 1: help you all that much.
Speaker 3: But it also depends to how you're interacting with that
Speaker 3: data that you are or those or the pieces of
Speaker 3: memory that you're working without a low level because you
Speaker 3: can also take a byte pointer or a void pointer
Speaker 3: whatever you want, like say I've allocated a big chunk
Speaker 3: of memory, and I want to pass that chunk of
Speaker 3: memory to a managed function that expects a spam like
Speaker 3: a SPAN of byte I can create. I can instantiate
Speaker 3: a SPAN byte, pass in that pointer to memory and say, hey,
Speaker 3: this is you know, two thousand and forty eight four thousand,
Speaker 3: ninety six bytes long. I now have a SPAN rapper,
Speaker 3: like a first SPAN wrapper around this chunk of memory
Speaker 3: that is quote unquote unsafe, and I can do all
Speaker 3: of my dot net goodie things to it.
Speaker 2: Right.
Speaker 1: Yes, SPAN is a wonderful, wonderful construct.
Speaker 3: And like I use that, for example to as part
Speaker 3: of encoding that ndiachex frame. I need to tell NDI
Speaker 3: here's how long the actual frame data is and all
Speaker 3: of the ancillary data, so like when this frame should
Speaker 3: be presented, that's how long this is. I have a
Speaker 3: parser at h two sixty four bitstream parser that is
Speaker 3: written in C sharp. It's all managed code that uses
Speaker 3: SPAN to go through and see, oh, here's where this
Speaker 3: part starts, here's where this part ends, here's what needs
Speaker 3: to be copied. So I'm doing all of that checking
Speaker 3: and it's I've got all the type safety and I've
Speaker 3: got all the bounds checking right, Yeah, so I'm not concerned.
Speaker 3: So I'm not as concerned about you know, overrunning underrunning,
Speaker 3: like as long as I'm as long as I'm instantiating
Speaker 3: with SPAN correctly. I've got all the type safety of
Speaker 3: C sharp.
Speaker 1: And that's SPAN. We've talked about that that span before,
Speaker 1: about span, and just remind us what it is from
Speaker 1: what I remember, it's sort of like a fixed position
Speaker 1: array of stuff that isn't going to move around under you, right.
Speaker 3: And it wraps it without doing memory copies.
Speaker 2: Yeah.
Speaker 3: So and then if you want, say you want to
Speaker 3: get you know, you've got a SPAN that's two that
Speaker 3: wraps a two megabyte bite array or a two megabyte
Speaker 3: bite pointer, and you want to get the data at
Speaker 3: offset five hundred and twelve K to oneenty twenty four K.
Speaker 3: But you don't want to copy that. You just want
Speaker 3: a you just want a slice of a slice of
Speaker 3: that to view it basically or to do something else
Speaker 3: with it, but you don't want to make a copy
Speaker 3: of it. You can say span dot slice I believe,
Speaker 3: and that will give you a SPAN without doing any
Speaker 3: memory copies that starts at that offset that you specify cool,
Speaker 3: so then you can do whatever you want to that nice, Yeah.
Speaker 2: Very nice.
Speaker 1: I remember talking to Mark Mark Heath of an i
Speaker 1: udio fame about spans when I first heard about them,
Speaker 1: and I think he implemented them behind the scenes in
Speaker 1: and audio and made it more performance and safer.
Speaker 3: Oh yeah, and it's it. What is really awesome about
Speaker 3: is you get you also get access to all the
Speaker 3: first class stuff like for each four loops. Yeah, and
Speaker 3: you're not having to do that manual You're not having
Speaker 3: to do the manual pointer arithmetic as well.
Speaker 2: Yeah.
Speaker 3: Again you can in unsafe mode. You can totally do
Speaker 3: that and it will, it behaves exactly. But you have
Speaker 3: to really think about it.
Speaker 1: Yeah, more than I want to think about plumbing.
Speaker 3: Yeah, but you think about it once and it's done.
Speaker 2: That's why we like the library builders too. Absolutely may
Speaker 2: make our lives easier for this stuff. I might include
Speaker 2: a link to an audio now on GitHub. I know
Speaker 2: I referred to it in the comment as on CodePlex,
Speaker 2: but they moved. Well.
Speaker 3: Remember once upon a time stuff used to it was CodePlex.
Speaker 3: Stuff also lived on code project for a long time.
Speaker 2: Yeah, code project. Yeah.
Speaker 3: Number of libraries that would just be like, hey, here's
Speaker 3: the DLL. Go have fun.
Speaker 2: Then one day, I mean GitHub has won. Right, It's
Speaker 2: almost not a conversation anymore, pretty much just the way
Speaker 2: it is. Well, remember their original tagline was social coding,
Speaker 2: and we realize how important that is, Like I look
Speaker 2: to the issue log to understand the health of a project,
Speaker 2: Like the way people talk about that code matters a lot.
Speaker 1: Yeah, oh yeah, intrinsics. Okay, what's that about.
Speaker 3: So not entirely related to p and voke, but it's
Speaker 3: a big deal when it comes to performance and dot net.
Speaker 3: And this is kind of one of the arguments about,
Speaker 3: you know, should we write our high performance code in
Speaker 3: C sharp or should we write our high performance code
Speaker 3: in a lower level language like C. And one of
Speaker 3: the items is I want to use hardware accelerated vector
Speaker 3: operations or single instruction multiple data SI MD instructions. Where
Speaker 3: this really comes into play is you know, think about
Speaker 3: like you're doing matrix transformations way outside my pay grade.
Speaker 3: I'm not even going to pretend that I'm good at
Speaker 3: that stuff. There are instructions that many modern CPUs support
Speaker 3: that allow you to say, you know, do a bunch
Speaker 3: of additions all at the same time, or like, you know,
Speaker 3: do four data rights at the same time instead of
Speaker 3: having to do you know a four each a four
Speaker 3: loop where it's like I'm going to process. Well, where
Speaker 3: I do it the most is transforming filling black for example. Sure,
Speaker 3: so big operation in my multiview is after every frame
Speaker 3: I have an I have four or five frame buffer
Speaker 3: setup that I draw to. And so after I'm done
Speaker 3: drawing to, before I draw the next frame, the first
Speaker 3: operation I need to do is fill black. So just clear,
Speaker 3: just blank out that frame for all rows and then
Speaker 3: for all columns exactly exactly. So if I do so,
Speaker 3: if I do the purely naive version where I'm just
Speaker 3: I'm literally writing black, you know, a black byte pixel
Speaker 3: across all the pixels in that frame buffer. On one
Speaker 3: of my target systems, that takes a millisecond. And now
Speaker 3: you think, okay, a millisecond. So what if I'm running
Speaker 3: at sixty frames per second, I get sixteen milliseconds in
Speaker 3: which to realize a frame. Yeah, that's not a lot
Speaker 3: of time, and so a full one to sixteenth of
Speaker 3: that time is now spent just filling the thing with black.
Speaker 3: If I go a little bit smarter and I say, okay,
Speaker 3: I know that I'm essentially writing a constant, So I'm
Speaker 3: just going to write a thirty two bit int, you know,
Speaker 3: and then do skip four bytes out of time. So
Speaker 3: I'll write four bytes skip, write four byte skip. That
Speaker 3: time goes down to about two hundred and seventy one
Speaker 3: or zero point two seven one milliseconds or two hundred
Speaker 3: and seventy one microseconds. Right. Nice, So we've already saved.
Speaker 3: So just doing that hack has saved us some time.
Speaker 3: If I do sixty four bits, so I do longs,
Speaker 3: I bring that time down to one hundred and ninety microseconds.
Speaker 2: Nice.
Speaker 3: If I use avx two, which is a single instruction
Speaker 3: multiple data extension for the X eighty six platform, that
Speaker 3: code is super crazy, and I will fully admit that chat,
Speaker 3: GPT and lot helped me write that and make it
Speaker 3: work at all.
Speaker 1: Ah excellent.
Speaker 3: Yes, but the original naive version took one millisecond one
Speaker 3: forty three microseconds. The SIMD version took seventy two microseconds.
Speaker 1: Well, jeez, that's significant.
Speaker 3: Think about this. Yeah, significant, not only significant, but it's
Speaker 3: in C sharp and I'm accessing and I'm accessing, say
Speaker 3: I'm accessing raw hardware essentially raw hardware instructions. So instead
Speaker 3: of having to write that performance critical code like normally
Speaker 3: if you wanted to access AVX or SIMB instructions, you
Speaker 3: would be going into C Like that's just that's just
Speaker 3: a done deal, dude.
Speaker 1: You can now write the trippiest screensaver ever, Yes, in
Speaker 1: C sharp. Yeah, I knew you would come to the
Speaker 1: killer app and sooner or later we can find it's
Speaker 1: finally here.
Speaker 3: We can finally do all the plasma moost scene effects
Speaker 3: we've always wanted.
Speaker 1: Absolutely in hardware, but so which you can then ingest
Speaker 1: into a v mix thing and you know, while you wait,
Speaker 1: you could just you know, be rendering that stuff.
Speaker 3: Well yeah, exactly. Another place the SAMD is important for too,
Speaker 3: is say I'm doing pixel format conversions where I know,
Speaker 3: I you know, I'm i've NDI likes YUV four two two,
Speaker 3: so I have. And the way the yav pixels work
Speaker 3: is basically the human eye is more sensitive to brightness
Speaker 3: than it is to changes in color. So what we're
Speaker 3: what we can do on most modern displays is say, hey,
Speaker 3: I'm going to encode my luminance. My brightness is going
Speaker 3: to be encoded at full resolution so I have nineteen
Speaker 3: twenty by ten eighty resolution for luma, and then for
Speaker 3: my color data. I can take advantage of the fact
Speaker 3: that the human eye kind of sucks, and I can
Speaker 3: do I can encode a macropixel and say, for every
Speaker 3: one bit of you and V data my chroma data,
Speaker 3: that will be decoded into two pixels worth of color data.
Speaker 3: So if I combine the luma and the chroma together,
Speaker 3: I get reproduction of color.
Speaker 1: Why am I thinking of Gaussian splatting right now? Do
Speaker 1: you know what that is?
Speaker 2: I don't.
Speaker 1: All right, I'm not even gonna we're gonna take that
Speaker 1: out Brandy, because it's only good if he's up on it.
Speaker 1: But I'll tell you while we're in an edit point. Okay,
Speaker 1: it's basically a way that instead of doing regular rendering,
Speaker 1: you can, oh, you know, like Gaussian blurring. He basically
Speaker 1: create these splats of starting with low resolution detail and
Speaker 1: then getting higher resolution over them, and for some way
Speaker 1: you can do like sixty frames per second rendering of
Speaker 1: a model in real time. Wow, And it's really amazing.
Speaker 1: So yeah, go look that Upusian splatting, Gaussian slat latti NG.
Speaker 1: There's some great demos out there, but it's coming. It's
Speaker 1: a technology that's coming.
Speaker 3: That sounded like something I needed a towel for.
Speaker 1: Absolutely, you will you will need a towel when you
Speaker 1: did this stuff. All right, So what haven't we talked
Speaker 1: about before you before we wrap up here, anything else?
Speaker 3: I the philosophical side of YP and VOC is so important,
Speaker 3: Like because I took a little bit of time after
Speaker 3: doing after making all these wrapper libraries you know, around
Speaker 3: the around NDI, around n Bank, even starting on doing
Speaker 3: rapper libraries around Apple's Video Toolbox, which that's their hardware
Speaker 3: and code. By the way, the Apple silicon for doing
Speaker 3: hardware encode and decode of H two sixty four, two
Speaker 3: sixty five and AVY one is like worth the price
Speaker 3: of admission of having the burden of voting a Mac.
Speaker 3: It's just those chips are insanely good.
Speaker 1: Mm hmm, even on a MacBook.
Speaker 3: Yeah, on even on a base M one, yea, even
Speaker 3: on a base M one MacBook Air. It's unbelievable the
Speaker 3: amount of horsepower you get out of that machine. Yeah, yeah,
Speaker 3: it's it's just unbelievable. It's just unfortunate that it's a Mac.
Speaker 2: Yeah, but any of those all those m chips are extraordinary.
Speaker 2: Yeah they are.
Speaker 3: Yeah, But the philosophical importance of why P and voke
Speaker 3: is such an awesome technology, why it's underrated is because
Speaker 3: say I'm in Java land and I wanted to call
Speaker 3: into a unmanaged library. Their story has gotten a little
Speaker 3: bit better, but you are still needing to write an
Speaker 3: unmanaged wrapper that your Java library can call into. So
Speaker 3: think about the process. I want to call the Nvidia
Speaker 3: API from Java, I need to write an intermediary layer
Speaker 3: in C that I can then call from Java in
Speaker 3: order to do my P and voke.
Speaker 2: Yeah.
Speaker 3: Whereas with dot net, I, as long as I get
Speaker 3: the function signature is right, and as long as I
Speaker 3: annotate everything correctly, the compiler in the back end figures
Speaker 3: all that stuff out for me.
Speaker 1: Mm hmm.
Speaker 3: So what you to take, Like you could literally take,
Speaker 3: and I've done this with a couple of APIs. Take
Speaker 3: the C header file for you know, an SDK of
Speaker 3: some sort, put it into chat GPT. Say I need
Speaker 3: P and vote calls for this.
Speaker 1: I was gonna suggest that, yep, And it's exactly what
Speaker 1: I was gonna do.
Speaker 3: Yeah, and bam, You've now got a complete wrapper for
Speaker 3: any of this stuff chat.
Speaker 1: GPT is great for transforming something into something else.
Speaker 3: Yeah.
Speaker 2: Yeah.
Speaker 3: But the reason this is really cool is think about
Speaker 3: what we've got here. Think about the original design goal
Speaker 3: of the C language. We wanted to have portable code
Speaker 3: because we had all of these different architectures, they all
Speaker 3: operated differently. We wanted we wanted to have encapsuley essentially
Speaker 3: our logic into a higher level language that human beings
Speaker 3: can understand, because let's face it, you know, looking at
Speaker 3: a line that says tax, what the hell does that do?
Speaker 2: Right?
Speaker 3: But if you look at you know, say, uh, you know,
Speaker 3: frame rate equals sixty. Oh okay, I have a general
Speaker 3: idea that I'm setting a frame rate variable that represents
Speaker 3: the frame rate of some operation, and that is equal
Speaker 3: to the value sixty. I, as a human can understand that.
Speaker 3: The problem with the problem, I would say with C,
Speaker 3: maybe not even the problem, but just the limitation with
Speaker 3: C is because it is such a low level language,
Speaker 3: and the design goals of it were to balance out
Speaker 3: the raw hardware speed with a bit of human creature comfort.
Speaker 3: You don't have a whole lot of safeties there. We
Speaker 3: try to rectify that a little bit with C plus
Speaker 3: plus but that is a whole mess that I don't
Speaker 3: even want to talk about. I've done enough C plus
Speaker 3: plus CLI that it makes me hurt. I still have
Speaker 3: nightmares from those days.
Speaker 1: It hurts. It hurts my brain just thinking about it.
Speaker 3: Oh.
Speaker 1: I think Kate Gregory is the only one that actually
Speaker 1: enjoys writing C plus plus Kate.
Speaker 3: Kate Gregory is a champion. She she's like, the fact
Speaker 3: that she can make that stuff work is just I've
Speaker 3: infinite respect just yea too. I can listen to her
Speaker 3: speak for days on that stuff.
Speaker 2: But you've also found a way to get the kind
Speaker 2: of farms we're talking that we need here to handle
Speaker 2: multiple streams and not need to live in C plus
Speaker 2: plus line.
Speaker 3: Yeah, or or just live in seaaland when you need
Speaker 3: to for those for them, it's the application, because like
Speaker 3: do we need to write every last I'll give you
Speaker 3: an example for my multiview. I've got a web api
Speaker 3: h gdp api that you can use to make to
Speaker 3: change everything about the multiview. If you want to assign sources,
Speaker 3: if you want to move boxes on the screen around
Speaker 3: you know, why do I need to write that in
Speaker 3: C plus plus right? Like, there's no good reason for.
Speaker 2: It, no reason, and plenty of reasons not.
Speaker 3: To, Yeah, exactly, not least of which are security, right,
Speaker 3: Like how it's just.
Speaker 2: The fragility, the maintainability, like it's all those things.
Speaker 1: Yeah, So nothing else today, kids, you've learned that p
Speaker 1: and voke is way more powerful than you thought it
Speaker 1: was ever going to be back when you were calling
Speaker 1: the Windows API from vb net or c sharp, you know,
Speaker 1: one point zero. Because now the platform have expanded and
Speaker 1: we still have p and vogue, so we can take
Speaker 1: our dot net code and go anywhere and get to
Speaker 1: the native hardware. That's that's the philosophy that Elias is
Speaker 1: talking about here, that we have to get out of
Speaker 1: Windows mode in dot net, yes, and into cross platform mode.
Speaker 1: And this is just one example of that.
Speaker 3: Well here's one last point. I just thought of this,
Speaker 3: and this is like a penny drop moment for me.
Speaker 3: Imagine you've got some legacy code and I'm talking like
Speaker 3: the author, and I've been in the scenario where it's
Speaker 3: like literally the author of that original code has passed on,
Speaker 3: so we can't even go drag them out of drag
Speaker 3: them out of retirement and ask them like what does
Speaker 3: this do maybe we've lost the source code, but we've
Speaker 3: got this, you know, but we've got this wrapped into
Speaker 3: a library of some sort, or we can get close enough, right,
Speaker 3: And we don't want to rewrite this old piece of
Speaker 3: logic that lives in a c library, lives in an
Speaker 3: unmanaged library, but we need to use it for some reason. Okay, cool,
Speaker 3: we can write a if you can write C sharp,
Speaker 3: you can call into that existing DLL and that can
Speaker 3: just live on as a rapper. And because we can
Speaker 3: target X eighty six thirty two, X eighty six sixty four,
Speaker 3: ARM thirty two, ARM sixty four whatever platforms dot Net
Speaker 3: will compile down to. This is now a means of
Speaker 3: preserving and carrying forward that old legacy code, you know,
Speaker 3: in a modern language with all the modern creature comforts.
Speaker 2: And if you want to bring cameras into your software,
Speaker 2: use POE cameras. Your life will be simpler. Oh yes,
Speaker 2: please do no custom hardware. It's just on the network.
Speaker 2: It has an IP address. You'll be fine.
Speaker 1: Mm hmm, all right, And NDI is your uncle Elias.
Speaker 1: Thank you so much. This has been a great talk
Speaker 1: for me personally, and I hope everyone else that's interested
Speaker 1: in audio and videos.
Speaker 3: Thank you, it's been an absolute pleasure. I think I'm
Speaker 3: going to go off and see if I can adjust
Speaker 3: my sound last settings.
Speaker 1: Yeah, playjazz dot mid while you're at playjazz dot mid.
Speaker 1: All right, We'll see you next time on dot dot net.
Speaker 1: Rocks is brought to you by Franklin's Net and produced
Speaker 1: by Pop Studios, a full service audio, video and post
Speaker 1: production facility located physically in New London, Connecticut, and of
Speaker 1: course in the cloud online at pwop dot com. Visit
Speaker 1: our website at d O T N E t R
Speaker 1: O c k S dot com for RSS feeds, downloads,
Speaker 1: mobile apps, comments, and access to the full archives going
Speaker 1: back to show number one, recorded in September two.
Speaker 2: Thousand and two.
Speaker 1: And make sure you check out our sponsors. They keep
Speaker 1: us in business. Now go write some code. See you
Speaker 1: next time. You got Jack Dully to see A summer
Speaker 1: time that means is home than my Texas a lie
Speaker 1: fresival