Speaker 1: How'd you like to listen to dot NetRocks with no ads? Easy?
Become a patron For just five dollars a month you
get access to a private RSS feed where all the
shows have no ads. Twenty dollars a month will get
you that and a special dot NetRocks patron mug. Sign
up now at Patreon dot dot NetRocks dot com. Hey,
Carl and Richard here with your twenty twenty four NDC schedule.
Speaker 2: Will be at as many NDC conferences as possible this year,
and you should consider attending no matter what. The Copenhagen
Developers Festival happens August twenty sixth through the thirtieth. Tickets
at Cphdevfest dot com.
Speaker 1: Ndcporto is happening October fourteenth through the eighteenth. The early
bird discount ends June fourteenth. Tickets at Ndcporto dot com.
Speaker 3: And we'll see you there, we hope.
Speaker 1: Hey, guess what it's Dot need rocks again. I'm Carl Franklin,
that Richard Campbell here for the nineteen hundredth and seventeenth time.
Speaker 2: It's World War One. It's World War one. It it's
officially started. Okay, we're well into it at this point.
Speaker 1: Yeah, you're right, yeah, yeah, right, So yeah, what's up
with you, mister Campbell. We just got back from Dev Intersection.
Had a great time there. Yeah, I had a great
time there. I was just writing up some of the pieces,
the keynotes and things. I mean, it's a crazy time, right,
Like the software is moving fast. Some of the demos
were astonishing. You had a good workshop, You're real happy
with it, very happy. Yeah, it's like thirty people in
it or something, and all about Blazer. Jeff Fritz did
a sort of an intro to Blazer workshop, and I
did a more more intermediate advanced workshop, and I also
had a talk called Gotcha's in Blazer Web Apps and
not at eight because there are some gotcha's. There are
some things that Microsoft intended to work a certain way
and then they didn't really work. So things that work around.
Speaker 2: It's not that bleeding edge of technology anymore though. I
mean you're kind of at V three. Things are sort
of maturing out, like it's almost a grown up.
Speaker 1: Yeah, And that's kind of why it was surprising that
we had gotcha's in the first place. But I think
they took everybody by surprise, not that there aren't workarounds
for it. But you know, it's not a reason not
to use Blazer, but you just need to know what
they are.
Speaker 2: I think you've also got a moving surface around you too.
I've got the comment I've got will tie you into
that pretty well. Yeah, because there's things on change in
all the time.
Speaker 1: And just to do a little plug, I'm also doing
that Gotcha's talk with on Sean Walker's virtual event on
October third. I don't have a time yet, but look
for that. Okay, okay, So let's get into it with
a little thing. We call it better no a framework.
Speaker 2: Awesome, roll it all right, man?
Speaker 3: What do you got?
Speaker 1: Well, you know, this started out as me sort of
pointing to little things in the dot net framework. It
was even before dot net core, and quickly ran out
of stuff to talk about. So it just turns into
you know, hey, what caught my interest this week? I'd pointed.
Speaker 2: It was not that quick. There was a period we
were doing three of these a week, and you went
through the whole flip and framework.
Speaker 1: Yeah, pretty much. I mean the stuff that the stuff
that was tricky.
Speaker 2: Yeah, several years worth of content.
Speaker 1: Yeah, you're right. Okay, So I found this event in
the twenty through the twenty sixth of January coming up here.
It's called Global Game Jam.
Speaker 2: Oh, that sounds fun.
Speaker 1: Globalgamejam dot org. And so just like when musicians get
together and have jam sessions and they feed on each
other and get creative together and good stuff comes out,
that's the whole idea with this. It's it's there's sort
of like four hundred and thirty nine thousand jammers. Wow,
this is to date, eighty nine thousand games created, one
hundred and thirty two countries participated. So it's it's all
online of course, and they have a discord and a
Twitch stream, and they have videos from last year keynote.
It's just it looks like a lot of fun. And
the thing is too that it's not just for gamers
who you know, who have experience like this is for
people who might be just game curious, like they don't
know how games are created at all.
Speaker 2: And the tooling these days is so good, like, yeah,
don't need a huge team, you can do it by yourself,
although I've seen lots of teams of three be very
successful and make it maybe a mobile game, make a
web game, like there's a lot of choices right.
Speaker 1: Yeah, so that looks really cool. I might pop in
and check it out and maybe create some games. Might
be fun.
Speaker 2: Yeah, that's a good idea.
Speaker 3: Yep.
Speaker 1: So who's talking to us today?
Speaker 2: Richard and Grab a comment of a show eighteen fifteen,
So about one hundred shows ago with our friend Debbie O'Brien.
Maybe you heard of her, Yeah, and we were talking
about play Right. This is back from October in twenty
two and I got a few good comments on the show,
but this one was very specific. Admittedly it's over a
year ago now. This is from Chris Martin. He said,
just getting into play right and it was great listening
to the episode of Dot and Rocks on it. I'm
currently getting this implemented for a Blazer app. What's the
babe for authenticating with azure ad in our tests? Thanks?
And Carl responded to this actually back then talking about
azure ad B two c of through Blazer training include
a link in the show notes for that particular show.
Although it's changing, it's been a year.
Speaker 1: Yeah, it's changed and stuff's changed.
Speaker 3: Yeah.
Speaker 1: We went through this period where we were using MSAL,
the Microsoft Authentication Library right as a way to simplify
accessing it, and now there's even more stuff that's like
I haven't even i mean even kept up with it.
That's how new it is.
Speaker 2: Well, they call it an intra now, and so in
some ways it's clearly better. I think definitely. My repertoire
for the show's coming up is a whole conversation about
managed identity. Yeah, specific And it's one thing to manage
identity in production. It's another thing to talk about managed
identity in tests and in the dev cycle. But you've
heard on the news this whole Microsoft Future Security Initiative
or Insecure Initiative, and a lot of that has to
do with undersecured dev resources, demos things like that. So
Microsoft is pushing hard for their own people to do
this better.
Speaker 1: Yeah, across all teams.
Speaker 2: Yeah yeah, and to be that's super exciting because they're
going to knock the snot out of this process, like
they're going to make it better for their own needs.
That means it's going to be better for all of us.
Speaker 1: I mean we're talking about people who write white papers,
who publish examples, who you know, maybe do this stuff
with three sixty the developer team obviously, Yeah, I know
I know my friend Jeff Fritz is in that.
Speaker 2: Yep, lots of folks that are buried in it right now.
And and again it's like there's a there's a feedback
loop here. Not only should they follow the practices, but
when the practices are just too awkward to use, they
know where those guys live, Like they'll lay in wait
in the bushes outside of the office building.
Speaker 1: They'll tackle too soon, too soon.
Speaker 2: No, No, I'm just saying, like I watched what happened
to WPF back in the day when the studio team
started implementing WPF, WPF got dramatically better, and it was
that back and forth.
Speaker 1: Hey, if you want to go way back when sql
server implemented the dot Net framework inside SQL servert got
better well, without a doubt, because the sequel team kicked
its butt.
Speaker 2: Yeah, no doubt. So anyway, I'm hopeful. Yeah, and Chris,
thank you so much for your commented. A copy of
music Cobe is on its way to you. And if
you'd like a copy of music code by I write
a comment on the website at dot at rocks dot
com or on Facebook. We publish every show there and
if you comment there in a reason the show. We'll
send you a copy music cope.
Speaker 1: He's code by still going strong and people using it
every day to be more productive when they code or
sleep or relax or focus or whatever. All right, so
let's bring in Wi O'Brien. Before I do that, though,
I need to mention that you can follow us on Twitter,
and of course we've been on ex Twitter for years.
But the cool kids are hanging out at Masdon. I'm
Carl Franklin at tech Hub dot Social.
Speaker 2: And I'm Riche Campbell at Masidon dot Angele.
Speaker 1: Okay, so Debbie has been on the show before, of course,
but if you don't remember, Wi O'Brien is a Principal
program manager at Microsoft. Wie has over fifteen years of
experience in front end development. She's worked as a tech
lead and consultant for many important and clients with various
technologies and often with a strong focus on performance, static
sites and testing. She has led teams both in house
and remotely, as well as giving workshops and training. Also
has many years of experience as a mentor for online
learning platforms, Treehouse and open classrooms. Debbie's also a Google
developer expert in web technologies, cloud andary Media Developer Expert
and next Ambassador. Boy, there's a mouthful, as well as
a GitHub Star alumni and a previous Microsoft Most Valuable
Professional and Developer Technologies Debbie, I think you need to
be a little more What should we say, you know,
just try try a little harder.
Speaker 4: I think I know, right, I'm trying to get the
YouTube Award. That's my next one, you know that like
big thing on one hundred thousand followers. I've got five thousand,
so got a long way to go. But yeah, I
need more awards.
Speaker 2: Yeah.
Speaker 1: I tell people the kind of the same way. I
just don't know what I want to do when I
grow up. I certainly haven't grown up yet.
Speaker 2: Why start now?
Speaker 1: Why start now? Yes, we're talking about Playwright, and Playwright
is sort of the canonical web testing and automation actually
platform that does snapshot testing, like we talked about with
our friend Simon Crop, and it does all sorts of
other things. Why don't you tell us what's on your
mind in terms of Playwright?
Speaker 4: Yeah? So, I mean everyone should already at this stage
be testing their applications, because there's absolutely no excuse in
the whole world why people are not testing right. Go
back a couple of years, like ten years ago, no
one was testing, and I completely understood that because testing
was hard. Testing to time testing, the slow testing costs money.
And now it's the opposite. If you're not testing your application,
you're losing money as a company. You're dealing with bugs
that you should and even have to deal with. So yeah,
I've been trying to get people to test their applications
and I think still there's people not testing. I don't
know how to fix Carl Richard, how do we fix this?
Speaker 1: I think most people assume and I'm going to assume
that they assume that, well, why should I test? Because
I know this routine or this method or creates this
UI result. Why do I need to prove it? Right now?
I can see that it works. But the question is, well,
how will you know when some other developer takes over
your project and does something on the back end that
changes the outcome? That's when testing becomes really valuable, right, yeah.
Speaker 4: And just changing like little like think about forms, right,
Forms are one of the hardest things to test manually.
And I don't know if you've ever done this a
manual test and you start manually and what I mean
by manual is going to the computer's a user and
filling it that form and then you find out the
typical For me, it happened to me today. I was
putting my name in. I have an apostrophe in my name. Well,
nobody tested the web and I couldn't put a specific
type of apostrophe in. It wasn't accepting it, so it
wouldn't accept my name, right, And nobody tested that, so
I couldn't. I couldn't just couldn't get in. I had
to like go to the actual computer. I couldn't do
it for my phone, had to go to the computer
where I had had more access. I was able to
do it. But when you're actually testing that out manually,
you're filling in all those things and then you press
submit and you get this error message and then something
is broken, and then you go in and you fix it,
and then you go and fill that it again, and
you keep doing that process, a repetible, horrible process that
I used to do, that you used to do back
in the old days, when you know, that was the
only way we could do things. But now you have
tools such as Playwright that basically is an extremely fast
you right, it is you times one hundred and that
can fill that form in one hundred times faster. So
that basically is automated. Once you write that test, that
form is filled in. And this is the cool thing
that you basically to that test. You just manually fill
in that form once and you do that by using
the code generator tool. You code gen the test by
filling in that form, it's going to generate all that
code for you, and then you just basically press play,
press play and on your CI it just runs anytime
the pull request is done in so if someone changes anything,
it's going to run that and it saves so much
time and it's so easy and quick to do it. Yeah.
Speaker 1: So the thing that occurs to me about web testing
tools like Playwright is that they have an advantage because
they can inspect the HTML, which is just text. You know,
they can inspect the output, whereas if you're trying to
do that, say on a Windows forms application or graphical application,
you have to do image you know, ocr for the
for the text and image processing and all of that
kind of stuff, and we just don't need to do
that with you get. You can still do screenshot testing, right.
You can say well, the screen should look like this,
but you have that the webliner allows you to be
a lot more accurate.
Speaker 4: It brings it closer to the user. And what we're
doing when we're testing Blaser applications, right, we're testing what
the user sees, what the user does. Simple thing is
testing dark mode and lightmode. This is the cool thing
right nowadays everyone has to have their website in two colors, right,
how can we test that? Well, we can test that
the CSS class is there and that can be done
true playwright, and then you can just easily have a test. Yeah,
that's dark mode, light moode. That's a simple test.
Speaker 3: Right.
Speaker 4: You do you care? Well, maybe you should if that's
important to you. What's more important is, you know, filling
out the forms, making money, putting articles in a shopping car,
actually purchasing things, and that's what the user does. What
buttons does the user need to press to get to
that payment? And if you can test that whole process
from that start to finish, the money is going to
be in your bank.
Speaker 2: Yeah. Well, I mean lots of friction you can remove
in that process. And turns out if it's easy to buy,
people do buy more. So you know there's money to
be made in creating good workflows that way, right. I
just think these are the tools that help you detect
where you're stumbling before you push it out there and
start losing money on it.
Speaker 1: Now. I use a tool called b unit from Egil
Hanson for Blazer components. Does that work in concert with playwright?
Is there something that does what b unit does in Playwright?
What's the whole ecosystem look like with that?
Speaker 4: So for component testing itself, we don't really support anything
outside of the experimental, which is kind of you and
like the component testing yourself. I'm not sure if you
can put a Blazer thing inside of vat application that's
something and push that up. That's something I've never tested out.
But we're basically like, when it comes to a Blazer,
you're testing what the user sees. That's already compiled, that's there,
it's visual, it's on that front end. So you're coponents
are making up a web page. Your components are making
a story, and that story is taking you. It's more
about the integration between those components. Sure, how they integrate
together and when on the page they do what they're
meant to do. That button is a component. For example,
that shopping cart is a component, and that all interacts
together to be able to get to that stage of
filling out the form, to fill in your delivery address
to get the product delivered to your home, etc.
Speaker 1: So you can still a b unit test to test
the individual components if you think you need to. But
what you're saying is playwright works the macro level. It
just works at the whole website. You fill in this form,
you expect this result. If you don't get it, then
the test breaks.
Speaker 4: Yeah, Like think about it. The user doesn't care about
your code, doesn't see your code. So unit tests is
not covering what the user sees and does. It covers
your code and is your code going to do the
job right? But end to end test integation tests, they're
covering what the use physically does in your website. And
it's like manual testing years ago when I used to
do this on a Friday right in a job ten
years ago. Everyone stop, stop we are doing and start
using the website as a user would use it. Go
and start doing stuff right and then fill out issues.
Oh this doesn't work, file boog, this doesn't work. File
a bug. That's ridiculous doing that nowadays, and still people
are doing that. Still people are paying manual people to
press buttons, and that should not be happening. So Playwright
is that manual person pressing the button. But that's much
faster than that manual, physical person that would have done
that ten years ago.
Speaker 2: One of the things I run into when we start
to build bigger websites is we have so many tests
that it slows down the CICD pipeline. I've gone to
great lengths to like break tests out across ponstible instances
and things to try and shorten that time up. Is
there tools in play right to make that easier.
Speaker 4: Yeah, so's there's things you can do. Like basically, you've
got to remember what's the slowest thing about testing when
it comes to to Playwright, it's the brows right, It's
it's you know, you're normally you should be testing as
well your website across multiple browsers, you know, across the
fire across Firefox, across Chromium. And once you're doing that,
you're spinning up that headless browser. Right. There's headless means
that there's no visual parts, there's no graphical interface, so
on CI you don't need to see something. The CI
basically spins up the browser without the actual graphical interface,
but it's still spinning up a browser instance for your tests.
And tests are running isolation, right, so that means you
know there's a new browser instance being spun up every
time there's a new test, because otherwise, you know your
cookies would jump from the same if it was the
same browser, right, and then you can messs of your
test you have flaky tests, you'd go crazy. So when
you have like a lot of tests, you're spinning up
all these browser things. You can do a couple of things, right.
You can go charting shot across multiple machines, which can
be good and can be bad depending on how you
set it up. Because if you shard too much, remember
you've still got to download the browsers for that shard, right,
So if you have eight shards, there's eight times you're
going to do that. Are you saving time or are you?
So sometimes you have to decide how many shards do
I need and you've got to play around with those numbers. Now,
another cool thing, cool thing that we have is the
Microsoft Playright service, which is not the open source part
of Playwright but the Azure part. So if you're in Azure,
you can basically use cloud based browsers. So now the
browsers that you normally are like downloading are in the cloud.
And I know it's like we all talk about the
cloud and we have to imagine this physical cloud and
the browsers are up there, and you're basically sending your
tests up to this cloud so it doesn't need to
download anything. What that means is you're just running your
tests and you're not doing the browser instance part, which
saves a hell of a lot of time. Now, obviously
this is an Azure feature, so it's it's costs money,
of course, but you save my in the long run.
If you're talking about scaling at an enterprise level, you're
talking about large scale. You want to run I don't know,
ten thousand tests and your pipeline is being slowed down
forty five minutes is taken. You want to take it
down to three minutes. You know, any company would pay for.
Speaker 2: That, right, you know, if I'm running tests in the cloud,
I'm paying for that. So the fact that you've made
me a dedicated service for this purpose, that sounds good
to me. That's just going to make fewer mistakes.
Speaker 1: Is that ratio like forty five minutes turns to three
minutes typical.
Speaker 4: No, that's just me making up a number from the
top of my head, because obviously everyone's tests are different
and you'd have to test it out. But the amount
of time saving across all the examples that we've kind
of run it on and then the clients that are
using it in the feedback they've been given us is
the amount of time. No one is the same, right,
but everyone has saved immense amount of time. And then
also what's really important to a lot of people as
well is being able to see, like, you know, to
see those those tests in a kind of like a
dashboard style, to be able to go back and forward
and then like have a report, right and see a
report of like this is what happened, this is where
it failed, and have a trace of that test on
your you know CEE I, on your on the on
in the cloud, so that basically you don't have to
you know, go to local and locally spin up that
that trace file and then debug it. You can basically
almost debugget in the cloud.
Speaker 1: Right.
Speaker 4: But yeah, so scaleability was an issue, and we solve
that issue because we also need to solve that issue
because otherwise we're losing time.
Speaker 1: That's great news.
Speaker 2: So testing as a service is that where we're going.
Speaker 4: I think we just need to fix testing in general.
And I'll be very truthful, we do not have it
all figured out in Microsoft. We're still learning because we
are so many people, so many teams. We're like so
many mini companies if you if you kind of think
about it that way, and everyone is trying to like,
you know, deliver products. Right, We've got a lot of products, right,
how do we test all these and how do we
create a better solution across Microsoft? And then how do
we let you all use that solution because if we
just test it into in ourselves, we don't get the
value from like, oh but what about this feature and
we need this so, you know, making sure that Playwright
is open source and then having those other services for
the enterprise scale really helps us to make a better product.
But yeah, we're not saying that we have everything sold
in Microsoft and we know everything. No, we're building things
as we go along, and we require like the Playwright
service is going to the reporting is in basically GA
this week, Right, It's like literally, it's new, so we
don't have it all figured out because someone might come
along and say, oh, we want this feature, Well cool,
tell us let us know. We'll build that feature for you,
because we might also think that feature is necessary for
other people. But yeah, everything is being built around making
lives easier, making testing easier. I think easier is our
big word here. Easier and getting more people to test.
Speaker 2: Yeah. Absolutely, And I could see it's still in preview too,
So it's just a hint that you know, you're all
learning by how people use it and what it's for.
But I appreciate the idea that you know, this is
the same angle as why would I build a bunch
of virtual machines if I can use an app service?
Why would I, you know, deploy my own caching if
I could use a reddish cash service like you go
as far up the stack as he can. That's cool,
that's I'm excited to see that, just because yeah, I
don't want to build these things. I'd like you to
do it for me please and charge me by the
test exactly.
Speaker 4: And I think if you have that many tests and
you're you're paying for that means you know you've got
some good coverage going on there that it's worth paying for, right,
it was just made.
Speaker 2: Too many pages, you know, Yeah?
Speaker 4: Maybe maybe.
Speaker 2: Well, I mean it's been two years as you were
on more or less, so it's lead to see Playwright
continuing do evolve Like that's cool.
Speaker 1: Yeah, so you have a special love for JavaScript frameworks.
How does Yeah, that's writing your bio. I'm not hating, Nana,
I'm just reading here, reading your bio. So how does
a JavaScript play into Playwright? If at all is any JavaScript?
Are there entry points for JavaScript in Playwright? If you
want to do some custom things? How does you know?
That's the question, I guess.
Speaker 4: I think what's important to understand about Playwright is that
it can be used across four languages. Right, So if
you were like myself, I'm a JavaScript developer or typescript
developer now is what the cool kids say, then I
can basically use that language because that's what I'm familiar with,
That's what my website is built in. So I'm naturally
going to write my tests using the language that I know.
But Carlin Richard, you're dot net people, right, and you
you're c sharp probably better than JavaScript, and your website
is probably built in Blazer, so you're gonna want to
write your tests in c sharp and not in JavaScript.
And I totally understand that. And that's why Playwright is
so cool because you can just do that, because you
can use Playwright as a dot net developer. And I
know this is dot net rocks, but let's talk about
the Java people and the Pithon people because they can
also write tests using Python and using Java and it's
the same API, and you don't have to learn new tricks,
and you know, you can have one company, which we do.
We're like in Microsoft, we have like you know, various
and we're using different languages depending on the teams and
depending on what we're testing and what people are comfortable with.
So we use dot Net, use Pithon, we use Java,
we use JavaScript. There are differences, of course, right, I'm
not going to lie and say it's the same across
the board. It can't be the same across the board
for various reasons. Because test runners MS test, you can't
compare the MS test to the pie test for example,
they have different stuff, right, And in JavaScript, we built
the test runner and play right, so it's a little
bit different. We've got a little bit more leeway on,
you know, doing certain things.
Speaker 1: And I just want to say for the record that
some of our best friends, right, JavaScript, Python and Java. Okay,
so isn't that what they say usually when somebody uses
you of anti say that some of our best friends.
Speaker 2: But either way, you are testing a web app, you're
just able to write the code for the test in
c sharp.
Speaker 1: Yeah.
Speaker 4: Yeah, because playwright just sees like what's out on the website, right,
what's what's being produced? The what the you know you
inspect code, right, and so you can see what the
user can see that accessibility role, right. It just sees
that that is a role of a button, and it's
looking for what a screen reader would read out or
what a user would see. You sees a button and
it clicks that button, and it does not care if
your button is written in blazer React you write heighten whatever.
Speaker 2: Yeah, no, and right, you know you don't want to
make writing the test the barrier, exactly like you wanted
to be as easy as possible. So I appreciate that
writing the language you want. I could write tests in Python.
That's one way to slow myself way down.
Speaker 4: It could be fun there. You might learn something, maybe.
Speaker 2: It's interesting you think about writing in different languages for
different tasks.
Speaker 1: Yeah, okay, that's cool. So break time. Want to take
a break.
Speaker 2: Let's take a little break.
Speaker 1: All right, We'll be right back after these very important messages.
Did you know you can easily migrate asp net web
apps to Windows containers on Aws. Use the app to
Container tool to containerize your iis websites and deploy to
AWS managed container services with or without Kubernetes. Find out
more about app to Container at aws, dot Amazon dot com, Slash,
dot Net, slash modernize, Attention dot netdeville. Looking for the
ultimate SDK to handle electronic document processing? Meet tx text Control.
Txtext controls your go to solution for seamless PDF generation,
secure electronic signatures, and efficient digital forms processing all within
your dot net applications. Empower your products with robust document
management capabilities, boost productivity, and deliver top notch solutions to
your clients. Trusted by developers worldwide, including me and Richard,
txtext control is the SDK that makes a difference. Check
out demos dot textcontrol dot com for live online demos
and see it in action. And we're back dot NetRocks on.
Carl Franklin. That's my buddy Richard Campbell and our friend
w O'Brien. We're talking about testing with playwright and just
a reminder that if you don't like those ads and
you don't want them, you can pay five bucks a
month and be a patron and you can get a
feed with ad free shows. All right, So we were
talking about we're talking about all sorts of stuff and
testing in different languages. But snapshot testing is something that
I learned about from Simon Crop. And so the idea
is that you have you have these following inputs and
you expect the following output and I get is that
primarily how playwright works or is it even simpler than that?
Speaker 4: So it's just another part of something playwright can do. Right,
So snapshots is like, you know, you're testing that something
looks the same that for example, this is really good
for Wednesday, like those banners up here at the top
of your you know, the Black Friday banners that appear,
and then what I like is there and you want
it to be there, but you wanted to, I don't know,
make sure it's not there after Black Friday eatin doesn't
make sense. So you could, like you know, write a
test that then I don't know, shows the banner, and
then make sure that it is exactly the same as
a banner right on the website. But yeah, there's a
lot of people using snapshot testing or visual regression testing
as a whole ballgame. It's not easy, right because because
across browser as well, the slight little differences, so you've
got to, like, you know, also make sure your pixel
differences are kind of the same, because Safari will do
something a little bit different to Firefox because that's just
how they do things. But definitely, I think if people
are getting started with testing, I wouldn't say, go and
make sure your website looks exactly the same as the
snapshots like everything, because you'll end up possibly things will
fail if you change that button. You've got to re
recreate those snapshots each time. So I would always say, like,
that's an extra, that's great and it's cool, but first
make sure that your user can do what the user
should be doing. Whatever it is on that website is
going to bring you money or make you click that
patron button that you talked about, whatever it is, test that.
Then do the accessibility testing, the regression testing, the API testing,
there's a whole ballgame of things that you can.
Speaker 2: Explore, sure, right, but I mean the snapshot test is
really about a look and feel thing rather than a
functionality thing.
Speaker 4: Yeah, it's about that it looks the same basically that
it hasn't changed that the designer didn't put a CSS
hidden on a button and now they can't see the
button for example. Right, But then if that was the case,
player wouldn't be able to click the button, so that
test would fail.
Speaker 2: It would still break anyway, exactly.
Speaker 4: But sometimes there are times when visual aggression testing is
really important, especially when using I don't know when they say,
like a lot of advertisements on your website and you
want to remove those and just see what the website
looks like out those You can mask those kind of things.
Speaker 2: Well, no websites are so twitchy these days, right, Like
thank goodness for pie hole. Otherwise there's stuff flying at
you in all directions.
Speaker 1: And you can also exclude things like you say, I
want this page to look like such and such with
the following data that it because we put in a
certain input. But you know, little things like the date
in the time, we don't want to We don't care
about that that can change.
Speaker 2: Yeah, I'm kind of staggered at how it would do
those visual representations you just talked about sort of headless
browsing where there is no visual component, but is do
something different when it wants to create a visualization.
Speaker 4: Well, it's just like taking a photo and then like
it takes a photo kind I'm just doing like an
example here, right, you take your phone out and you
take a photograph of your screen of the website, and
then you change your code, and then it takes another
photograph and then it compares those two overlays one in
front of the other, and it kind of sees like
is there a pixel difference? Has the button moved over there?
What's it expecting and where is the button now? And
I'm using button as an example, but it could be anything, right.
Speaker 2: But it is pixel perfect it.
Speaker 4: Yeah, but you can adjust that, right, You can say
I want it to be like so many pixels away
because I don't know it's an animation that's gonna like,
you know, I don't do stuff, and you know you
want it to be able to move a little bit
or whatever.
Speaker 1: So I wasn't really thinking of taking screenshots, screenshot testing,
snapshot testing something I learned from Simon because he has
a tool called Verify, and it's really just testing the
HTML output of a page. So like a form, Right,
you fill in a form, you press a button, it's
going to give you a summary of that data. And
that is what it tests that the output of this
matches what we expected to match. So I think I
think we might be talking about two different things.
Speaker 4: We're probably talking about different Yeah.
Speaker 2: But it is that in that case, it is literally
a text comparison to the HTML. Yeah, Like if I
was doing visual representation comparison, I could write the page
different ways but have to present the same way.
Speaker 1: Yep.
Speaker 2: Not that I'm good enough to do that, but you
know that would be interesting.
Speaker 1: Yeah, you want to make sure that this did with
this ID has this content right, which is different than
you know, comparing uh screenshat, which is also valuable. I
mean absolutely, There's just so many ways to go about testing.
Speaker 2: Yeah, I'm just wondering about the brittleness of tests, right,
Like most people dive into testing the first round of
tests they build. The next time they do enough to
do the app, every single test is broken and they right,
why did I do this?
Speaker 1: Yep?
Speaker 2: And you Debbie jump in on this, like, how do
you write non brittle tests?
Speaker 4: I think you just have to constantly test. You've got
to like make sure that I don't know, I think,
I think use the tools that are there. I would say,
use the tools that are the code generator, right, the
code gen will generate the test view. I think a
lot of people going back the previous days would have
wrote their test using things like XPath, using things like
CSS selectors, right, And that was the right thing to
do back then. But our front end, and I don't
know about you guys in the Blazer world, but we
CHANGEAVASCRIPT are known for We're changing things all the time, right,
We're constantly like changing, so our htmil is changing. And
if you're going to change that, the x path is
going to change, which means your test is now going
to be brittle. It's going to break just changing your
CSS class. New developer comes along and he changes a
CS class. If your website and your tests break, well,
that's not good, right, So you need to avoid things
like that. And the way to do that is, you know,
use the test generator tools to generate those locators. Locators
is how you locate an element on the page. So
how do I all locate that button on the page.
You use a get by roll button, and that button
has a name called submit, and that is tested. It's
also tested for accessibility, and it's easier than for you
to go out and just do whatever you want with
that button, change the CSS of it, put it in
a different place. Play right's going to look for that
button with a roll of button call submit, and it
doesn't care where it does in your page because that's
not important. What's important is that the user can click it,
and the user can get to that next section of
the page or whatever it needs to do with it.
So I think making sure you test as close to
the user as possible will save a lot of problems
in the long run when it comes to brittle tests.
Speaker 1: M So instead of on Fridays you guys all get
together and push buttons, now you're getting together and updating
your tests or maybe you're doing that every time you
check in something, right, yeah, and make sure your tests
are working.
Speaker 4: And what's really cool is is like the code Generator
tool is very cool because you literally like it launches
up a browser window, right, and you put in your
website address and you start using a use as a
user would, and you generate that code, and then you've
got tools like get up copilot, right, and you can
do really funky things at copilot because you can turn
around and then say a test step for my tests,
and it's going to like literally take the code that
you've just written, that you've just generated, actually you haven't
written it, you've just generated, and it's going to put
it into test steps for you, and it's going to
do all these kind of other stuff that you want
to do it or put it into a described block
or you know, you can help get a copilot can
also help you to improve the code. So as a developer,
you're not sitting there just like writing the boring parts
of testing. You're using the tools to your advantage.
Speaker 2: Mm hmm.
Speaker 1: Good.
Speaker 2: Yeah, And I think definitely I remember in Strange Loop.
I think the third go around of writing tests for
Strange Loo, we've gotten better at writing tests that just
we're going to be more tolerant, like you're still checking
the value the way you want to, but you're not
fussing over you know, unnecessary elements of that. That means
that the slightest change in the test fails. Yeah. I
think it's really an art to write good tests and
stuff that will give you check the stuff you need
to check without being infrastructure where minor changes the things
just derail all the tests.
Speaker 4: I think that's the difference between unit testing, right, whereas
unit testing really is so close to the code that
it's like you change something in the code or your
test is going to break. With play right, really, you
can change something in the code and your test shouldn't
break unless you change, say, like what the user sees
and instead of a submit button, you put upload, and
now it's going to like look for submit and it
finds upload because that they messed up the translation or whatever, or.
Speaker 2: You've removed I or you've removed a field or like you.
I mean, there's also good test fails where it's like
I made this change and the test shows that I
made this change. Now I can remove the test or
change the test. Yes, I don't mind it when it
is clear on stuff. What I mind is, hey, you know,
we wrote three hundred tests around this set of pages,
and I changed this CSS structure and every test is
vailing now.
Speaker 4: And that's the difference between unit tests, right, because you
write that test and then you can't really see what's
going on, right, you're stuck in code land. But in Playwright,
when you have that failing test, you spin up a
trace of your test, and that's like basically be able
to go think of a video, but better so a
video with dev tools, a video that you can manually
step between every single step and go back and forward
in time at your own pace, and have a DMB
snapshot that you can just pop out and inspect the
dev tools and say, right, I know what happened here.
Somebody puts CSS hidden. I'm just gonna use that example
because that actually happened to me on one of them
that I was testing on the podcast website, and it
put CSS hitting them playright couldn't find it because it
isn't visible to the user. So if the user can
click it, but the code is there, the code didn't change.
The CSS change and the CSOs could have been in
a different file to the code, right well.
Speaker 2: And so that could have gone the other way. Right,
you've hid it the test pass and you deploy it,
but the users can't work that. Yeah, you want to
be able to catch that too. It's it's it's always
going to be a battle.
Speaker 1: Can can Playwright penetrate to the level where it can
see components of the shadow? Don Yeah, it just works.
Speaker 4: It just works, iphrames, it just works.
Speaker 1: Just wow. Not well that was a simple answer.
Speaker 4: Sorry, I try to make it more complicated.
Speaker 1: Yes, no, yes, no question, I can't tell. That's great. Yeah,
that's great because that that has been a problem.
Speaker 4: In the past, you know it has. That's why we
had to solve it. Because you know, problems need to
be solved.
Speaker 1: Yeah.
Speaker 2: Do you find that developers are using playwright in their
local cycle or is this purely part of the c
i CD pipeline.
Speaker 4: I think it has to be both, because you need to.
I always think that unless a website is finished, which
I've very rarely seen, what is product finished exactly exactly,
So unless it's something that you are never going to
touch ever again in your life and you're going to
write those tests and it's just on CICD for whatever reason.
But no, nothing has ever finished. Things are always changing.
So running your tests locally as you're building some code,
as you're adding additions, as you're changing things always makes sense.
Some people don't. Some people push it to CEICD so
they can just get their code up there and they're
not testing it locally, and then like you know, Friday,
they make all the changes, they push and then they
can't go home because, like you know, something broke. So
running it locally always always helps, and having the tools
like it's it's when when tests are quick and easy
to run, it means people don't mind running them locally.
When tests, if your test takes forty five minutes to run,
as a developer, I'm not going to run it. I'm
not going to run it locally. I might run the
one test that I'm working on, but it might affect
other things the code I'm working on.
Speaker 2: So and that's sort of that integration test that after
the fact. Yeah, now you go check the integration effects.
Speaker 4: So I always say, like new code you're building, I'm
building this new feature, then I really want to work
on those tests. I'm writing those tests. As I'm building
the feature. I write the tests, and then I'm testing
that and then I you know, it's now deployed and
then it's running on CICD and it keeps running. Yeah,
So it's a mixture of things.
Speaker 2: I mean, a CICD part is not not optional. Before
this heads towards deployment or load testing or the next stage.
It must be thoroughly tested. It's just a question of
how much of that can you pull down to the
local workstation for that DEV cycle, the impacts that would
have on them. I guess it would be great to
organize it in such a way that you could only
need to test the piece you're working it.
Speaker 1: Are there any common misassumptions or barriers to entry or
gotcha's that people will find misassumptions, Maybe that people will
run into things that they assume that it does that
it doesn't do, or what's the typical getting started barrier
that you see people having.
Speaker 2: I think that miss assumption is a word. Look, sorry,
you looked up? I did?
Speaker 1: I can't.
Speaker 2: I just like, what have you met me? You've appreciate
met me? Debbie, I'm sorry, We're okay.
Speaker 4: So I think what's really important to note is that
tools change and evolve over time, and things get better
and we improve things a lot. So sometimes people will
say I'm going to go write a test and they
are too. I'm gonna use the word lazy, but it's just,
you know, that's my word, but there could be another
way of describing it. They're too lazy to read the documentation.
So what they do is they just go and they say,
I'm going to write a test. I don't want to
read docs. So what do I do? And what do
we all do? We just Google right and you just
throw it into Google or now we got the new Google.
We got chat gtpt right, and we say, how do
I write a test in Playwright? And some times the
results we get are wrong and we don't care because
we don't know because we didn't read the docs. We
copy and we paste that code and we put it
into our tests and then we go, my test is breaking,
and I don't know why my test doesn't work, and
I don't know.
Speaker 1: What it works when it shouldn't work exactly.
Speaker 4: So this is really important to kind of make sure
you're using the latest version that you know, I mean,
Playwright is you know, the first time Playwright came out
three years ago, the locators were different to what they
are now. And you know, sometimes chat GTPT isn't up
to date, it hasn't been trained in the latest model,
so it's going to give you an old version.
Speaker 3: Right.
Speaker 4: Someone on stack overflow maybe wrote that a couple of
years ago. Maybe doesn't know much about testing. You have
no idea of their knowledge level. You're just copying their
code and their answer, assuming that they're all experts and
that you're not an expert. But really, if you just
go into the documentation and did the same search and
the documentation, you would get that answer. We get a
better answer.
Speaker 1: Are there are there really good examples in the documentation
of you know, for people getting started their walkthroughs or
that kind of thing.
Speaker 4: So I'm going to be very very honest and tell
you that I did never basically ever install sea sharp
on my computer until literally a couple of months ago.
Of all guils, I built the getting started docs for
c sharp, and I took it from start to finish,
and I mean from start to installing on my computer.
Speaker 1: Wow.
Speaker 4: So if I, as a non developer, can write a
playwright test in dot net by following the getting started guide,
then a dot net developer should be able to do
it ten times faster than me and should be very
successful in a very short space of time.
Speaker 1: Well done you nice.
Speaker 2: Thanks, Yeah, that's.
Speaker 4: Actually really easy to install.
Speaker 3: Do it?
Speaker 4: I realized it's actually a purple button. You just press
it and then it just installs. I did not know that.
Speaker 2: Yeah, I mean we always had this problem. But there's
too many ways and so did some of them are
hard and yeah, if you picked it.
Speaker 4: You know what I did, I asked chat and you
know what it did. It sent me down the rum path.
Speaker 1: Oh man, you go to the docks.
Speaker 2: Who's lazy? Yeah?
Speaker 1: Right, it's like have an intelligent mentor that sometimes loses
its mind.
Speaker 2: Well, it gave you an answer, it just wasn't the
easiest answer, right, Like, that's that's a tough concept for
a piece of software to figure out.
Speaker 1: Gott to change your system? Prompted to can you not
give me any dumb answers? That'd be great. If you
don't do that, you're only going to get dumb answers.
Speaker 4: I mean the answer sounded good to me if it
felt familiar in my world. It was like, you know, yeah, homebrew,
just homebrew it all up. Yeah, that's that's me, Mark
that that does it. But no, you don't need that
was it was?
Speaker 2: It was it Hunter that told you. It's like we
made a button.
Speaker 1: There's a button.
Speaker 4: Yeah, why don't you know about the button?
Speaker 1: One thing that I've noticed as an aside here that
Chat GPT is terrible at is CSS?
Speaker 2: Oh boy?
Speaker 1: Yeah, don't try to solve a CSS problem with chat GP.
Speaker 2: Problem. Now I have two problems.
Speaker 1: I have a problem factory.
Speaker 2: Yeah, I tried it with Kubernetes and a head of
container problems. Yeah, CSS is terrible. I'm sorry. Or did
I have a problem orchestrator. It's one of those, you.
Speaker 1: Know, I might as well ask the JavaScript JavaScript web
queen here. CSS visualization tools exist so that you can
see a tree view of your application so you can
more easily diagnose CSS problems.
Speaker 4: I mean, we just live in death tools, and like
the death tools is in me, and even like you know.
Speaker 1: You really think so, yeah, you think the dev tools
are amazing for CSS. I mean, I guess you do
see the tree, see.
Speaker 4: Everything you see like accessibility for your components. You can
see the color. You can even like modify it and
say like that's that a color is not accessible. But
if I can just like scroll it across a bit,
this is an accessible color, which means you can go
back to your designer and say, hey, that gray it's
just not going to work for me, but if you want,
can we make it this grey? Because that would work
accessibility wise, and then they just kind of get you know,
they love their colors and their stuff, so they go, oh,
that gray is a little bit different.
Speaker 1: Yeah, I guess what I'm really looking for, And maybe
the dev tools haven't have just missed this. But you know,
when you're several layers down, like maybe ten or eleven
or twelve layers down, and you have some CSS that's
all messed up and it's not what you expected, can
you like querry that and say, yeah, where did this
particular attribute come from in the CSS tree?
Speaker 2: Why is this hidden?
Speaker 1: Yeah? Why is it hidden? Why is it purple? Why
is it not purple?
Speaker 4: I think a lot of people write very very bad CSS,
and that is a weird problem that you need to solve,
and deaf tools can't solve that, and there's no tool
I can solve that. I tried to solve that in
a company and I literally re architected the whole thing
in the amount of times, like CSS, this is being
used across twenty components and you change it once and
it's going to affect so many others, but but you
don't know about it because this file live somewhere else,
and it's just being cold and bundled together. I mean,
tailwind kind of solves that a little bit. If you
wanted to get out on that road, That's what I
would have used and used.
Speaker 1: Maybe there's a good AI solution out there waiting on
the horizons that you can just ask, why is the
button not purple? There you go or whatever, and it'll
find it in the tree and.
Speaker 4: Bob shrunkle AI is going to solve all our problems.
Speaker 1: All the problems, all the problems I know. Yeah, co
pilot for CSS. Here it comes you laugh, I bet
it's coming. Well.
Speaker 2: That brings up a great question, Daddy, what's next for Playwright? Like,
is how long's your to do list?
Speaker 4: I think the too list is never ending and nobody
can ever know because like when we last spoke, I
never knew that, you know, cloud based browsers were going
to be a thing. That was never because we didn't
we didn't have that scaling problem. We didn't have that
many people using Playwright, so we didn't have all this
feedback from all these people and companies and community. I mean,
we went from zero members in discord to over thirty
thousand members since since we spoke, basically right, so all
of those companies are now using playwrights. You need different solutions,
so we basically use GitHub. Playwright is open source and
people file issues and say we have this problem, we
want this tool, we want this and everything that we
built is based on up votes from our users and
you just go to that file an issue and up voted.
Speaker 2: How organized these issues are technically collecting feedback like I
could see you guys are taking care of this part
of the inner loop, like because people are asking for features,
they things they think are bugs but may or may
not be like mm hmm exactly. So really are developing
out in the in public. Basically anybody can contribute. Anybody
can read lots of comments too, like you're clearly prosecuting
these like what do you think of this? What about that? Way?
You know, how would we solve it this way?
Speaker 4: Because we have to understand, right, sometimes people say I
need this feature, like we'll explain to us because I'm
not seeing what you're seeing, so we need to understand, like,
you know, what is it that we why do you
want this? Why is it important?
Speaker 1: Or the question that I'll microsoft people ask, why would
you ever want to do that? It's it's not a
threatening question. It's a real question, like why, what's your why?
Speaker 4: Yeah, because then we can say, well, oh that sounds great,
let's you know, put some time towards that and work
on that one instead of this feature, right, or fix
this boog instead of that bog or whatever.
Speaker 2: Yeah, yeah, totally. And I said just and you know,
you could write the code too, like if you're if
you're that interested like that, you can contribute.
Speaker 4: We have had we've had some external contributors, especially in
the last releases, and I think now get up has
done some really nice thing where you can kind of
see their names and their picture and it's really cool.
So that even like is nice, So yeah, do yeah.
Speaker 2: To contribute to a project like this, that would be
really fun because.
Speaker 4: We are a very small team, so we can't do
everything that people want us to do because it's just
not possible, so we can only like say.
Speaker 2: It's a triayeah, yes, I get that. That makes a
lot of sense.
Speaker 1: Very cool, awesome sense, like a fun project to contribute to.
If you're looking first, you've.
Speaker 2: Had a lot of contributors. Holy man, look at this list.
Speaker 4: Yeah, hundreds. We have a great community. I told you.
And Discord Yeah, oh, there you are.
Speaker 2: I see your commit.
Speaker 1: Well, Debbie, it's been an absolute delight having you on
the show again, and congratulations on playwright success. And we
hope our listeners will check it out.
Speaker 4: I mean they have to, it's not we hope. You
got to be strict to Carl, you gotta tell them.
Speaker 1: All right, guys, my next episode.
Speaker 4: You have all should have written at least one test,
and I want.
Speaker 1: To see some comments on hiver right.
Speaker 4: I would say I would challenge I would challenge anyone
who thinks or has never written a playwright test. I
would say take seven minutes. I won't say seven, not
ten's a little bit more than five. Seven minutes. Go
to the getting started guide and the documentation of the
dot net and basically write that test. And you don't
even have to have a website. Just put in any
website and just just go through it and write that test.
Seven minutes and you'll have a test written to your developer.
You'll get a pay rise in a second.
Speaker 1: All right, there you go. Let's take make this about money,
all right, Debbie, thanks a.
Speaker 4: Lot again, Thank you.
Speaker 1: All right, we'll talk to you next time on dot
net rocks. Dot net rocks is brought to you by
Franklin's Net and produced by Pop Studios, a full service audio,
video and post production facilities located physically in New London, Connecticut,
and of course in the cloud online at pwop dot com.
Visit our website at d O T N E t
R O c k S dot com for RSS feeds, downloads,
mobile apps, comments, and access to the full archives going
back the show number one recorded in September two thousand
and two. And make sure you check out our sponsors.
They keep us in business. Now go write some code.
See you next time you got jas middle Vans
Speaker 3: Home the