WEBVTT

1
00:00:05.160 --> 00:00:09.119
Hey, folks, welcome back to
another episode of JavaScript Jabber. This week,

2
00:00:09.160 --> 00:00:13.199
on a panel, we have Dan
shapere Hi from a very hot Tel

3
00:00:13.240 --> 00:00:18.600
Aviv. We also have Steve Edwards. Hello from a very hot Tel Aviv

4
00:00:18.600 --> 00:00:24.000
style of Portland. Right, it's
I'm like really hot here. Like,

5
00:00:24.320 --> 00:00:28.000
yeah, I'm Charles Maxwood from top
end devs. Yeah, it's it's been

6
00:00:28.039 --> 00:00:32.200
getting warm here too. I think
it's all over the place, maybe maybe

7
00:00:32.240 --> 00:00:36.399
out where our guest is too.
We have a special guest this week.

8
00:00:36.439 --> 00:00:44.439
It's uh, I'm not even sure
how to say your name, ven that's

9
00:00:44.479 --> 00:00:48.079
me, Yes, called Vinnie for
sure. Yeah, Vinnie. I'm pretty

10
00:00:48.119 --> 00:00:53.079
used to being called Vinnie, to
be honest. Yeah, my cousin Vinny

11
00:00:53.600 --> 00:00:57.359
exactly. That's kind of how started
whenever I started speaking to American people.

12
00:00:58.560 --> 00:01:03.159
But yes, Venice is from Stockholm, Sweden, where I wish it was

13
00:01:03.200 --> 00:01:11.040
warmer. It's been a very rainy
summer, especially rainy and way worse than

14
00:01:11.200 --> 00:01:15.920
normal Swedish summers, even though Sweden
is not necessarily known for the good weather.

15
00:01:17.280 --> 00:01:19.760
Yeah, I was gonna say,
I'm kind of wishing for that here.

16
00:01:19.799 --> 00:01:23.000
But I saw a headline a couple
of days ago in the newspaper that

17
00:01:23.040 --> 00:01:26.400
said that you tie is no longer
in a drought, and in fact,

18
00:01:26.439 --> 00:01:30.640
all of our reservoirs are over full. So that's maybe we don't need the

19
00:01:30.719 --> 00:01:36.200
rain. I don't know anyway.
Yeah, we brought you on to talk

20
00:01:36.200 --> 00:01:40.359
about performance, and you shared an
article with me before these guys got on.

21
00:01:40.480 --> 00:01:45.079
I know you've also been chatting with
Dan. So I'm just gonna let

22
00:01:45.079 --> 00:01:47.280
you guys take the lead as far
as where we go, and then I'll

23
00:01:47.359 --> 00:01:52.319
chime in with my basic question since
I am not a performance expert like you

24
00:01:52.359 --> 00:01:56.640
guys are. Yeah, sounds good. You guys you mean those two because

25
00:01:56.640 --> 00:02:02.560
I'm not the performance expert either.
Ah. Yes, you know, performance

26
00:02:02.599 --> 00:02:07.480
is kind of important. I don't
think everybody can should or can be an

27
00:02:07.480 --> 00:02:09.800
expert. But and I guess this
is one of the things that we'll be

28
00:02:09.840 --> 00:02:15.759
talking about. It's not something that
you can ignore either. Let's put it

29
00:02:15.800 --> 00:02:20.759
this way, right, Yeah,
Yeah, it's uh, it's it's one

30
00:02:20.800 --> 00:02:23.639
of those topics that it's like the
more you know, the more you realize

31
00:02:24.360 --> 00:02:29.759
things go deeper, and you can't
get pretty deep, you know what.

32
00:02:29.919 --> 00:02:32.039
Yeah, but one example I've heard
frequently mentioned, and I've thought about it

33
00:02:32.080 --> 00:02:37.439
myself is you know, I'll look
at stack overflow posts and people talk about,

34
00:02:37.439 --> 00:02:39.400
Okay, what's the best way to
loop through an array that's more performer,

35
00:02:40.039 --> 00:02:45.039
or what's the best way to you
know, handle large data sets in

36
00:02:45.159 --> 00:02:49.400
code and and sort of stuff like
that, and people will obsess over these

37
00:02:50.719 --> 00:02:53.159
little things that will increase performance,
and then they've got a six meg image

38
00:02:53.159 --> 00:02:59.000
file downloading on the same page everything
anyway, you know. So, so

39
00:02:59.400 --> 00:03:02.319
I think it's safe to say that
when you're talking performance, it's sort of

40
00:03:02.319 --> 00:03:07.080
got to be comprehensive and not just
focused on you know, code performance I

41
00:03:07.080 --> 00:03:12.879
think, or bundling of code and
that kind of stuff. It also needs

42
00:03:12.919 --> 00:03:15.240
to serve a purpose at the end
of the day, which again is something

43
00:03:15.240 --> 00:03:21.400
that I guess we will be talking
about. But performance is not is it's

44
00:03:21.439 --> 00:03:25.560
not an ego trip. It's not
about you know, bragging rights. It's

45
00:03:25.599 --> 00:03:32.080
about serving your customers, your users. So anything that you do needs to

46
00:03:32.120 --> 00:03:38.120
be with that front and center.
If it actually brings value to your users

47
00:03:38.120 --> 00:03:45.120
and your customers or or it doesn't, and if it doesn't then it's kind

48
00:03:45.120 --> 00:03:51.639
of pointless. Yeah, yeah,
absolutely, And the whole micro benchmarking thing

49
00:03:51.879 --> 00:03:57.120
is it was a very good example, like it's very easy to look into

50
00:03:57.159 --> 00:04:02.080
like things that sound very buoyant,
flamboyant, and you kind of missed the

51
00:04:02.120 --> 00:04:09.759
target where the actual hurt is,
which is on the user's experience. Also,

52
00:04:10.000 --> 00:04:13.479
you know, in the context of
JavaScript, it's lies, damn lies

53
00:04:13.560 --> 00:04:19.959
and micro benchmarks. Yes, that's
a good one. Yeah, because the

54
00:04:20.040 --> 00:04:28.399
way that the JavaScript engines work,
the modern JavaScript engines and the optimizers they

55
00:04:28.480 --> 00:04:33.639
contained. I've read some quote unquote
horror stories about you know, people drawing

56
00:04:34.439 --> 00:04:44.199
conclusions from micro benchmarks that were effectively
not just meaningless, but in fact holy

57
00:04:45.560 --> 00:04:50.519
misleading. Let's put it this way, like people wrote loops, but because

58
00:04:50.759 --> 00:04:57.439
it had no side effect, as
it were, the optimizer kind of optimized

59
00:04:57.480 --> 00:05:01.959
the loop into nothing and then what
are you actually even measuring? So so

60
00:05:02.160 --> 00:05:08.360
yeah, but put to put it
bluntly, I wish my problems were how

61
00:05:08.560 --> 00:05:15.560
fast you know, some loop in
jobscript works, although occasionally it does actually

62
00:05:15.720 --> 00:05:23.319
happen. And I'll finish with that. That this part I actually contributed back

63
00:05:23.920 --> 00:05:30.519
to the Prometheists client for Node and
I don't know if you're familiar with it.

64
00:05:30.639 --> 00:05:36.319
It's a mechan it's a system or
service for monitoring and alerting and stuff

65
00:05:36.399 --> 00:05:44.480
like that. And the optimization was
actually an optimization about how to loop and

66
00:05:44.639 --> 00:05:54.199
build the response string because in that
particular case, via profiling, I proved

67
00:05:54.240 --> 00:06:00.399
that it actually did make significant difference. So it can, but it's usually

68
00:06:00.759 --> 00:06:06.079
does anyway. Now from my titter
chatter now over to venis yeah, the

69
00:06:06.959 --> 00:06:13.600
I mean the benchmarks and has to
serve a purpose, right, And if

70
00:06:13.639 --> 00:06:17.759
you just benchmar like if you're micro
focusing on things, they might not be

71
00:06:17.879 --> 00:06:23.920
painting a good picture for even what
you're even trying to measure. This actually

72
00:06:24.199 --> 00:06:28.720
brings to a very interesting conversation that
I've had a chance to have a JAS

73
00:06:28.800 --> 00:06:33.240
nation in Amsterdam couples of a couple
of weeks back with Ryan from Solid Jazz

74
00:06:33.600 --> 00:06:42.800
and Attila. So we were talking
about how can one build well structured benchmark

75
00:06:43.079 --> 00:06:46.959
to test different frameworks on some aspects, right, And we started talking about

76
00:06:47.800 --> 00:06:54.319
how it would be good to have
something that can bring TODs web virus because

77
00:06:54.319 --> 00:06:59.399
it's something we already kind of have
standardized, and that's how we can natural

78
00:06:59.480 --> 00:07:05.879
measure actual impact and like break down
different threshold like different kind of common problems

79
00:07:06.000 --> 00:07:13.680
where you can try to build correlations
and build a good delta between different frameworks,

80
00:07:13.759 --> 00:07:15.639
like what kind of strategy spays off
the most, right, Because Solid

81
00:07:15.680 --> 00:07:19.079
does it one way, he React
does it another, So it cannot necessarily

82
00:07:19.680 --> 00:07:24.680
measure them apples to apples in a
way, And so you need to try

83
00:07:24.680 --> 00:07:28.519
to establish like a good baseline of
how can you benchmark those kinds of things

84
00:07:28.560 --> 00:07:31.240
that are in the end just deliver
the kind of similar results, but they

85
00:07:31.360 --> 00:07:34.639
do it differently, so you're not
really trying to benchmark in a way the

86
00:07:34.720 --> 00:07:40.040
framework itself, but different strategies that
they use. Yeah, it's an interesting

87
00:07:40.079 --> 00:07:43.439
conversation. First of all, it's
worth mentioning that Ryan is kind of a

88
00:07:43.519 --> 00:07:46.959
regular guest here on the podcast.
We've had in quite a number of times.

89
00:07:47.279 --> 00:07:51.439
He's an incredible smart person, like
you said, is the creator of

90
00:07:51.560 --> 00:08:00.720
solid Solid Start, is also kind
of the CEO of Signals really popularize that,

91
00:08:01.079 --> 00:08:05.639
and is also one of the people
most knowledgeable about frameworks in general.

92
00:08:05.800 --> 00:08:11.120
I mean, like his hobby is
to get his hands on, you know,

93
00:08:11.279 --> 00:08:16.360
more or less every framework out there
and then do all sorts of comparative

94
00:08:16.680 --> 00:08:22.000
testing and analysis and whatnot. So, yes, if anybody is knowledgeable about

95
00:08:22.079 --> 00:08:26.560
how to best compare performance and other
aspects of frameworks, it's probably ryan.

96
00:08:28.079 --> 00:08:33.879
It's also worth mentioning a tool created
by the Quick people called Mitosis, which

97
00:08:33.080 --> 00:08:41.120
you can actually use it to compile
like sort of pseudo code like React like

98
00:08:41.200 --> 00:08:48.639
pseudocode into various frameworks, and then
you can kind of more easily build the

99
00:08:48.720 --> 00:08:54.320
same application using different frameworks and then
be able to compare it. Because,

100
00:08:54.879 --> 00:08:58.919
like you were saying, if we
want to get away from micro benchmarks and

101
00:08:58.039 --> 00:09:05.639
we want to actually compare real applications
built in various frameworks, we run right

102
00:09:05.720 --> 00:09:11.720
into the problem of the overhead of
building and maintaining sophisticated applications in a variety

103
00:09:11.759 --> 00:09:16.879
of frameworks. And you know who
wants to do that. I just do

104
00:09:16.080 --> 00:09:22.000
want to mention that there's an alternative
approach if you're interested in the performance of

105
00:09:22.080 --> 00:09:26.600
various frameworks, and I actually gave
a talk about that at several conferences,

106
00:09:26.919 --> 00:09:33.200
including I think jays Nation a year
before, which is to use rum data

107
00:09:33.720 --> 00:09:37.879
to compare the performance of frameworks.
So you're not looking like at a specific

108
00:09:37.960 --> 00:09:43.720
application, you're looking across all websites
built with a particular framework, and then

109
00:09:43.840 --> 00:09:50.759
you're not so much saying like how
fast will my application be with this framework.

110
00:09:52.240 --> 00:09:56.440
You're more looking at how likely am
I going to how likely is it

111
00:09:56.559 --> 00:10:01.720
that I'll be able to build a
fast website or web app using a particular

112
00:10:01.799 --> 00:10:09.759
framework, And you might say this
framework is more likely to produce faster websites

113
00:10:09.840 --> 00:10:15.039
or web apps, and this one
is less likely, you know, not

114
00:10:15.240 --> 00:10:20.159
surprising. The one that is least
likely is angular, and the ones that

115
00:10:20.279 --> 00:10:28.600
are most likely are Quick, solid
spelled you know these. But it's really

116
00:10:30.200 --> 00:10:37.639
you really need to be careful about
mixing correlation and causation. Like, for

117
00:10:37.799 --> 00:10:43.200
example, people who use Quick are
people who are more likely to be concerned

118
00:10:43.200 --> 00:10:52.360
about web performance. So are they
producing faster websites because they're using Quick or

119
00:10:52.480 --> 00:10:56.960
are they producing faster websites because there
are people that care about web performance and

120
00:10:58.200 --> 00:11:01.919
then so they know what to do
and they also happen to be using Quick

121
00:11:01.279 --> 00:11:05.799
for that reason. So you need
to be kind of careful but doing these

122
00:11:05.879 --> 00:11:09.960
types of conclusions. That's a very
good point actually, because it's one of

123
00:11:09.000 --> 00:11:13.679
the things that I like you come
to realize when you start collecting run data.

124
00:11:15.600 --> 00:11:20.159
So I established I helped sublished out
time out, time out. I

125
00:11:20.279 --> 00:11:22.679
know that we've defined this in other
episodes. But run data what is it.

126
00:11:24.279 --> 00:11:30.360
It's real usermetrics. So you have
two different sides of performance monitoring.

127
00:11:30.480 --> 00:11:33.120
So you have what is called the
lab and what it's called the rum side.

128
00:11:33.720 --> 00:11:39.080
So the lab is just you know, CICD lighthouse and whatever you do

129
00:11:39.240 --> 00:11:43.440
to make sure that you don't perform
aggressions before changes actually go live to users.

130
00:11:45.360 --> 00:11:48.440
And on the run side you have
different providers and you can even use

131
00:11:48.519 --> 00:11:54.000
like GA Google Analytics even have like
collect automatic luber vitals. Yeah, it's

132
00:11:54.200 --> 00:12:01.559
it's basically the question of are you
testing your performance in a synthetic lab style

133
00:12:01.759 --> 00:12:09.440
setting, maybe even on your own
computer while you're developing and comparing that way,

134
00:12:09.039 --> 00:12:16.679
or are you collecting performance data from
the field, from actual, real

135
00:12:16.879 --> 00:12:22.000
user sessions. So obviously you can
collect data for your own particular website,

136
00:12:22.120 --> 00:12:26.320
especially if you have enough traffic coming
in. But the interesting thing, and

137
00:12:26.399 --> 00:12:31.960
we actually had Rik Viskomi from Google
here to talk about it. Google actually

138
00:12:31.080 --> 00:12:39.480
collects data from all sessions on Chrome
and they put it into this database called

139
00:12:39.720 --> 00:12:45.799
Chrome User Experience Report or CRUs for
short. Here and Steve usually makes a

140
00:12:45.960 --> 00:12:50.399
joke about the crux of the matter
or the crux of the issue or whatever.

141
00:12:50.799 --> 00:12:54.679
Yeah, exactly. And the nice
thing about what Google does is that

142
00:12:54.840 --> 00:13:03.200
they actually give everybody essentially access to
this data. And also they attach all

143
00:13:03.279 --> 00:13:07.200
sorts of meta data to this data, like you know which framework was used

144
00:13:07.240 --> 00:13:11.679
to build which website and whatnot,
so you can do all sorts of slicing

145
00:13:11.759 --> 00:13:18.080
and dicing, so you can look
at the performance of you know, your

146
00:13:18.159 --> 00:13:24.279
own website or competing websites or eCos
websites in general, in particular geos,

147
00:13:24.559 --> 00:13:31.519
or particularly particular types of devices,
or created using particular frameworks, using particular

148
00:13:31.600 --> 00:13:35.240
libraries, et cetera. It's interesting
data for those of us who like to

149
00:13:35.279 --> 00:13:39.159
geek out on performance. Yeah,
that's a that's a really good point to

150
00:13:39.240 --> 00:13:43.639
start. So for like for building, for building web websites, like,

151
00:13:43.759 --> 00:13:46.600
things can go wrong in many different
ways even when people use the same framework

152
00:13:46.639 --> 00:13:52.679
and two point exactly done like some
people when building like because also like when

153
00:13:52.679 --> 00:13:56.000
you pick reacts, most people pick
reactors, and one of the most like

154
00:13:56.200 --> 00:14:01.399
is probably now these demos used framework
out. Really, I don't keep up

155
00:14:01.440 --> 00:14:05.840
with the trends, but oh yeah, for sure. Yeah, so's it's

156
00:14:05.639 --> 00:14:09.720
it's just because of the sheer volume
you have, right of people writing like,

157
00:14:09.840 --> 00:14:11.559
there would be all kinds of quality
out there. It's kind of the

158
00:14:11.679 --> 00:14:16.320
game. It's a case in point
effectively, just to throw the numbers out,

159
00:14:16.720 --> 00:14:22.840
there are many as many websites and
web apps being built in React as

160
00:14:22.919 --> 00:14:26.600
all the other frameworks put together.
Yeah, yeah, I would, I

161
00:14:26.600 --> 00:14:28.600
would imagine so and so in that
mix. There would be a lot of

162
00:14:30.080 --> 00:14:31.919
so when you're trying to divide,
like if you're building some sort of collection

163
00:14:33.039 --> 00:14:37.559
tool based on craucks and you're trying
to divide persontiles for each framework, the

164
00:14:37.159 --> 00:14:41.159
React having so many more samples,
right, it would kind of push the

165
00:14:41.279 --> 00:14:46.320
data towards the left, or rather
towards the right side, where you're going

166
00:14:46.399 --> 00:14:52.399
to have more worse metrics overall,
just because of how many just the sheer

167
00:14:52.480 --> 00:14:58.360
volume of it. Yeah, that's
one aspect. Another aspect is the fact

168
00:14:58.559 --> 00:15:03.159
that a lot of React websites,
like there's a long tail of React websites,

169
00:15:03.240 --> 00:15:09.960
websites either built like long ago or
built with particular focus in mind.

170
00:15:09.480 --> 00:15:13.399
So, for example, if you're
building a website in React, then you're

171
00:15:13.519 --> 00:15:20.519
not using service side rendering or static
side generation if you're only using client side

172
00:15:20.559 --> 00:15:26.919
rendering, which means that you're building
the dorm representation of the website on the

173
00:15:28.000 --> 00:15:33.960
client side, then by definition,
effectively you're not going to have good cowed

174
00:15:33.039 --> 00:15:39.320
vitals, which is the way that
Google measures loading performance. You might have

175
00:15:41.279 --> 00:15:45.600
other aspects of performance that are good, like I don't know, responsiveness or

176
00:15:45.600 --> 00:15:48.679
stuff like that, but in terms
of loading performance, if you're just using

177
00:15:50.759 --> 00:15:56.240
client side rendering of CSR, it's
not going to be good. And the

178
00:15:56.320 --> 00:16:00.399
reality is that a lot of reacts
websites you see s are some of them

179
00:16:00.440 --> 00:16:04.879
don't care. If they're not indexed, then maybe they do. They don't

180
00:16:04.960 --> 00:16:11.000
care, But if you're looking at
the metrics that Google is collecting, that's

181
00:16:11.039 --> 00:16:17.639
what you see. Yeah, And
it's interesting that the whole pivoting that we

182
00:16:17.679 --> 00:16:19.639
are now having tools the service side
as well, because one of the things

183
00:16:19.720 --> 00:16:23.799
that like I've I've worked I've worked
for Spotify and I worked for Klana before

184
00:16:23.960 --> 00:16:30.480
Spotify, and in both places have
set up the like monitoring tools and start

185
00:16:30.519 --> 00:16:36.320
collecting round data and all this kind
of stuff. And for CSR, one

186
00:16:36.360 --> 00:16:40.639
of the strategies that people have is
normally like cold splitting lie that's pretty much

187
00:16:40.639 --> 00:16:44.559
your only or one of the few
things you can do to try to improve

188
00:16:44.679 --> 00:16:48.600
that first paint and then in the
LCP and even that, like when I

189
00:16:48.799 --> 00:16:52.960
run an experiment once trying to understand
especially for CLAN when I was working at

190
00:16:53.000 --> 00:16:57.440
KLANA, I was trying to build
benchmarks on how can we make sure that

191
00:16:57.519 --> 00:17:02.240
we have as a third party,
how can you make sure to affect the

192
00:17:02.360 --> 00:17:06.920
loading of the site you're integrated with
the you know the least of how can

193
00:17:06.960 --> 00:17:10.359
you make sure that you're not the
one who is causing the harm? And

194
00:17:11.160 --> 00:17:15.319
I've ran an experiment on on on
doing cold splitting. And back then,

195
00:17:15.680 --> 00:17:19.000
mind you, it was gosh,
it was like twenty six, twenty eighteen,

196
00:17:19.039 --> 00:17:25.759
I think or twenty seventeen, so
it was I've like did like a

197
00:17:25.880 --> 00:17:30.759
full cold splitting kind of like manualal
splitting, splitting chunks everywhere and like lazy

198
00:17:30.799 --> 00:17:36.240
loading some stuff, And back then
it was kind of a revelation. Now

199
00:17:36.279 --> 00:17:40.039
there's I think more people understand why. But like when you do that many

200
00:17:40.160 --> 00:17:45.079
chunks, things can get very congested
when you're loading, when you're loading websites

201
00:17:45.240 --> 00:17:48.680
and doing a lot of JavaScript small
chunks. At the time, it was

202
00:17:48.680 --> 00:17:51.640
a lot in compression size as well, and you're also going to be hogging

203
00:17:52.000 --> 00:17:55.880
you know, CPU time and back
then not even using HDP two if I

204
00:17:55.920 --> 00:17:59.720
remember correctly, if you though it
was already available, So you know,

205
00:17:59.839 --> 00:18:06.880
like that the whole critical render path
was absolutely destroyed us in congestion. And

206
00:18:07.039 --> 00:18:10.680
I think back then we didn't even
have the priloscanner to try to like hacken

207
00:18:10.759 --> 00:18:15.599
to as like priority and all this
kind of stuff. Yeah, so it

208
00:18:15.799 --> 00:18:18.279
was like back then it was a
revelation on yeah, you can have too

209
00:18:18.359 --> 00:18:22.759
much of a good thing and then
like start working with prioritization of chances and

210
00:18:22.839 --> 00:18:26.519
this kind of stuff. Yeah.
We actually had Robin Marx recently to talk

211
00:18:26.559 --> 00:18:30.680
about this whole network thing and how
it often behaves in a way that's not

212
00:18:30.839 --> 00:18:37.039
expected and to bring it to a
more general aspect and kind of related to

213
00:18:37.200 --> 00:18:42.119
the main topic that we've yet to
focus on. I would say that one

214
00:18:42.200 --> 00:18:51.400
of my main things regarding performance that
I always say is that you've got to

215
00:18:51.599 --> 00:18:56.759
measure. You've got to measure in
order to decide what to focus on,

216
00:18:57.119 --> 00:19:02.599
and after you make it change,
you got to measure to verify that the

217
00:19:02.759 --> 00:19:10.640
change you made actually improves things and
doesn't even potentially degrade them. And I

218
00:19:10.720 --> 00:19:14.440
can give a case in point,
like you know, if your CSS is

219
00:19:14.519 --> 00:19:18.680
small, a lot of times you
will hear that you should inline that CSS

220
00:19:18.799 --> 00:19:23.240
into your HTML to avoid the extra
round trip to bring the CSS because CSS

221
00:19:23.599 --> 00:19:27.200
is rendered blocking, so you want
to get it down as quickly as possible.

222
00:19:27.680 --> 00:19:36.279
And I've seen cases where inlining the
CSS actually degraded performance. And you

223
00:19:36.359 --> 00:19:38.839
know, and without going into the
details of why, the fact is that

224
00:19:40.039 --> 00:19:45.039
once you see that it's actually degraded, you roll it back, yes,

225
00:19:45.279 --> 00:19:52.559
because you know it is measuring the
whole measuring its Like I always mention,

226
00:19:52.920 --> 00:19:57.240
whenever you start working with performance,
you always start from the data. You

227
00:19:57.400 --> 00:20:00.519
always starts. If you don't have
collection story, if you don't have a

228
00:20:00.559 --> 00:20:04.119
good data story, you have to
start collecting data from your users. That

229
00:20:04.279 --> 00:20:07.720
is right, so because because lab
data doesn't really tell you the whole story,

230
00:20:07.759 --> 00:20:12.279
as you've been breaking down earlier,
So you need to understand how is

231
00:20:12.319 --> 00:20:17.240
the actual user experience out there,
and you need to understand even what kind

232
00:20:17.240 --> 00:20:18.920
of data do you want to collect, which kind of brings us into the

233
00:20:19.359 --> 00:20:26.200
topic we've been discussing as well on
the article and the whole thing of understanding

234
00:20:26.200 --> 00:20:29.480
your product and where you want to
collect data and what they're collecting, and

235
00:20:29.680 --> 00:20:34.599
you know, building the story around
performance from that perspective first instead of just

236
00:20:34.720 --> 00:20:38.200
trying to micro optimize. First,
is how you're going to make sure to

237
00:20:38.240 --> 00:20:47.079
deliver impactful results to users. So
yeah, So, I mean I read

238
00:20:47.240 --> 00:20:49.519
part of the article and yeah,
and what I'm wondering, you know,

239
00:20:49.720 --> 00:20:53.559
just jumping in. Then you talked
in your article about lab versus rum and

240
00:20:53.720 --> 00:20:59.200
things like that and how to measure
it. But I mean, if I'm

241
00:21:00.240 --> 00:21:03.319
if I'm really new to this,
right, and you know, Dan mentioned

242
00:21:03.400 --> 00:21:07.920
having measurements and knowing what your numbers
are so that you can, you know,

243
00:21:08.160 --> 00:21:12.759
verify that you had the effect that
you wanted. How do you start

244
00:21:12.799 --> 00:21:15.480
gathering that? I'm assuming that's kind
of your first step, right, is

245
00:21:15.559 --> 00:21:19.960
gathering that information, whether it's in
a lab setting or rum setting. Yeah,

246
00:21:21.279 --> 00:21:27.480
I mean, very often, if
you're like engaged in a performance project

247
00:21:29.039 --> 00:21:33.039
or related project, you'll be under
kind of the gun to show results,

248
00:21:33.640 --> 00:21:41.119
and it's kind of tempting to start
optimizing things. And whenever I was engaged

249
00:21:41.279 --> 00:21:47.359
in projects like that, I quickly
pushed back and said, no, we've

250
00:21:47.440 --> 00:21:52.680
got to get the data first,
We've got to have the graphs. We've

251
00:21:52.720 --> 00:21:59.119
got to have the measurements because otherwise
you're just working blind. And if you

252
00:21:59.319 --> 00:22:06.440
want an extra incentive when it comes
to your time for your annual review or

253
00:22:06.680 --> 00:22:11.160
half annual review and you want to
like prove your worth to the company,

254
00:22:11.720 --> 00:22:17.119
having a graph that shows like this
is what I did. It's a line

255
00:22:17.200 --> 00:22:22.319
that goes up down depending on what
you're actually graphing is a great thing that

256
00:22:22.559 --> 00:22:26.039
you know, you want to have
when you're trying to get a raise or

257
00:22:26.200 --> 00:22:33.359
something like that. So I literally, in various occasions pushed back against management

258
00:22:33.440 --> 00:22:40.000
who were trying to get me to
start optimizing before we actually had good measurements

259
00:22:40.039 --> 00:22:44.279
in place. And and to your
question, Shark, We've had some guests

260
00:22:44.359 --> 00:22:48.839
on the show to talk about you
know these days. There are really two

261
00:22:49.039 --> 00:22:53.039
good, really good ways that I
have to mention to get good metrics.

262
00:22:53.680 --> 00:23:00.480
One is if you is basically just
use the Google the Google Search console.

263
00:23:00.400 --> 00:23:04.960
In the Google Search Console, they
have a Cowork Vitals panel and you can

264
00:23:06.079 --> 00:23:12.400
get you know, information about pages
that have performance issues. It's it's not

265
00:23:12.599 --> 00:23:18.039
an ideal and optimal rum solution.
You know. For example, they average

266
00:23:18.119 --> 00:23:23.519
the results over twenty eight over a
period of a month effectively. So improvements

267
00:23:23.680 --> 00:23:30.240
you will make will take time to
show, and likewise, degradations will take

268
00:23:30.359 --> 00:23:34.079
time until they actually manifest themselves.
But you know, it's a good starting

269
00:23:34.160 --> 00:23:40.640
point. Another good starting point is
that they're like an issues mentioned before,

270
00:23:40.720 --> 00:23:45.240
there are a lot of third party
tools and services out there that are fairly

271
00:23:45.400 --> 00:23:52.079
straightforward to integrate into your website.
Some of them are even like partially free.

272
00:23:53.400 --> 00:23:59.519
But you know, we've had people
from Centry, they have a rum

273
00:23:59.640 --> 00:24:07.279
solution, we had people from gun
Ray, gun Akama, We've had people

274
00:24:07.319 --> 00:24:11.960
from Akamai. They have the impulse
that there are a lot of there a

275
00:24:11.119 --> 00:24:15.839
lot of great tools out there that
are fairly straightforward to integrate and that you

276
00:24:15.960 --> 00:24:21.680
can start getting data from. You
know, yeah, you do. You

277
00:24:21.880 --> 00:24:26.079
have a lot of options like debug, beear and rum vision as a speed

278
00:24:26.160 --> 00:24:30.000
curve. But there is the question
real quick, we're talking about tools,

279
00:24:30.599 --> 00:24:34.720
how often do you see tools because
a lot of these tools are basically jumping

280
00:24:34.839 --> 00:24:38.759
JavaScript into your page, right,
so I know Google Analytics is a classic.

281
00:24:40.440 --> 00:24:44.799
So how many times do you see
the tool that you're putting into measure

282
00:24:44.880 --> 00:24:48.880
performance hurting your performance because of everything
that's putting in your site to measure performance

283
00:24:48.400 --> 00:24:56.440
fun you ask, I have heard
horror stories with ot L and UH integrated.

284
00:24:57.400 --> 00:25:00.359
I'm not gonna yeah, I'm not
gonna name it any games, but

285
00:25:00.440 --> 00:25:07.720
I haven't I have heard interesting stories
when trying to implement open telemetry, and

286
00:25:08.279 --> 00:25:14.279
that's one of the examples where it
can go pretty bad in one way when

287
00:25:14.319 --> 00:25:17.200
you're trying to measure Yeah, I
remember, now if I had a service

288
00:25:17.200 --> 00:25:18.720
that you could pay for, I
haven't seen it a while where whether they

289
00:25:18.720 --> 00:25:23.359
would run stuff like that on the
server so you know you're not dumping JavaScript

290
00:25:23.440 --> 00:25:27.559
into your page. I never used
that reading about it, seeing it as

291
00:25:27.599 --> 00:25:30.880
an alternative. Most of the back
ends have services like that too. And

292
00:25:32.359 --> 00:25:36.400
sure they don't give you like the
core web vitles right because they're not Yeah,

293
00:25:36.440 --> 00:25:38.960
they're front end things because yeah,
because their front end measurements not back

294
00:25:40.039 --> 00:25:44.599
end measurements. But they give you
some information about how long it takes to

295
00:25:45.240 --> 00:25:48.720
Yeah, but I will say this, you know, like Venius mentioned,

296
00:25:48.759 --> 00:25:55.400
you kind of need to be careful. But most of the rum providers that

297
00:25:55.599 --> 00:26:00.640
actually measure coed vitos, you know
they know about with vitals, they've done

298
00:26:00.880 --> 00:26:08.000
work to ensure that whatever they're providing
has has little or no impact on your

299
00:26:08.119 --> 00:26:12.200
score exactly. Yeah, if one
of them does, it will be it

300
00:26:12.279 --> 00:26:15.799
will pretty quickly come out. But
you know, just google it, your

301
00:26:15.880 --> 00:26:19.359
friends, search what people say about
it, and you know, try it

302
00:26:19.440 --> 00:26:25.079
out yourself and see what happening.
Yeah, and on the whole topic of

303
00:26:25.240 --> 00:26:27.319
actually, on the whole topic of
back end, I've wrote an article last

304
00:26:27.680 --> 00:26:33.079
yeah for Perth Calendar about server timings, and I think there's there's a good

305
00:26:33.279 --> 00:26:37.680
amount of things you can do to
help understand and like end to ends sort

306
00:26:37.680 --> 00:26:41.000
of tracing when it comes to even
to back end and stuff. I mean,

307
00:26:41.039 --> 00:26:45.599
we don't have it as well standardized
as web vitls, but understanding because

308
00:26:45.759 --> 00:26:51.160
one of the things that a performance
specialist becomes is like it comes in three

309
00:26:51.279 --> 00:26:53.960
parts, right, Like it's a
salesman, is a data analytics person that

310
00:26:55.240 --> 00:26:57.279
has to understand what kind of data
you're trying to collect and understand the data

311
00:26:57.359 --> 00:27:00.839
collected in order to visualize it well. And you know, a good an

312
00:27:00.880 --> 00:27:06.279
engineer side as well. So the
back and part. There is a whole

313
00:27:06.319 --> 00:27:11.880
lot of attributions that you can build
within the back inside. Unfortunately not as

314
00:27:11.960 --> 00:27:17.799
well standardized as web vials but but
that's actually a good opportunity right where one

315
00:27:17.839 --> 00:27:22.599
can try to define good metrics within
back end. But like when you're trying

316
00:27:22.640 --> 00:27:26.720
to understand especially for instance, when
you're now starting to have more react to

317
00:27:26.759 --> 00:27:32.720
being around on the severn right with
Next, the no neuros versions of Next,

318
00:27:32.759 --> 00:27:37.079
the newest version of React, and
things are kind of moved will be

319
00:27:37.200 --> 00:27:38.799
moved a bit more to the back
end and processing part time is going to

320
00:27:38.839 --> 00:27:45.759
be spent more. You know that
TTFB window, and it does have a

321
00:27:45.920 --> 00:27:48.160
great amount of benefits that you can
get out of here, but some things

322
00:27:48.200 --> 00:27:53.920
can be a bit unexpected, like
so you can have some of the performance

323
00:27:53.960 --> 00:27:59.960
degradation as well on some types of
websites out there, because not all web

324
00:28:00.079 --> 00:28:03.640
sites, perf have trying to differ
the same kind of experience. Does not

325
00:28:03.839 --> 00:28:10.079
all parts of a virus matter equally, so do all the websites, And

326
00:28:10.359 --> 00:28:14.559
so if there is if you are
working on something that is very sensitive to

327
00:28:14.640 --> 00:28:18.599
that CTFB window, you really want
to make sure to understand from the server

328
00:28:18.759 --> 00:28:22.039
timings, you know, from behind
a curtain network curtain what's going on,

329
00:28:22.400 --> 00:28:25.920
and you can collect that data as
well. Yeah, and and by the

330
00:28:25.960 --> 00:28:27.920
way, like I said. We
had Robin Marks on the show, and

331
00:28:29.119 --> 00:28:33.000
one of the things that came up
is basically that if you really care about

332
00:28:33.079 --> 00:28:38.359
TTFB, and especially if you've got
a global audience, then you probably want

333
00:28:38.440 --> 00:28:45.000
to be using a CDN and then
the TTFB of your own server is not

334
00:28:45.160 --> 00:28:49.960
irrelevant, but is you know,
less less impactful I call it. Let's

335
00:28:51.000 --> 00:28:56.319
put it this way, but I
would actually like to pull us back a

336
00:28:56.359 --> 00:29:00.720
little bit and looking at your article
which I have read. Uh, one

337
00:29:00.759 --> 00:29:04.319
of the things that you wrote about
that really resonated with me because it really

338
00:29:04.440 --> 00:29:11.279
matches my own experience is about getting
buy in from management. Can you talk

339
00:29:11.319 --> 00:29:17.039
about that a little bit? Yeah, absolutely. It is the salesman part

340
00:29:17.160 --> 00:29:22.240
of the of the triad, So
it is it is like, in a

341
00:29:22.319 --> 00:29:26.640
way, you really have to understand
the product you're working with. So if

342
00:29:26.680 --> 00:29:30.400
you're working in a product team,
one of the things that you constantly hear

343
00:29:30.920 --> 00:29:34.400
and I've certainly faced it myself as
well when I was working on product teams,

344
00:29:34.519 --> 00:29:41.759
is you have a certain pressure to
ship features constantly. So sometimes sometimes

345
00:29:41.839 --> 00:29:47.119
and sometimes more than others, you
will have pressure looming and trying to solve

346
00:29:47.599 --> 00:29:52.079
backlog stuff, which some in most
cases performance kind of falls under you.

347
00:29:53.119 --> 00:29:56.319
You it's hard to get that buy
in coming from management. So one way

348
00:29:56.359 --> 00:30:00.680
you can start doing it is,
of course, as we mentioned, COLAC

349
00:30:00.799 --> 00:30:03.799
data, and with that data you
collect, you want to try to understand

350
00:30:03.839 --> 00:30:08.279
correlations between you know, pain points
of your users and how your product is

351
00:30:08.359 --> 00:30:15.039
performing. Because when you're trying to
build work within performance, you really want

352
00:30:15.079 --> 00:30:18.880
to understand on if you work on
a certain part of application, you will

353
00:30:18.960 --> 00:30:23.240
deliver the most well the most optimal
results for the perform for the products to

354
00:30:23.359 --> 00:30:27.200
perform. So you're not only trying
to cover the engineering side, but you

355
00:30:27.359 --> 00:30:30.599
will like when getting buying, you
want to make sure to cover the product

356
00:30:30.640 --> 00:30:34.759
side. You know that the shipping
good metrics not only for EPMs and for

357
00:30:34.880 --> 00:30:41.200
your performance metrics, but also understand
how those who affect your product metrics.

358
00:30:41.599 --> 00:30:44.880
So the business side also matters.
So you have to wear kind of like

359
00:30:45.039 --> 00:30:49.400
both hats and build that bridge between
engineering and product. That's how you can

360
00:30:51.119 --> 00:30:56.680
strengthen the buying you get from management. I totally agree with that, and

361
00:30:56.839 --> 00:31:02.160
it kind of reminds me of an
interest seeing experience that I've had quite a

362
00:31:02.240 --> 00:31:07.680
number of years ago. And I
won't be naming names but I remember talking

363
00:31:07.799 --> 00:31:11.720
with this person. I was working
at a certain company and I was speaking

364
00:31:11.799 --> 00:31:18.119
with one of the people in product
and I asked that person why their product

365
00:31:18.160 --> 00:31:30.559
specifications never include any performance requirements and
they answered first he said, I don't

366
00:31:30.839 --> 00:31:37.240
know how to specify performance requirements.
And I get that, And it's become

367
00:31:37.319 --> 00:31:44.079
easier these days again thanks to Google
with co wored vitals we have. They've

368
00:31:44.240 --> 00:31:49.720
made the industry aware of a relatively
small set of well defined metrics that you

369
00:31:49.799 --> 00:31:56.440
know, you can tell that person
from product Hey read this article. I

370
00:31:56.559 --> 00:32:00.400
actually even gave a talk about it
at the previous employer to people to kind

371
00:32:00.400 --> 00:32:07.000
of teach them about how performance is
measured and the impact of performance and whatnot.

372
00:32:07.599 --> 00:32:10.759
So's that was his first point.
And the second point, which really

373
00:32:10.839 --> 00:32:15.720
made me laugh, he said,
but I expect our developers to write the

374
00:32:15.839 --> 00:32:22.119
code in the most performed way possible, because that's what developers do. And

375
00:32:22.240 --> 00:32:27.000
I really started laughing, because you
know, we work, like you said,

376
00:32:27.200 --> 00:32:30.079
we're always in a rush to deliver
features. We're always under the gun.

377
00:32:31.119 --> 00:32:38.759
We're always facing short time frames and
not enough resources, and certain aspects

378
00:32:38.839 --> 00:32:43.160
like performance, can you know,
go out the window in these types of

379
00:32:43.240 --> 00:32:50.079
scenarios if you're pressured to deliver a
feature and you know you won't be really

380
00:32:50.200 --> 00:32:55.119
worrying about the performance impact if you
hardly have time to finish the requirements that

381
00:32:55.240 --> 00:33:05.480
are actually in the spec and and
yeah, so so if you want.

382
00:33:05.759 --> 00:33:10.759
And that's from my perspective why management
buying is so important because very often we

383
00:33:10.920 --> 00:33:15.359
hear about people saying, hey,
can we do stuff like that grassroots?

384
00:33:15.440 --> 00:33:20.079
Can we introduce quality like from the
bottom up and stuff like that? And

385
00:33:20.200 --> 00:33:23.559
the answer is no, because if
you're going to be engaged in a large

386
00:33:23.640 --> 00:33:30.039
scale, ongoing project that's going to
require a lot of resources, time and

387
00:33:30.119 --> 00:33:34.119
effort, maybe even money, if
you're let's say buying rum tools and other

388
00:33:34.720 --> 00:33:40.519
and other things, then management needs
to be aligned with that otherwise it just

389
00:33:40.759 --> 00:33:46.599
can't happen absolutely absolutely Yeah, and
then becomes a fight right over what your

390
00:33:46.599 --> 00:33:52.119
engineering team interests are and where your
product team trying to ship and and you

391
00:33:52.200 --> 00:33:55.400
know all the things that product teams
have agreed on for quarters. So it

392
00:33:55.559 --> 00:34:00.519
just does become like you need to
get a lot more involved with the site

393
00:34:00.519 --> 00:34:05.519
and understand what kind of values matter
the most for the product side. And

394
00:34:05.920 --> 00:34:09.639
that's why it becomes a salesman pitch
because it's not just blindly trying to web

395
00:34:09.719 --> 00:34:15.440
viders have done wonders for that conversation
start. Like, I think most people

396
00:34:15.519 --> 00:34:20.960
nowadays understand some aspects of revisers are
at the very least when you mention it

397
00:34:21.159 --> 00:34:24.400
that it matters for CEO, it
matters for the product. So that's great,

398
00:34:24.639 --> 00:34:30.480
that's really like because I still remember
from the days of Klana, we're

399
00:34:30.559 --> 00:34:34.679
having to build, you know,
like what metrics matter the most for our

400
00:34:34.719 --> 00:34:38.519
product? Was was an actual task
you had to So many metrics are available

401
00:34:38.559 --> 00:34:42.960
to you and so little knowledge around
how they affect different parts of the product.

402
00:34:44.480 --> 00:34:51.199
But when it comes to building those
kind of values and translating them to

403
00:34:51.760 --> 00:34:55.480
product, it becomes onto a where
it's very important to understand the product that

404
00:34:55.559 --> 00:35:00.000
you're working on, because like,
not all products will have like as I

405
00:35:00.039 --> 00:35:04.719
mentioned a bit earlier, like not
all products will have the same kind of

406
00:35:04.920 --> 00:35:08.320
values. Or you can't apply all
metrics blindly to all products because some metrics

407
00:35:08.440 --> 00:35:12.760
just don't matter as much to certain
type of products. For instance, me

408
00:35:12.920 --> 00:35:17.679
working at now, I've overcause one
of the products that I'm currently working very

409
00:35:17.719 --> 00:35:25.320
closely with is the configurator that they
have and the configurators as a as a

410
00:35:25.360 --> 00:35:30.159
product is just an actual product team
that cares for it, and it does

411
00:35:30.320 --> 00:35:36.159
have very very different business metrics.
Product metrics then one would think about when

412
00:35:36.199 --> 00:35:39.119
you think about conversion and all this
kind of stuff. So how the product

413
00:35:39.159 --> 00:35:44.639
behaves in real time is a lot
more important than how it loads, because

414
00:35:45.400 --> 00:35:47.119
you can do a lot of stuff
to try to trick it loading, but

415
00:35:47.280 --> 00:35:51.559
once it is loaded, that's the
thing. Like you have the I in

416
00:35:51.639 --> 00:35:55.920
P metric for all that product is
like king that's the most important part IP

417
00:35:57.159 --> 00:36:00.079
and cls, so you know you
need to and which metrics is going to

418
00:36:00.079 --> 00:36:04.519
be delivered the most impact all the
types of products you're trying to ship.

419
00:36:04.639 --> 00:36:08.079
So you can build you know,
KPIs and slas with downstream teams and try

420
00:36:08.159 --> 00:36:14.119
to have this kind of consolidated governance
model or the least just understand how things

421
00:36:14.119 --> 00:36:17.519
impact your application differently, and from
that part within the engineering, you can

422
00:36:17.639 --> 00:36:23.280
understand also how to tie up those
engineering metrics with what kind of product metrics

423
00:36:23.360 --> 00:36:27.519
matters the most, and you can
try to build correlations from that. I

424
00:36:27.719 --> 00:36:31.840
totally agree with what you just said. An interesting case that I had experience

425
00:36:31.960 --> 00:36:38.079
with was when I was working WIS. There was the WIS editor that you

426
00:36:38.159 --> 00:36:45.480
would use to build websites using withywig
dragging and dropping things around, and you

427
00:36:45.719 --> 00:36:51.519
had the actual websites that you built
with the WIS editor. So for the

428
00:36:51.639 --> 00:36:57.239
websites, the most important aspect was
the loading performance as measured by let's say

429
00:36:57.360 --> 00:37:02.880
LCP Largest Content for pain but for
the editor itself, I mean, obviously

430
00:37:04.039 --> 00:37:08.320
you don't want it to take half
an hour to load, but most customers

431
00:37:09.360 --> 00:37:14.639
didn't care so much about how long
it took the editor to load. What

432
00:37:14.920 --> 00:37:19.519
was more important for them was that
once the editor was loaded, it was

433
00:37:19.719 --> 00:37:24.519
really responsive and snapping, that when
you drag things around, they would drag

434
00:37:24.639 --> 00:37:30.760
smoothly and not like you know,
with this jitter or trailing after the cursor

435
00:37:30.079 --> 00:37:34.960
or stuff like that. So you
really need to be cognizant about what it

436
00:37:35.199 --> 00:37:39.039
is that is important to your users
in the context of performance, and what

437
00:37:39.199 --> 00:37:44.119
it is that you want to optimize, and again going back to the aspect

438
00:37:44.559 --> 00:37:50.320
of getting buy in from management that
it's actually worthwhile to invest the effort because

439
00:37:50.400 --> 00:37:54.079
one of the things that I like
to say is that performance is a journey,

440
00:37:54.199 --> 00:37:58.760
not a destination. It's not like
I'm going to invest a little bit

441
00:37:58.800 --> 00:38:02.079
of time and effort get their performance
house in order, and then we can

442
00:38:02.199 --> 00:38:07.599
forget about it forever because it's done. That's not how it works. You

443
00:38:07.239 --> 00:38:13.960
need to put systems in place to
identify regressions because regressions will happen, and

444
00:38:14.119 --> 00:38:16.679
then when you detect a regression,
you're going to have to invest effort in

445
00:38:16.840 --> 00:38:22.440
fixing that regression. So it's ongoing
effort. And by the way, in

446
00:38:22.559 --> 00:38:27.400
this context, I'm going to post
a good link, so again, thank

447
00:38:27.480 --> 00:38:30.760
you to the people at Google.
Their web dot dev website has a section

448
00:38:30.960 --> 00:38:39.400
called case studies where they where they
post a lot of case studies about how

449
00:38:39.519 --> 00:38:45.679
improving performance has impact variety of businesses
and products. So you can actually go

450
00:38:45.920 --> 00:38:52.599
in there find services that are similar
to what you're building, and then be

451
00:38:52.719 --> 00:38:55.639
able to bring that to management and
say, here, you know, this

452
00:38:55.880 --> 00:39:00.320
company X does something that's similar to
what we're doing. This is how improving

453
00:39:00.400 --> 00:39:05.679
performance has impacted their business, their
bottom line, and this is why it's

454
00:39:05.760 --> 00:39:10.800
worthwhile for us to invest this effort
as well. Yeah, yeah, absolutely,

455
00:39:12.000 --> 00:39:15.880
and there. You know. It's
funny that back in the days where

456
00:39:15.960 --> 00:39:20.960
I actually have an article from that
time. I'm posting here on the chat

457
00:39:21.119 --> 00:39:28.519
and the building building and then performance
monitoring that was for lab That the article

458
00:39:28.639 --> 00:39:31.000
that are based based there. So
it's like build before the time when we

459
00:39:31.079 --> 00:39:35.840
had c I c D with Lighthouse. So I was trying to integrate Lighthouse

460
00:39:35.840 --> 00:39:38.079
into c CD manually and build my
own C I c D server with Lighthouse.

461
00:39:38.440 --> 00:39:42.159
One of the things the Lighthouse allowed
I think it still allows us to

462
00:39:42.199 --> 00:39:45.280
do. I haven't dug into the
Lighthouse source code for quite some time.

463
00:39:45.880 --> 00:39:50.960
It was to define different weights for
different metrics. So when you get a

464
00:39:51.079 --> 00:39:54.559
Lighthouse score from from from within your
c I c D, you can just

465
00:39:54.639 --> 00:40:00.039
find, for instance, you for
if you your application Maths is more I

466
00:40:00.159 --> 00:40:05.679
pay much as more for application,
you can customize the weights of the score

467
00:40:05.760 --> 00:40:07.880
you get from Lighthouse based on that. At least you could back then.

468
00:40:08.079 --> 00:40:10.880
I'm not entirely sure if you still
can because I haven't dug into the source

469
00:40:10.920 --> 00:40:15.320
code for some time, but it's
one of the things that back then top

470
00:40:15.400 --> 00:40:20.079
and source so you could probably do
whatever you want if you. Yeah,

471
00:40:20.920 --> 00:40:22.960
the effort. Yeah, But like
back then when I was doing that kind

472
00:40:23.000 --> 00:40:28.159
of integration, is that defining for
for instance, back then, gosh,

473
00:40:28.440 --> 00:40:31.559
like one of the metrics that we
used to have, I mean l CPFCP

474
00:40:31.679 --> 00:40:36.519
still where we're around, so like
those as a third party at Tallana was

475
00:40:36.599 --> 00:40:38.679
the ones that matter the most.
So I like, you can find tune

476
00:40:38.960 --> 00:40:43.480
the scores based on the weights that
you want, you know, matter the

477
00:40:43.559 --> 00:40:46.639
most for you. So we kind
of talked about it already, but still

478
00:40:46.960 --> 00:40:52.880
one of the sections in your article
is about metrics and KPIs and s l

479
00:40:52.960 --> 00:40:59.039
as. Can you elaborate about this? Yeah, absolutely, So the so

480
00:40:59.159 --> 00:41:01.400
government, if you try and establish
governor if you try governance, and or

481
00:41:02.039 --> 00:41:07.159
if you're trying to get a better
understanding even from what you're trying to ship.

482
00:41:07.480 --> 00:41:12.519
And normally we don't I mean we
don't work in asolation. And as

483
00:41:12.559 --> 00:41:15.760
you're working for a very very small
company, normally you have teams either above

484
00:41:15.840 --> 00:41:20.599
you or below you, so you
are d downstream or you are working teams

485
00:41:20.639 --> 00:41:24.159
downstream, so a graph service or
an API or a back end. So

486
00:41:25.599 --> 00:41:30.599
you you after you understood which kind
of metrics master the most for you and

487
00:41:30.760 --> 00:41:36.000
for your product, and you're building
starting to build the correlations. One of

488
00:41:36.039 --> 00:41:40.840
the natural next steps is to build
KPI so key performance indicators for your application

489
00:41:42.280 --> 00:41:46.039
with a lens of performance of course, and building also a service level agreements

490
00:41:46.079 --> 00:41:52.039
as a as against teams that might
affect your performance. So this comes with

491
00:41:52.119 --> 00:41:57.039
in you know, the any KRE
person would know those by HARP. But

492
00:41:58.079 --> 00:42:04.639
it's one of the things that really
up to strengthen your because it's pretty much

493
00:42:04.800 --> 00:42:07.639
what you said, it's not a
race. It's a continuous thing, and

494
00:42:07.719 --> 00:42:10.400
you will have regressions, and whenever
you have regressions, you first of all

495
00:42:10.480 --> 00:42:15.519
want to document those. And you
always want to document both regressions and any

496
00:42:15.639 --> 00:42:20.280
performance improvements that you shape because you
want to understand how things you know worked

497
00:42:20.320 --> 00:42:25.800
over time or not. And having
those esslays is a good way to communicate

498
00:42:25.880 --> 00:42:30.599
downstream and sure to have more resilience
on things that you're trying to ship.

499
00:42:30.639 --> 00:42:35.639
And because some parts of the performance
when you work in a big product,

500
00:42:35.719 --> 00:42:39.360
some parts of the performance spectrum is
not directly coming from you. One of

501
00:42:39.440 --> 00:42:43.400
the things, for instance, that
we have and I think most people out

502
00:42:43.400 --> 00:42:47.480
there will know this is GTM and
GTM can really you know, if you

503
00:42:47.559 --> 00:42:52.039
have a marketing marketing team that's pushing
a lot of tags on DTM that can

504
00:42:52.239 --> 00:42:58.119
really mess you up on performance and
it's not even coming from your code base

505
00:42:58.239 --> 00:43:01.639
necessarily. And that's Google tech Manager, right, Google tach Manager. Yeah,

506
00:43:01.800 --> 00:43:06.079
So so Google Tech Manager is a
brilliant thing. It's a really really

507
00:43:06.480 --> 00:43:10.360
interesting product. But anything can be
misused and such as the such as the

508
00:43:12.320 --> 00:43:15.000
nature with GTM in some teams and
one of the marketers junk drawer. It

509
00:43:15.079 --> 00:43:20.599
really is. But yeah, yeah, and one of the things that I'm

510
00:43:20.639 --> 00:43:24.199
currently trying to invest in a lot
of time in the governance model that I'm

511
00:43:24.199 --> 00:43:28.280
trying to build this is around GTM
as well. But then how can you

512
00:43:28.400 --> 00:43:32.599
even it's like those the other side
with the GTM side is a completely different

513
00:43:32.599 --> 00:43:37.320
set of skills, So how can
you make sure that you build a connection

514
00:43:37.880 --> 00:43:40.320
on things that both of you can
align on and understand. So that's where

515
00:43:40.599 --> 00:43:45.119
the KPIs and slas come in,
and you can you can really make sure

516
00:43:45.159 --> 00:43:49.199
that you can have a proper well
structured process around it. By the way,

517
00:43:49.360 --> 00:43:54.159
we did have I think Adam Bradley
from a builder io who spoke about

518
00:43:54.199 --> 00:44:00.519
the tool that they build called party
Town which can which can reduce the impact

519
00:44:00.679 --> 00:44:06.920
of third party scripts and pixels and
whatnot by moving them off of the main

520
00:44:07.039 --> 00:44:12.400
thread onto a web worker. The
big challenge is that it's a pretty manual

521
00:44:12.480 --> 00:44:15.840
effort. You know, we spoke
about needing management buy in in order to

522
00:44:16.559 --> 00:44:24.239
approve effort. That's one of those
things. It's not a drop in solution

523
00:44:24.519 --> 00:44:30.360
that you can just put in your
product and watch the improvement. You probably

524
00:44:30.480 --> 00:44:36.679
need to do some manual integration work
and in order to get the real benefits.

525
00:44:36.800 --> 00:44:40.599
But when you do, the benefits
can be substantial. In the case

526
00:44:40.760 --> 00:44:46.719
of Next Insurance, where I previously
worked, by putting in party town,

527
00:44:46.840 --> 00:44:54.000
we were able to dramatically improve our
I n P. The number of pages

528
00:44:54.280 --> 00:45:00.599
that have good I n P scores
went from about something like less than half

529
00:45:00.679 --> 00:45:08.239
our pages to effectively all our pages
thanks thanks to using party tap. Yeah,

530
00:45:08.519 --> 00:45:15.119
because Yeah, Unfortunately, a lot
of these third party scripts can have

531
00:45:15.280 --> 00:45:21.199
significant impact on the responsiveness of the
website. They run a lot of JavaScript

532
00:45:21.320 --> 00:45:24.480
on the main thread unless you know
you can figure it otherwise, and that

533
00:45:24.800 --> 00:45:32.599
impacts how responsive the pages to user
interactions. Unfortunately. Yeah, and that's

534
00:45:32.599 --> 00:45:37.840
why building a good attribution model is
is like absolutely fundamental. It's one of

535
00:45:37.880 --> 00:45:42.880
the things that I'm obsessing as a
late because once you have the metrics up

536
00:45:43.159 --> 00:45:45.480
on the monitor into that's a great
start. You're off to a great start.

537
00:45:45.519 --> 00:45:49.280
You are starting to understand, you
know, the nature of a product

538
00:45:49.320 --> 00:45:52.599
and you're starting to understand how the
things you do impact it. But then

539
00:45:52.639 --> 00:45:57.760
again, as previously mentioned, like
performance comes from a performance recradition can come

540
00:45:57.800 --> 00:46:02.360
from different passive web you have the
bugbear I should release a great, great

541
00:46:02.480 --> 00:46:09.480
article just a few weeks ago about
the impact of Chrome plugins. So if

542
00:46:09.519 --> 00:46:15.880
you have like different different plugins within
Google Chrome, they can also affect your

543
00:46:15.440 --> 00:46:20.119
your performance, and some more than
others. And there's a brittiant article there.

544
00:46:20.159 --> 00:46:23.639
I'll see if I can find before
these recording ends. But it's really

545
00:46:23.679 --> 00:46:28.679
it's a really good example on how
performance sometimes you can have some degradations coming,

546
00:46:29.239 --> 00:46:31.880
you know, on your rear usermetrics, all nonetheless and coming from different

547
00:46:32.000 --> 00:46:37.400
kinds of things, even out of
control. So building good attributions means not

548
00:46:37.480 --> 00:46:42.639
only having the metrics, but you
know, web vitals have an actual attribution

549
00:46:42.800 --> 00:46:45.719
build that you can do use where
you can send more information about each one

550
00:46:45.760 --> 00:46:50.159
of the metrics you're collecting. So
if you have you know, i NP

551
00:46:50.559 --> 00:46:55.280
you will show you if you will
like details about the love data long animation

552
00:46:55.400 --> 00:47:00.800
frame, and you can have a
better intelligence on where is it coming from.

553
00:47:00.920 --> 00:47:04.039
Is it coming from a third party, is it coming from or even

554
00:47:04.079 --> 00:47:07.159
a comm extension. Is it coming
from Google Tag Manager or something like that,

555
00:47:07.920 --> 00:47:13.360
and that can give you more even
more power we're talking about. Yeah,

556
00:47:13.559 --> 00:47:19.800
a lot of the tools provided by
run providers have kind of attribution capabilities

557
00:47:19.880 --> 00:47:23.880
built in, so you know,
you don't necessarily need to be an expert.

558
00:47:24.599 --> 00:47:29.840
But that's actually another point that I
want to mention is that it does

559
00:47:30.039 --> 00:47:37.000
require some expertise. I don't expect
that you know to if you want to

560
00:47:37.079 --> 00:47:44.039
do some performance improvements on your website, it does require some know how.

561
00:47:44.519 --> 00:47:47.119
So either you know, you find
it really interesting and you invest the time

562
00:47:47.719 --> 00:47:53.000
and an effort and learn it.
Alternatively, you can bring in an expert,

563
00:47:53.079 --> 00:47:57.599
maybe hire an expert if you're large
enough. If you're not, you

564
00:47:57.719 --> 00:48:02.000
can bring in a consultant, somebody
like Roberts, which we mentioned before we

565
00:48:02.119 --> 00:48:07.480
started recording. We actually had him
as a guest on the show. Can

566
00:48:07.559 --> 00:48:10.239
bring on somebody like that to do
an audit for you and kind of point

567
00:48:10.280 --> 00:48:17.519
you at the right directions. Yeah, I'm curious with you mentioned Chrome extensions

568
00:48:17.559 --> 00:48:22.239
as an example, and you have
attribution that that tells you when you gather

569
00:48:22.400 --> 00:48:27.039
the information. Hey, look,
you know people who have this extension are

570
00:48:27.079 --> 00:48:32.000
having this impact right one way,
one way or the other. What do

571
00:48:32.079 --> 00:48:35.280
you do about it? I mean, can you just add it to your

572
00:48:35.360 --> 00:48:39.760
CI for some of your tests to
check for the performance to get you know,

573
00:48:40.119 --> 00:48:44.559
once you've set those performance standards,
do you put a banner up that

574
00:48:44.719 --> 00:48:49.920
detects, Hey, you're running this
Chrome extension and it impacts you know,

575
00:48:50.519 --> 00:48:53.559
slow down your thing? I mean, yeah, what do you do with

576
00:48:53.639 --> 00:48:57.280
it? Yeah? No, So
first of all, yes, you can

577
00:48:57.440 --> 00:49:02.320
put it as part of your test
environment for sure. The question is is

578
00:49:02.480 --> 00:49:08.880
the slowdown happening because just the fact
that the extension is there, like gives

579
00:49:08.920 --> 00:49:15.880
this extension just doing generally a bad
thing, or is it something that's kind

580
00:49:15.880 --> 00:49:22.440
of a bad into react, you
know, some sort of reaction or or

581
00:49:23.119 --> 00:49:28.800
like, you know, your page
specifically doesn't work well with this extension for

582
00:49:28.920 --> 00:49:35.199
some reason. Now, if it's
the latter, then obviously, if it's

583
00:49:35.239 --> 00:49:39.119
and it's a popular extension, then
you may want to invest effort in trying

584
00:49:39.239 --> 00:49:44.119
to fix it. Like, obviously
you can't fix the extension, but you

585
00:49:44.199 --> 00:49:47.280
can try to fix your stuff if
it's the former. If it's just a

586
00:49:47.360 --> 00:49:52.119
problem with the extension, basically it
is what it is. You may want

587
00:49:52.320 --> 00:49:59.760
to put some notice on your website. I wouldn't pop up an alert that's

588
00:50:00.159 --> 00:50:07.880
is overly intrusive, but like you
know, having something in your knowledge base

589
00:50:08.559 --> 00:50:14.760
so at least your support people would
be cognizant of the problem and if a

590
00:50:14.840 --> 00:50:17.880
customer complaints, then they can ask
them, you know, are you using

591
00:50:17.960 --> 00:50:23.519
this extension and if so, please
don't. But it's like a fact of

592
00:50:23.639 --> 00:50:30.840
life, right, yeah, So
on that, we have a great talk

593
00:50:30.360 --> 00:50:36.480
from last year's Perth now by team
Verike. He mean is the name of

594
00:50:36.599 --> 00:50:43.119
talk is noise noise canceling rumor or
something like that. I posted a link

595
00:50:43.159 --> 00:50:50.480
here on the chat and that talks
about what can you do to eliminate noise

596
00:50:50.599 --> 00:50:53.599
out of your metrics? And that's
like, that's why having this kind of

597
00:50:53.679 --> 00:50:58.400
attribution and understanding your attribution, and
you're very right like this this kind of

598
00:50:58.559 --> 00:51:00.480
and this is one of the problems
when you to performance as a subject is

599
00:51:00.519 --> 00:51:07.920
because it is somewhat of a very
steep vertical for people to get into.

600
00:51:09.079 --> 00:51:13.599
But one of the things you can
do is once you to start understanding these

601
00:51:13.679 --> 00:51:16.519
kind of things is to understand also
how to remove noise from a metrics and

602
00:51:16.639 --> 00:51:23.440
understand better how your actual personiles are
split up onto and understanding the distributions because

603
00:51:23.440 --> 00:51:29.840
it also, like building better attributions
for a product, comes from bringing back

604
00:51:29.880 --> 00:51:32.400
to the whole, like tying up
with your product values, right, it

605
00:51:32.519 --> 00:51:37.280
comes to understanding what kind of market
markets means much as the most for your

606
00:51:37.280 --> 00:51:42.280
product. Right, So you don't
want to really measure globally because the bigger

607
00:51:42.320 --> 00:51:45.800
the metrics scope you're trying to have, the harder it is for you to

608
00:51:45.920 --> 00:51:50.000
find attributions or the hard it is
for you to understand them because you're trying

609
00:51:50.039 --> 00:51:53.599
to deal with something that is too
large of a context. And it's a

610
00:51:53.679 --> 00:51:59.199
lot easier when you're trying to slice
their data up. And like let's say,

611
00:51:59.760 --> 00:52:01.800
for certain products, certain markets are
a lot more important, and for

612
00:52:02.000 --> 00:52:06.320
that product, certain pages or fear
functionalities are a lot more important. So

613
00:52:06.400 --> 00:52:10.280
you definitely want to slice your analytics
and your data and your run metrics in

614
00:52:10.400 --> 00:52:15.639
such a way where you can cut
through the noise that you gather globally into

615
00:52:16.079 --> 00:52:22.239
something that builds better attribution models and
builds better relations out of it. So

616
00:52:22.440 --> 00:52:25.360
that like removing this kind of noise
from a metrics is also a really really

617
00:52:25.400 --> 00:52:32.400
good way to avoid I'll give a
different example for you know, it might

618
00:52:32.559 --> 00:52:43.079
be that certain geos have performance issues. So for example that again talking about

619
00:52:43.159 --> 00:52:49.840
Wiggs Global Company, we saw very
different performance profiles for example for North America

620
00:52:50.119 --> 00:52:53.960
and for South America. On the
other hand, you need to be careful

621
00:52:54.000 --> 00:52:58.559
with these sorts of things because for
example, you might say, hey,

622
00:52:58.800 --> 00:53:04.679
something really happens bad happened in Colombia
for some reason, I'm seeing a significant

623
00:53:04.719 --> 00:53:08.840
degradation, and then you realize,
hey, I only have like eight sessions

624
00:53:08.920 --> 00:53:17.159
a day, so if just one
bad session can skew my my data for

625
00:53:17.599 --> 00:53:22.159
that geo, So you kind of
need to be careful. And again this

626
00:53:22.320 --> 00:53:27.840
is something that you also talked about
in the article about how you need to

627
00:53:27.920 --> 00:53:34.480
be careful with blind spots, like
how you know, some data can hide

628
00:53:34.599 --> 00:53:38.639
or obscure some other data and cause
you to miss important aspects. Yeah,

629
00:53:39.639 --> 00:53:43.559
yeah, yeah, it's like I
bring within the ark, I bring to

630
00:53:43.960 --> 00:53:47.559
the same talk by Tim and it
is like it is one of the things

631
00:53:47.880 --> 00:53:54.239
that it comes again from the triad
right where you that's from the data analytical

632
00:53:54.360 --> 00:54:00.719
part where you need to understand the
alternal the data you have, because otherwise

633
00:54:01.000 --> 00:54:06.360
you might be looking too too much
noise, and then it's harder to work

634
00:54:06.440 --> 00:54:09.519
with that data. Right Like,
then your performance work that you're trying to

635
00:54:10.039 --> 00:54:15.199
carve out of the data might even
be is aligned, So when you're trying

636
00:54:15.239 --> 00:54:17.800
to ship performance, you might not
even see the needle moving properly because you're

637
00:54:17.880 --> 00:54:23.400
just being affected by by noisy run
data. I will say one thing,

638
00:54:23.800 --> 00:54:29.639
and I think we can summarize this
particular aspect with it, is that I'm

639
00:54:29.679 --> 00:54:32.920
also a member of the W three
C WHERE Performance Working Group, and there's

640
00:54:32.960 --> 00:54:38.320
always tension there with regard to attribution
because you've got the run providers who are

641
00:54:38.440 --> 00:54:45.880
pushing to get ever more accurate and
detailed attribution data that they can externalize and

642
00:54:45.000 --> 00:54:50.800
expose to their customers. And on
the other hand, you've got the browser

643
00:54:51.400 --> 00:54:58.400
manufacturers who are concerned about privacy issues. So you know, with too much

644
00:54:58.519 --> 00:55:04.559
attribution that opens the door to all
sorts of fingerprinting techniques and stuff like that.

645
00:55:05.119 --> 00:55:08.000
So you kind of need to strike
a good balance there, or put

646
00:55:08.039 --> 00:55:12.599
another way, you know, yet
another reason why we can't have good things.

647
00:55:13.599 --> 00:55:15.760
Yeah, that's a good point.
Well, there's valid concerns, so

648
00:55:16.840 --> 00:55:21.119
yeah, it is a valid concern. It is very good. They're both

649
00:55:21.519 --> 00:55:25.519
valid concerns. I really want to
know what's causing my headaches, but yeah,

650
00:55:25.679 --> 00:55:30.559
I also want to protect my users. Yeah, there's a very good

651
00:55:30.599 --> 00:55:34.519
ways to anonymize that data, and
most most collection tools will make sure to

652
00:55:34.599 --> 00:55:37.440
do so. But that is a
very if you're building your own collection tool,

653
00:55:37.480 --> 00:55:42.599
that's a very valid concern to have. Yeah, and again it's also

654
00:55:42.639 --> 00:55:47.679
a question for the browser manufacturers whether
or not to provide certain APIs or not,

655
00:55:49.480 --> 00:55:52.400
because you know, obviously, if
you could ensure that only good actors

656
00:55:52.519 --> 00:55:57.679
actually use those APIs, that would
be wonderful, but unfortunately the world doesn't

657
00:55:57.719 --> 00:56:02.159
work that. Yeah. Absolutely,
I think we're nearing the end of our

658
00:56:02.199 --> 00:56:07.880
show. Is there anything in particular
that you wanted to mention that we haven't

659
00:56:07.079 --> 00:56:15.760
mentioned so far? I mean,
I can basically summarize that working with performance

660
00:56:15.840 --> 00:56:21.239
in a company large or small,
it starts with data, and understanding that

661
00:56:21.360 --> 00:56:24.239
data is naturally the second step because
a lot of times you're very eager to

662
00:56:25.119 --> 00:56:29.880
get you know, get down with
actual code and try and move that.

663
00:56:30.119 --> 00:56:34.199
You know, sometimes it can be
a bit too early, and you're if

664
00:56:34.239 --> 00:56:37.599
you try to work on something that
is not going to be shifting not only

665
00:56:37.679 --> 00:56:43.239
the actual application performance towards the right
way, but not affecting the right product

666
00:56:43.280 --> 00:56:46.199
metrics. You know, it's it's
very hard to get that buying again and

667
00:56:46.400 --> 00:56:50.159
just gets harder to get a performance
work off the ground. Of course,

668
00:56:50.159 --> 00:56:54.440
if you're working with a mature engineering
organization, that can be a different story,

669
00:56:54.960 --> 00:56:59.239
but most people are not in that
kind of stage, right, So

670
00:56:59.719 --> 00:57:05.039
we're with attributions first and data and
analyzing it. It might seem the most

671
00:57:05.119 --> 00:57:07.639
boring part, but it's the most
important one as well, So make sure

672
00:57:07.679 --> 00:57:15.280
to just understand your data before trying
to work and performance. Good deal,

673
00:57:15.440 --> 00:57:17.800
All right, Well let's go ahead
and do our picks before we do that,

674
00:57:17.960 --> 00:57:22.639
though, where do people find you
on the internet if they're thinking,

675
00:57:22.760 --> 00:57:29.400
oh, I want to know more? Absolutely, I am mostly nowadays on

676
00:57:29.559 --> 00:57:37.679
Twitter x X. Everybody knows what
you mean when you say Twitter. I'm

677
00:57:37.960 --> 00:57:40.920
okay, yeah, it's just I
am not yet most of the times I

678
00:57:42.000 --> 00:57:45.760
forgot. It's been renamed to be
honestly, but I think everybody would prefer

679
00:57:45.880 --> 00:57:53.719
to forget there. I am Webb
Tita, and I mean if I have

680
00:57:53.800 --> 00:57:59.400
a pretty easy name to look on
LinkedIn, but that I spend most of

681
00:57:59.440 --> 00:58:02.920
the time talking about performance on twater. That's where most people confinm web tweeter

682
00:58:04.760 --> 00:58:09.039
with t w I t R.
It's a bit u, it's a bit

683
00:58:09.079 --> 00:58:15.119
shortened. Cool. All right,
Well, let's do it and do our

684
00:58:15.199 --> 00:58:20.360
picks. Steve, you want to
start us off going for the high points

685
00:58:20.400 --> 00:58:24.079
first, Okay, uh, before
I get to the dad jokes of the

686
00:58:24.119 --> 00:58:27.760
week. I do have one pick. And this has been around for a

687
00:58:27.800 --> 00:58:30.639
while and I've heard other people talk
about it extensively, but it's something I

688
00:58:30.719 --> 00:58:36.519
finally started digging into, and that's
the warp terminal. For years, I've

689
00:58:36.519 --> 00:58:40.880
always just used the built in terminal
app that comes on Mac. And you

690
00:58:40.960 --> 00:58:45.760
know, I've gotten fairly good at
at you know, switching me tabs tabs

691
00:58:45.760 --> 00:58:50.079
and I've got you know, uh
tab group set up and all that.

692
00:58:50.239 --> 00:58:52.920
But I had heard a lot about
it, so I started playing around the

693
00:58:52.960 --> 00:58:58.920
warp terminal and it's really really slick, makes it very easy to find things.

694
00:58:58.960 --> 00:59:02.000
There's all kinds of enhance since you
can create custom workflows, which is

695
00:59:02.039 --> 00:59:07.760
sort of like custom commands where you
pass in parameters and you know you can

696
00:59:07.280 --> 00:59:12.000
it's pretty easy to customize your prompt
to show like what get branch you're on,

697
00:59:12.159 --> 00:59:15.119
and how what the difference is between
you and your remote in terms of

698
00:59:15.199 --> 00:59:20.519
normal commit colors. There's all kinds
of customizations you can do. I know

699
00:59:20.559 --> 00:59:22.199
a lot of people use it term
two and I've used that in the past,

700
00:59:22.280 --> 00:59:28.559
but I really like the WARP terminal
just in what I've played around.

701
00:59:28.840 --> 00:59:32.760
You can find it at warp dot
dev on the internets. I'm gonna jump

702
00:59:32.840 --> 00:59:37.400
in here because warp is a recent
thing that I've picked up as well,

703
00:59:37.519 --> 00:59:44.400
and yeah, all the things that
Steve said. It also has some AI

704
00:59:45.199 --> 00:59:52.519
so you can basically tell it what
you wanted to do and kind of like

705
00:59:52.559 --> 00:59:58.159
an AI prompt, and it sometimes
works, sometimes it doesn't work quite how

706
00:59:58.199 --> 01:00:04.519
you want. Sometimes it also interprets
my get commands as or my commands with

707
01:00:04.599 --> 01:00:07.840
them, like you know, using
the rail cli and it'll say, oh,

708
01:00:08.480 --> 01:00:10.519
what you're saying is you want to
do this, and I'm like yes,

709
01:00:10.599 --> 01:00:15.159
because that is literally the command to
do that. So it's not perfect,

710
01:00:15.239 --> 01:00:19.880
but you know, sometimes it's handy
when you're trying to remember the exact

711
01:00:20.360 --> 01:00:23.000
combination of said AWK and GP that
you want in order to get the thing.

712
01:00:23.840 --> 01:00:28.400
So anyway, yeah, it also
has some team stuff where you can

713
01:00:28.480 --> 01:00:31.320
use it with teams and share workflows
and share out pit if. I haven't

714
01:00:31.400 --> 01:00:34.639
use any of that. I haven't
had a chance to play any either.

715
01:00:34.760 --> 01:00:37.119
But the downside that some people don't
like is that you have to create an

716
01:00:37.159 --> 01:00:40.519
account and they say why you need
to do that, and so some people

717
01:00:40.559 --> 01:00:44.760
don't like that. But anyway,
it's it's really pretty cool. I really

718
01:00:44.920 --> 01:00:47.480
liked it. Yep, all right, dad, Jokes of the week.

719
01:00:49.239 --> 01:00:52.400
So I've never understood while people wear
black when they want to be sneaky,

720
01:00:53.000 --> 01:01:01.760
they should just wear leather armor because
it's made of hide. Didn't even get

721
01:01:01.800 --> 01:01:04.880
a smile at it, Dan,
all Right, I gotta keep working here,

722
01:01:05.199 --> 01:01:07.239
so it's gonna see somebody laughed.
Don't laugh, it only encourages him.

723
01:01:08.840 --> 01:01:14.320
Uh So, I took my kids
to see Disney on Ice and it

724
01:01:14.440 --> 01:01:21.159
really sucked. He was just some
old dead guy in a box I had.

725
01:01:21.440 --> 01:01:27.079
I actually responded to that, Yes
that I said that I actually expected

726
01:01:27.119 --> 01:01:30.880
a dead mouse rather than a dead
true. Well, if you notice on

727
01:01:30.960 --> 01:01:35.239
Twitter somebody responded with an actual cartoon
of that same joke, and it shows

728
01:01:35.280 --> 01:01:38.039
these people, uh looking, here
is Walt Disney in a clear coffin on

729
01:01:38.159 --> 01:01:43.400
I. It's really pretty morbid,
but it's very funny too. And then

730
01:01:43.480 --> 01:01:51.760
finally, uh, what famous military
general was killed by a cannon? Napoleon

731
01:01:51.800 --> 01:01:59.960
blown apart? Those are my jokes
of the week. All right, well,

732
01:02:00.079 --> 01:02:01.639
thank you for elevating my week,
Dan, what are your picks?

733
01:02:06.800 --> 01:02:09.719
First? Would like to unmute.
I would like to remind us all that

734
01:02:09.920 --> 01:02:16.599
Napoleon actually started as a gunnery officer
and one got one of his big promotions

735
01:02:17.440 --> 01:02:22.679
because you know, there was this
demonstration and they were trying to mob the

736
01:02:24.119 --> 01:02:34.119
parliament and he basically dispersed the crowd
by firing the cannons point blank into the

737
01:02:34.239 --> 01:02:39.599
crowd. Yeah, that got him
to become a general. So yeah,

738
01:02:42.079 --> 01:02:50.000
anyway, So about being blown apart
by Bonaparte? Uh yeah, anyway,

739
01:02:51.239 --> 01:02:53.719
So I don't have that much in
a way of picks this week. So

740
01:02:53.960 --> 01:03:00.400
one thing that I have is that
Matt Pocock, who we've had added as

741
01:03:00.440 --> 01:03:05.199
a guest on our show to talk
about Typescript, has written a book about

742
01:03:05.840 --> 01:03:09.559
Surprise, Surprise typescript, And since
I consider Matt to be one of the

743
01:03:09.840 --> 01:03:15.400
foremost experts on this topic, this
is a really good thing, especially given

744
01:03:15.519 --> 01:03:22.000
that to the online version is available
for free. So I'm actually going to

745
01:03:22.280 --> 01:03:25.840
share the link here right now,
and we'll also obviously put it in the

746
01:03:25.960 --> 01:03:30.960
show notes. And the dog is
barking, so hopefully you can hear what

747
01:03:30.079 --> 01:03:40.880
I'm actually saying. But here's that
link to that book online. Highly recommended.

748
01:03:42.840 --> 01:03:50.239
Making effective use of typescript is incredibly
useful and incredibly powerful, and it's

749
01:03:50.360 --> 01:03:54.079
something that's kind of expected from web
developers these days, you know, I

750
01:03:54.199 --> 01:04:00.079
think it can be. It's a
first statement to say that almost all of

751
01:04:00.159 --> 01:04:06.480
us have moved off of you know, untyped JavaScript and into typescript. So

752
01:04:08.920 --> 01:04:15.760
that's that would be my first pick. My second pick is the fact that

753
01:04:16.840 --> 01:04:27.639
Google has kind of integrated AI into
the browsers sort of, so they're actually

754
01:04:27.760 --> 01:04:36.440
experimenting with the Window dot AI.
It's currently I think it's last time I

755
01:04:36.599 --> 01:04:41.280
checked, it was only in the
beta version of Chrome and not in the

756
01:04:41.360 --> 01:04:45.159
release version, and you needed to
enable a flag to get it to work.

757
01:04:45.679 --> 01:04:54.360
But it's basically what they've done is
that they've baked Gemini Micro whatever they

758
01:04:54.440 --> 01:05:01.760
call it, into the browser itself, so that you've got U a large

759
01:05:01.880 --> 01:05:09.360
language model that's a small large language
model that's effectively baked into the browser itself

760
01:05:09.880 --> 01:05:15.639
and runs locally so that you don't
need to be concerned with paying the bills

761
01:05:15.760 --> 01:05:21.679
for running it in the cloud,
which turns out is how NVIDI is making

762
01:05:21.760 --> 01:05:28.519
a ton of money and all the
AI startups are burning through cash by paying

763
01:05:28.599 --> 01:05:36.480
all their money to Nvidia. So
being able to run models locally is a

764
01:05:36.840 --> 01:05:44.599
very interesting technological development and it will
be interesting to see what happens with that

765
01:05:45.000 --> 01:05:50.320
and how various websites are able to
leverage this technology once it becomes mainstream and

766
01:05:50.440 --> 01:05:54.800
open to all. Yeah, I
mean, considering how well the original gem

767
01:05:54.880 --> 01:05:58.960
and I roll out went, I
think this should go very smoothly without any

768
01:05:59.039 --> 01:06:03.400
hitches or anything exactly. And also
there's the fact that it's a model that's

769
01:06:03.519 --> 01:06:09.880
not optimized for any particular use case. It's kind of a general model,

770
01:06:10.480 --> 01:06:15.159
so there's also that aspect to factor
in. I don't know what you want

771
01:06:15.239 --> 01:06:19.079
to do with AI, but you
know, it does open up some interesting

772
01:06:19.840 --> 01:06:28.519
opportunities for user interactions that's that's put
it this way in websites, Yeah,

773
01:06:28.559 --> 01:06:33.159
because it has because it's a Gemini
micro or whatever, it has much smaller

774
01:06:33.199 --> 01:06:36.840
context on what it was trained on. But it's still very good to as

775
01:06:36.880 --> 01:06:42.000
a as a transformer, as a
as an LLM. If you are curious

776
01:06:42.000 --> 01:06:45.719
about that stuff, I definitely suggest
following Jason Mays, the webi lead from

777
01:06:45.800 --> 01:06:49.199
Google, that he has a lot
of great demos on what you can do

778
01:06:49.280 --> 01:06:56.119
with that stuff, because not only
the web ai APIs the upcoming, but

779
01:06:56.199 --> 01:07:00.320
you can also load models with web
assembly and web GP you they can use

780
01:07:00.360 --> 01:07:03.159
your own models kind of locally.
So that's pretty that's it does a lot

781
01:07:03.239 --> 01:07:09.199
of good stuff can Yeah. The
only downside there is that downloading running the

782
01:07:09.320 --> 01:07:13.559
model locally, like I said,
solves a lot of the coast issues.

783
01:07:14.079 --> 01:07:18.320
But it's also these models can be
even the smaller models can be fairly large,

784
01:07:18.400 --> 01:07:25.639
so it can downloading. So downloading
them locally that can that can be

785
01:07:26.199 --> 01:07:28.880
that can you know, we were
talking about performance, that could be a

786
01:07:28.960 --> 01:07:30.440
performance issue. I was going to
say that could hurt your performance. I

787
01:07:30.440 --> 01:07:33.440
can hurt your performance, But then
again, it depends on the nature of

788
01:07:33.519 --> 01:07:36.280
a product, right because if your
look, for instance, working with something

789
01:07:36.320 --> 01:07:39.920
like Photoshop on the web, they
do kind of they kind of do that

790
01:07:40.159 --> 01:07:45.239
and it's not about the loading times
but execution times in that case. Yeah,

791
01:07:45.800 --> 01:07:48.239
which then you're actually gains significant boost, right because it's local, so

792
01:07:48.360 --> 01:07:56.599
the latency is non existence. Well, and then I guess it's my picks.

793
01:07:57.320 --> 01:07:59.639
Yeah you can go, then I'll
go yes, So go ahead,

794
01:08:00.679 --> 01:08:03.639
Okay, So for my picks,
I have two things. Since we're on

795
01:08:03.760 --> 01:08:10.920
the topic of AI and mL,
I've being started to dive into mL training

796
01:08:11.039 --> 01:08:15.440
and machine learning, building models and
this kind of stuff, and I actually

797
01:08:15.559 --> 01:08:18.399
chose to do it with Alexia as
a language. It's a brilliant language as

798
01:08:18.520 --> 01:08:24.720
just as fellow Brazilian I'm also from
Brazil. It's a really nice functional language

799
01:08:24.760 --> 01:08:27.319
to get started with. And one
of the things is that, especially come

800
01:08:27.319 --> 01:08:30.520
from JavaScript, there is not as
much tooling fatigue because when you within Alexi,

801
01:08:30.640 --> 01:08:34.920
things are kind of like very coherent
and there are you know, all

802
01:08:35.000 --> 01:08:39.119
the options are kind of built by
the same people, so everything kind of

803
01:08:39.199 --> 01:08:44.039
speaks not and no point intend the
same language, so things kind of flow

804
01:08:44.319 --> 01:08:45.800
very easily. So it's not like, forri instance, you've tried to learn

805
01:08:45.880 --> 01:08:51.239
Python and then choosing an mL library
to do things on top of and compatibility

806
01:08:51.279 --> 01:08:55.560
issues and all this kind of stuff. So that's my technical pick, and

807
01:08:56.039 --> 01:09:00.720
non technical I would I'm all finishing
the last season of freet and if you

808
01:09:00.800 --> 01:09:03.119
haven't watched that series, it's really
good to watch. With the cases though,

809
01:09:03.119 --> 01:09:06.039
if the kids is old enough,
and it's a very nice, like

810
01:09:06.159 --> 01:09:11.600
halted series to watch kind of tries
to be like hearted in the bocalytical setting,

811
01:09:12.000 --> 01:09:16.000
which is an interesting thing. Awesome. Well, I'm going to throw

812
01:09:16.079 --> 01:09:18.319
in a couple of picks on my
own. First of all, I just

813
01:09:18.399 --> 01:09:21.159
want to point out, if you
are interested in Alixir, we have an

814
01:09:21.159 --> 01:09:25.199
Elixir podcast. It's called a Elixer
Mix. I am not on that one,

815
01:09:25.279 --> 01:09:29.880
but those guys are awesome and they
cover things very very well. I've

816
01:09:29.920 --> 01:09:34.159
also talked to Jose on multiple occasions
from when he was in the Ruby community

817
01:09:34.199 --> 01:09:39.359
and then when he started doing Elixir
stuff, so it's it's a very very

818
01:09:39.399 --> 01:09:45.600
cool language. And if you're whenever
I hear about the Elixir, it's usually

819
01:09:45.640 --> 01:09:47.720
in the context of how great it
is, and to my shame, I've

820
01:09:47.760 --> 01:09:51.000
yet to learn it. Maybe I
should start listening to that podcast. Then,

821
01:09:51.680 --> 01:09:56.199
yeah. Yeah, it's a very
very not so hard language to get

822
01:09:56.199 --> 01:10:00.119
into. Even though it's fully functional, it's a very very easy function to

823
01:10:00.199 --> 01:10:05.960
the programming language. Still getting too
very approachable. I'm gonna start out with

824
01:10:06.680 --> 01:10:10.920
game pick. I always pick a
board game or a card game. When

825
01:10:11.079 --> 01:10:13.840
pick a card game, it's called
six NIM that's n I m M.

826
01:10:15.319 --> 01:10:16.920
Now. When I looked it up
on board game Geek, it says that

827
01:10:17.079 --> 01:10:20.119
it is the same game as Take
five. It has a bunch of other

828
01:10:20.279 --> 01:10:29.359
names uhrendi, which looks like is
Italian category five, Take six blah blah

829
01:10:29.359 --> 01:10:32.720
blah blah. Anyway, board game
Geek rates it or has a weight of

830
01:10:32.800 --> 01:10:39.199
one point one nine says eight and
older can play it, which is probably

831
01:10:39.600 --> 01:10:45.760
pretty accurate. Really, just quickly, what it is is everybody plays a

832
01:10:45.800 --> 01:10:48.319
card face down. So you put
it. You put out four cards.

833
01:10:48.359 --> 01:10:53.920
There are four piles. Then you
put out your card face down, and

834
01:10:53.960 --> 01:10:56.520
then you go from lowest to highest
and you put your card. You put

835
01:10:56.560 --> 01:11:00.439
the card onto whatever pile it would
go on, right, So it's it's

836
01:11:00.720 --> 01:11:05.159
whatever number is below it and nearest
to it, right. So you know,

837
01:11:05.720 --> 01:11:09.479
if you put if there's a thirty
four out there, and you put

838
01:11:09.520 --> 01:11:11.680
out a thirty five, it's going
to go on the thirty four. You

839
01:11:11.760 --> 01:11:14.039
put out a forty, the forty
will go on it. As long as

840
01:11:14.359 --> 01:11:16.479
there's nothing between thirty five and forty
out there, it'll go on the other

841
01:11:16.560 --> 01:11:19.199
card, right, And it's always
the highest card in the pile that you're

842
01:11:19.199 --> 01:11:24.119
playing on. When you play the
sixth card on a pile, you get

843
01:11:24.159 --> 01:11:28.439
all five cards in the pile,
and that sixth card that you play becomes

844
01:11:28.640 --> 01:11:32.079
the start of the next pile.
The different cards are worth different points,

845
01:11:32.760 --> 01:11:36.840
And the only other I guess rule
is is if you play a number that

846
01:11:36.960 --> 01:11:41.680
is lower than any of the piles
out there, then you just take whichever

847
01:11:41.800 --> 01:11:44.840
pile you want and replace it with
your card. So it's like you played

848
01:11:44.880 --> 01:11:46.399
the sixth card. But in a
lot of times there's a one or two

849
01:11:46.520 --> 01:11:50.319
card pile that only is worth one
or two points, and so you take

850
01:11:50.359 --> 01:11:53.640
that pile because you're trying to get
the lowest number of points. You play

851
01:11:53.680 --> 01:11:57.319
till somebody gets to sixty six.
Game's over there. You know how to

852
01:11:57.359 --> 01:12:00.760
play it. It was a lot
of fun. Numbers are numbered one to

853
01:12:01.119 --> 01:12:06.159
one oh four, I think,
and so yeah, you just deal out

854
01:12:06.239 --> 01:12:10.880
ten cards to every player and you
put the rest off to the side,

855
01:12:11.159 --> 01:12:13.760
so some of the some of the
numbers aren't gonna be out there, but

856
01:12:14.199 --> 01:12:15.640
anyway, it was a lot of
fun. I think we played it in

857
01:12:15.720 --> 01:12:19.279
what ten to twenty minutes. I
mean, it's real, real quick play.

858
01:12:19.439 --> 01:12:23.800
But if you look up for a
fast fun game six nim or take

859
01:12:23.880 --> 01:12:28.239
five or whatever it's called where you
live, that was super fun game.

860
01:12:29.880 --> 01:12:32.119
I'm also going to pile on the
AI stuff, so I've started getting into

861
01:12:32.239 --> 01:12:36.760
writing AI code. I've been playing
with it in both Ruby and JavaScript.

862
01:12:38.600 --> 01:12:42.039
I have to say that a lot
of the tools for AI are actually nicer

863
01:12:42.079 --> 01:12:48.119
and Ruby. That might surprise some
people, but anyway, what I'm looking

864
01:12:48.199 --> 01:12:50.600
to do, and so keep an
eye out for this, I bought the

865
01:12:50.640 --> 01:12:57.680
domains AI for Ruby fo R Ruby
and AI for JavaScript, and so if

866
01:12:57.720 --> 01:13:01.479
you're uh and I'm gonna I'm put
together a newsletter for both of those topics.

867
01:13:02.520 --> 01:13:08.920
I'm also going to be putting on
a summit, probably the week after

868
01:13:09.159 --> 01:13:14.279
Labor Day here in the US,
which is the first Monday of September,

869
01:13:14.520 --> 01:13:16.439
so it'll probably be Friday and Saturday. I'll do the Ruby one first and

870
01:13:16.479 --> 01:13:19.840
then the JavaScript right after that,
and then two weeks after that, So

871
01:13:19.960 --> 01:13:24.119
toward the end of September beginning of
October, I'm going to do a three

872
01:13:24.159 --> 01:13:26.600
month boot camp and I'm going to
be teaching people how to do JAVA or

873
01:13:26.680 --> 01:13:31.560
how to do AI. But the
difference between my boot camp and and it'll

874
01:13:31.560 --> 01:13:34.079
be a three month boot camp,
so we're going to get into prompt engineering

875
01:13:34.199 --> 01:13:38.479
and uh, you know, building
chatbots and right were we're going to be

876
01:13:38.560 --> 01:13:44.039
using the APIs and you know,
we might do some model training, but

877
01:13:44.159 --> 01:13:45.840
it's going to be fairly lightweight,
you know, so you don't have to

878
01:13:45.880 --> 01:13:48.520
know the math, you don't have
to you know, have a deep understanding

879
01:13:48.560 --> 01:13:54.279
of the models. This is how
do you add AI to your app,

880
01:13:55.079 --> 01:13:58.439
you know, your web app basically, but it could be a desktop app

881
01:13:58.520 --> 01:14:00.960
or something else if you're writing it
Ruby or John script. So and if

882
01:14:00.960 --> 01:14:03.960
you're if you're you know, primary
language is a different language. Maybe it's

883
01:14:03.960 --> 01:14:10.439
Elixir. I'm I'm almost certain they
have libraries that attached to the same stuff.

884
01:14:10.960 --> 01:14:14.840
And so you can probably figure it
out. And I may or may

885
01:14:14.880 --> 01:14:16.039
not be able to point you in
the right direction. But but this is

886
01:14:16.079 --> 01:14:21.520
what we're gonna do. And so
yeah, so uh AI for JavaScript and

887
01:14:21.600 --> 01:14:25.720
AI for Ruby dot com. If
you go to those websites, you'll be

888
01:14:25.760 --> 01:14:28.479
able to sign up for the newsletter
for free. You'll be able to sign

889
01:14:28.560 --> 01:14:32.479
up for the summit for free.
If you want the videos after the summit,

890
01:14:32.600 --> 01:14:38.039
that that's what you're paying for on
those and then the boot camp will

891
01:14:38.319 --> 01:14:41.439
will cost money as well. But
I want to talk to people make sure

892
01:14:41.439 --> 01:14:45.279
that they're in a position actually take
advantage of the boot camp because it's not

893
01:14:45.319 --> 01:14:47.640
going to be a cheap thing.
So anyway, that's what I'm working on

894
01:14:47.760 --> 01:14:53.720
these days, and really I'm really
digging it. It's it's fun, fun,

895
01:14:53.880 --> 01:14:58.840
fun stuff. Another pick that I
have this is a Ruby Rogues episode.

896
01:14:58.840 --> 01:15:00.960
I'll put a link to it in
the show note. But we recently

897
01:15:01.039 --> 01:15:08.560
talked to Obi Fernandez and he has
a book about building AI stuff. It's

898
01:15:08.960 --> 01:15:15.279
language agnostic. I don't think we've
released the Ruby Rogues episode yet. We

899
01:15:15.399 --> 01:15:20.199
might have, but oh yeah,
it is up here. I'll put the

900
01:15:26.479 --> 01:15:30.920
here it is, I'll get the
link in here. But anyway, I'm

901
01:15:30.279 --> 01:15:36.199
really really loving the AI stuff.
I think it's amazing. I love the

902
01:15:36.319 --> 01:15:42.119
only thing that I guess JavaScript has
that Ruby doesn't is TensorFlow jas right,

903
01:15:42.479 --> 01:15:46.079
there's no TensorFlow Ruby, so you
have to interface through JavaScript or Python.

904
01:15:47.840 --> 01:15:53.119
But beyond that, yeah, I'm
I'm really enjoying just what you can put

905
01:15:53.159 --> 01:15:57.439
together with it, and I think
a lot of the capabilities that come out

906
01:15:57.439 --> 01:16:03.199
of it go well beyond just the
prompt for something like chat GPT. Obie

907
01:16:03.239 --> 01:16:09.479
actually explains it very well. What
he's done is he's built essentially virtual AI

908
01:16:09.840 --> 01:16:16.119
assistance that they take the prompt,
but then they also have API capabilities into

909
01:16:16.359 --> 01:16:20.279
systems that you use, like your
email and stuff, and so you can

910
01:16:20.399 --> 01:16:25.960
actually write a prompt where it'll go
find an email for you, or respond

911
01:16:26.039 --> 01:16:28.960
to certain kinds of emails for you, or things like that. And so

912
01:16:30.279 --> 01:16:32.720
you know, you start getting into
Okay, I'm not going to just prompt

913
01:16:32.800 --> 01:16:35.359
you to write something out or give
me an answer. I'm going to prompt

914
01:16:35.359 --> 01:16:41.439
you to actually go do something useful. So anyway, I have pontificated on

915
01:16:41.600 --> 01:16:45.680
that longer than I needed to,
and I am very much enjoying what I've

916
01:16:45.680 --> 01:16:48.640
got. There one other pick that
I have, and this is a technology

917
01:16:48.720 --> 01:16:55.119
pick. It's something that is put
out by base Camp or thirty seven Signals.

918
01:16:56.119 --> 01:17:01.560
It's called Turbinative, and it's essentially
a way of raping your web applications

919
01:17:01.600 --> 01:17:06.199
into a native app that you can
deploy to the app stores on Android and

920
01:17:06.239 --> 01:17:11.840
iOS. And I have been very
very happy with what I've been able to

921
01:17:11.920 --> 01:17:15.720
do with that without having to write
a full on native app. And I

922
01:17:15.840 --> 01:17:19.359
can see the pros and cons,
right, I mean, if you're not

923
01:17:19.479 --> 01:17:23.520
on the web, it won't work
right, and you know, maybe you

924
01:17:23.600 --> 01:17:27.000
can use some local storage or things
like that to make it do what you

925
01:17:27.079 --> 01:17:31.279
want and maybe load anyway, but
the majority of it just requires you to

926
01:17:31.680 --> 01:17:35.920
operate, you know, connected to
the Internet. But most people are operating

927
01:17:35.960 --> 01:17:42.520
on their phones connected to the internet
anyway, and so you know, anyway,

928
01:17:42.560 --> 01:17:45.239
for what that's worth, I'm really
really enjoying that looking at ways to

929
01:17:45.279 --> 01:17:51.560
get my web apps onto other systems
though beyond phones, like the Firestick TV

930
01:17:51.720 --> 01:17:56.039
and things like that. But Firestick
TV has a way of wrapping web apps

931
01:17:56.079 --> 01:18:00.840
anyway, and so anyway, that's
just another area that I'm I'm diving into

932
01:18:00.000 --> 01:18:02.840
and I might put together a boot
camper a course on that as well.

933
01:18:02.920 --> 01:18:10.560
But anyway, so turber Native is
my last pick for Hey, I and

934
01:18:10.600 --> 01:18:14.399
Alex say you have an X and
bumblebee. So if you were trying to

935
01:18:15.039 --> 01:18:18.359
get like familiar with that subject training
models and running models and ex. Simbo

936
01:18:18.399 --> 01:18:27.199
will because you covered cool, I
just well, there we go. It

937
01:18:27.319 --> 01:18:30.880
made the window go away and I
couldn't click on it. All right,

938
01:18:30.960 --> 01:18:35.079
good deal. Well thanks for coming. Uh I'm always hesitating the same name,

939
01:18:38.319 --> 01:18:41.359
Niches. Thank you for coming.
This is This has been really cool

940
01:18:41.479 --> 01:18:44.239
and I love kind of just getting
into Hey, you know these are the

941
01:18:45.000 --> 01:18:49.199
steps and kind of levels to performance
and how to read the data and then

942
01:18:49.239 --> 01:18:54.199
how to make your case with the
data. Yeah, my pleasure. Always

943
01:18:54.239 --> 01:18:59.079
happy to talk about it. All
right, Well we'll go ahead and wrap

944
01:18:59.119 --> 01:19:00.279
it up here. Until next time, folks, max out