1
00:00:02,040 --> 00:00:05,440
Speaker 1: So welcome everyone to another episode of Adventures in DevOps.

2
00:00:05,480 --> 00:00:08,960
Happy new Year, Warren, Happy new year. Thanks for joining

3
00:00:09,000 --> 00:00:09,400
me today.

4
00:00:10,279 --> 00:00:14,000
Speaker 2: Thank you for having me back. Always a pleasure. Jillian,

5
00:00:14,240 --> 00:00:16,399
welcome back. How are you good?

6
00:00:16,480 --> 00:00:18,320
Speaker 3: Good? How are you? And Happy New Year?

7
00:00:18,920 --> 00:00:19,679
Speaker 4: I'm everyone.

8
00:00:20,559 --> 00:00:27,760
Speaker 1: I'm pretty excited about today's episode because we have your background, AJ.

9
00:00:28,320 --> 00:00:30,160
So our guest today is a J Funk and his

10
00:00:30,280 --> 00:00:37,320
background is just so cool. So you're a software engineer

11
00:00:37,439 --> 00:00:43,320
for Rainforest, you are a you live in California, you're

12
00:00:43,359 --> 00:00:46,840
a snowboarder, and you are a previous d one and

13
00:00:47,039 --> 00:00:48,960
professional baseball player.

14
00:00:49,000 --> 00:00:49,799
Speaker 2: Is that all correct?

15
00:00:50,320 --> 00:00:55,560
Speaker 5: That is correct. I've taken an interesting career path, so yeah,

16
00:00:55,960 --> 00:00:58,439
definitely lots of stuff to do you here, and I'm

17
00:00:58,479 --> 00:01:01,359
in the Lake Tahoe area in California around the Porta

18
00:01:01,399 --> 00:01:05,480
in California and Nevada, So yeah, it's all true.

19
00:01:06,159 --> 00:01:08,120
Speaker 2: What position did you play in baseball?

20
00:01:08,640 --> 00:01:09,400
Speaker 5: I was a pitcher?

21
00:01:09,799 --> 00:01:11,519
Speaker 2: A pitcher, Okay, cool.

22
00:01:11,319 --> 00:01:16,799
Speaker 3: Oh that's what I know what it is. I was

23
00:01:16,799 --> 00:01:18,519
about to be like, I've told you what we're talking

24
00:01:18,560 --> 00:01:20,400
about here, but I know what the picture is.

25
00:01:21,760 --> 00:01:23,599
Speaker 2: It's all about the little connections, Jillian.

26
00:01:24,120 --> 00:01:25,000
Speaker 3: It really is.

27
00:01:26,400 --> 00:01:26,920
Speaker 2: So cool.

28
00:01:27,760 --> 00:01:30,319
Speaker 1: A j tell us a little bit about your role

29
00:01:30,439 --> 00:01:34,799
at Rainforest and what Rainforest does, and I think I'm

30
00:01:34,799 --> 00:01:41,439
interested to hear what led you to go from pitching

31
00:01:41,599 --> 00:01:44,120
Major League Baseball to pitching code?

32
00:01:45,760 --> 00:01:50,480
Speaker 5: Yeah for sure. Yeah, so I started writing code when

33
00:01:50,640 --> 00:01:53,680
when I was a kid, really always as a hobby.

34
00:01:54,319 --> 00:01:57,719
My plan life was always to play baseball. You know,

35
00:01:57,799 --> 00:02:00,200
once you become an adult and you realize that's not

36
00:02:00,239 --> 00:02:04,280
always such a viable career path. I started realizing that

37
00:02:04,359 --> 00:02:06,159
this thing I do as a hobby I could do

38
00:02:06,239 --> 00:02:09,680
as a career. So just started kind of dabbling and

39
00:02:09,719 --> 00:02:12,439
realizing that I actually really enjoyed doing this, and it

40
00:02:12,560 --> 00:02:16,319
was a big pivot from you know, just athletics all

41
00:02:16,360 --> 00:02:21,240
the time. So yeah, now I ended up at Rainforest QA.

42
00:02:21,360 --> 00:02:24,319
I've been here for seven and a half years now.

43
00:02:24,879 --> 00:02:28,159
Oh wow, Yeah, it's been a long time, and it's

44
00:02:28,400 --> 00:02:31,240
it's been really fun. I work with really really great people,

45
00:02:31,840 --> 00:02:37,199
really intelligent experienced engineers, really great culture. We're distributed all

46
00:02:37,240 --> 00:02:39,080
over the world, so it's fun. We get to go

47
00:02:39,479 --> 00:02:41,719
meet up with each other every once in a while

48
00:02:42,000 --> 00:02:45,039
in some random place in the world, and so it's

49
00:02:45,120 --> 00:02:48,800
it's a really enjoyable place to work. I specialize in

50
00:02:48,800 --> 00:02:51,120
front and development, So I spent a lot of time

51
00:02:51,159 --> 00:02:54,680
with the product and design team shaping how our product works.

52
00:02:55,080 --> 00:02:58,680
And you know, as a quality assurance company, it's really

53
00:02:58,719 --> 00:03:01,639
important for us to have a really high bar of

54
00:03:01,759 --> 00:03:05,960
quality and reliability for a product. Obviously, if our app breaks,

55
00:03:06,240 --> 00:03:08,280
why are you going to trust us to make sure

56
00:03:08,280 --> 00:03:14,080
that your app doesn't break? It's reasonable, yeah, right, and

57
00:03:14,639 --> 00:03:17,479
you know, to meet that high bar, we use rainforest

58
00:03:17,560 --> 00:03:20,439
to test Rainforest. So being able to eat your own

59
00:03:20,479 --> 00:03:23,159
dog food on a daily basis is a really really

60
00:03:23,199 --> 00:03:26,680
good position to be in. It allows you to identify

61
00:03:26,680 --> 00:03:29,680
pain points in the user experience before your users do.

62
00:03:30,120 --> 00:03:32,879
Hopefully that's not always true, but we do our best,

63
00:03:33,560 --> 00:03:35,439
and it also means we spend a lot of time

64
00:03:35,479 --> 00:03:38,199
thinking about the best way to do QA, both from

65
00:03:38,240 --> 00:03:42,800
a kind of philosophical standpoint and from a practical implementation

66
00:03:43,080 --> 00:03:49,479
standpoint reality in other words, so this experience obviously helps

67
00:03:49,599 --> 00:03:52,639
guide our product roadmap, but it's also led me to

68
00:03:52,639 --> 00:03:55,560
develop a lot of strong opinions on how product and

69
00:03:55,599 --> 00:03:59,280
engineering team should shape their testing strategies, what kind of

70
00:03:59,280 --> 00:04:02,560
tools they should you'd be using, and how they can

71
00:04:02,560 --> 00:04:06,759
ship code quickly and continuously without having a sacrifice quality

72
00:04:08,240 --> 00:04:08,639
for sure.

73
00:04:08,680 --> 00:04:11,919
Speaker 1: For me, that seems like one of those rabbit holes

74
00:04:11,919 --> 00:04:15,080
that it's really hard to find the balance on of

75
00:04:15,120 --> 00:04:19,399
like what's the minimum level of testing that you should

76
00:04:19,480 --> 00:04:24,079
be doing, and then like what's a what's an adequate

77
00:04:24,199 --> 00:04:29,399
level to get into where you're actually still getting good

78
00:04:29,480 --> 00:04:31,279
use for your time, you know, because obviously you can

79
00:04:32,399 --> 00:04:36,480
throw something at it that tests every possible combination or

80
00:04:37,439 --> 00:04:39,959
or path through your application that could ever be taken,

81
00:04:40,000 --> 00:04:42,879
and you're going to reach a point of diminishing returns there.

82
00:04:42,879 --> 00:04:44,759
Speaker 2: So how do you figure out what that right spot is?

83
00:04:45,519 --> 00:04:50,160
Speaker 5: Yeah? Absolutely, the myth of one hundred percent test coverage, right, Yeah, Yeah,

84
00:04:50,360 --> 00:04:53,600
that we we all strive for. I think they teach

85
00:04:53,639 --> 00:04:57,319
you very early on that, like one hundred test coverage

86
00:04:57,360 --> 00:05:00,439
is what we should be doing with every every time

87
00:05:00,439 --> 00:05:03,079
we push code there, we should make sure everything is covered.

88
00:05:03,519 --> 00:05:05,639
The reality is that's just not possible, right, What does

89
00:05:05,639 --> 00:05:07,439
that even mean? Does that mean every line of code

90
00:05:07,519 --> 00:05:11,800
is covered? Does that mean every possible edge case is covered? Like,

91
00:05:11,879 --> 00:05:14,160
you're never going to get all of those things. So

92
00:05:14,199 --> 00:05:18,639
the trick is finding that balance, right, We want to

93
00:05:18,639 --> 00:05:22,480
make sure that we have confidence in the thing that

94
00:05:22,519 --> 00:05:25,720
we're shipping that both the thing we're shipping works and

95
00:05:25,759 --> 00:05:28,360
that we're not breaking anything else. But we also want

96
00:05:28,399 --> 00:05:30,279
to be able to move quickly. We don't want this

97
00:05:30,399 --> 00:05:34,879
to get into our way, and so the main ways

98
00:05:34,879 --> 00:05:38,439
we go about that is really thinking about what your

99
00:05:38,439 --> 00:05:42,560
testing strategy is, right, What are you testing? What is

100
00:05:42,600 --> 00:05:46,040
your bar for quality? Because in reality sometimes we're okay

101
00:05:46,079 --> 00:05:49,240
if some things break, especially when we're in the early

102
00:05:49,279 --> 00:05:56,160
stages of prototyping something it's a beta feature, etc. And

103
00:05:57,920 --> 00:06:02,120
what are the layer of your testing, right, so we

104
00:06:02,199 --> 00:06:05,240
can we can think about our testing strategy as layers,

105
00:06:05,759 --> 00:06:11,680
with pictured as three layers, but more as a period. Right.

106
00:06:11,759 --> 00:06:17,480
So your foundation of your testing strategy is your unit tests,

107
00:06:17,480 --> 00:06:20,439
something we're all familiar with. We're using code to test

108
00:06:20,480 --> 00:06:21,800
code and small chunks.

109
00:06:21,879 --> 00:06:22,040
Speaker 2: Right.

110
00:06:22,079 --> 00:06:25,000
Speaker 5: What a unit is is totally up to you. We

111
00:06:25,040 --> 00:06:28,720
can define it as a single function or some class

112
00:06:28,720 --> 00:06:32,759
component whatever. The top of our pyramid is our end

113
00:06:32,800 --> 00:06:37,759
to end or our UI tests. That's testing our application

114
00:06:38,000 --> 00:06:41,279
in a kind of real world scenario. As we go

115
00:06:41,399 --> 00:06:46,639
up the pyramid, these tests are more comprehensive, we can

116
00:06:46,680 --> 00:06:50,920
rely on them more, but they're slower, they're more expensive. Right,

117
00:06:51,279 --> 00:06:53,199
So at the base of our pyramid, we have all

118
00:06:53,240 --> 00:06:55,959
of these unit tests that we could run constantly. So

119
00:06:56,040 --> 00:06:58,959
at Rainforest, we run these every single time you push

120
00:06:59,000 --> 00:07:01,399
code it. Just run them because it's cheap. We don't

121
00:07:01,399 --> 00:07:04,160
really care if you're lazy like me. I like to

122
00:07:04,240 --> 00:07:06,680
just push my code up and oh, something broke, I

123
00:07:06,759 --> 00:07:09,480
didn't notice, because I don't want to run the whole

124
00:07:09,480 --> 00:07:12,839
test suite all the time on my local machine. But

125
00:07:13,480 --> 00:07:17,639
as you move up that pyramid from unit tests, the

126
00:07:17,680 --> 00:07:20,120
middle of that pyramid would be our integration test, so

127
00:07:20,199 --> 00:07:23,680
basically testing multiple chunks of these units and how they

128
00:07:23,720 --> 00:07:26,759
interact with each other. That could be maybe one of

129
00:07:26,759 --> 00:07:30,079
your micro services talking to another micro service or something

130
00:07:30,120 --> 00:07:32,040
like that. As we get to the top of the

131
00:07:32,040 --> 00:07:34,240
pyramid and we start running these end to end tests,

132
00:07:34,680 --> 00:07:38,639
that's where I think strategy becomes much more important because,

133
00:07:39,120 --> 00:07:41,759
like I mentioned, they're slower, so we actually care about

134
00:07:41,759 --> 00:07:43,959
when we run these. We need to be more strategic

135
00:07:44,079 --> 00:07:48,759
about how often we run them. And they are more expensive,

136
00:07:49,040 --> 00:07:50,720
so we don't want to just blow a bunch of

137
00:07:51,120 --> 00:07:54,240
a whole bunch of money on them, right, So finding

138
00:07:54,279 --> 00:07:56,680
the balance between those things is the real trick.

139
00:07:57,199 --> 00:07:59,519
Speaker 4: I like that you pulled out test pyramid and not

140
00:07:59,639 --> 00:08:04,600
one of the other newer hipster trends of the test

141
00:08:06,079 --> 00:08:10,519
diamond or a test climb model gone with the tried

142
00:08:10,560 --> 00:08:12,920
and true test pyramid. I mean, at least that's how

143
00:08:12,959 --> 00:08:15,480
I've always seen it. And you know, it's also really

144
00:08:15,480 --> 00:08:18,959
interesting that you bring up the one hundred percent test

145
00:08:18,959 --> 00:08:22,279
coverage is not possible at least from like anecdotal experience

146
00:08:22,319 --> 00:08:25,639
for me, I found it's almost like a preto distribution

147
00:08:26,360 --> 00:08:29,639
and follows the eighty twenty rule where if you wanted

148
00:08:29,639 --> 00:08:31,839
to have one hundred percent test coverage, it would actually

149
00:08:31,920 --> 00:08:33,879
require an infinite amount of time.

150
00:08:34,600 --> 00:08:41,120
Speaker 5: Absolutely, yeah, we certainly strive to have all that test coverage,

151
00:08:41,120 --> 00:08:44,039
but I think the reality of one hundred percent test

152
00:08:44,080 --> 00:08:48,360
coverage is more along the lines of how you define it.

153
00:08:48,440 --> 00:08:52,399
Like your user workflows, Right, so the typical examples like

154
00:08:52,440 --> 00:08:57,080
your log and flow. What are the the main outcomes

155
00:08:57,120 --> 00:09:00,759
of the log in it's successful, log in, failed, log in?

156
00:09:00,960 --> 00:09:04,360
Maybe I forgot your password? And do you have test

157
00:09:04,399 --> 00:09:09,080
coverage and to end test coverage on that flow? If so,

158
00:09:09,279 --> 00:09:12,799
we could usually consider that cover. Do I have a

159
00:09:12,960 --> 00:09:15,840
unit test that covers every combination of things that I

160
00:09:15,879 --> 00:09:20,159
could type into that box. Absolutely not, but having some

161
00:09:20,279 --> 00:09:22,039
sort of test covered on it to make sure that

162
00:09:22,039 --> 00:09:25,519
it actually loads in some kind of real world scenario

163
00:09:25,759 --> 00:09:27,519
gives me much more confidence.

164
00:09:30,399 --> 00:09:31,960
Speaker 2: With a lot of things like logins.

165
00:09:32,080 --> 00:09:35,559
Speaker 1: And there's so many areas these days in developing software.

166
00:09:35,600 --> 00:09:42,200
You're using SaaS products as a as the mechanism for

167
00:09:42,279 --> 00:09:43,159
implementing that.

168
00:09:43,759 --> 00:09:45,279
Speaker 2: What's your approach for.

169
00:09:47,039 --> 00:09:52,159
Speaker 1: Dealing with that external dependency? Because you can you can

170
00:09:52,240 --> 00:09:54,960
mock it, or you can try to stimulate it, or

171
00:09:55,000 --> 00:09:56,120
you can actually call it.

172
00:09:56,519 --> 00:09:57,919
Speaker 2: What do you what do you think about those?

173
00:09:58,240 --> 00:10:00,320
Speaker 4: I feel like that the job at a me. Well,

174
00:10:00,440 --> 00:10:03,200
I see you, I see you're coming there.

175
00:10:04,399 --> 00:10:08,480
Speaker 2: Now now I'm trying to Yeah, I got nothing came

176
00:10:08,559 --> 00:10:09,120
back to you.

177
00:10:12,919 --> 00:10:18,639
Speaker 5: Yeah. So I think what you're asking is when we

178
00:10:18,840 --> 00:10:22,480
run any kind of tests, take We'll stick with unit

179
00:10:22,519 --> 00:10:26,360
tests for for the time being. There's a context that

180
00:10:26,399 --> 00:10:32,240
they run inside of, right, and if we wanted to

181
00:10:32,279 --> 00:10:35,919
test our app as close as possible to reality, we

182
00:10:35,960 --> 00:10:39,759
would test it in production on an actual machine with

183
00:10:39,840 --> 00:10:43,159
an actual human, which some people do. Right. There's obviously

184
00:10:43,200 --> 00:10:46,240
downsides to that testing and production probably don't have to

185
00:10:46,240 --> 00:10:49,399
get into why that's not a great idea.

186
00:10:49,720 --> 00:10:52,720
Speaker 3: And the entire video game industry is wrong.

187
00:10:58,039 --> 00:11:01,240
Speaker 5: And yes they are wrong. I mean, I'm not even

188
00:11:01,279 --> 00:11:02,200
sure that's true anymore.

189
00:11:02,240 --> 00:11:04,639
Speaker 4: Like they don't even release games, right, It's just like

190
00:11:04,759 --> 00:11:07,039
content that you click download on and you pay for

191
00:11:07,120 --> 00:11:09,799
and then the game comes later or something like. I

192
00:11:09,919 --> 00:11:11,799
think that's what the game industry has gone towards.

193
00:11:11,960 --> 00:11:13,559
Speaker 3: That does tend to be how it goes. But I

194
00:11:14,320 --> 00:11:16,240
still feel like they have the users doing an awful

195
00:11:16,279 --> 00:11:19,840
lot of acceptance testing in the video game industry and

196
00:11:19,919 --> 00:11:22,960
I'm like, too cheap for this nonsense. But anyways, I'm

197
00:11:22,960 --> 00:11:25,799
trying not to be rail the entire conversation today, so

198
00:11:25,919 --> 00:11:27,759
we can we can skip right on over that.

199
00:11:28,320 --> 00:11:31,519
Speaker 5: No, I totally agree. It's like you do. Even when

200
00:11:31,559 --> 00:11:34,519
you do get a game, it's an incomplete game that's buggy,

201
00:11:34,600 --> 00:11:36,080
and you play it for an hour and you know,

202
00:11:36,120 --> 00:11:39,960
I'm never playing this again. So I'm a late adopter

203
00:11:40,080 --> 00:11:42,559
when it comes to these things. I wait till the

204
00:11:42,600 --> 00:11:49,720
Internet stops screaming about it and I started downloading. But yeah,

205
00:11:49,879 --> 00:11:53,399
when obviously we don't want to test our applications in

206
00:11:53,480 --> 00:11:55,679
production because we're smarter than that, and we have the

207
00:11:55,679 --> 00:12:00,559
ability to test these things in different environments. When we

208
00:12:00,919 --> 00:12:04,080
are running things like unit tests, we're kind of stuck

209
00:12:04,279 --> 00:12:09,840
inside of this artificial context. Right. If you're just running

210
00:12:09,879 --> 00:12:15,120
code to test code, it's inside of that specific code environment.

211
00:12:15,200 --> 00:12:18,720
It's given us inputs and outputs. Even as we go

212
00:12:18,879 --> 00:12:21,399
up the chain to some things that like to call

213
00:12:21,480 --> 00:12:24,240
themselves and the end tests which I kind of disagree with,

214
00:12:24,279 --> 00:12:28,720
which would be things like dombased testing, you're still stuck

215
00:12:28,759 --> 00:12:30,279
inside some kind of context.

216
00:12:30,440 --> 00:12:30,639
Speaker 1: Right.

217
00:12:30,720 --> 00:12:33,480
Speaker 5: So if the DOM, if you're not familiar, it's the

218
00:12:33,559 --> 00:12:39,200
Document Object Model, and it's essentially the application in interface

219
00:12:39,240 --> 00:12:41,960
that we have with the browser. So it's how our

220
00:12:42,039 --> 00:12:45,879
JavaScript code talks to the browser, how we manipulate things,

221
00:12:45,919 --> 00:12:49,120
how we read things from the browser. And so the

222
00:12:49,159 --> 00:12:53,399
important nuance here is that our code interacts with the DOM.

223
00:12:53,799 --> 00:12:57,200
A human being doesn't interact with the DOM. Right when

224
00:12:57,200 --> 00:12:59,440
you go click a button, you don't go talk to

225
00:12:59,480 --> 00:13:03,879
the DOM, interact with the user phase user interface. So

226
00:13:04,120 --> 00:13:07,600
we want to get our tests as close to actual

227
00:13:07,679 --> 00:13:10,000
end to end tests as possible, right, A human looking

228
00:13:10,039 --> 00:13:13,879
at the screen, a human interacting with the screen. And

229
00:13:15,039 --> 00:13:18,720
if we're not in that production environment, every step we

230
00:13:18,799 --> 00:13:21,399
take away from that gets us further from reality, right,

231
00:13:21,440 --> 00:13:27,639
it gives us this false sense of security. Sometimes there's

232
00:13:28,200 --> 00:13:32,000
a really common example with with DOM based tools, might

233
00:13:32,080 --> 00:13:36,360
be click this button, did it work right? Well, just

234
00:13:36,399 --> 00:13:39,000
because you can't interact with that button through the DOM

235
00:13:39,480 --> 00:13:42,559
doesn't mean your user can actually interact with that button right.

236
00:13:43,039 --> 00:13:45,879
There might be you know, some kind of overlay over

237
00:13:45,960 --> 00:13:48,080
my button. The button might be off of the screen.

238
00:13:48,720 --> 00:13:50,360
But when I asked the DOM, can I click the button,

239
00:13:50,399 --> 00:13:53,200
it says, yeah, we clicked it. It worked. Ship the

240
00:13:53,240 --> 00:13:55,200
code and now no one can log into your AP

241
00:13:55,240 --> 00:13:58,120
because no one can click the button right, And so

242
00:13:58,399 --> 00:14:02,440
doing ours as much as we can to get to

243
00:14:02,519 --> 00:14:06,080
that real world scenario, creating testing and staging environments that

244
00:14:06,159 --> 00:14:10,039
mirror production as much as possible, and loading these things

245
00:14:10,120 --> 00:14:15,639
into virtual machines with operating systems instead of a headless browser,

246
00:14:15,679 --> 00:14:18,039
which is, you know, basically a browser with no UI,

247
00:14:18,480 --> 00:14:21,600
and interacting with it in a way that human doesn't

248
00:14:21,639 --> 00:14:24,840
interact with it just gets us further away from reality.

249
00:14:25,679 --> 00:14:28,320
Speaker 4: I mean, it's interesting you bring that up, and I'm

250
00:14:28,360 --> 00:14:30,720
sort of now I'm intrigued if you maybe want to

251
00:14:30,799 --> 00:14:34,320
roast all what we've been telling our customers. So obviously,

252
00:14:34,399 --> 00:14:37,399
where where we provide a third party product for our

253
00:14:37,440 --> 00:14:40,679
customers for a log in and access control, so they have,

254
00:14:41,240 --> 00:14:44,519
you know, providing them the off needs there, and I

255
00:14:44,559 --> 00:14:46,360
think the biggest advice that we end up giving them

256
00:14:46,440 --> 00:14:50,399
is like we are already testing that thing, Like, you know,

257
00:14:50,559 --> 00:14:54,440
don't focus on this. You're wasting your time duplicating our testing.

258
00:14:54,720 --> 00:14:57,360
If you felt the need to do that, it's almost

259
00:14:57,360 --> 00:15:00,320
like you don't trust us with our product, and then

260
00:15:00,399 --> 00:15:03,279
you probably should question why you're using that solution in

261
00:15:03,320 --> 00:15:06,120
the first place. If you get to that point, you know,

262
00:15:06,200 --> 00:15:10,320
that's actually a conversation more than it's a technical solution. However,

263
00:15:10,320 --> 00:15:12,600
we do find some customers still have a need to

264
00:15:12,639 --> 00:15:14,759
go a little bit further and the thing that we've done,

265
00:15:14,879 --> 00:15:17,240
I don't know this is the right answer, but we

266
00:15:17,320 --> 00:15:20,200
provide a given it is a SAS product, we provide

267
00:15:20,200 --> 00:15:23,000
a clone of our service as a container that can

268
00:15:23,080 --> 00:15:27,559
run that it is trimmed down, only has minor features,

269
00:15:27,600 --> 00:15:30,600
but allows the flows that you're going to test or

270
00:15:30,600 --> 00:15:35,000
you want to actually verify available without having to go

271
00:15:35,039 --> 00:15:37,360
through all the complexity that the service actually provides.

272
00:15:38,440 --> 00:15:40,240
Speaker 5: Yeah, I mean, I think that's a good compromise. Right.

273
00:15:40,279 --> 00:15:43,519
So in this whole strategy, of fending balance between our

274
00:15:44,120 --> 00:15:49,720
confidence and our velocity and shipping. The reality is our

275
00:15:49,759 --> 00:15:52,320
testing environs are not going to match our production environments

276
00:15:52,320 --> 00:15:55,200
all the time, right, and a lot of times we're

277
00:15:55,240 --> 00:16:01,159
constrained by resources. So I think in a situation like that,

278
00:16:01,159 --> 00:16:03,600
that totally makes sense. You know, some kind of pared

279
00:16:03,639 --> 00:16:07,519
down version of your production application. I think the important

280
00:16:07,519 --> 00:16:11,879
thing is how you're testing it, right. If we are

281
00:16:11,919 --> 00:16:14,080
able to just kind of like strip down our product

282
00:16:14,240 --> 00:16:18,399
and test the bare bones version of it, as long

283
00:16:18,440 --> 00:16:20,919
as we are in an environment, right, they're clicking on

284
00:16:20,960 --> 00:16:23,120
it through I don't know, a web browser or whatever

285
00:16:23,200 --> 00:16:25,879
it might be, versus just like you know, running some

286
00:16:25,960 --> 00:16:28,039
script in the background. I think that's a really good

287
00:16:28,080 --> 00:16:30,799
balance between those two things. The key here is that

288
00:16:30,799 --> 00:16:34,720
we're still doing and testing right. I imagine it's you

289
00:16:34,759 --> 00:16:37,840
know that someone's typing in the box, a button's being clicked,

290
00:16:38,039 --> 00:16:40,559
there's an HTTP request or whatever it might be to

291
00:16:40,639 --> 00:16:42,919
an API that reads from a database, and we're checking

292
00:16:42,919 --> 00:16:46,559
out all of these things actually work together. So yeah,

293
00:16:46,720 --> 00:16:48,279
I think that's that's a good compromise.

294
00:16:49,720 --> 00:16:51,240
Speaker 4: I think one of the things that actually comes up

295
00:16:51,279 --> 00:16:54,600
a lot is maybe just on a slight tangent is

296
00:16:54,639 --> 00:16:58,200
people are so focused on and testing they never stop

297
00:16:58,320 --> 00:17:02,399
to question should we like for that particular flow, is

298
00:17:02,840 --> 00:17:05,440
that where the value is for our company? Is it

299
00:17:05,799 --> 00:17:07,880
really where we should put a lot of resources? And

300
00:17:07,920 --> 00:17:11,559
do find that those that you're working with or your

301
00:17:11,559 --> 00:17:15,519
customers may or may not know where the highest value

302
00:17:15,680 --> 00:17:19,240
testing should be done, And then that's a conversation or

303
00:17:19,279 --> 00:17:19,839
maybe it's.

304
00:17:19,680 --> 00:17:24,240
Speaker 5: Something that your tool provides. Yeah. Absolutely, And I think

305
00:17:24,319 --> 00:17:27,279
the trick between the trick for that is doing it

306
00:17:27,279 --> 00:17:32,599
early right. So, if you have a large application in

307
00:17:32,640 --> 00:17:34,480
a code base and you have not written any end

308
00:17:34,599 --> 00:17:38,039
n tests, it's hard to determine where to start, right

309
00:17:38,440 --> 00:17:42,079
versus when if you start early on, it kind of

310
00:17:42,079 --> 00:17:44,599
writes itself. Right. The first day is your login, you

311
00:17:44,640 --> 00:17:50,119
have some login coverage. Determining where the highest value is

312
00:17:50,119 --> 00:17:53,720
is certainly up to each usually the product team, Right,

313
00:17:53,759 --> 00:17:57,160
what do we care most about not breaking? And can

314
00:17:57,200 --> 00:18:00,400
we create some kind of smoke test that spans all

315
00:18:00,440 --> 00:18:03,359
of these right, So each one of these tests has

316
00:18:03,400 --> 00:18:08,039
a certain level of granularity to it. A good smoke

317
00:18:08,079 --> 00:18:10,799
test might be can I log into my application? Can

318
00:18:10,839 --> 00:18:13,759
I create a thing? Can I delete a thing? And

319
00:18:13,839 --> 00:18:18,359
things just like generally work. Those initially are your highest

320
00:18:18,440 --> 00:18:22,119
value tests because I know that my app actually loads

321
00:18:22,200 --> 00:18:24,960
in reality, right, regardless of what my unit tests say.

322
00:18:25,880 --> 00:18:30,559
After that, it's usually defined by what those user flows are. Right. So,

323
00:18:30,720 --> 00:18:33,480
as you're scoping something out with your product team, here's

324
00:18:33,519 --> 00:18:37,039
this new feature that we're building, it's really important in

325
00:18:37,119 --> 00:18:42,000
your planning process to include that right right tests for it.

326
00:18:42,640 --> 00:18:47,400
These are things that we usually as developers, kind of

327
00:18:47,440 --> 00:18:49,720
bake into our estimates, Right, I have the right unit

328
00:18:49,759 --> 00:18:53,839
tests for this. At Rainforest, we've shifted more towards baking

329
00:18:53,880 --> 00:18:56,880
and Rainforest tests for these things. We obviously have unit tests,

330
00:18:57,839 --> 00:19:01,480
But getting the coverage at the time of implementation or

331
00:19:01,519 --> 00:19:04,160
the time of release or whatever that might be, is

332
00:19:04,279 --> 00:19:07,359
usually your best bet to get that. If I have

333
00:19:07,480 --> 00:19:11,680
a large, large application and I don't have that coverage yet,

334
00:19:12,119 --> 00:19:15,400
it is certainly a balancing act figuring out what should

335
00:19:15,440 --> 00:19:18,079
be test first, Right, So I would certainly start with

336
00:19:18,119 --> 00:19:22,079
those kind of smoke tests, and then your highest used

337
00:19:22,319 --> 00:19:26,519
features is usually a really good place to start. The

338
00:19:26,559 --> 00:19:29,559
pitfall that you go into is putting too much nuance

339
00:19:29,599 --> 00:19:31,559
in a lot of these tests, right, What if they

340
00:19:31,559 --> 00:19:33,599
click into this and click out of that and then

341
00:19:33,680 --> 00:19:38,000
open this menu and whatever. Keeping them very coherent and

342
00:19:38,279 --> 00:19:40,920
legible and kind of focused on the thing that they're

343
00:19:40,960 --> 00:19:45,440
testing is the important piece of having efficient tests that

344
00:19:45,480 --> 00:19:46,599
you can maintain over time.

345
00:19:46,920 --> 00:19:49,440
Speaker 4: I saw Will smarting there, and I know he's just

346
00:19:49,599 --> 00:19:52,880
entered into a new, glorious position at his organization, so

347
00:19:52,880 --> 00:19:56,559
maybe he has some unique insight that he's interested in

348
00:19:57,039 --> 00:19:58,759
blessing else with No.

349
00:19:59,000 --> 00:20:02,160
Speaker 1: I was curious because this is an opportunity for me

350
00:20:02,200 --> 00:20:05,920
to throw in a buzzword that's trending, and so once

351
00:20:06,000 --> 00:20:09,599
the episode is transcribed, we'll just go viral on that.

352
00:20:09,720 --> 00:20:12,079
Speaker 2: So does AI play a role in.

353
00:20:12,079 --> 00:20:15,960
Speaker 1: Helping figure out that type of user flow? In the

354
00:20:16,000 --> 00:20:19,039
different like odd places you can end up.

355
00:20:20,400 --> 00:20:25,039
Speaker 5: Good question? That's my knowledge. Yet, you know, as really

356
00:20:25,039 --> 00:20:27,400
good at some things and really bad at some things,

357
00:20:27,440 --> 00:20:31,359
and we haven't quite figured out how to make it

358
00:20:31,720 --> 00:20:34,640
give it enough context to understand how it should go

359
00:20:34,640 --> 00:20:38,559
about testing your app. Right, we do have some really

360
00:20:38,599 --> 00:20:44,160
cool AI tools that rain for us. They don't determine

361
00:20:44,200 --> 00:20:46,960
what your test coverage should be. Rather that's kind of

362
00:20:47,039 --> 00:20:49,640
left up to you and then it helps you write

363
00:20:49,680 --> 00:20:53,359
the test. So what we have is entering a prompt,

364
00:20:53,440 --> 00:20:56,400
you know, say, it could be something pretty generic. Log

365
00:20:56,480 --> 00:20:58,960
in and Adam add an item to the cart and

366
00:20:59,039 --> 00:21:01,680
check out something like that, and it will generate your

367
00:21:01,759 --> 00:21:06,759
reinforce steps for you. So during execution, AI is left

368
00:21:06,759 --> 00:21:08,720
out of it. Right, It does the initial generation and

369
00:21:08,720 --> 00:21:12,680
then we just execute things normally and then we have

370
00:21:12,759 --> 00:21:16,640
some self healing functionality. So it fails on something that

371
00:21:16,680 --> 00:21:19,480
we generated, we're going to try and regenerate those steps.

372
00:21:20,119 --> 00:21:24,640
And what's really nice about that is since Rainforest is

373
00:21:24,720 --> 00:21:30,319
a visual tool, we identify things on the screen based

374
00:21:30,359 --> 00:21:34,920
on screenshots. Right, it's possible for you to make slight

375
00:21:35,039 --> 00:21:38,200
visual changes, and now that image doesn't quite match up,

376
00:21:38,279 --> 00:21:40,359
your test might fail. You don't want to have to

377
00:21:40,359 --> 00:21:43,599
go back in and retake all of their screenshots. But

378
00:21:43,880 --> 00:21:46,799
since it's generated by AI, it could go back follow

379
00:21:46,799 --> 00:21:49,119
the same steps and realize this is what the button

380
00:21:49,200 --> 00:21:52,240
is here. It would be really cool if it could

381
00:21:52,960 --> 00:21:54,839
kind of add that test coverage for you, or like

382
00:21:54,920 --> 00:21:58,279
tell you what you should be testing. We've poked at

383
00:21:58,319 --> 00:22:01,240
that a few times and it's obviously just really dumb

384
00:22:01,279 --> 00:22:03,920
in that aspect and doesn't really give you anything useful.

385
00:22:04,559 --> 00:22:06,720
It's like, yeah, go go test all the things and

386
00:22:06,759 --> 00:22:09,119
make sure things work and it's like cool, yeah, I

387
00:22:09,279 --> 00:22:17,440
knew that, thank you, Yes they get smart. I feel like.

388
00:22:17,519 --> 00:22:20,279
Speaker 4: Part of the answer is also the domain you're in.

389
00:22:20,640 --> 00:22:22,640
I know, something we haven't talked about is like really,

390
00:22:22,640 --> 00:22:25,319
at the top of the test pyramid is exploratory testing,

391
00:22:25,640 --> 00:22:30,480
whereas like add your creative human instincts to where bugs

392
00:22:30,480 --> 00:22:33,119
could potentially pop up while you're looking at an interface

393
00:22:33,279 --> 00:22:37,440
or API. And I don't think we're doing anything wrong

394
00:22:37,599 --> 00:22:40,359
in the creation of AI and LM models. It's removing

395
00:22:40,359 --> 00:22:44,160
the creativity from them, and I think that that harms

396
00:22:44,240 --> 00:22:47,319
us here. But there has been one area, especially within

397
00:22:47,680 --> 00:22:53,759
things like protocol creation or SDKs interfaces for the services,

398
00:22:54,359 --> 00:22:57,799
and that's I think the keyword is fuzzing. So trying

399
00:22:57,960 --> 00:23:01,640
an LM, any sort of AI can spam with almost

400
00:23:01,640 --> 00:23:04,920
a more intelligent root for strategy about what sorts of

401
00:23:04,920 --> 00:23:08,920
inputs tend to break your interface or your service or

402
00:23:08,920 --> 00:23:12,640
your product and then use that as a potential test

403
00:23:13,119 --> 00:23:15,519
that you can commit longer term. And again it's not

404
00:23:15,559 --> 00:23:17,400
for everything, like I don't think it really works so

405
00:23:17,519 --> 00:23:21,119
much in a UI world, but definitely depending on what

406
00:23:21,119 --> 00:23:23,319
your service or interface is doing. Stuff in the crypto

407
00:23:23,400 --> 00:23:26,519
space cryptography, not blockchain.

408
00:23:28,240 --> 00:23:31,319
Speaker 1: Just to be clear, I feel like that was a

409
00:23:31,359 --> 00:23:33,119
dig there, Warren, What are you trying to say?

410
00:23:34,880 --> 00:23:37,000
Speaker 4: You know, it's not the sort of thing I want

411
00:23:37,039 --> 00:23:38,640
to bring up on an episode.

412
00:23:38,640 --> 00:23:42,480
Speaker 3: Well, you don't want a record.

413
00:23:42,559 --> 00:23:47,000
Speaker 4: Huh yeah, definitely not on a record. We do cryptography

414
00:23:47,599 --> 00:23:49,759
because we're really into security and deep there, and we

415
00:23:49,799 --> 00:23:52,640
do not building our own crypto, but hi, we're very

416
00:23:52,680 --> 00:23:57,599
high users of it. Everything JAWT creation or jot creation,

417
00:23:58,400 --> 00:24:01,160
every single different kind of algorithm strategy we end up

418
00:24:01,200 --> 00:24:04,319
utilizing these, and so finding where we're not using libraries

419
00:24:04,319 --> 00:24:09,759
effectively is certainly an area that we've potentially looked into. Actually,

420
00:24:09,799 --> 00:24:12,119
according to our company by laws, we're not allowed to

421
00:24:12,119 --> 00:24:16,799
do anything regarding cryptocurrency, Like it's actually not allowed by

422
00:24:16,799 --> 00:24:18,720
the country of Switzerland for us to get involved in

423
00:24:18,720 --> 00:24:21,119
any way. We can't accept payments, we can't pay people

424
00:24:21,119 --> 00:24:25,240
in crypto. We can't even think about consulting for companies

425
00:24:25,279 --> 00:24:28,720
that I want to do something crypto related.

426
00:24:29,400 --> 00:24:30,319
Speaker 2: That's discrimination.

427
00:24:34,000 --> 00:24:36,160
Speaker 3: I work at HPC and I'm pretty sure like some

428
00:24:36,240 --> 00:24:38,279
of the admins will just kind of use like a

429
00:24:38,279 --> 00:24:40,720
little bit of the compute power from different clusters that

430
00:24:40,759 --> 00:24:44,599
they have to be running different crypto schemes, but I haven't.

431
00:24:44,680 --> 00:24:46,759
I haven't like one hundred percent caught anybody. But I'm

432
00:24:46,759 --> 00:24:48,799
just I'm waiting for the day. I'm waiting for it.

433
00:24:49,640 --> 00:24:50,920
Not to tell on them, I just want to know

434
00:24:50,960 --> 00:24:52,920
because I'm super nosy and like I just like knowing

435
00:24:52,960 --> 00:24:53,519
things like this.

436
00:24:53,839 --> 00:24:55,279
Speaker 2: No, I'm going to tell on as long as they

437
00:24:55,359 --> 00:24:56,720
cut you in. Yeah.

438
00:24:58,319 --> 00:25:00,160
Speaker 3: It's like when my you know, when my website not

439
00:25:00,240 --> 00:25:03,440
hacked by that Chinese jewelry store and I was like, guys, like,

440
00:25:03,519 --> 00:25:05,200
if you if you would just give me a cut,

441
00:25:05,240 --> 00:25:06,920
this would be fine. It was nice.

442
00:25:08,119 --> 00:25:10,039
Speaker 4: I mean I think that's I think that really is

443
00:25:10,039 --> 00:25:14,680
expert advice from our resident mL expert here, because.

444
00:25:16,880 --> 00:25:17,559
Speaker 3: Like just making.

445
00:25:19,519 --> 00:25:21,720
Speaker 4: Yeah no, because AS just came out and said that

446
00:25:21,920 --> 00:25:27,400
the strategy of sharing reservations across customer a WOS accounts,

447
00:25:27,400 --> 00:25:30,880
Like if you're a consultant that does bundling, for instance,

448
00:25:30,960 --> 00:25:34,200
reservations or compute reservations, you no longer can pass along

449
00:25:34,240 --> 00:25:36,559
that savings to the customer. I mean, what are they

450
00:25:36,559 --> 00:25:38,720
going to do with all this excess capacity now other

451
00:25:38,839 --> 00:25:41,799
than some good old fashioned bitcoin mining.

452
00:25:43,119 --> 00:25:45,920
Speaker 3: I don't know, I don't know. I mean we could

453
00:25:45,920 --> 00:25:49,799
be making drugs for autoimmune diseases and cancer, or or

454
00:25:50,160 --> 00:25:52,400
you could be making some call part cash. I don't know.

455
00:25:54,000 --> 00:25:54,400
Speaker 1: It all.

456
00:26:00,160 --> 00:26:02,920
Speaker 3: Such both it's not in either. Or there's plenty of

457
00:26:02,960 --> 00:26:06,519
compute power these guys have. You know, they're spinning up

458
00:26:06,519 --> 00:26:08,880
plenty of a W Westerns. They're not going to notice

459
00:26:09,440 --> 00:26:12,319
if that last ten is using crypto.

460
00:26:14,200 --> 00:26:17,279
Speaker 1: So when this episode launches and we all get blocked

461
00:26:17,279 --> 00:26:20,599
from a respective AWS accounts, we can just reflect on

462
00:26:20,640 --> 00:26:21,640
this moment fondly.

463
00:26:22,720 --> 00:26:27,160
Speaker 4: So, I mean, for the record, AWS is going to

464
00:26:27,200 --> 00:26:32,519
block you because the ROI on utilizing cloud resources to

465
00:26:32,640 --> 00:26:35,960
mind crypto is so low that you're pretty much just

466
00:26:36,319 --> 00:26:40,160
paying AWS. But it is a good indication that there

467
00:26:40,240 --> 00:26:42,880
is malicious activity happening on your account. So it is

468
00:26:42,880 --> 00:26:46,039
something that they will for sure investigate, and that I

469
00:26:46,119 --> 00:26:48,720
think is as much as a tangent on this that

470
00:26:48,839 --> 00:26:51,400
I want to go down for today.

471
00:26:51,920 --> 00:26:53,839
Speaker 3: Right, Yeah, I think we should talk about the low

472
00:26:53,880 --> 00:26:56,880
code with Rainforest. I love low code stuff. How did

473
00:26:56,920 --> 00:27:00,440
how did this come about? And like how does it work?

474
00:27:00,480 --> 00:27:01,799
I just I want to know all about it.

475
00:27:02,920 --> 00:27:07,079
Speaker 5: Yeah, for sure. So when I first joined Rainforest over

476
00:27:07,119 --> 00:27:09,640
seven years ago, our model is a bit different. We

477
00:27:09,680 --> 00:27:12,240
had a bunch of human testers. It was kind of

478
00:27:12,240 --> 00:27:15,279
the gig uber model of I have something I want

479
00:27:15,319 --> 00:27:17,799
to test. Here's my test cases. They're all written in

480
00:27:17,799 --> 00:27:21,000
plain English, and we'll provide a bunch of humans for

481
00:27:21,039 --> 00:27:26,319
you to go test your application, right, including some exploratory

482
00:27:26,319 --> 00:27:28,720
stuff like you mentioned go click all over this page

483
00:27:28,960 --> 00:27:32,200
and try and find problems with it. And that worked

484
00:27:32,240 --> 00:27:36,359
really well. It was true and the end testing, we

485
00:27:36,440 --> 00:27:40,720
load your app in a in a virtual machine inside

486
00:27:40,720 --> 00:27:43,640
of a web browser. They're actually clicking the buttons and

487
00:27:43,680 --> 00:27:46,720
confirming there's things on the screen. But what we found

488
00:27:46,839 --> 00:27:50,240
is that humans are inefficient and expensive, as we all know.

489
00:27:50,279 --> 00:27:53,880
That's why we have automation, right, and so we kind

490
00:27:53,880 --> 00:27:56,920
of shifted over to automation, but we wanted to do

491
00:27:56,960 --> 00:27:59,599
something a bit different from what everyone else was doing,

492
00:27:59,640 --> 00:28:04,680
which is these code based tools uh dom based interactions,

493
00:28:05,079 --> 00:28:07,680
and instead we built it all on the visual layer.

494
00:28:08,759 --> 00:28:12,440
So the way it works is you go in, you

495
00:28:12,480 --> 00:28:16,119
load your app and essentially, just like tix stree shots

496
00:28:16,119 --> 00:28:20,039
of things, right click on this type into this field.

497
00:28:21,039 --> 00:28:24,039
I can give it an ai an AI prompt and say,

498
00:28:24,079 --> 00:28:26,559
you know, log in and check check out in the

499
00:28:26,640 --> 00:28:31,480
cart and then when you execute things, it loads into

500
00:28:31,480 --> 00:28:34,480
the same environment. Right, you have your your staging environment.

501
00:28:34,480 --> 00:28:39,920
Hopefully you have some seed data with login information. You

502
00:28:39,920 --> 00:28:42,319
can load that all in the rainforest. It goes in

503
00:28:42,400 --> 00:28:46,559
and runs this whole workflow for you. The output of

504
00:28:46,599 --> 00:28:50,160
it is a video of the thing being tested. Results

505
00:28:50,279 --> 00:28:55,880
on each step, things like HTTP logs, jobscript, console logs,

506
00:28:56,160 --> 00:28:59,279
all the information that you need to actually debug things

507
00:28:59,319 --> 00:29:01,559
when something instead of it just saying you know, like

508
00:29:01,599 --> 00:29:03,799
in a unit test and it's like failure like one

509
00:29:03,880 --> 00:29:08,559
does not equal to And so by doing things at

510
00:29:08,559 --> 00:29:12,200
that visual level, it offers a lot of flexibility. The

511
00:29:12,240 --> 00:29:15,240
first thing is that we're out stuck inside of the browser. Right,

512
00:29:16,400 --> 00:29:20,599
we do primarily focus on web based testing, but that

513
00:29:20,640 --> 00:29:23,960
does not mean you're stuck inside of the browser. It

514
00:29:24,079 --> 00:29:27,240
means you can do things like install a Chrome extension. Right,

515
00:29:27,319 --> 00:29:30,559
open another tab in your browser, install a Chrome extension,

516
00:29:30,799 --> 00:29:34,599
interact with that extension. Because while you're still inside the browser,

517
00:29:34,680 --> 00:29:37,400
you're outside of the scope of that web page where

518
00:29:37,440 --> 00:29:42,799
you usually are interacting just through the domin you can

519
00:29:43,400 --> 00:29:46,839
you know, install some type of desktop application and test

520
00:29:46,880 --> 00:29:50,039
it through there. Because since we're working at the visual layer,

521
00:29:50,119 --> 00:29:53,000
it doesn't care what you're testing. It doesn't care what

522
00:29:53,039 --> 00:29:55,839
your tech stack is. It just cares that it loads

523
00:29:55,880 --> 00:30:00,240
in the machine. And it also offers a lot of

524
00:30:01,599 --> 00:30:07,599
like more flexible and robust in avoiding flakiness and brittleness

525
00:30:07,599 --> 00:30:11,440
to small changes. We have fallback methods. As much as

526
00:30:11,480 --> 00:30:14,880
I've been kind of hammering that testing with the DOM

527
00:30:15,000 --> 00:30:17,480
is not a great idea, we do offer DOM fallbacks

528
00:30:18,119 --> 00:30:20,559
because sometimes it makes sense. Sometimes I don't care about

529
00:30:20,559 --> 00:30:22,920
the visual appearance of the button, and all I care

530
00:30:23,000 --> 00:30:28,519
about is that there's a button there. Right. In reality,

531
00:30:28,720 --> 00:30:32,359
there are variables that we can't control. Right. A very

532
00:30:32,400 --> 00:30:36,920
common scenario is my marketing team is running experiments. Every

533
00:30:36,920 --> 00:30:39,440
time I load this page, the button says something different,

534
00:30:39,480 --> 00:30:42,240
it looks different, and so we don't want to tie

535
00:30:42,640 --> 00:30:48,480
the visual appearance to the past fail result of this test.

536
00:30:48,559 --> 00:30:50,920
So I'll use something else. I'll use a DOM selector.

537
00:30:51,240 --> 00:30:54,279
We also have like AI search. You could say something

538
00:30:54,359 --> 00:30:56,359
like the button, the log and button at the bottom

539
00:30:56,400 --> 00:31:00,200
of the page. And so the important point here is

540
00:31:00,240 --> 00:31:03,440
you don't write any code whatsoever. We have an intuitive

541
00:31:03,519 --> 00:31:06,039
UI that you do all of this through, which means

542
00:31:06,079 --> 00:31:08,359
you don't need skilled engineers to do it right. A

543
00:31:08,359 --> 00:31:12,240
lot of teams have QA engineers that their job is

544
00:31:12,279 --> 00:31:17,079
to just write tests all the time. Other teams, you

545
00:31:17,319 --> 00:31:20,559
are read the engineer is responsible for writing these tests,

546
00:31:20,680 --> 00:31:24,480
but they need very specific domain knowledge, right. I need

547
00:31:24,519 --> 00:31:28,240
to know about the thing that I'm testing, like from

548
00:31:28,279 --> 00:31:30,519
a product standpoint, what does this thing do? I need

549
00:31:30,559 --> 00:31:33,119
to understand the code. I need to know how to

550
00:31:33,160 --> 00:31:37,319
write these tests with a no code solution. Anybody can

551
00:31:37,440 --> 00:31:42,440
do this, right. It's up to your team who owns quality.

552
00:31:42,440 --> 00:31:46,839
Who owns these tests. For us, it is usually the

553
00:31:47,279 --> 00:31:50,359
engineer that is shipping the code we write the reinforest

554
00:31:50,359 --> 00:31:53,119
tests with it, or our product and design team owns

555
00:31:53,119 --> 00:31:55,599
it because, like I was saying, they're very tied to

556
00:31:55,640 --> 00:31:57,839
our user workflows, right, They're like, this is how we

557
00:31:57,920 --> 00:32:01,200
design this thing. Engineers are going to build it, and

558
00:32:01,240 --> 00:32:04,200
then any of us that have knowledge about how this

559
00:32:04,319 --> 00:32:07,839
userflow is supposed to work can own this test. So

560
00:32:08,000 --> 00:32:11,480
it makes it much easier to both write and maintain

561
00:32:11,559 --> 00:32:14,079
your tests over time. And then you know, if that

562
00:32:14,119 --> 00:32:16,759
person leaves your company and has all of that domain

563
00:32:16,799 --> 00:32:18,960
knowledge and how it works, you just need someone who

564
00:32:19,000 --> 00:32:20,880
knows how the app works and they can update your

565
00:32:20,880 --> 00:32:21,680
test suite.

566
00:32:21,880 --> 00:32:24,240
Speaker 4: Now, I feel like one of the biggest mistakes that

567
00:32:24,279 --> 00:32:27,279
I keep seeing over the course of my career is

568
00:32:27,400 --> 00:32:32,079
as companies grow, they tend to have more, allegedly more software,

569
00:32:32,240 --> 00:32:35,119
more code, which may or may not end up in

570
00:32:35,119 --> 00:32:39,400
a giant ball of mud mass or even an extensive

571
00:32:39,480 --> 00:32:44,440
number of quote unquote micro services that communicate and really

572
00:32:44,480 --> 00:32:48,640
depend on each other. And there was always this challenge

573
00:32:48,880 --> 00:32:52,920
by someone who wanted to have a test that required

574
00:32:53,119 --> 00:32:57,680
somehow interacting with all of these components, and they never

575
00:32:57,680 --> 00:33:00,200
really could understand that one of the whole point so

576
00:33:00,359 --> 00:33:03,880
micro services was to isolate testing. But I think we

577
00:33:03,920 --> 00:33:06,599
live in the reality, which is there are some companies

578
00:33:06,640 --> 00:33:09,079
that do have a giant ball of mud that have

579
00:33:09,559 --> 00:33:12,880
thousands of binaries that have to be installed on running servers.

580
00:33:13,200 --> 00:33:15,920
Is there a strategy that I don't I don't mean to,

581
00:33:16,200 --> 00:33:17,200
you know, pick on your company.

582
00:33:17,240 --> 00:33:17,559
Speaker 5: I don't know.

583
00:33:17,599 --> 00:33:19,119
Speaker 4: If I don't think there is a strategy. I think

584
00:33:19,160 --> 00:33:22,160
the strategy is right micro services. But I can imagine that.

585
00:33:22,519 --> 00:33:24,960
You know, as a SaaS company, the last thing we

586
00:33:24,960 --> 00:33:28,200
want to tell our customers is, yeah, have you tried

587
00:33:28,240 --> 00:33:29,200
not having that problem?

588
00:33:31,920 --> 00:33:34,400
Speaker 3: Is there a package manager that tends to do this

589
00:33:34,440 --> 00:33:37,079
solution that I see that what is everybody's favorite.

590
00:33:37,559 --> 00:33:41,599
Speaker 4: Yeah, distributed monolith always works. Publish all your binaries that

591
00:33:41,759 --> 00:33:44,079
remotely depend on each other to a third party solution

592
00:33:44,279 --> 00:33:47,480
and then pull those out at run time always works.

593
00:33:47,599 --> 00:33:51,240
Best solution ever. Maybe Aja, you have some insight here

594
00:33:51,319 --> 00:33:54,039
on either something that works or something that works with

595
00:33:54,119 --> 00:33:56,799
Rainforest QA to deal with those situations, or maybe you

596
00:33:56,880 --> 00:33:58,799
just you know, it's not something that is handled today.

597
00:33:59,599 --> 00:34:01,759
Speaker 5: Yeah. Sure, we do have a bunch of different micro

598
00:34:01,839 --> 00:34:05,359
services running, and I think I'm going to refer back

599
00:34:05,359 --> 00:34:08,599
to the testing pyramid. Right, is we test each one

600
00:34:08,599 --> 00:34:12,639
of those micro services in isolation. Absolutely, Maybe we test

601
00:34:13,239 --> 00:34:14,559
as we go up to the next layer of our

602
00:34:14,599 --> 00:34:18,800
integration test, we test some interactions between them, right, the

603
00:34:18,920 --> 00:34:24,440
kind of core you know, handshake interactions, whatever is the

604
00:34:25,159 --> 00:34:28,159
main functionality of these two microservices talking to each other.

605
00:34:28,199 --> 00:34:30,639
Maybe we have some tests there, But at the end

606
00:34:30,679 --> 00:34:34,199
of the day, these end to end tests are comprehensive. Right,

607
00:34:34,920 --> 00:34:38,719
If anything in that micro service architecture is failing, presumably

608
00:34:38,800 --> 00:34:41,599
my test is going to fail. And at the end

609
00:34:41,639 --> 00:34:44,280
of the day, all we really care about in theory

610
00:34:44,519 --> 00:34:47,199
is what the user gets when they're interacting with it.

611
00:34:47,840 --> 00:34:50,360
So if I'm just clicking a button, maybe there's a

612
00:34:50,440 --> 00:34:54,000
thousand micro services that are involved in this, and maybe

613
00:34:54,000 --> 00:34:57,000
I'm not directly testing each one of those, but by

614
00:34:57,039 --> 00:34:59,920
implementing it as an end to end test, I am

615
00:35:00,079 --> 00:35:02,559
very confident that they're all working because my test passed.

616
00:35:03,519 --> 00:35:06,480
And so being smart about how you implement each one

617
00:35:06,519 --> 00:35:10,360
of those layers in an efficient way. Right, lots of

618
00:35:10,519 --> 00:35:14,360
unit tests on each micro service and then this overarching

619
00:35:14,400 --> 00:35:17,679
test that just make sure everything is working together is

620
00:35:17,760 --> 00:35:19,280
usually a way to go about this.

621
00:35:19,679 --> 00:35:22,440
Speaker 4: I mean, maybe it's a technical implementation question, like where

622
00:35:22,480 --> 00:35:25,760
is the environment running that the rainforest tests are actually

623
00:35:25,760 --> 00:35:28,679
executing us? Is this some sort of binary or CLI

624
00:35:28,760 --> 00:35:32,519
that's run on the client side, or are they sharing

625
00:35:32,599 --> 00:35:36,400
with you a set of micro services with deployment instruction

626
00:35:36,559 --> 00:35:38,840
so that you can run them within your own infrastructure.

627
00:35:39,679 --> 00:35:43,440
Speaker 5: Sure, Usually our requirement is that you need to be

628
00:35:43,440 --> 00:35:46,280
able to access it via a web URL. So the

629
00:35:46,639 --> 00:35:49,800
kind of standard way that a Rainforests is run is

630
00:35:49,840 --> 00:35:53,239
we provide you a VM, and that VM has a

631
00:35:53,239 --> 00:35:56,639
browser on it, so I can specify Chrome on Windows

632
00:35:56,679 --> 00:36:00,400
eleven and the first step of your test is going

633
00:36:00,480 --> 00:36:04,199
to be a navigation navigates this URL. This is where

634
00:36:04,239 --> 00:36:08,599
my web app lives. There are some different use cases

635
00:36:08,599 --> 00:36:11,800
where you can absolutely go download a Binari and install

636
00:36:11,880 --> 00:36:14,440
it and do whatever you want with it or out

637
00:36:14,440 --> 00:36:19,079
of the box functionality is to primarily test those web apps,

638
00:36:19,840 --> 00:36:22,440
so that's where we focus. But you certainly have the

639
00:36:22,480 --> 00:36:25,000
flexibility to do whatever you want with those vms.

640
00:36:25,280 --> 00:36:27,719
Speaker 4: No, I mean, I think I think that approaches genius. Basically,

641
00:36:27,719 --> 00:36:29,960
it's out of scope for setting up the environment unless

642
00:36:30,000 --> 00:36:31,880
you want it to be in scope of which you know.

643
00:36:32,039 --> 00:36:35,280
Then it's a it's a virtual machine. Go to town

644
00:36:35,440 --> 00:36:37,119
on how they want to deal with it there.

645
00:36:37,719 --> 00:36:41,360
Speaker 5: Right, and by kind of forcing people to give it

646
00:36:41,360 --> 00:36:46,599
a public URL, we're nudging them towards good practices. Right

647
00:36:46,639 --> 00:36:49,400
for sure is set up a staging environment a QA

648
00:36:49,559 --> 00:36:52,840
environment and make it mirror production as much as possible,

649
00:36:52,920 --> 00:36:55,719
which includes being able to navigate to it in a URL.

650
00:36:56,599 --> 00:36:58,920
And these are small things that we see with some

651
00:36:58,960 --> 00:37:00,800
of our new clients are like, well, I don't have

652
00:37:00,840 --> 00:37:04,840
a staging environment, and sure, I guess you could load

653
00:37:04,840 --> 00:37:08,039
your production environment in there, but let's show you how

654
00:37:08,079 --> 00:37:10,800
to test this properly and not shoot yourself on the foot.

655
00:37:11,360 --> 00:37:13,559
Speaker 4: I'm like, I love that you're saying that. The number

656
00:37:13,559 --> 00:37:16,000
one feedback I've always seen here is we can't expose

657
00:37:16,000 --> 00:37:19,599
our non production environment publicly, Like people can't know what

658
00:37:19,599 --> 00:37:23,159
we're currently working on. They will use that information maliciously

659
00:37:23,199 --> 00:37:24,760
against our company in some way.

660
00:37:26,239 --> 00:37:27,679
Speaker 5: Yeah, like what are they going to do with it?

661
00:37:27,719 --> 00:37:27,920
Speaker 3: Though?

662
00:37:27,960 --> 00:37:33,000
Speaker 5: You know, like if I have seen some interesting mistakes,

663
00:37:33,039 --> 00:37:35,880
like you know, maybe we're cloning our production database and

664
00:37:35,960 --> 00:37:40,719
not sanitizing sensitive information from it or something like that. Yes,

665
00:37:40,760 --> 00:37:44,360
absolutely you're doing some bad things, but there are certainly

666
00:37:44,400 --> 00:37:48,280
ways to do this. And I'm of the opinion who

667
00:37:48,320 --> 00:37:50,760
cares if people are in your testing environment, Like worst

668
00:37:50,760 --> 00:37:55,000
case they blow up your testing environment and whatever.

669
00:37:55,280 --> 00:37:58,039
Speaker 1: And in that case you figured out how they could

670
00:37:58,039 --> 00:37:59,880
blow up your production environment.

671
00:37:59,519 --> 00:38:02,159
Speaker 5: Without losing pride exactly.

672
00:38:02,679 --> 00:38:04,920
Speaker 2: Yeah, I think there's.

673
00:38:07,039 --> 00:38:11,639
Speaker 1: That's, you know, like part of the undocumented learning curve

674
00:38:11,800 --> 00:38:15,480
of working in this industry, you know, because people who

675
00:38:15,480 --> 00:38:18,559
are early in their careers think things like, oh, I

676
00:38:18,559 --> 00:38:23,480
shouldn't expose staging until you know, they learned that that's

677
00:38:23,519 --> 00:38:25,239
actually probably a good thing.

678
00:38:25,320 --> 00:38:27,280
Speaker 2: But like, nowhere in any.

679
00:38:28,519 --> 00:38:32,599
Speaker 1: Computer science or course or or boot camp or anything

680
00:38:33,000 --> 00:38:34,920
do they cover these kinds of things, and so I

681
00:38:34,920 --> 00:38:39,960
think that's actually like a really valuable add on service

682
00:38:40,519 --> 00:38:43,280
that you know, you get from rain forest, or that

683
00:38:43,360 --> 00:38:46,639
you get from working with people who are more experienced,

684
00:38:46,719 --> 00:38:49,880
is just like learning that tribal knowledge that's going to

685
00:38:49,920 --> 00:38:52,159
help you out later in your career, so you don't

686
00:38:52,159 --> 00:38:55,119
have to reinvent the wheel and solve problems that we

687
00:38:55,159 --> 00:38:56,559
actually solved thirty years ago.

688
00:38:57,039 --> 00:38:58,360
Speaker 2: I mean, that's will.

689
00:38:58,480 --> 00:39:01,480
Speaker 4: I mean we've unfortunately had to append our documentation with

690
00:39:01,599 --> 00:39:05,480
like here explicitly are the sensitive pieces of data that

691
00:39:05,519 --> 00:39:08,760
are relevant to our third party application. This is sensitive.

692
00:39:08,760 --> 00:39:11,599
This is sensitive, Like this is not sensitive. This is

693
00:39:11,639 --> 00:39:13,880
like the application I need not sensitive. Like, do not

694
00:39:13,920 --> 00:39:15,880
try to encrypt this, Do not try to secure it,

695
00:39:16,599 --> 00:39:18,400
because people will try like how do I do this?

696
00:39:18,440 --> 00:39:21,239
I'm like, you can't stop it, Like this has to

697
00:39:21,280 --> 00:39:24,719
be public on your website, in your application. People have

698
00:39:24,760 --> 00:39:26,199
to be able to see it. You're not going to

699
00:39:26,239 --> 00:39:28,400
be able to get around that. And I feel like

700
00:39:28,920 --> 00:39:31,639
it's more than just experience. I feel like there's a

701
00:39:31,719 --> 00:39:35,400
whole level of pragmatism there, like weighing the cost versus

702
00:39:35,480 --> 00:39:38,599
the reward of actually trying to sanitize that piece of information.

703
00:39:39,159 --> 00:39:41,519
And having a third party testing service, as you mentioned,

704
00:39:42,440 --> 00:39:44,679
just reinforces that in a way, like you are going

705
00:39:44,719 --> 00:39:48,239
to have to expose that to be tested, must show

706
00:39:48,360 --> 00:39:50,079
that it's not actually sensitive information.

707
00:39:52,880 --> 00:39:55,079
Speaker 5: Yeah, definitely. To me, it just reminds me of like

708
00:39:55,119 --> 00:39:57,079
the myth of the one hundred percent test coverage, right,

709
00:39:57,079 --> 00:40:00,559
It's like, can we one hundred percent encrypt everything? Absolutely not?

710
00:40:00,719 --> 00:40:03,840
Like people, your end users need to see this information.

711
00:40:04,280 --> 00:40:06,639
I see. I've seen some interesting attempts to kind of

712
00:40:06,679 --> 00:40:10,840
obfuscate those things, Like I've seen some libraries that prevent

713
00:40:10,880 --> 00:40:13,719
you from opening the davascript console, for example, and it's like,

714
00:40:13,760 --> 00:40:15,920
what are you hiding in there? Maybe maybe you should

715
00:40:15,960 --> 00:40:17,400
just not put sensitive things.

716
00:40:19,760 --> 00:40:22,800
Speaker 2: Here's here's a wild thought. I just don't do that.

717
00:40:23,199 --> 00:40:26,239
Speaker 3: Credentials like encoded in the HTML on your page.

718
00:40:26,360 --> 00:40:26,639
Speaker 5: Maybe.

719
00:40:27,880 --> 00:40:31,039
Speaker 4: I mean, I can't believe you two are joking about this, honestly,

720
00:40:31,199 --> 00:40:34,639
like one of the most common attacks against So I

721
00:40:34,639 --> 00:40:35,000
don't do.

722
00:40:35,119 --> 00:40:37,239
Speaker 3: You why so I can joke about all this because

723
00:40:37,280 --> 00:40:40,840
none of them It's okay. Tone You'll have.

724
00:40:42,559 --> 00:40:45,199
Speaker 4: Jillian, You'll have plenty of opportunity to get your models

725
00:40:45,280 --> 00:40:49,039
encoded with AWS access keys in secrets, and then you

726
00:40:49,159 --> 00:40:51,320
just ask the model, Hey, can I have an access

727
00:40:51,360 --> 00:40:53,679
key and secret that are valid that work for any

728
00:40:53,679 --> 00:40:54,920
aw's account.

729
00:40:55,159 --> 00:40:58,239
Speaker 3: I did actually accidentally push my AWS credentials to get

730
00:40:58,280 --> 00:41:00,920
huged once, and like the amounted emails that I got

731
00:41:00,960 --> 00:41:04,880
from AWS was just like it was unreal. It was

732
00:41:05,039 --> 00:41:06,719
very It was a very bad day for me. It

733
00:41:06,760 --> 00:41:09,480
was a very very bad day. So I've done other

734
00:41:09,519 --> 00:41:11,800
stupid things, but I don't do the same stupid things,

735
00:41:11,800 --> 00:41:14,119
so I can sit here and be very smug about this.

736
00:41:14,519 --> 00:41:14,679
Speaker 5: Like.

737
00:41:18,320 --> 00:41:21,360
Speaker 1: That is like that is I've always I've often wondered

738
00:41:21,400 --> 00:41:25,760
about that, Like the speed that AWS and other malicious

739
00:41:25,800 --> 00:41:30,119
people can identify that you committed an AWS access key

740
00:41:30,199 --> 00:41:31,480
to a GitHub repo.

741
00:41:31,840 --> 00:41:34,679
Speaker 3: It was instant. It was right then, because as soon

742
00:41:34,719 --> 00:41:36,679
as I did it, I was like, oh no, and

743
00:41:36,760 --> 00:41:38,880
tried to you know, and like try to like make

744
00:41:38,880 --> 00:41:42,079
the gi hub repil private, and no, it was instant.

745
00:41:42,159 --> 00:41:43,960
They knew, they knew it was out there.

746
00:41:44,480 --> 00:41:46,440
Speaker 4: Yeah, I mean it's bad. I mean I think I

747
00:41:46,480 --> 00:41:49,000
saw a bunch of statistics on this that for AWS

748
00:41:49,119 --> 00:41:51,639
keys and on GitHub, it's about thirty seconds to two

749
00:41:51,679 --> 00:41:56,000
minutes having been exposed in the repository anywhere in any format,

750
00:41:56,079 --> 00:41:59,360
So like commit at the beginning of the repository where

751
00:41:59,800 --> 00:42:01,880
it was there but then got removed, so it's not

752
00:42:02,039 --> 00:42:03,880
in plain text anymore. You have to go back through

753
00:42:03,920 --> 00:42:08,239
the get history still about two minutes. Then there's exposure

754
00:42:08,360 --> 00:42:11,840
on like stack overflow and places like I don't know

755
00:42:11,840 --> 00:42:14,800
who uses Facebook in connection with their work, but that

756
00:42:14,880 --> 00:42:18,320
was another place, and then Instagram and read it somewhere

757
00:42:18,360 --> 00:42:20,960
between two and four or five days, and then there's

758
00:42:21,000 --> 00:42:22,760
a couple other ones where it's six and more.

759
00:42:23,920 --> 00:42:24,960
Speaker 5: Some of those you have to.

760
00:42:24,920 --> 00:42:28,559
Speaker 4: Thank, like GitHub for like they'll actually discover secrets there.

761
00:42:28,599 --> 00:42:31,800
So if you provide a third party application that has credentials,

762
00:42:31,800 --> 00:42:34,960
like at authors, we have our secret keys registered there,

763
00:42:35,000 --> 00:42:38,199
so if one of our customers exposes keys for our

764
00:42:38,360 --> 00:42:42,360
service on GitHub, we'll get notified, automatically revoke those keys

765
00:42:42,400 --> 00:42:44,760
and send them an email telling them that they did

766
00:42:44,760 --> 00:42:48,400
something that they probably did not want to do, multiple

767
00:42:48,400 --> 00:42:51,360
times if that's if necessary, because that's happened as well.

768
00:42:54,320 --> 00:42:57,199
Speaker 1: I want to switch topics here real quick, AJ, because

769
00:42:57,239 --> 00:43:01,639
you've been with Rainforest QA for over seven years now,

770
00:43:02,800 --> 00:43:08,719
which is unusual in the tech industry, So I'm curious

771
00:43:08,719 --> 00:43:12,800
about what are the what are the things that you

772
00:43:12,920 --> 00:43:18,280
look for in a job, that have been fulfilled at Rainforest,

773
00:43:18,320 --> 00:43:19,639
that keep you there that long.

774
00:43:21,360 --> 00:43:25,679
Speaker 5: Yeah, for sure. First and foremost is the people. Actually,

775
00:43:25,679 --> 00:43:29,320
when I was interviewing with Rainforest, the last person I

776
00:43:29,360 --> 00:43:32,679
talked to told me something like I say, at Rainforest

777
00:43:32,800 --> 00:43:34,960
because of the people, And I was like, okay, that's

778
00:43:35,119 --> 00:43:42,239
that's a way whateveryone said, And then I quickly drank

779
00:43:42,280 --> 00:43:44,320
the kool aid, I think, and I found myself saying

780
00:43:44,320 --> 00:43:46,320
that on interviews and I'm like, I know, this sounds

781
00:43:46,360 --> 00:43:50,920
like a load of crap, and I think, you know,

782
00:43:50,960 --> 00:43:54,280
the hiring process is super super important, right, Yeah, both

783
00:43:54,320 --> 00:43:57,440
finding people that are qualified for the job obviously, but

784
00:43:57,559 --> 00:44:01,159
are good culture fits. We have a pretty small team,

785
00:44:01,320 --> 00:44:04,320
so there's nowhere to hide. If you are not doing

786
00:44:04,360 --> 00:44:06,280
your job where you're not up to par, you're going

787
00:44:06,320 --> 00:44:10,840
to be exposed pretty quickly, which leads us to have

788
00:44:11,039 --> 00:44:15,840
a very reliable team. You know, we are distributed globally,

789
00:44:16,039 --> 00:44:19,199
so there's a lot of handoff. You know, I'm going

790
00:44:19,239 --> 00:44:22,280
to sleep, you're waking up. Here's what I did, and

791
00:44:22,320 --> 00:44:24,519
I trust when I wake up that you're just going

792
00:44:24,599 --> 00:44:27,000
to have this thing done. And if you're not one

793
00:44:27,039 --> 00:44:29,559
of those people, you're probably not going to fit at Rainforest.

794
00:44:30,519 --> 00:44:35,119
So really really qualified, experienced, smart, reliable people makes life

795
00:44:35,199 --> 00:44:38,760
so much easier. And then the other piece of it is,

796
00:44:39,480 --> 00:44:41,800
you know the mission that we're on, the technology that

797
00:44:41,840 --> 00:44:46,400
we're building. I think when I was first exposed to it,

798
00:44:46,559 --> 00:44:48,719
the first time I ship code with Rainforest, it was

799
00:44:48,800 --> 00:44:50,679
kind of like, Wow, how have I been shipping code

800
00:44:50,760 --> 00:44:53,159
before this? And the answer was I was probably breaking

801
00:44:53,199 --> 00:44:56,320
things all of the time and you don't notice until

802
00:44:56,719 --> 00:44:59,599
user catches it in production two days later or whatever.

803
00:45:00,679 --> 00:45:04,760
And it's something that I'm really passionate about. I think

804
00:45:04,800 --> 00:45:07,760
as a front and engineer, we get really caught up

805
00:45:07,800 --> 00:45:11,320
on the details. Right, There's all these visual layers, there's

806
00:45:11,639 --> 00:45:14,880
these very specific human interactions. I like building things that

807
00:45:14,960 --> 00:45:20,400
humans are actually interacting with, and that kind of naturally

808
00:45:20,480 --> 00:45:24,119
leads you to a quality assurance mindset, Right, I want

809
00:45:24,159 --> 00:45:26,360
everything to be perfect all of the time. How do

810
00:45:26,440 --> 00:45:29,840
I ensure this? And so the combination of really great

811
00:45:29,840 --> 00:45:32,639
people and working on something that I'm actually really passionate

812
00:45:32,639 --> 00:45:35,320
about and I want to see the rest of the

813
00:45:35,360 --> 00:45:38,519
world adopt these correct ways of testing things, in my

814
00:45:38,599 --> 00:45:42,599
opinion of course, just makes it makes it easy to

815
00:45:42,639 --> 00:45:43,199
work area.

816
00:45:43,920 --> 00:45:47,239
Speaker 2: Right on. That's cool, that's cool.

817
00:45:47,519 --> 00:45:51,159
Speaker 1: Is there you guys obviously do a lot of front

818
00:45:51,280 --> 00:45:55,840
end type testing. Is there a particular industry or vertical

819
00:45:56,000 --> 00:46:00,320
that you have got a lot of experience it's in,

820
00:46:00,519 --> 00:46:03,039
or something that has worked really well that makes a

821
00:46:03,039 --> 00:46:08,960
really cool story.

822
00:46:07,599 --> 00:46:10,599
Speaker 5: Something I've been involved in that makes a cool story. Oh,

823
00:46:10,679 --> 00:46:12,440
I don't know if I have a good answer for you. Honestly,

824
00:46:12,480 --> 00:46:14,440
I've been, like I said, I've been at Rainforest for

825
00:46:14,480 --> 00:46:17,440
so long. That's all I didn't think about. I guess, right.

826
00:46:17,559 --> 00:46:21,079
Speaker 1: Do you do you attract like a certain like customers

827
00:46:21,119 --> 00:46:25,440
with like financial apps or with like web based gaming apps,

828
00:46:25,519 --> 00:46:28,079
or is there a particular vertical that tends to gravitate

829
00:46:28,199 --> 00:46:29,679
towards your service?

830
00:46:31,800 --> 00:46:33,480
Speaker 5: I think not really, And I think that's one of

831
00:46:33,480 --> 00:46:35,239
the things that makes it cool is it's a very

832
00:46:35,400 --> 00:46:41,199
generic testing tool. Right There are some some limitations, but

833
00:46:41,280 --> 00:46:43,519
in general, if you could load your app on a machine,

834
00:46:43,519 --> 00:46:47,400
you could probably test it with Rainforest, not caring about

835
00:46:47,400 --> 00:46:50,800
what the tech stack is those kinds of things. So

836
00:46:51,039 --> 00:46:55,519
there's a very wide range of users that we have from. Yeah,

837
00:46:55,559 --> 00:47:01,079
there's some financial some financial companies do some What I

838
00:47:01,079 --> 00:47:04,079
always finds interesting kind of like testing visually things like

839
00:47:04,480 --> 00:47:08,199
spreadsheet style apps like their tables and things like that.

840
00:47:09,079 --> 00:47:11,800
And then we have some really cool like visual tools

841
00:47:12,000 --> 00:47:16,400
like like drag and drop interfaces where you're building things

842
00:47:16,440 --> 00:47:20,880
like you know, lego style building, where there's probably, to

843
00:47:20,920 --> 00:47:24,000
my knowledge, not any other great way to test something

844
00:47:24,320 --> 00:47:27,039
like that, Like what you say, are all my legos

845
00:47:27,079 --> 00:47:29,400
on the page? Yeah they are? Are they kind of

846
00:47:29,440 --> 00:47:31,559
oriented this way? Like? Yeah they are? But how does

847
00:47:31,559 --> 00:47:35,119
it look right? What does the user see? So the

848
00:47:35,519 --> 00:47:40,360
real sweet spot is really visual based applications because I

849
00:47:40,360 --> 00:47:42,880
don't think there's other great solutions for them out there.

850
00:47:43,440 --> 00:47:47,760
But in general, being a kind of generic visual testing application,

851
00:47:47,960 --> 00:47:50,320
it really applies to anything right on.

852
00:47:51,920 --> 00:47:56,760
Speaker 1: For a lot of web based front ends, it's all

853
00:47:57,119 --> 00:48:00,639
no JS based. Do you have a favorite no JS

854
00:48:01,840 --> 00:48:06,159
type tool? Are you like a React fan or next

855
00:48:06,280 --> 00:48:07,800
JS or view give.

856
00:48:07,679 --> 00:48:08,679
Speaker 2: A personal preference?

857
00:48:09,639 --> 00:48:14,519
Speaker 5: Yes, I am a Ract fanboy for sure. I started.

858
00:48:14,840 --> 00:48:17,639
Uh you know, we rewind all the way back to

859
00:48:17,679 --> 00:48:19,960
like the j Query days and stuff, right, I see that,

860
00:48:20,000 --> 00:48:22,880
and I have nightmares still we have some we actually

861
00:48:22,920 --> 00:48:25,360
have some of that floating around and our like like

862
00:48:25,400 --> 00:48:28,119
our admin applications and stuff where it's like a rails

863
00:48:28,199 --> 00:48:30,599
back in and they're like, yeah, we got jQuery in there.

864
00:48:31,519 --> 00:48:33,119
And then my first thought was always like, well, like

865
00:48:33,159 --> 00:48:35,480
how do you test that jQuery? And the answer is

866
00:48:35,519 --> 00:48:38,719
we don't. Uh throw a couple of wait for us

867
00:48:38,719 --> 00:48:43,320
tests that and call it good. And I started. I

868
00:48:43,360 --> 00:48:47,599
started with Angular back in the day, all right, back

869
00:48:47,719 --> 00:48:51,599
Angular one anyways was kind of the reverse of React.

870
00:48:51,679 --> 00:48:54,440
We're like, we're gonna put your JavaScript in your HTML.

871
00:48:54,800 --> 00:48:57,440
React took the approach ever to put your HTML and

872
00:48:57,480 --> 00:49:00,639
your JavaScript, you know, just smush it all together. And

873
00:49:00,679 --> 00:49:04,159
it's come a very very long way, I must say. So. Yeah,

874
00:49:04,199 --> 00:49:07,440
I find working with React very easy and intuitive, and

875
00:49:07,519 --> 00:49:12,679
it's very nice that the general JavaScript community has supported

876
00:49:12,679 --> 00:49:15,760
that and has pushed that forward because especially, you know,

877
00:49:15,840 --> 00:49:18,480
with with all software and technology, but especially in front

878
00:49:18,480 --> 00:49:23,559
of development, it's really easy to pick the wrong tool

879
00:49:23,840 --> 00:49:27,360
long term, Right, I picked this thing, it's great, and

880
00:49:27,400 --> 00:49:28,760
then we find a better way to do it, and

881
00:49:28,760 --> 00:49:32,039
they just abandon the project. Right. This is true with

882
00:49:32,199 --> 00:49:35,519
I mean anything open source, and we've run into this

883
00:49:35,559 --> 00:49:38,440
a lot of times, right, even with open source testing tools.

884
00:49:39,480 --> 00:49:45,239
And actually we had a very large Enzyme test suite

885
00:49:45,280 --> 00:49:48,159
on our React application, and we ran into something like this.

886
00:49:49,239 --> 00:49:52,679
There was a new way of testing React apps, which

887
00:49:52,800 --> 00:49:57,280
was the React Testing library, and Enzyme kind of said, yep,

888
00:49:57,320 --> 00:49:59,000
that's a better way to do it. We're going to

889
00:49:59,039 --> 00:50:02,159
stop supporting at after I forgot version like REX sixteen

890
00:50:02,239 --> 00:50:06,119
ORRAC seventeen, like, well we want to upgrade to React seventeen.

891
00:50:06,199 --> 00:50:11,119
It's like, well none of your enzyme tests work, so yeah, exactly,

892
00:50:11,199 --> 00:50:14,079
exactly too bad for us. And so now you start

893
00:50:14,079 --> 00:50:16,840
weighing the options of well, how do we upgrade? Right?

894
00:50:16,880 --> 00:50:19,840
Do we just say let's not upgrade, which is going

895
00:50:19,880 --> 00:50:22,320
to bite you really quickly, right, especially at the pace

896
00:50:22,599 --> 00:50:26,039
all these JavaScript libraries remain updated. I want that new

897
00:50:26,079 --> 00:50:29,800
shiny thing, I want support for that thing, and I

898
00:50:29,800 --> 00:50:31,760
don't want to be stuck in the past. The more

899
00:50:31,800 --> 00:50:33,760
you get stuck in the past, the harder it is

900
00:50:33,800 --> 00:50:37,079
to catch up with everything else. Right, And so our

901
00:50:37,119 --> 00:50:43,079
options were basically rewrite all these however many thousand enzyme tests,

902
00:50:44,519 --> 00:50:50,480
or we could just nuke them all, which is it

903
00:50:50,559 --> 00:50:53,440
reminds me of like these these memes I see about

904
00:50:53,440 --> 00:50:55,639
like junior engineers and the intern where they're like they're

905
00:50:55,639 --> 00:50:58,800
commit messages. I knuked all the tests because they were failing,

906
00:50:58,800 --> 00:51:01,639
and I kind of make them pass turn true and

907
00:51:01,679 --> 00:51:03,519
all the tests because you know, that's one of the.

908
00:51:03,519 --> 00:51:05,320
Speaker 3: Past, the only reasonable way to do things.

909
00:51:05,840 --> 00:51:12,000
Speaker 5: Yeah, and it sounds kind of like an overreaction, but

910
00:51:12,079 --> 00:51:14,559
as we started to kind of think about these testing philosophies,

911
00:51:14,599 --> 00:51:18,159
we're like, we have end and test coverage on all

912
00:51:18,199 --> 00:51:21,679
of these things, right, And a lot of the front

913
00:51:21,760 --> 00:51:27,480
end tests, even though they're unit tests, they they load

914
00:51:27,599 --> 00:51:30,280
things in a headless browser and we're kind of recreating

915
00:51:30,360 --> 00:51:32,800
what an end end test does. So we chose to

916
00:51:32,880 --> 00:51:35,400
keep all of our actual unit tests, all the kind

917
00:51:35,440 --> 00:51:39,400
of business logic that didn't use enzyme new call the

918
00:51:39,480 --> 00:51:42,360
enzyme tests, and just lean into our rainforest test because

919
00:51:42,360 --> 00:51:44,239
we know if the rainforest tests are passing, we don't

920
00:51:44,280 --> 00:51:48,719
need all of these redundant tests anymore. And instant productivity

921
00:51:48,760 --> 00:51:50,599
boosts Like I don't have to maintain all of these

922
00:51:50,599 --> 00:51:53,880
things anymore. I don't have to upgrade them. I could

923
00:51:53,960 --> 00:51:55,519
just get them out of my way and I can

924
00:51:55,599 --> 00:51:59,199
upgrade all my dependencies. And because we have really good

925
00:51:59,320 --> 00:52:02,000
end and testsage, we could do that confidently and know

926
00:52:02,039 --> 00:52:06,559
that we're not breaking things. So, yeah, choosing dependencies can

927
00:52:06,559 --> 00:52:09,079
be quite tricky sometimes, especially in the job script world.

928
00:52:09,679 --> 00:52:12,280
Speaker 4: Did you find some places that you still wanted to

929
00:52:12,320 --> 00:52:15,960
reintroduce some of the React testing library for I don't know,

930
00:52:16,039 --> 00:52:20,760
component level testing of the UI or have you kept

931
00:52:20,800 --> 00:52:23,159
with like one hundred percent of the decision to not

932
00:52:23,599 --> 00:52:27,079
have that layer of testing anymore regarding the UI components

933
00:52:27,079 --> 00:52:30,519
because you focus on the full picture and the end

934
00:52:30,559 --> 00:52:34,119
testing for the user flow and also whatever you have

935
00:52:34,840 --> 00:52:36,400
with the interaction with the back end.

936
00:52:37,320 --> 00:52:39,239
Speaker 5: Yeah, we still have some of it, and we drew

937
00:52:39,239 --> 00:52:43,280
the line at user interactions. Right. So React has this

938
00:52:43,360 --> 00:52:47,280
idea of hooks, which are basically just chunks of logic.

939
00:52:47,519 --> 00:52:50,000
It's just a function that I can use inside of

940
00:52:50,000 --> 00:52:55,079
a component. We stopped having any React testing library tests

941
00:52:55,559 --> 00:52:58,559
that were actual user interactions, no clicking on things, and

942
00:52:58,599 --> 00:53:01,360
instead we used it to test the functionality the logic

943
00:53:01,480 --> 00:53:05,800
of those hooks. So it's essentially a unit test, but

944
00:53:05,840 --> 00:53:09,360
it's testing a specific React thing, and it requires the

945
00:53:09,400 --> 00:53:12,679
testing library to do that. Everything else kind of gets

946
00:53:12,719 --> 00:53:15,920
hoisted up to the end to end testing level, and

947
00:53:16,000 --> 00:53:19,440
it's nice to just say, hey, designer, hay, product manager, like,

948
00:53:19,760 --> 00:53:22,679
go at this test coverage while I'm busy hacking on things,

949
00:53:22,679 --> 00:53:24,159
and I don't have to worry about this anymore.

950
00:53:24,840 --> 00:53:27,159
Speaker 2: So that's actually a great Jillian.

951
00:53:27,639 --> 00:53:29,440
Speaker 3: Oh, I was just gonna say, I'm so impressed with

952
00:53:29,559 --> 00:53:31,960
people who can keep up with like the UI and

953
00:53:32,079 --> 00:53:35,320
JavaScript plan because I've tried. I've tried like a few

954
00:53:35,360 --> 00:53:38,599
times and it just then everything changed. I was like,

955
00:53:38,599 --> 00:53:40,239
all right, I'm not doing this anymore, and we go

956
00:53:40,440 --> 00:53:42,599
I'm gonna go do high performance computing that hasn't changed

957
00:53:42,639 --> 00:53:43,400
in like thirty years.

958
00:53:43,440 --> 00:53:47,159
Speaker 5: It's gonna be Yeah, you definitely start feeling like Sissyphis.

959
00:53:47,239 --> 00:53:50,599
You're just pushing that rock up the hill and every

960
00:53:50,599 --> 00:53:52,480
time you get to the top, someone tells you that

961
00:53:52,519 --> 00:53:53,920
you're actually on the wrong hill.

962
00:53:57,880 --> 00:54:01,239
Speaker 1: I was gonna say, that seems like a really interesting

963
00:54:03,199 --> 00:54:05,719
approach that I hadn't thought of when we initially started talking.

964
00:54:05,760 --> 00:54:09,840
But like you, you can replace having to write a

965
00:54:09,880 --> 00:54:12,920
lot of your tests in your React app by using

966
00:54:13,239 --> 00:54:17,159
Rainforest right. By just focusing on what the end user

967
00:54:17,239 --> 00:54:20,519
experience is and testing for that, you can save yourself

968
00:54:20,519 --> 00:54:22,440
from having to write a lot of tests in the

969
00:54:22,480 --> 00:54:23,559
React standard library.

970
00:54:23,920 --> 00:54:26,639
Speaker 4: So that's where the trade off is though, right, because

971
00:54:26,760 --> 00:54:30,960
these tests then are testing more functionality at once and

972
00:54:31,000 --> 00:54:33,119
so if there is a problem, you don't necessarily know

973
00:54:33,679 --> 00:54:36,400
like which line of code is causing the issue or

974
00:54:36,480 --> 00:54:39,559
which interaction there is, so you know there really is

975
00:54:39,599 --> 00:54:42,760
Like how valuable is that flow? I think? And that's

976
00:54:42,800 --> 00:54:44,719
something that, as you pointed out Agent, like you sort

977
00:54:44,719 --> 00:54:47,079
of have to determine upfront, like where is the value

978
00:54:47,119 --> 00:54:49,039
of your testing and how do you get the most

979
00:54:49,119 --> 00:54:50,800
value out of which pieces.

980
00:54:50,519 --> 00:54:53,679
Speaker 5: You're adding and where you're validating an et cetera. And so, yeah,

981
00:54:53,679 --> 00:54:54,039
I mean.

982
00:54:53,960 --> 00:54:56,960
Speaker 4: In your case, and tis weren't actually providing the right

983
00:54:57,039 --> 00:55:00,440
value in the first place, so definitely switch them over.

984
00:55:01,199 --> 00:55:04,320
Speaker 5: Yeah, absolutely. And it's kind of a question of redundancy too, right,

985
00:55:04,920 --> 00:55:08,000
Like is redundancy good? Sometimes like I can be really

986
00:55:08,000 --> 00:55:09,920
really sure and I can have some extra confidence that

987
00:55:10,000 --> 00:55:12,440
the things isn't going to break, But most of the

988
00:55:12,440 --> 00:55:15,719
time it just slows us down. Right. I find that

989
00:55:15,840 --> 00:55:19,840
the often the best time to add more unit test

990
00:55:19,880 --> 00:55:23,960
coverage is when something breaks, right, because if my end

991
00:55:23,960 --> 00:55:27,679
to end tests are all passing, but something's broken, very

992
00:55:27,719 --> 00:55:29,559
often it's some kind of edge case, right, It's some

993
00:55:29,679 --> 00:55:33,000
either some weird user behavior, some weird input, some weird

994
00:55:33,239 --> 00:55:36,519
sequence of events, and those things are usually better captured

995
00:55:36,519 --> 00:55:41,039
in a unit test because it's it's easier to kind

996
00:55:41,039 --> 00:55:45,440
of implement that specific scenario, that specific line of code

997
00:55:45,480 --> 00:55:48,559
that is the offender here versus creating you know, a

998
00:55:48,599 --> 00:55:50,400
whole new end the end test and just cover some

999
00:55:50,559 --> 00:55:53,360
edge case. Those tests are going to just get longer

1000
00:55:53,400 --> 00:55:58,079
and longer and just be kind of confusing. Honestly, it's like, well,

1001
00:55:58,079 --> 00:56:00,199
why am I just like clicking in all the these

1002
00:56:00,280 --> 00:56:02,960
random spots doing these things trying to cover these edge cases?

1003
00:56:03,000 --> 00:56:04,960
Like just reread a unit test for it and call good.

1004
00:56:06,199 --> 00:56:08,559
Speaker 3: I really like the emphasis on, you know, testing for

1005
00:56:08,639 --> 00:56:13,440
business logic and just in general having not everything controlled

1006
00:56:13,679 --> 00:56:16,039
by the engineers, because I find for myself, you know,

1007
00:56:16,079 --> 00:56:18,039
like I'll write something and then I'll hand it off

1008
00:56:18,079 --> 00:56:20,280
to a user and then they immediately start using it

1009
00:56:20,320 --> 00:56:22,159
in some way that I didn't even think of, and then,

1010
00:56:22,239 --> 00:56:23,400
you know, and then we do like a couple of

1011
00:56:23,440 --> 00:56:25,800
rounds of this. So being able to cut back on

1012
00:56:25,880 --> 00:56:29,800
that person who writes thing, who does not actually use

1013
00:56:29,880 --> 00:56:32,119
the thing, and then just immediately being able to push

1014
00:56:32,159 --> 00:56:33,199
it off to an end user.

1015
00:56:34,159 --> 00:56:39,679
Speaker 5: Yeah, absolutely, Yeah, And things like testing and staging environments

1016
00:56:39,719 --> 00:56:41,840
are great for this. We push code of those environments

1017
00:56:41,880 --> 00:56:44,599
all the time, give them to a PM and say

1018
00:56:44,960 --> 00:56:47,880
go run and try and try and break this thing, right.

1019
00:56:48,599 --> 00:56:50,519
Don't always want to do that in production, right, like

1020
00:56:50,559 --> 00:56:52,719
if the thing's not fully baked, I don't want to

1021
00:56:52,760 --> 00:56:55,760
corrupt something in the database or whatever. And so having

1022
00:56:55,800 --> 00:56:58,800
places to push that and have people early in the

1023
00:56:58,840 --> 00:57:03,480
process on this, finding the major bugs, the minor bugs,

1024
00:57:03,559 --> 00:57:08,239
the stylistic bugs is super super valuable then having one

1025
00:57:08,239 --> 00:57:09,400
of your users find it later.

1026
00:57:09,760 --> 00:57:12,920
Speaker 2: Right on. So you live in Tahoe. Do you get

1027
00:57:12,920 --> 00:57:13,599
outdoors a lot?

1028
00:57:14,599 --> 00:57:17,599
Speaker 5: I do. I live here with my wife and my dog.

1029
00:57:17,760 --> 00:57:21,280
He's a lab husky mix, so he kind of thrives

1030
00:57:21,280 --> 00:57:23,840
in the summer, thrives in the winter. There's lots of

1031
00:57:23,880 --> 00:57:27,440
snow out right now, and so we're outside pretty much

1032
00:57:27,440 --> 00:57:31,079
every day, you know, snowboarding, hiking, kayaking and all that

1033
00:57:31,199 --> 00:57:31,760
kind of stuff.

1034
00:57:32,039 --> 00:57:33,800
Speaker 2: All Right on, how long have you lived in the

1035
00:57:33,840 --> 00:57:34,519
Tahoe area.

1036
00:57:35,960 --> 00:57:39,360
Speaker 5: I've been here for about five years now. I grew

1037
00:57:39,440 --> 00:57:41,239
up in the San Francisco Bay area and I was

1038
00:57:41,280 --> 00:57:44,760
part of the great COVID migration out here. We always

1039
00:57:44,880 --> 00:57:49,079
wanted to get here eventually, and I was lucky. I

1040
00:57:49,119 --> 00:57:52,400
was still working at rainforest at the time and already remote.

1041
00:57:52,440 --> 00:57:56,800
So the transfer up here to from remote near an

1042
00:57:56,840 --> 00:57:59,199
office to remote it actually doesn't matter how far you

1043
00:57:59,199 --> 00:58:02,639
are from the office, was very easy. And we're also

1044
00:58:02,760 --> 00:58:05,400
real fortunate that we were not the only ones doing

1045
00:58:05,440 --> 00:58:08,000
this migration. So we've made lots of friends that were like, yeah,

1046
00:58:08,039 --> 00:58:09,719
we lived down the street from you in the city

1047
00:58:09,760 --> 00:58:12,519
and we all live here now. So it's a very

1048
00:58:12,519 --> 00:58:14,880
different life, but we love it and I don't think

1049
00:58:14,880 --> 00:58:15,480
we're leaving.

1050
00:58:15,920 --> 00:58:19,039
Speaker 2: Ah, that's cool. Yeah, right on, Tahoe is a beautiful area.

1051
00:58:19,599 --> 00:58:20,679
Speaker 5: Yeah, it really is.

1052
00:58:23,400 --> 00:58:25,519
Speaker 2: Cool. All right. Should we move on to some picks

1053
00:58:27,679 --> 00:58:30,519
before we do any final thoughts on.

1054
00:58:30,519 --> 00:58:34,840
Speaker 1: QA rainforest tips guidance that you want to leave us

1055
00:58:34,840 --> 00:58:35,320
with aj.

1056
00:58:37,280 --> 00:58:40,880
Speaker 5: I think just recapping is finding the balance between confidence

1057
00:58:40,920 --> 00:58:45,960
and velocity, right, everybody needs to set their own bar

1058
00:58:46,239 --> 00:58:51,679
for quality, like how what is my ratio between confidence

1059
00:58:51,719 --> 00:58:55,440
and velocity? Determining that for yourself is the most important

1060
00:58:55,480 --> 00:58:59,679
thing here and keeping in mind that not only is

1061
00:58:59,679 --> 00:59:01,280
it a lot, but a lot of times it's the

1062
00:59:01,280 --> 00:59:03,960
sanity of your engineers, right, Like, we don't want to

1063
00:59:03,960 --> 00:59:07,639
spend all of our time writing tests, So finding that

1064
00:59:07,719 --> 00:59:12,440
balance and doing things in an efficient way is the

1065
00:59:12,480 --> 00:59:13,559
key to success.

1066
00:59:14,800 --> 00:59:19,400
Speaker 1: Right on. And I think that's very use case specific too,

1067
00:59:19,440 --> 00:59:23,199
you know, because the right answer for a financial app

1068
00:59:23,639 --> 00:59:27,239
is going to be very different than the right answer

1069
00:59:27,280 --> 00:59:27,639
for like.

1070
00:59:27,639 --> 00:59:28,679
Speaker 2: A social media app.

1071
00:59:29,280 --> 00:59:34,159
Speaker 1: Absolutely cool, All right, Jillian calling you out first? What'd

1072
00:59:34,199 --> 00:59:35,239
you bring for a pick today?

1073
00:59:36,159 --> 00:59:38,880
Speaker 3: I am going to pick Drive by Dave Kellett. It

1074
00:59:39,000 --> 00:59:42,559
is a sci fi graphic novel and I think it's

1075
00:59:42,599 --> 00:59:45,599
on it's I think it's like releasing the fourth one

1076
00:59:45,920 --> 00:59:47,880
this summer. But it's so good and it's so nice

1077
00:59:47,920 --> 00:59:50,840
and wholesome, which is very nice because like, I really

1078
00:59:50,880 --> 00:59:53,920
like sci fi, but I don't really like violence or

1079
00:59:53,960 --> 00:59:59,239
gore or you know, ikey fluids or I don't like

1080
00:59:59,239 --> 01:00:00,639
any of that. Okay, I don't like any of it.

1081
01:00:00,760 --> 01:00:03,039
And this is just so wholesome and adorable and the

1082
01:00:03,039 --> 01:00:06,280
main character is very cute. So that's it. That's the pick.

1083
01:00:07,320 --> 01:00:09,119
I've got a whole bunch of copies for Christmas and

1084
01:00:09,159 --> 01:00:11,519
I'm like making people read them and I'm gonna have

1085
01:00:11,639 --> 01:00:15,239
like a little little intigraphic novel cult going on soon enough.

1086
01:00:15,880 --> 01:00:23,719
Speaker 2: Great, all right, Warren, would you bring? Yeah?

1087
01:00:23,760 --> 01:00:28,280
Speaker 4: So I just got back from a long hiatus being

1088
01:00:28,320 --> 01:00:30,639
away from the show I was on vacation, and so

1089
01:00:30,719 --> 01:00:34,480
I think this pick is a really accurate, very short book.

1090
01:00:35,119 --> 01:00:41,639
I highly recommend Dowdishing by Laozza, which is the founder

1091
01:00:41,679 --> 01:00:46,000
of Taoism spelt Taoism in case in case. You've seen

1092
01:00:46,000 --> 01:00:48,400
it written but never pronounced before. And there's just so

1093
01:00:48,559 --> 01:00:51,639
much good stuff that is in the book that can

1094
01:00:51,639 --> 01:00:57,039
be applied to everyday life, working environment, etc. It's incredibly short.

1095
01:00:57,079 --> 01:00:59,320
There's only like one hundred and eight principles or so,

1096
01:01:00,079 --> 01:01:03,480
and it starts off great with the dow that can

1097
01:01:03,519 --> 01:01:06,320
be told. Is not the eternal tao Like, you can't

1098
01:01:06,360 --> 01:01:09,800
write down the whole truth. There is something that's never said.

1099
01:01:09,920 --> 01:01:12,800
It's impossible to convey everything. And I know it sounds

1100
01:01:12,840 --> 01:01:15,760
so philosophical, you know, to go down this path, but

1101
01:01:16,000 --> 01:01:18,679
I feel like going through these really helps to put

1102
01:01:18,719 --> 01:01:22,840
into perspective thinking outside the box with solving certain problems

1103
01:01:22,960 --> 01:01:25,599
or interactions or the communication we have every day.

1104
01:01:26,280 --> 01:01:31,599
Speaker 2: Highlight right on, cool heja, what you got for us?

1105
01:01:32,239 --> 01:01:35,960
Speaker 5: Yeah. My reading and listening choices are kind of all

1106
01:01:35,960 --> 01:01:38,760
over the map. But I did have an interesting one recently.

1107
01:01:38,800 --> 01:01:42,039
It was called The Light Eaters. It's about plants and

1108
01:01:42,079 --> 01:01:46,039
specifically this idea of plant intelligence. So obviously intelligence is

1109
01:01:46,079 --> 01:01:48,559
a loaded word. They're not intelligent like you and I

1110
01:01:48,599 --> 01:01:52,119
either not debating QA strategies and things like that, but

1111
01:01:52,159 --> 01:01:54,639
they do have a lot of intelligent like behavior. You know,

1112
01:01:54,679 --> 01:01:59,360
they communicate, they recognize their kin, they hear sounds, they

1113
01:01:59,400 --> 01:02:03,599
transform themselves based on the visual appearance of the environment

1114
01:02:03,639 --> 01:02:06,719
around them, And so I found it really interesting and

1115
01:02:06,719 --> 01:02:09,199
gave me a lot to think about, especially you know,

1116
01:02:09,239 --> 01:02:11,480
when I'm out in nature with the with the wife

1117
01:02:11,519 --> 01:02:14,199
and dog, just kind of staring at trees and stuff.

1118
01:02:15,119 --> 01:02:18,519
Speaker 4: Check out that are intelligent for sure, one hundred percent

1119
01:02:18,719 --> 01:02:20,840
totally with you. There's a there's a good one. If

1120
01:02:20,880 --> 01:02:23,559
you are out and there's plants or grass being cut

1121
01:02:23,800 --> 01:02:27,719
and you notice the smell of you know, freshly cut grass.

1122
01:02:27,760 --> 01:02:28,320
Speaker 5: What is that?

1123
01:02:28,559 --> 01:02:32,440
Speaker 4: It's a fear intently, a fear pheromone that's been sent

1124
01:02:32,440 --> 01:02:35,800
off to warn other grass that that there is danger around.

1125
01:02:35,960 --> 01:02:37,840
Like that is the sign of intelligent life.

1126
01:02:38,679 --> 01:02:41,840
Speaker 5: Yeah, for sure. There's lots of super interesting examples in

1127
01:02:41,880 --> 01:02:45,719
this book, and just like plants acting like animals essentially,

1128
01:02:45,760 --> 01:02:49,159
and it's it's kind of mind blowing experience.

1129
01:02:51,440 --> 01:02:53,320
Speaker 1: I read a book recently and I can't remember which

1130
01:02:53,320 --> 01:02:57,000
one it was, but I've been studying mushrooms a lot lately,

1131
01:02:57,039 --> 01:03:02,320
and this book showed where mushrooms actually act as a

1132
01:03:02,360 --> 01:03:08,480
communication agent for trees in the forest, and so like

1133
01:03:08,559 --> 01:03:12,280
a specific you know, set of insects can start attacking

1134
01:03:12,280 --> 01:03:15,159
trees on one end of the forest, and then the mushroom,

1135
01:03:15,480 --> 01:03:19,599
because it's the mycelium that grows underneath the entire forest floor,

1136
01:03:20,079 --> 01:03:23,920
will relay that information to the other trees in the forest.

1137
01:03:24,000 --> 01:03:26,840
And so by the time the insects work their way

1138
01:03:26,840 --> 01:03:31,079
down to those trees, that those trees are producing a

1139
01:03:31,159 --> 01:03:36,239
sind or a pheromone that actually repels the insects by

1140
01:03:36,239 --> 01:03:37,079
the time they get there.

1141
01:03:37,119 --> 01:03:38,400
Speaker 2: And I thought that was super cool.

1142
01:03:39,239 --> 01:03:42,920
Speaker 3: That is cool. I'm in on a like a mushroom

1143
01:03:42,960 --> 01:03:46,719
and foraging Facebook group and everybody just like takes pictures

1144
01:03:46,760 --> 01:03:49,159
of fun mushrooms that they find when when they're out

1145
01:03:49,159 --> 01:03:52,400
and about, and it's it's just such a nice little

1146
01:03:52,400 --> 01:03:54,920
group because it's so chill. That's it. There's like, there's

1147
01:03:54,960 --> 01:03:57,320
no drama, there's no nothing, it's just look at this mushroom.

1148
01:03:57,360 --> 01:04:01,400
Speaker 1: I found there's an app called Naturalist that I use

1149
01:04:02,000 --> 01:04:04,760
for that you can take a picture of not just mushrooms,

1150
01:04:04,760 --> 01:04:07,360
but anything you find that you can't identify, and then

1151
01:04:07,440 --> 01:04:10,039
upload it to I Naturalists and it will it will

1152
01:04:10,039 --> 01:04:13,000
try to auto detect what it is for you, but

1153
01:04:13,039 --> 01:04:16,599
then other people will come in and confirm or tell

1154
01:04:16,639 --> 01:04:17,599
you what that actually is.

1155
01:04:17,800 --> 01:04:18,599
Speaker 2: That was pretty cool.

1156
01:04:19,639 --> 01:04:20,920
Speaker 3: I used to do that a lot as a kid.

1157
01:04:20,960 --> 01:04:22,679
I'd have like the field guides and go out with

1158
01:04:22,719 --> 01:04:25,039
my field guide and try to like identify all the plants.

1159
01:04:25,400 --> 01:04:26,559
But now we have an app for that.

1160
01:04:27,159 --> 01:04:31,119
Speaker 1: There's an app for that always will what's your what's

1161
01:04:31,159 --> 01:04:36,079
your pick? My pick is There's a series on Netflix

1162
01:04:36,159 --> 01:04:39,800
called Kunk on Earth and I thought my sense of

1163
01:04:39,840 --> 01:04:45,880
humor was like really really dry, but this lady, she's amazing,

1164
01:04:45,880 --> 01:04:46,239
takes it.

1165
01:04:46,320 --> 01:04:47,920
Speaker 2: She takes it to a whole new level.

1166
01:04:48,480 --> 01:04:52,239
Speaker 1: This series is just hilarious because she sits down it's

1167
01:04:52,320 --> 01:04:56,119
you know, like a the History of Earth basically, but

1168
01:04:56,320 --> 01:05:01,800
she'll sit down with legitimate world renow experts in their

1169
01:05:01,840 --> 01:05:05,159
field and ask them the most off the wall questions.

1170
01:05:05,199 --> 01:05:07,280
And that, to me, thet was the highlight of the series.

1171
01:05:07,360 --> 01:05:09,679
Is just the looks on their faces when she would

1172
01:05:09,719 --> 01:05:13,000
ask them these questions that had absolutely nothing to do

1173
01:05:13,079 --> 01:05:17,039
with what they were an expert in, but super entertaining series.

1174
01:05:17,400 --> 01:05:21,159
Definitely ten out of ten stars. Kunk on Earth on Netflix,

1175
01:05:22,360 --> 01:05:24,880
and you know, there's actually two other things. There's Kunk on.

1176
01:05:26,599 --> 01:05:28,840
Speaker 4: Britain I think, and then there's like one on Christmas

1177
01:05:29,039 --> 01:05:32,239
and Shakespeare, so you too.

1178
01:05:32,639 --> 01:05:33,239
Speaker 5: Oh sweet.

1179
01:05:33,320 --> 01:05:34,960
Speaker 2: I will have to check those out because I love

1180
01:05:35,000 --> 01:05:35,760
her sense of humor.

1181
01:05:40,400 --> 01:05:42,719
Speaker 3: I've never heard that's fun.

1182
01:05:42,920 --> 01:05:45,559
Speaker 1: Yeah, it's very very accurate.

1183
01:05:47,599 --> 01:05:48,000
Speaker 2: All right.

1184
01:05:48,719 --> 01:05:50,760
Speaker 1: That brings us to the end of the episode. Thank

1185
01:05:50,800 --> 01:05:54,719
you everyone for listening. Jillian Warren, thank you for joining

1186
01:05:54,719 --> 01:05:57,480
me hosting the show. And aj thanks for coming on

1187
01:05:57,519 --> 01:05:59,320
the show man. It's been a pleasure talking to you.

1188
01:06:00,000 --> 01:06:00,800
Speaker 5: Thanks so much for having me.

1189
01:06:00,800 --> 01:06:03,280
Speaker 1: It was a lot of fun right on. Glad to

1190
01:06:03,320 --> 01:06:05,800
hear that, and I will see everyone next week.

1191
01:06:09,679 --> 01:06:09,719
Speaker 5: M