1
00:00:02,000 --> 00:00:05,480
Speaker 1: Go wild. So that's how the episode starts with Jillian Wild.

2
00:00:07,000 --> 00:00:11,080
Speaking of which, staying on the internet, Yeah too late,

3
00:00:11,480 --> 00:00:16,280
it's too late. Yeah, okay, Speaking of which, you are

4
00:00:16,399 --> 00:00:19,280
back on the show with us. You're back in the US,

5
00:00:20,359 --> 00:00:21,800
So it's been going on.

6
00:00:23,760 --> 00:00:26,399
Speaker 2: Well, I mean I moved continent, so that was that

7
00:00:26,519 --> 00:00:27,399
was a pretty big thing.

8
00:00:27,600 --> 00:00:29,359
Speaker 3: Now I'm in the US full time, which.

9
00:00:29,160 --> 00:00:32,159
Speaker 2: Is exciting, Like just two hours north of Boston and

10
00:00:32,320 --> 00:00:34,560
a little town that probably nobody has ever heard of

11
00:00:34,600 --> 00:00:36,640
in New Hampshire. But it's been fun.

12
00:00:36,759 --> 00:00:38,600
Speaker 3: It's been it's been a big adjustment, but it's been

13
00:00:38,600 --> 00:00:39,240
a really good time.

14
00:00:40,320 --> 00:00:44,039
Speaker 1: Yeah. From you were in Doka.

15
00:00:43,799 --> 00:00:45,880
Speaker 3: Right Doha?

16
00:00:46,399 --> 00:00:50,600
Speaker 1: Oh yeah, yeah, in the Middle East, So yeah, Doha

17
00:00:50,759 --> 00:00:54,520
to New Hampshire. There's some adjustment there.

18
00:00:54,840 --> 00:00:57,359
Speaker 2: There, there is. There's a lot more color here, like

19
00:00:57,479 --> 00:00:58,799
just in general the story.

20
00:01:00,640 --> 00:01:03,320
Speaker 1: I'm sure. Yeah, I've not been to Dolla, but I

21
00:01:03,320 --> 00:01:05,840
have been to Dubai several times, and I think they're

22
00:01:05,879 --> 00:01:06,719
fairly similar.

23
00:01:06,879 --> 00:01:09,040
Speaker 2: Yeah. Yeah, it's like less than an hour's plane ride.

24
00:01:09,079 --> 00:01:10,040
They're about the same thing.

25
00:01:10,640 --> 00:01:15,599
Speaker 1: Yeah. Cool. So back in the US permanently or is

26
00:01:15,599 --> 00:01:17,599
this a temporary thing for you?

27
00:01:17,680 --> 00:01:19,359
Speaker 3: Permanently? No living here now?

28
00:01:19,879 --> 00:01:22,640
Speaker 1: Right on. Well cool, Well, welcome back home.

29
00:01:23,239 --> 00:01:24,640
Speaker 3: Thank you. It's been nice.

30
00:01:24,640 --> 00:01:28,079
Speaker 2: I'm very happy to be back, and it's nice that professionally,

31
00:01:28,120 --> 00:01:29,760
it's so much better for me too. I don't have

32
00:01:29,799 --> 00:01:32,120
to work split shifts and that's great, and I get

33
00:01:32,159 --> 00:01:34,359
to be back on the podcast and talk to you guys,

34
00:01:34,400 --> 00:01:35,599
which is obviously.

35
00:01:35,200 --> 00:01:35,879
Speaker 3: The biggest win.

36
00:01:36,719 --> 00:01:37,280
Speaker 1: Of course.

37
00:01:37,400 --> 00:01:40,359
Speaker 2: Yeah, of course, go harass my clients in Boston. Now

38
00:01:40,400 --> 00:01:42,599
that's where most of them are at and here we are.

39
00:01:42,879 --> 00:01:45,040
Speaker 4: Oh but two hours like and you know, so you

40
00:01:45,079 --> 00:01:47,599
said maybe no one knows about that, but I actually

41
00:01:47,599 --> 00:01:50,879
made that drive fair number of times up to Lake Ossipe,

42
00:01:51,000 --> 00:01:55,200
So there's there's a whole bunch of town there that

43
00:01:55,239 --> 00:01:58,719
I could probably rattle off. And maybe maybe you're living

44
00:01:58,719 --> 00:02:01,319
in one of those. That's a long I get from

45
00:02:02,680 --> 00:02:03,640
New Hampshire downtown.

46
00:02:04,000 --> 00:02:06,400
Speaker 2: I'm not driving, though, so it doesn't bother me. I

47
00:02:06,480 --> 00:02:08,360
drive to the bus station and then one of those

48
00:02:08,479 --> 00:02:11,560
very nice buses takes me into Boston. I'm not trying

49
00:02:11,560 --> 00:02:13,319
to drive in Boston, Like, that's not a thing that

50
00:02:13,319 --> 00:02:13,800
we're doing.

51
00:02:15,039 --> 00:02:17,360
Speaker 1: It doesn't really matter, right because you're on the East

52
00:02:17,439 --> 00:02:19,560
coast where they actually have public transit.

53
00:02:20,840 --> 00:02:25,199
Speaker 2: No, the bus is actually nice, Like I don't actually

54
00:02:25,240 --> 00:02:26,960
mind taking the bus. It has like the really nice

55
00:02:26,960 --> 00:02:29,840
big seats and it goes right into South Station and

56
00:02:29,879 --> 00:02:31,759
then I don't know, most of my clients are like

57
00:02:31,759 --> 00:02:33,120
ten fifteen minutes from there.

58
00:02:34,680 --> 00:02:36,400
Speaker 1: That's cool. I do kind of miss that. I miss

59
00:02:36,479 --> 00:02:38,479
that every once in a while of just seeing and

60
00:02:38,520 --> 00:02:39,919
working with people face to face.

61
00:02:41,479 --> 00:02:43,400
Speaker 2: I do. I actually do like that, Like I wouldn't

62
00:02:43,479 --> 00:02:45,599
even really want to have a hybrid job, but showing

63
00:02:45,680 --> 00:02:48,120
up like I don't know quarterly and just going to

64
00:02:48,120 --> 00:02:50,560
go see everybody. Usually it's for some type of meetup.

65
00:02:50,599 --> 00:02:53,400
There's like, you know, a big meeting of the mind's

66
00:02:53,400 --> 00:02:55,759
happening or a conference or a talk or something like that,

67
00:02:55,840 --> 00:02:58,159
and then I'll go for sure.

68
00:02:59,439 --> 00:03:02,840
Speaker 1: So what was it like moving across continents? Did you

69
00:03:02,879 --> 00:03:08,240
like bring like couches and mattresses and furniture and did no?

70
00:03:09,159 --> 00:03:12,800
Speaker 3: No, no, it's all I don't know.

71
00:03:12,879 --> 00:03:15,159
Speaker 2: I guess like living overseas, all the furniture and stuff

72
00:03:15,199 --> 00:03:17,439
got treated as kind of disposable because there was enough

73
00:03:17,479 --> 00:03:20,319
of those moves. And the last place that we were

74
00:03:20,360 --> 00:03:22,360
at was the last place that we were at was

75
00:03:22,360 --> 00:03:25,080
mostly furnished anyway, So I didn't have a lot of furniture.

76
00:03:25,599 --> 00:03:28,800
So I bought like furniture when I bought the house

77
00:03:28,800 --> 00:03:30,800
that I'm living in now right on.

78
00:03:31,520 --> 00:03:35,840
Speaker 1: Yeah, cool, We did that whenever we moved to Oregon

79
00:03:35,879 --> 00:03:38,800
from Arizona. The people that bought our house in Arizona

80
00:03:39,479 --> 00:03:42,000
wanted to buy most of the furniture too, So when

81
00:03:42,000 --> 00:03:46,120
we got up here, we had almost no furniture. And

82
00:03:46,240 --> 00:03:47,639
so the first thing we had to do was, you know,

83
00:03:47,680 --> 00:03:53,479
go drop thousands of dollars on furniture. And I can't

84
00:03:53,520 --> 00:03:56,159
say we made the best decisions, you know, because like

85
00:03:56,199 --> 00:04:00,680
you're like desperate for a couch and a mass so

86
00:04:01,479 --> 00:04:04,199
you don't really research it or take time to say

87
00:04:05,560 --> 00:04:09,319
do I really like this? You just like say I

88
00:04:09,319 --> 00:04:12,520
think I'll like this, and then three months later you're like,

89
00:04:14,280 --> 00:04:16,160
I probably should have gone a different direction.

90
00:04:16,519 --> 00:04:17,680
Speaker 5: You're an amateur mover.

91
00:04:17,839 --> 00:04:18,160
Speaker 2: That's fun.

92
00:04:18,360 --> 00:04:20,279
Speaker 3: Yeah, I mean I do, because I'm insane.

93
00:04:20,399 --> 00:04:22,800
Speaker 2: Like I would just buy some of the Amazon like

94
00:04:22,839 --> 00:04:25,360
Futon mattresses, like the kind that I take when I

95
00:04:25,360 --> 00:04:27,399
go camping, and I would throw those on the floor

96
00:04:27,439 --> 00:04:29,439
with some pillows, and that's what we would have until

97
00:04:29,480 --> 00:04:32,519
I found a couch that I obsessed research and I

98
00:04:32,560 --> 00:04:35,920
have like a whole decision matrix of couch versus size

99
00:04:35,959 --> 00:04:37,600
in the living room versus how I'm gonna like this

100
00:04:37,759 --> 00:04:40,079
versus different seasons, and how I'm gonna like it? How

101
00:04:40,160 --> 00:04:43,959
much can the destroy it? Like, there's a lot.

102
00:04:43,800 --> 00:04:45,600
Speaker 3: That goes into these decisions for.

103
00:04:45,519 --> 00:04:47,279
Speaker 5: Me, So you figured it out.

104
00:04:47,680 --> 00:04:50,040
Speaker 3: That's not how I do anything, ever.

105
00:04:50,519 --> 00:04:53,600
Speaker 4: The futon sofa and I also used to buy a

106
00:04:53,680 --> 00:04:57,439
queen size blow up mattress because you know that's fairly

107
00:04:57,519 --> 00:05:00,360
cheap and can help you a lot there with even

108
00:05:00,360 --> 00:05:02,240
knowing what your situation is going to be. If you

109
00:05:02,319 --> 00:05:05,879
can't find a quick futon sofa and then yeah, then

110
00:05:05,879 --> 00:05:08,839
you've got as much time as you're willing to sacrifice

111
00:05:08,920 --> 00:05:13,000
into living into this terrible situation until you actually define

112
00:05:13,000 --> 00:05:15,360
the mattress you're looking for, the furniture you're willing to

113
00:05:16,040 --> 00:05:19,040
I got a lot of black previously because I used

114
00:05:19,040 --> 00:05:26,199
to use my moving boxes as partial furniture. This box

115
00:05:26,279 --> 00:05:28,800
turned sideways is like a shelf. You stack some boxes

116
00:05:28,839 --> 00:05:30,720
on top, and then you have a bureau made out

117
00:05:30,759 --> 00:05:31,319
of cardboard.

118
00:05:31,639 --> 00:05:32,319
Speaker 5: There, he does.

119
00:05:35,519 --> 00:05:37,360
Speaker 2: Although I am willing to fight you on the air

120
00:05:37,360 --> 00:05:40,519
mattresses because air mattresses are like one hundred bucks now,

121
00:05:40,600 --> 00:05:43,600
and my kids break them fairly regularly, so they don't

122
00:05:43,639 --> 00:05:45,600
last long enough for us, which is why I have

123
00:05:45,720 --> 00:05:47,360
moved on, which is why I moved on to the

124
00:05:47,399 --> 00:05:51,959
futon mattress like lifestyle, and I'm willing to commit to that.

125
00:05:52,120 --> 00:05:53,519
Speaker 3: Yeah, for life.

126
00:05:53,680 --> 00:05:54,079
Speaker 2: That's it.

127
00:05:54,560 --> 00:05:56,560
Speaker 3: Nothing bad is going to happen to host mattresses ever,

128
00:05:57,120 --> 00:05:57,399
right on.

129
00:05:57,680 --> 00:06:02,160
Speaker 1: Yeah, cool, Well, welcome back. Excited to have you back

130
00:06:02,199 --> 00:06:02,959
on the show.

131
00:06:03,959 --> 00:06:06,279
Speaker 3: And thank you for letting me back.

132
00:06:07,199 --> 00:06:12,439
Speaker 1: Yeah, of course, anytime, especially for today's episode because we're

133
00:06:12,480 --> 00:06:18,800
talking about the twenty twenty four Door Report. Because I mean,

134
00:06:18,920 --> 00:06:21,879
by the time this episode releases, I'm sure everyone will

135
00:06:21,879 --> 00:06:25,160
have read all one hundred and twenty pages and this

136
00:06:25,199 --> 00:06:27,199
is going to be a pretty boring episode because you

137
00:06:27,240 --> 00:06:31,240
will have digested all the information already. But for the

138
00:06:31,560 --> 00:06:33,800
two or three people who didn't read it, we're here

139
00:06:33,879 --> 00:06:35,759
to get you at the speed so that you can

140
00:06:36,680 --> 00:06:39,600
converse with everyone else as if you spent just as

141
00:06:39,680 --> 00:06:41,399
much time reading it as they did.

142
00:06:42,560 --> 00:06:44,720
Speaker 2: So I think instead of people reading it, we're all

143
00:06:44,720 --> 00:06:49,000
going to have to brains and then the report will

144
00:06:49,040 --> 00:06:51,839
automatically be wrong because it says there ai ho's and

145
00:06:51,920 --> 00:06:56,439
how as much have been impact as was expected, except

146
00:06:56,439 --> 00:07:00,519
in Drug Discovery, where it's been insane. But what could happen?

147
00:07:01,720 --> 00:07:03,920
Speaker 4: I honestly, while I was reading it, I had this

148
00:07:04,040 --> 00:07:06,680
fear like, oh, you like, our whole episode could have

149
00:07:06,839 --> 00:07:11,079
just been feeding the report into notebook LM and having

150
00:07:11,160 --> 00:07:15,079
that running instead of the legitimate adventures and DevOps for

151
00:07:15,279 --> 00:07:18,680
our you know, faithful viewers here.

152
00:07:20,800 --> 00:07:27,319
Speaker 3: Well, I haven't actually read, so.

153
00:07:27,360 --> 00:07:29,759
Speaker 5: I did go through. I did go through almost all

154
00:07:29,800 --> 00:07:30,040
of it.

155
00:07:30,439 --> 00:07:32,839
Speaker 4: There were some pages at the end that I skipped

156
00:07:32,839 --> 00:07:36,160
that was just repeating a little bit about what Dora

157
00:07:36,319 --> 00:07:39,560
does and for those that actually don't know, Dora was

158
00:07:39,560 --> 00:07:43,879
a separate company that Google bought in twenty eighteen that

159
00:07:44,000 --> 00:07:48,439
was focused on the delivery metrics that we all know

160
00:07:48,720 --> 00:07:51,600
and now love in and around engineering to have.

161
00:07:52,480 --> 00:07:53,399
Speaker 5: I mean, there's nothing else.

162
00:07:53,560 --> 00:07:55,199
Speaker 4: I say, there's nothing else, although there are a bunch

163
00:07:55,199 --> 00:07:57,839
of companies that have started to experiment with a sort

164
00:07:57,879 --> 00:08:00,800
of revision of them called Space, which I don't have

165
00:08:00,839 --> 00:08:06,639
a lot of experience with, but does handle additional aspects

166
00:08:06,680 --> 00:08:08,879
that some of the Dora metrics are missing.

167
00:08:09,800 --> 00:08:11,839
Speaker 1: Is space another acronym for something?

168
00:08:12,319 --> 00:08:15,079
Speaker 4: Yeah, you know, because people love acronyms and they don't

169
00:08:15,079 --> 00:08:17,680
mean anything, and I'm sure it stands for something, but

170
00:08:19,000 --> 00:08:22,240
I think realistically most organizations can do a better job

171
00:08:22,319 --> 00:08:25,800
by just understanding where their metrics are currently at and

172
00:08:25,839 --> 00:08:28,439
focusing on improving them irrelevant of what they are, because

173
00:08:28,480 --> 00:08:32,799
no metric is entirely perfect. Because of the companies that

174
00:08:32,879 --> 00:08:36,600
I've worked with in my history, often what happens is

175
00:08:36,840 --> 00:08:39,360
they say, oh, we're we have great Dora metrics, or

176
00:08:39,399 --> 00:08:41,399
we even have bad door metrics, and we need to

177
00:08:41,399 --> 00:08:44,000
do better improving them. And then I look at how

178
00:08:44,000 --> 00:08:47,399
they're capturing those metrics and they couldn't be more wrong.

179
00:08:47,639 --> 00:08:50,200
Like you look at them and they're say, let's say,

180
00:08:50,320 --> 00:08:53,559
using feature flags and whatnot, and so you're looking at

181
00:08:53,559 --> 00:08:55,919
your time to roll out changes in production. But those

182
00:08:56,200 --> 00:08:59,200
changes in production have never been tested, they're never being used,

183
00:08:59,240 --> 00:09:02,360
and they're behind flags that aren't even being that don't

184
00:09:02,399 --> 00:09:05,080
make that functionality available to your customers. And so it's like,

185
00:09:05,080 --> 00:09:07,039
can you really even say it's in the production. And

186
00:09:07,159 --> 00:09:09,279
yet these companies are saying, oh, yeah, we roll out

187
00:09:09,320 --> 00:09:11,679
our code immediately. It's all in craud, it's all being used,

188
00:09:11,919 --> 00:09:15,399
and so our values for that are really high. So

189
00:09:15,519 --> 00:09:17,480
I mean, I think in those cases that it doesn't

190
00:09:17,480 --> 00:09:20,120
matter what the metric is if you're recording it in

191
00:09:20,120 --> 00:09:22,919
a way which doesn't encourage good engineering practices.

192
00:09:24,399 --> 00:09:26,480
Speaker 2: So I think maybe we should backtrack that a little bit.

193
00:09:26,519 --> 00:09:27,600
What is a Dora metric?

194
00:09:30,080 --> 00:09:34,399
Speaker 1: Dora is this little girl in a cartoon with a

195
00:09:34,440 --> 00:09:40,360
monkey that gets chased by a fox, and evidently she

196
00:09:40,399 --> 00:09:44,240
does reporting in between episodes of making cartoons.

197
00:09:46,080 --> 00:09:48,440
Speaker 3: That's fair. I'll bet she'd be a better reporter than most.

198
00:09:49,519 --> 00:09:55,480
Speaker 1: Yeah, that was a reference to Dora the Explorer. For

199
00:09:55,519 --> 00:09:58,960
anyone who didn't get that joke, or.

200
00:09:59,080 --> 00:10:02,440
Speaker 5: Maybe before my time, it was gone.

201
00:10:03,200 --> 00:10:04,879
Speaker 1: Yeah, it was when my kids were young, so it's

202
00:10:04,879 --> 00:10:06,759
probably pretty old at this point.

203
00:10:07,159 --> 00:10:09,919
Speaker 2: My oldest really went through like a phase where she

204
00:10:09,960 --> 00:10:10,720
really loved Dora.

205
00:10:10,960 --> 00:10:12,000
Speaker 3: She was like three or something.

206
00:10:12,000 --> 00:10:14,000
Speaker 2: That was very cute. It was one of the shows

207
00:10:14,000 --> 00:10:16,200
that I actually liked too. Not all of them, but

208
00:10:16,279 --> 00:10:17,360
some of them are adorable.

209
00:10:18,080 --> 00:10:22,120
Speaker 1: I'm sorry, I totally derailed the topic. What was your question, Jillian?

210
00:10:22,480 --> 00:10:24,879
Speaker 2: Yeah, what is it? Door? We're talking about Dora metrics

211
00:10:24,879 --> 00:10:26,759
and it's like, well, what is a Dora metric? First

212
00:10:26,759 --> 00:10:28,039
of all, I mean, I suppose we can all kind

213
00:10:28,039 --> 00:10:30,360
of infer it, right, It's some type of metric that

214
00:10:30,399 --> 00:10:33,320
we're using to and for how devopsy we really all are,

215
00:10:33,559 --> 00:10:36,559
Like who's got the highest score? Is there a scoreboard

216
00:10:36,639 --> 00:10:38,879
that we can we can enter ourselves into, Like what

217
00:10:38,919 --> 00:10:39,600
are we doing here?

218
00:10:40,240 --> 00:10:42,799
Speaker 4: So there was actually like four official ones that everyone

219
00:10:43,080 --> 00:10:46,720
could could be using. And this is where the test

220
00:10:46,759 --> 00:10:48,720
really comes because I don't know if I remember all

221
00:10:48,720 --> 00:10:49,159
of them off.

222
00:10:49,159 --> 00:10:49,679
Speaker 5: On my head.

223
00:10:50,159 --> 00:10:53,320
Speaker 4: But there is a mean time to resolution when a

224
00:10:53,360 --> 00:10:56,720
failure happens and you need to go and resolve it

225
00:10:56,799 --> 00:11:00,360
and get your production state back to working according to

226
00:11:00,440 --> 00:11:05,200
your SLAS. Then there's cycle time to deploy changes from

227
00:11:05,240 --> 00:11:09,480
the time you start working on them. There's change failure rate,

228
00:11:09,639 --> 00:11:14,159
how often failures actually happen because of the changes you're making.

229
00:11:14,200 --> 00:11:16,960
And I missed one and I'm hoping Will knows which

230
00:11:16,960 --> 00:11:17,600
one I missed.

231
00:11:18,200 --> 00:11:23,840
Speaker 1: Yeah, we have change lead time, deployment frequency with frequency yeah,

232
00:11:24,440 --> 00:11:28,200
then the change fail rate and deployment recovery time.

233
00:11:28,759 --> 00:11:30,320
Speaker 5: Yeah.

234
00:11:31,000 --> 00:11:33,279
Speaker 1: Yeah. So it's interesting. It's one hundred and twenty pages

235
00:11:33,600 --> 00:11:39,799
of texts that just boils down to those four metrics.

236
00:11:40,120 --> 00:11:42,159
Speaker 5: So normally, normally that's what they talk about.

237
00:11:42,799 --> 00:11:44,759
Speaker 4: And so the question is that since they've been doing

238
00:11:44,799 --> 00:11:48,600
this what's new every year that is relevant because once

239
00:11:48,639 --> 00:11:50,840
you sort of share that these are the metrics that

240
00:11:50,879 --> 00:11:53,759
every engineering team could be utilizing to decide like how

241
00:11:53,799 --> 00:11:57,080
great they are. And they talk about benchmarks and where

242
00:11:57,519 --> 00:12:02,679
elite teams are versus maybe behind the curve teams that

243
00:12:02,799 --> 00:12:05,519
have a long way to go, what changes from year

244
00:12:05,519 --> 00:12:10,039
to year. And so this year they dove a lot

245
00:12:10,080 --> 00:12:13,600
more into three areas. The first one was they started

246
00:12:13,600 --> 00:12:17,240
talking about a new metric the amount of rework you have,

247
00:12:17,679 --> 00:12:19,879
so when you thought something was completed and you have

248
00:12:19,960 --> 00:12:22,679
to go back. And this is really interesting because it

249
00:12:22,679 --> 00:12:24,679
was always something that made a lot of sense to

250
00:12:24,720 --> 00:12:27,399
me and wasn't included something else that they don't really

251
00:12:27,399 --> 00:12:29,519
talk about. It's like the number of support tickets you get,

252
00:12:29,559 --> 00:12:30,960
like whether or not, Like there may not be a

253
00:12:30,960 --> 00:12:33,440
production incident in any way, but your customers may be

254
00:12:33,480 --> 00:12:35,559
concerned and confused, your users.

255
00:12:35,240 --> 00:12:37,440
Speaker 5: May not know what's going on, and that's not really included.

256
00:12:37,799 --> 00:12:40,960
Speaker 4: So they talk about expanding the metrics and maybe breaking

257
00:12:40,960 --> 00:12:43,679
them down into different areas that are all independent of

258
00:12:43,720 --> 00:12:46,279
each other. And that's sort of what's important, right, Metrics

259
00:12:46,320 --> 00:12:49,720
that depend on each other or have high correlation or

260
00:12:49,799 --> 00:12:52,919
high cohesion tend not to be as effective because they measure.

261
00:12:52,720 --> 00:12:53,200
Speaker 5: The same thing.

262
00:12:54,039 --> 00:12:57,879
Speaker 4: Then the report moves on to in depth review of

263
00:12:58,320 --> 00:13:02,120
the impact AI in our industry and the world, really

264
00:13:02,159 --> 00:13:04,840
about engineering teams and software, and the last part talks

265
00:13:04,879 --> 00:13:07,679
a lot about platform engineering and its impact.

266
00:13:09,120 --> 00:13:11,600
Speaker 1: Yeah, before we dig into those, I think the thing

267
00:13:11,639 --> 00:13:16,120
that stood out to me most in here is the

268
00:13:16,159 --> 00:13:20,720
section applying insights from Dora, and it says the recommended

269
00:13:20,759 --> 00:13:24,120
approach is to identify the outcome you would like to improve,

270
00:13:24,919 --> 00:13:29,080
measure your baseline or current state, and then I feel

271
00:13:29,080 --> 00:13:34,960
like this is where so many places start falling down.

272
00:13:35,399 --> 00:13:38,120
So the next step is to develop a hypothesis about

273
00:13:38,120 --> 00:13:40,759
what might get you to your closer state, and then

274
00:13:41,039 --> 00:13:45,000
agree to a plan for improvement, do the work, and

275
00:13:45,039 --> 00:13:47,399
then the last one measure the progress that you've made.

276
00:13:47,600 --> 00:13:50,480
I feel like that's where we fail a lot of times.

277
00:13:50,559 --> 00:13:52,639
You know, we like say, hey, we're going to make

278
00:13:52,639 --> 00:13:56,279
this better, and then we start working on it and

279
00:13:56,360 --> 00:13:59,960
never take a step back to say did we actually

280
00:14:00,080 --> 00:14:03,519
make it better? And if not, let's rip that out,

281
00:14:03,600 --> 00:14:07,639
And so over time we end up dragging all of

282
00:14:07,639 --> 00:14:12,799
these failed attempts with us adding you know, layers of

283
00:14:13,759 --> 00:14:19,919
complexity or bureaucracy or whatever the addition was, and we

284
00:14:19,960 --> 00:14:22,000
just keep dragging it along with us, even though it

285
00:14:22,039 --> 00:14:27,000
doesn't work for us or improve the situation for us.

286
00:14:27,559 --> 00:14:27,759
Speaker 2: Yeah.

287
00:14:27,799 --> 00:14:29,440
Speaker 4: No, I mean I think we bring that up a

288
00:14:29,480 --> 00:14:32,759
lot in this podcast, and I think the report really

289
00:14:32,799 --> 00:14:36,000
dives into it in the executive summary section. Which is

290
00:14:37,000 --> 00:14:41,159
an experiment is a hypothesis that is time bound with

291
00:14:42,039 --> 00:14:46,200
retrospective at some point. And often I think we do

292
00:14:46,279 --> 00:14:48,759
see a lot of organizations say, oh, we could try

293
00:14:48,799 --> 00:14:52,080
this thing, which is great, like you should absolutely experiment,

294
00:14:52,639 --> 00:14:55,320
but then there is no review, like was it even

295
00:14:55,360 --> 00:14:56,039
a good thing.

296
00:14:55,879 --> 00:14:56,480
Speaker 5: For us to do?

297
00:14:56,960 --> 00:15:00,840
Speaker 4: And that's like there is no like any experiment is

298
00:15:00,879 --> 00:15:03,480
a success, even if it rejects your hypothesis, if you

299
00:15:03,639 --> 00:15:06,320
come out with some learning. The only failed experiment is

300
00:15:06,320 --> 00:15:09,519
when you start it and never stop. And I think

301
00:15:09,519 --> 00:15:12,080
that's what a lot of organizations end up doing. They

302
00:15:12,159 --> 00:15:15,000
try something out, they experiment with a different team structure,

303
00:15:15,799 --> 00:15:18,399
even something that we believe is good. You know. I

304
00:15:18,399 --> 00:15:20,679
think it was a few years ago that the concept

305
00:15:20,840 --> 00:15:24,519
of team toypapologies came out with different types of teams

306
00:15:24,600 --> 00:15:28,679
and how they will work, stream aligned teams, platform teams,

307
00:15:29,759 --> 00:15:33,279
complex subsystem teams, etc. And it's a great idea to

308
00:15:33,320 --> 00:15:37,159
shift an organizational structure to focus on these, But even

309
00:15:37,200 --> 00:15:39,360
if it's a thing that everyone agrees on, you should

310
00:15:39,399 --> 00:15:42,080
still be like, we believe that a different org structure

311
00:15:42,080 --> 00:15:45,519
would make sense for us, and then measure why that

312
00:15:45,679 --> 00:15:47,720
is the case at some point later, like did it

313
00:15:47,759 --> 00:15:51,039
actually have an impact? Because if you don't do that,

314
00:15:51,120 --> 00:15:52,720
someone else is just going to say later, oh, that

315
00:15:52,879 --> 00:15:54,360
was bad for us, and you're not going to have

316
00:15:54,399 --> 00:15:57,679
any data to really back that up. A common argument

317
00:15:57,720 --> 00:16:03,519
against that has been the notion like rapid reteaming or

318
00:16:03,600 --> 00:16:06,200
dynamic reteaming. I think it's a book on the topic, actually,

319
00:16:06,360 --> 00:16:09,720
which is this concept ofok, like you swarm on an idea,

320
00:16:09,799 --> 00:16:11,759
you just get it to completion, and the you, you know,

321
00:16:11,879 --> 00:16:13,919
di spand the team and create a new team to

322
00:16:13,960 --> 00:16:16,600
focus on it, and without really understanding when your core

323
00:16:16,679 --> 00:16:19,759
businesses or why you're making the change, just arbitrarily making

324
00:16:19,840 --> 00:16:22,639
changes your organization, you don't know if it's going to

325
00:16:22,720 --> 00:16:23,639
be successful in the end.

326
00:16:24,120 --> 00:16:27,000
Speaker 2: So my favorite way of thinking about applying the scientific

327
00:16:27,039 --> 00:16:29,519
method is when it involves people, because the only way

328
00:16:29,519 --> 00:16:31,519
that you can actually do that is to spin off

329
00:16:31,559 --> 00:16:35,879
parallel universes and have this whole double lunch where you

330
00:16:36,000 --> 00:16:38,679
have all these simulations of different people, and then you

331
00:16:38,720 --> 00:16:40,480
have your crystal ball where you're like, well, how did

332
00:16:40,480 --> 00:16:43,440
this actually work and how many times? And then you

333
00:16:43,440 --> 00:16:45,399
have the different iterations, and then you would have different

334
00:16:45,399 --> 00:16:48,440
like branching models of the different iterations too, and since

335
00:16:48,480 --> 00:16:50,679
you can't actually do that, you just get to guess.

336
00:16:50,679 --> 00:16:54,320
But at least with technology, we could have isolated environments

337
00:16:54,360 --> 00:16:58,440
where we really applied the scientific method. But you can't

338
00:16:58,440 --> 00:17:01,559
so much with people because people are chaos.

339
00:17:02,039 --> 00:17:03,440
Speaker 5: I mean, you're absolutely right, Jillian.

340
00:17:03,480 --> 00:17:05,680
Speaker 4: I feel like most companies don't have the sort of

341
00:17:05,720 --> 00:17:07,799
scale to be able to do this, and it's a

342
00:17:07,839 --> 00:17:10,599
mistake to believe, especially if you're like selling to businesses,

343
00:17:10,720 --> 00:17:14,680
that you have the ability to segregate a clear control

344
00:17:14,759 --> 00:17:17,359
group and a test and multiple test groups to validate

345
00:17:17,400 --> 00:17:19,559
what's going on. I do remember there was a not

346
00:17:19,720 --> 00:17:23,680
great book that referenced one of the major tech companies

347
00:17:25,039 --> 00:17:28,279
in the world today, and they had talked extensively about

348
00:17:28,319 --> 00:17:32,920
horse trading, individual countries, populations, as far as users go

349
00:17:33,240 --> 00:17:38,920
to perform human psychological experiments on the information that the

350
00:17:39,079 --> 00:17:40,160
company was providing them.

351
00:17:40,160 --> 00:17:42,720
Speaker 5: To see how humans would deal with that sort of.

352
00:17:42,759 --> 00:17:46,559
Speaker 4: Information, and so like at scale, you know, there is something,

353
00:17:46,599 --> 00:17:49,039
but that's not like most of the companies, Like it's

354
00:17:49,039 --> 00:17:50,839
not my company. I don't think it's well as company.

355
00:17:50,880 --> 00:17:53,000
I'm sure it's not yours, Agilian, Like most people don't

356
00:17:53,000 --> 00:17:54,839
work at a company where this is actually a real

357
00:17:54,839 --> 00:17:57,160
thing that could be done at a human level scale.

358
00:17:58,279 --> 00:18:02,359
Speaker 2: No, I'm also uncomfortable with idea of like the human experimentation,

359
00:18:02,599 --> 00:18:08,160
Like my scenario was a lot more fun because I

360
00:18:08,160 --> 00:18:09,680
would just like to point that out there.

361
00:18:11,319 --> 00:18:14,079
Speaker 1: So what you're saying is it's okay to experiment with

362
00:18:14,200 --> 00:18:17,039
humans as long as they're in parallel universes, just not

363
00:18:17,160 --> 00:18:17,799
in this one.

364
00:18:18,599 --> 00:18:23,440
Speaker 2: Yeah, fine, I mean we agreed, that's all fine, Like

365
00:18:23,440 --> 00:18:25,759
we all like the crossover events and the Arrowverse, like

366
00:18:25,799 --> 00:18:26,559
this is what we do.

367
00:18:27,200 --> 00:18:27,960
Speaker 5: Okay, I wouldn't.

368
00:18:27,960 --> 00:18:30,559
Speaker 4: I wouldn't have mentioned the arrow Verse, but you know,

369
00:18:30,680 --> 00:18:33,920
I uh yeah, I mean I I'll just have done

370
00:18:34,119 --> 00:18:37,079
quite heavily in science fiction and fantasy, for sure. I

371
00:18:37,079 --> 00:18:39,559
Mean I always liked the question like did you did

372
00:18:39,640 --> 00:18:40,359
you enjoy.

373
00:18:40,160 --> 00:18:41,319
Speaker 5: Being an older sibling?

374
00:18:41,920 --> 00:18:43,640
Speaker 4: Because I only have younger siblings and I'm like I

375
00:18:43,640 --> 00:18:45,680
honestly don't know how to answer that question. I've never

376
00:18:45,720 --> 00:18:47,000
been a younger sibling, so I have.

377
00:18:46,960 --> 00:18:48,880
Speaker 2: To see this is what I mean. We can't spin

378
00:18:48,920 --> 00:18:51,359
you off like we can't. We can't create like another

379
00:18:51,440 --> 00:18:53,480
experiment with you as a person where then you have

380
00:18:53,640 --> 00:18:57,119
the perception of having older siblings, but it doesn't work.

381
00:18:57,960 --> 00:18:59,839
I could spend it, Listen, I have a neuroscience to

382
00:19:00,200 --> 00:19:02,240
could spend like a lot of time mark doing about this,

383
00:19:02,359 --> 00:19:02,960
but we may.

384
00:19:04,599 --> 00:19:07,440
Speaker 3: Want the actual devox portion of the show.

385
00:19:07,920 --> 00:19:10,279
Speaker 4: I mean, I'm just wondering if there's, like you're secretly

386
00:19:10,319 --> 00:19:14,640
developing some sort of medical treatment here that like would

387
00:19:14,640 --> 00:19:16,960
cause you to forget your memories and still be able

388
00:19:16,960 --> 00:19:21,079
to persist in a new hypothetical reality where you could actually.

389
00:19:22,960 --> 00:19:28,000
Speaker 1: My second time, oh that exists, Go smoke some weed,

390
00:19:34,039 --> 00:19:36,359
wake up in a room with a bunch of empty

391
00:19:36,400 --> 00:19:39,799
cereal bowls and boxes of fruity pebbles, going Wait, right.

392
00:19:43,440 --> 00:19:46,640
Speaker 5: Is memory loss associated with potsmoking?

393
00:19:47,759 --> 00:19:49,400
Speaker 1: I've heard that it is. I don't know that for

394
00:19:49,640 --> 00:19:50,079
a fact.

395
00:19:50,720 --> 00:19:52,920
Speaker 4: I mean, maybe maybe we're missing out on this prime

396
00:19:53,519 --> 00:19:59,160
experimental tool that we could be using in legitimate professional.

397
00:19:58,839 --> 00:20:02,680
Speaker 2: Applications, absolutely legitimate, I'm sure we could get lots of

398
00:20:02,720 --> 00:20:07,680
sign up for that and it'll be fine.

399
00:20:09,960 --> 00:20:11,519
Speaker 3: Well, ethical dilemmas there.

400
00:20:11,960 --> 00:20:13,920
Speaker 4: Well, what they did in my university is they used

401
00:20:13,920 --> 00:20:15,440
to run these experiments all the time.

402
00:20:15,559 --> 00:20:17,519
Speaker 5: And all they do is they say that.

403
00:20:17,440 --> 00:20:19,799
Speaker 2: You know, well, you have students, the students like care

404
00:20:19,799 --> 00:20:20,119
about that.

405
00:20:20,480 --> 00:20:23,200
Speaker 4: And you pay them, Yeah, students that you pay like

406
00:20:23,240 --> 00:20:26,960
twenty dollars per study, and they will for sure sign up,

407
00:20:27,200 --> 00:20:30,400
sign away all of their rights and do whatever unethical

408
00:20:30,440 --> 00:20:32,920
thing you want in your treatment. No co.

409
00:20:33,759 --> 00:20:34,839
Speaker 3: Yeah, that's how that goes.

410
00:20:35,480 --> 00:20:44,480
Speaker 1: Wow, I'm glad we don't get real time analytics of

411
00:20:44,519 --> 00:20:50,559
our episodes. I don't know. Maybe maybe this would be

412
00:20:50,559 --> 00:20:59,160
a maybe this would actually increase our ratings. I don't know. Meanwhile,

413
00:21:00,000 --> 00:21:00,240
don't know.

414
00:21:00,279 --> 00:21:03,160
Speaker 2: But the whole like organizational team building and all that stuff,

415
00:21:03,200 --> 00:21:06,319
all of those are very interesting to me because like,

416
00:21:06,839 --> 00:21:11,720
you can't really test it properly, and the only proper

417
00:21:11,720 --> 00:21:13,920
way to test anything is to use the scientific method,

418
00:21:14,039 --> 00:21:17,160
so you know. But then it's like, all right, so

419
00:21:17,200 --> 00:21:18,799
you can't test it properly, but what can you do

420
00:21:18,799 --> 00:21:21,119
to kind of approximate it? Like you know, obviously this

421
00:21:21,200 --> 00:21:23,440
is not black and white there, they're like ways you

422
00:21:23,440 --> 00:21:28,160
can you know, evaluate the performance of you know, of

423
00:21:28,160 --> 00:21:30,200
team organizations or any of that kind of stuff.

424
00:21:30,200 --> 00:21:31,839
Speaker 3: And for the purposes of.

425
00:21:31,880 --> 00:21:34,440
Speaker 2: This talk, we should probably you know, just just stick

426
00:21:34,480 --> 00:21:36,359
with reality of like, Okay, these are the things that

427
00:21:36,400 --> 00:21:38,240
people are doing and what are they doing. So was

428
00:21:38,279 --> 00:21:41,480
there anything in this report that discusses like, well, this

429
00:21:41,519 --> 00:21:43,319
is how your team should be organized if you have

430
00:21:43,440 --> 00:21:45,880
you have a platform team and an AI team and a.

431
00:21:47,559 --> 00:21:49,680
Speaker 5: So it definitely tries to people the team.

432
00:21:50,200 --> 00:21:52,759
Speaker 4: So I think this is one of those things where

433
00:21:52,839 --> 00:21:58,039
it's research oriented, not so much prescriptive or evaluating what happened.

434
00:21:58,079 --> 00:22:00,559
So the best it comes out with is this is

435
00:22:00,759 --> 00:22:03,359
what some people are doing, and this is the data,

436
00:22:03,680 --> 00:22:06,359
and here's us trying to maybe explain why the data

437
00:22:06,480 --> 00:22:09,319
is what it is. So there is an aspect of

438
00:22:09,480 --> 00:22:12,880
a belief that a lot of organizations are creating something

439
00:22:12,960 --> 00:22:16,440
that they're calling platform engineering or platform engineering teams.

440
00:22:16,440 --> 00:22:19,319
Speaker 5: And obviously some of those organizations just took.

441
00:22:19,160 --> 00:22:22,839
Speaker 4: Their age old infra team, didn't change anything or the

442
00:22:22,880 --> 00:22:25,720
people or the mindset and just rebrand, you know, slapped

443
00:22:25,720 --> 00:22:28,519
a new label on it and said this is a

444
00:22:28,519 --> 00:22:30,640
platform engineering team.

445
00:22:31,200 --> 00:22:33,880
Speaker 5: We're goal then. So I mean, obviously.

446
00:22:33,480 --> 00:22:38,240
Speaker 4: Those people and what they're executing on and their approach.

447
00:22:38,359 --> 00:22:40,480
It didn't change, and so they're likely going to be

448
00:22:40,519 --> 00:22:42,200
in the laggard bucket.

449
00:22:42,160 --> 00:22:43,400
Speaker 5: And low performers.

450
00:22:43,440 --> 00:22:45,359
Speaker 4: And then there's other teams that are other companies that

451
00:22:45,400 --> 00:22:48,559
really focused on how do we elevate what we're doing,

452
00:22:48,839 --> 00:22:50,359
how do we And one of the things they really

453
00:22:50,359 --> 00:22:52,160
care about is if we look back at the door metrics,

454
00:22:52,200 --> 00:22:55,680
is maybe something called developer productivity and what does that

455
00:22:55,720 --> 00:22:58,400
even mean realistically? But maybe it's the number of lines

456
00:22:58,400 --> 00:22:59,880
of code that you push out. They don't really talk

457
00:23:00,039 --> 00:23:02,759
about this, but it's how people feel. So maybe it's

458
00:23:02,759 --> 00:23:05,240
the number four requests or the number of GERA tickets

459
00:23:05,240 --> 00:23:08,119
that got completed. And no one uses Gira obviously anymore,

460
00:23:08,160 --> 00:23:10,960
so maybe they're using like linear, so linear tickets that

461
00:23:11,039 --> 00:23:14,880
got completed, and we can evaluate number of tickets that

462
00:23:14,880 --> 00:23:19,599
got completed equals velocity thereby productivity, and so did our

463
00:23:19,680 --> 00:23:22,920
organizational change that we made cause an increased number of

464
00:23:23,039 --> 00:23:27,119
tickets that get completed? And when it comes to platform engineering,

465
00:23:27,680 --> 00:23:32,160
the sort of conclusion is that it doesn't have as

466
00:23:32,160 --> 00:23:35,920
big of impact as we wanted to or believe it should.

467
00:23:36,039 --> 00:23:36,799
Speaker 5: And even in the.

468
00:23:36,720 --> 00:23:39,640
Speaker 4: Best companies it about averages out to if you have

469
00:23:39,720 --> 00:23:42,559
some sort of platform engineering team and they're doing a

470
00:23:42,599 --> 00:23:47,119
great job, it amounts to about five percent productivity increase

471
00:23:47,279 --> 00:23:52,079
per engineer. So that's five percent across your organization basically,

472
00:23:52,759 --> 00:23:55,039
and I think this is a really interesting number if

473
00:23:55,079 --> 00:23:58,000
you think a lot about it, because it means that

474
00:23:58,279 --> 00:24:02,400
if you have twenty engineers, you can hire one person

475
00:24:02,720 --> 00:24:06,559
to power your platform engineering team, because if you hire

476
00:24:06,880 --> 00:24:09,240
more than one person, then you're not going to be

477
00:24:09,240 --> 00:24:12,559
able to increase the productivity of those engineers any more

478
00:24:12,559 --> 00:24:15,240
than that. The five percent times twenty is one hundred

479
00:24:15,440 --> 00:24:19,119
percent of the amount of work that could possibly be done.

480
00:24:19,279 --> 00:24:20,680
And of course you don't want to have a team

481
00:24:20,720 --> 00:24:23,200
of one person. You probably have a team of four people.

482
00:24:23,559 --> 00:24:26,559
So realistically, if we just look at the numbers, having

483
00:24:26,680 --> 00:24:30,039
a platform engineering team before you have one hundred person

484
00:24:30,119 --> 00:24:34,400
organization means that you are spending too much time focusing

485
00:24:34,480 --> 00:24:36,720
on whatever the platform engineering.

486
00:24:36,440 --> 00:24:39,000
Speaker 5: Team is doing and optimizing for things that the.

487
00:24:39,039 --> 00:24:43,960
Speaker 4: Actual who your users are, the engineers, developers, product engineers

488
00:24:44,759 --> 00:24:47,079
don't get the benefit out of it. You can't increase

489
00:24:47,079 --> 00:24:48,000
the productctivity.

490
00:24:48,039 --> 00:24:50,480
Speaker 1: Further, Yeah, I would agree with that because I think

491
00:24:50,599 --> 00:24:53,359
in those like the scenario were describing. You know, before

492
00:24:53,400 --> 00:24:57,640
you have one hundred engineers. I think for most companies

493
00:24:57,680 --> 00:25:00,799
in my experience, before that, you don't even really know

494
00:25:01,119 --> 00:25:05,079
what you're doing as a company. So it's it's hard

495
00:25:05,200 --> 00:25:11,559
to optimize for something when you don't know what that

496
00:25:11,640 --> 00:25:12,200
something is.

497
00:25:13,079 --> 00:25:14,880
Speaker 5: Yeah, for sure, Julian, you're gonna say something.

498
00:25:15,279 --> 00:25:16,640
Speaker 2: Oh, I was just going to say, you were talking

499
00:25:16,680 --> 00:25:20,160
about the difference between platform engineering teams, which is kind

500
00:25:20,160 --> 00:25:23,960
of a I would say, maybe like a newer phrase,

501
00:25:24,160 --> 00:25:26,519
and infrastructure teams, and I was just wondering, what do

502
00:25:26,559 --> 00:25:27,759
you what do you kind of see is the difference

503
00:25:27,799 --> 00:25:28,759
between the two of those.

504
00:25:29,319 --> 00:25:31,079
Speaker 4: I mean, this is this is where like you know,

505
00:25:31,119 --> 00:25:34,599
it's because well, I mean no, I mean, I think

506
00:25:34,599 --> 00:25:36,440
it's a really great question because at least from my

507
00:25:38,440 --> 00:25:40,400
engineering experience, and I feel like I'm a little bit

508
00:25:40,480 --> 00:25:43,720
unique here because when I started, some of the things

509
00:25:43,759 --> 00:25:48,000
I was doing, I would definitely say came into effect

510
00:25:48,039 --> 00:25:52,400
because of the mindset of DevOps was growing up now

511
00:25:52,440 --> 00:25:55,960
getting out of it seed stages, and the work I

512
00:25:56,119 --> 00:25:59,839
was doing was specifically to eliminate teams in the organization

513
00:26:00,200 --> 00:26:06,319
that we're called platform engineering teams, and now almost twenty

514
00:26:06,400 --> 00:26:10,319
years later, we're now creating platform engineering teams and calling

515
00:26:10,359 --> 00:26:13,440
the platform engineers because I think everyone no one remembered

516
00:26:13,480 --> 00:26:16,720
that this term had been used previously, and they're doing

517
00:26:16,720 --> 00:26:17,400
similar things.

518
00:26:17,400 --> 00:26:19,440
Speaker 5: But we've automated a lot of.

519
00:26:19,400 --> 00:26:23,359
Speaker 4: Whey so much to the point of where we feel

520
00:26:23,400 --> 00:26:26,720
like the job now could exist again, but doing different things.

521
00:26:26,920 --> 00:26:29,000
So when I say infrastructure, what I had meant was

522
00:26:29,079 --> 00:26:32,880
is maybe writing some scripts to automate the installation of

523
00:26:32,920 --> 00:26:38,880
some software on bare metal racks for our servers that

524
00:26:38,920 --> 00:26:42,519
you got installed. Often they may even be responsible for

525
00:26:42,640 --> 00:26:45,640
setting installing the blades and putting together the server blades

526
00:26:45,680 --> 00:26:48,119
in the data center, but for sure writing the scripts

527
00:26:48,319 --> 00:26:51,400
to manage that and maybe send alerts and whatnot. And

528
00:26:51,480 --> 00:26:55,720
today I doubt any platform engineering team is doing that work.

529
00:26:55,759 --> 00:26:58,839
I mean, even when they have an on prem solution

530
00:26:58,960 --> 00:27:01,519
and an on prem data center, they're likely not even

531
00:27:01,559 --> 00:27:05,519
building the blades. You're outsourcing data center creation. Even if

532
00:27:05,559 --> 00:27:08,000
you have a physical onpreme data center, you are relying

533
00:27:08,000 --> 00:27:10,440
on these third party companies that come in build it

534
00:27:10,480 --> 00:27:13,480
for you, of which then you're likely not even installing

535
00:27:13,480 --> 00:27:16,359
the base software, but then you're installing some sort of

536
00:27:16,640 --> 00:27:19,319
native system on top of that, either at Kubernetes.

537
00:27:18,839 --> 00:27:20,039
Speaker 5: Or Coros or something like that.

538
00:27:21,240 --> 00:27:25,319
Speaker 1: I think I think the end product for both of

539
00:27:25,359 --> 00:27:30,279
those teams is the same, you know, to create a

540
00:27:30,839 --> 00:27:34,559
place for the engineering teams you support to run their applications.

541
00:27:34,559 --> 00:27:37,880
I think the big difference between the two is how

542
00:27:37,920 --> 00:27:41,920
they accomplish that goal or as an infrastructure engineer will

543
00:27:42,720 --> 00:27:50,480
be more hands on and more tightly interfaced with the

544
00:27:50,480 --> 00:27:57,279
infrastructure components themselves versus a platform engineering team. It abstracts

545
00:27:57,319 --> 00:28:03,240
themselves away from those those complexities and will rely on

546
00:28:04,079 --> 00:28:08,400
like either one or more platform engineering style tools that

547
00:28:08,799 --> 00:28:11,839
automate or abstract those tasks away from them. So I

548
00:28:11,880 --> 00:28:13,920
think I think the end product is the same as

549
00:28:13,960 --> 00:28:16,519
the approach of how you build that end product distinguishes

550
00:28:16,559 --> 00:28:17,440
the two teams.

551
00:28:17,880 --> 00:28:18,519
Speaker 5: Yeah, for sure.

552
00:28:18,599 --> 00:28:20,920
Speaker 4: I mean I think my CEO and I were talking

553
00:28:20,920 --> 00:28:23,200
about this recently and she brought up there's like a

554
00:28:23,240 --> 00:28:26,720
lot of different other things here that probably aren't necessarily

555
00:28:26,920 --> 00:28:30,359
normally associated with it, but things like building golden paths

556
00:28:30,440 --> 00:28:34,519
of sets of frameworks and tech stacks that we decide

557
00:28:34,519 --> 00:28:37,079
to use, but you know, someone has to do the

558
00:28:37,200 --> 00:28:40,400
difficult work of actually configuring it. Because these things are

559
00:28:40,599 --> 00:28:44,119
too unopinionated about what you're utilizing is one of them.

560
00:28:44,799 --> 00:28:48,039
Maybe you're utilizing some sort of shared digital or software

561
00:28:48,119 --> 00:28:51,279
driven infrastructure, infrastructure as code and coming up with those

562
00:28:51,559 --> 00:28:54,759
common components that go together would be another one. Sometimes

563
00:28:54,799 --> 00:28:58,839
there's an integration with IT system so SSO based integration

564
00:28:59,039 --> 00:29:02,640
from your IT provider, whatever it is in Google Workspace,

565
00:29:02,720 --> 00:29:05,319
et cetera, to power as SO in your organization, and

566
00:29:05,319 --> 00:29:07,880
the integration between other apps you have, or the management

567
00:29:07,880 --> 00:29:11,279
of the cloud accounts if you're using AWS or another

568
00:29:11,279 --> 00:29:14,640
cloud provider. And obviously those things didn't exist twenty years ago.

569
00:29:15,039 --> 00:29:18,119
Speaker 2: So I'm not opposed to using like the newest buzzword,

570
00:29:18,240 --> 00:29:20,400
mostly because I do that every couple of years on

571
00:29:20,480 --> 00:29:23,160
my LinkedIn to get myself a raised Like that's all

572
00:29:23,200 --> 00:29:26,000
fine with me, But I specifically don't like the word

573
00:29:26,039 --> 00:29:28,839
platform because it's just it's too general. And then I'm

574
00:29:28,839 --> 00:29:32,319
in you know, these meetings where you have like the scientists,

575
00:29:32,319 --> 00:29:34,960
and then you have like the infrastructure people, and then

576
00:29:35,000 --> 00:29:37,440
you have like the drug discovery, and you have all

577
00:29:37,440 --> 00:29:39,599
these different groups and they all use the word platform

578
00:29:39,640 --> 00:29:42,160
and it means something completely different to each group, and

579
00:29:42,200 --> 00:29:44,720
then everybody's just having their own conversations, and they don't

580
00:29:44,720 --> 00:29:48,000
seem to realize that there's like four different conversations going

581
00:29:48,039 --> 00:29:50,319
on because we're all using the same word, but it

582
00:29:50,359 --> 00:29:54,799
does not mean the same thing. That's my rant about platform,

583
00:29:55,000 --> 00:29:57,680
which I suppose doesn't maybe have a whole lot to

584
00:29:57,680 --> 00:29:59,200
do with the Dora report, but I don't like it.

585
00:29:59,200 --> 00:30:00,440
Speaker 3: It's too general a word.

586
00:30:01,599 --> 00:30:01,880
Speaker 2: I think.

587
00:30:01,920 --> 00:30:04,839
Speaker 5: I think you're you're totally on a good point there.

588
00:30:04,880 --> 00:30:07,279
Speaker 4: Actually, So last year I ran a seminar at one

589
00:30:07,279 --> 00:30:10,480
of the DevOps Days conferences trying to define actually what

590
00:30:10,559 --> 00:30:14,920
a platform is, and after an hour talking on the subject,

591
00:30:14,960 --> 00:30:16,759
the best we were able to come up with is

592
00:30:17,160 --> 00:30:22,920
it's something horizontal that supports something else, which.

593
00:30:22,759 --> 00:30:23,039
Speaker 1: Is just.

594
00:30:25,799 --> 00:30:27,720
Speaker 2: The beams of my house.

595
00:30:27,759 --> 00:30:29,799
Speaker 3: That's it.

596
00:30:29,839 --> 00:30:31,200
Speaker 5: Isn't that the foundation though?

597
00:30:31,759 --> 00:30:34,160
Speaker 3: So I mean, I don't know if the analogy broke

598
00:30:34,279 --> 00:30:37,160
down what you're talking about.

599
00:30:37,880 --> 00:30:39,200
Speaker 5: Well, we we got platform.

600
00:30:39,640 --> 00:30:41,400
Speaker 4: I think the best analogy we had is if you

601
00:30:41,440 --> 00:30:45,160
think about the gates for getting on a train, like

602
00:30:45,200 --> 00:30:47,319
the plat those are the platforms. You stand on the

603
00:30:47,359 --> 00:30:50,759
platform to get to the height of the entrance to

604
00:30:50,799 --> 00:30:53,279
the train. So the train if you had to board

605
00:30:53,319 --> 00:30:56,599
it from the ground layer or floor you know whatever

606
00:30:56,680 --> 00:30:59,319
from the earth. Uh, it's too far away. It's not

607
00:30:59,440 --> 00:31:02,200
well designed from that regard, and so the platform lifts

608
00:31:02,200 --> 00:31:05,400
you up to be able to utilize that technology as

609
00:31:05,440 --> 00:31:09,240
efficiently as possible. And what it doesn't handle is you

610
00:31:09,319 --> 00:31:12,079
think about that as a platform, it's very difficult to

611
00:31:12,240 --> 00:31:16,680
raise up that concrete slab or lower it. If the

612
00:31:16,680 --> 00:31:18,640
height of the train changes, or if the types of

613
00:31:18,680 --> 00:31:20,519
the trains change, or if the train is in a

614
00:31:20,519 --> 00:31:22,720
different location, you may have to extend the platform.

615
00:31:23,119 --> 00:31:25,559
Speaker 5: And so this is I think Will sort of brought

616
00:31:25,559 --> 00:31:25,960
this up.

617
00:31:26,599 --> 00:31:28,680
Speaker 4: One of the things that's actually highlighting the report is

618
00:31:28,720 --> 00:31:34,160
that platform engineering teams, what they build at first, tackles

619
00:31:34,200 --> 00:31:36,519
a lot of the low hanging fruit, and that's things

620
00:31:36,519 --> 00:31:38,680
that are easy, that have a lot of high return

621
00:31:38,720 --> 00:31:41,799
on investment that the engineering teams may be struggling with. Right,

622
00:31:41,839 --> 00:31:43,839
you see the same problem a bunch of times, and

623
00:31:43,839 --> 00:31:46,119
then you develop a solution to help solve them. Maybe

624
00:31:46,119 --> 00:31:51,240
it's a script or a bunch of open TOFU files

625
00:31:51,279 --> 00:31:55,400
that have the same coupled infrastructure together. But since technology

626
00:31:55,440 --> 00:31:58,720
is not static, the needs of the team slightly change

627
00:31:58,720 --> 00:32:01,279
over time, and so whatever you built is now out

628
00:32:01,319 --> 00:32:04,559
of date. It now is actually causing a cost and

629
00:32:04,599 --> 00:32:06,839
a burden to the teams to continue to use it

630
00:32:06,960 --> 00:32:10,720
because it's too opinionated, because it's a wrapper on top

631
00:32:10,759 --> 00:32:13,200
of what was actually available in the world before, and

632
00:32:13,279 --> 00:32:18,000
so actually platform teams in a one to three year timeframe,

633
00:32:18,359 --> 00:32:24,279
cost usually if not well maintained overall, actually hinder engineering

634
00:32:24,359 --> 00:32:28,599
teams for being able to deliver stuff effectively, and then

635
00:32:28,680 --> 00:32:31,039
until they realize, oh, we're actually getting in the way.

636
00:32:31,039 --> 00:32:33,880
We need to build a real product to actually deliver effectively,

637
00:32:34,200 --> 00:32:37,240
and then turn that around and extend the platform and

638
00:32:37,319 --> 00:32:41,440
improve it over time until it reaches this more effective

639
00:32:41,519 --> 00:32:44,559
long term vision of being more agnostic and being able

640
00:32:44,599 --> 00:32:49,759
to just like raise up very small amount but consistently

641
00:32:49,839 --> 00:32:54,359
across for whatever their user group is seems reasonable.

642
00:32:54,759 --> 00:32:56,200
Speaker 3: Yeah, it seems like a good way to at least

643
00:32:56,200 --> 00:32:56,799
reason about it.

644
00:32:58,680 --> 00:33:00,480
Speaker 2: If we're going to try to set up mental models

645
00:33:00,519 --> 00:33:03,039
and not going to use the word I almost use

646
00:33:03,079 --> 00:33:05,400
the word frameworks, but trying to not anymore.

647
00:33:09,440 --> 00:33:12,200
Speaker 1: I'll take overloaded buzzwords for one thousand anlycs.

648
00:33:12,480 --> 00:33:15,359
Speaker 4: Yeah, so, I mean this actually came up in one

649
00:33:15,359 --> 00:33:18,039
of the communities I'm in. We're trying to define what

650
00:33:18,079 --> 00:33:21,079
the difference between code and configuration.

651
00:33:20,720 --> 00:33:25,200
Speaker 3: Is and it's a very blurry line now for sure.

652
00:33:25,359 --> 00:33:30,440
Speaker 4: Right, And so my point was exactly as yours was, Jillian,

653
00:33:30,480 --> 00:33:32,920
which is like, it's just a word that helps convey

654
00:33:33,079 --> 00:33:35,519
some sort of point that we're trying to make to

655
00:33:35,720 --> 00:33:38,119
help get us from where we are to somewhere else.

656
00:33:38,160 --> 00:33:41,200
We're just using the vocabulary we have to try to

657
00:33:41,240 --> 00:33:44,839
convey our thoughts and the most conducive way we know.

658
00:33:44,960 --> 00:33:47,240
And of course different people understand those things to be

659
00:33:47,240 --> 00:33:51,440
different and apparently also have very strong opinions on what

660
00:33:51,519 --> 00:33:54,160
those things actually do mean. And if you try to

661
00:33:54,240 --> 00:33:58,680
challenge it, like you know, doesn't matter, they do sometimes.

662
00:33:59,599 --> 00:34:01,680
Speaker 5: Not too happy about challenge.

663
00:34:01,759 --> 00:34:04,519
Speaker 4: You know, they associate themselves with who they are based

664
00:34:04,559 --> 00:34:06,079
off of the tools they're using. And if you said, oh,

665
00:34:06,079 --> 00:34:10,599
you're just a configurator, you know, that may drive a

666
00:34:10,679 --> 00:34:14,199
stake in some people's you know, personal perspective of themselves.

667
00:34:14,679 --> 00:34:18,480
Speaker 1: Yeah, for sure, that's that could go very deeply into

668
00:34:18,519 --> 00:34:22,480
a whole other topic. But a lot of people in

669
00:34:22,519 --> 00:34:27,519
our industry, they do tie their professional identity to the tools,

670
00:34:27,519 --> 00:34:32,639
like I'm a terrorform person or or I'm an antiable person,

671
00:34:32,880 --> 00:34:36,559
or i use AWS and it's one of the things

672
00:34:36,599 --> 00:34:41,719
I try to share with people whenever I talk to

673
00:34:41,760 --> 00:34:46,239
them about their careers is don't tie yourself to that technology.

674
00:34:46,320 --> 00:34:49,760
Tie yourself to the problems that you're solving, because in

675
00:34:49,800 --> 00:34:55,159
the course of your career, those technologies are going to change,

676
00:34:56,000 --> 00:34:58,119
but the problems you solve will be the same. And

677
00:34:58,159 --> 00:35:02,719
it's a very subtle but effective trick to making sure

678
00:35:02,800 --> 00:35:08,840
that you stay relevant for additional career options over the years.

679
00:35:09,760 --> 00:35:13,639
Speaker 2: That's why my title is not Biopearl software developer anymore.

680
00:35:15,280 --> 00:35:16,639
I don't know what it is right now, but it's

681
00:35:16,639 --> 00:35:22,159
something that it's not biopearl. It's not catalysts.

682
00:35:22,719 --> 00:35:23,079
Speaker 3: I don't know.

683
00:35:23,079 --> 00:35:24,960
Speaker 2: I don't even remember what any of the old biogragmatics

684
00:35:25,000 --> 00:35:26,880
frameworks now, but they're all pearl based.

685
00:35:27,880 --> 00:35:29,559
Speaker 1: Nice. How could you let that go?

686
00:35:30,159 --> 00:35:30,679
Speaker 2: I don't.

687
00:35:32,519 --> 00:35:33,679
Speaker 3: I don't know. I don't know.

688
00:35:34,039 --> 00:35:36,800
Speaker 2: Yeah, I would think most people will kind of. I

689
00:35:36,800 --> 00:35:39,280
guess if you're talking to like newer professionals, I could

690
00:35:39,320 --> 00:35:40,599
see how that could be a problem.

691
00:35:41,480 --> 00:35:41,840
Speaker 3: I don't know.

692
00:35:41,880 --> 00:35:44,360
Speaker 2: I don't think people in the field for a long

693
00:35:44,360 --> 00:35:47,239
time would associate themselves with the particular tool, but I

694
00:35:47,239 --> 00:35:49,320
could be wrong about that.

695
00:35:49,320 --> 00:35:53,119
Speaker 1: That Pearl is one of those, like the old school.

696
00:35:52,880 --> 00:35:56,440
Speaker 3: Pearl for like a long long time.

697
00:35:56,800 --> 00:35:59,199
Speaker 1: Oh yeah, and they'll still argue with you today about

698
00:35:59,280 --> 00:36:02,480
how great it is is and how humanity is doomed

699
00:36:02,480 --> 00:36:04,719
because we've moved on to something else.

700
00:36:06,960 --> 00:36:09,400
Speaker 2: That was the first programming class I actually ever took.

701
00:36:10,119 --> 00:36:11,960
I had no idea what they were talking about, and

702
00:36:12,000 --> 00:36:14,880
the professor just launched into this whole Pearl versus Python thing,

703
00:36:14,960 --> 00:36:17,079
and I was like, I just want to program some robots,

704
00:36:17,159 --> 00:36:20,000
Like I don't even know what you're talking about, but

705
00:36:20,039 --> 00:36:20,719
it stuck with me.

706
00:36:20,880 --> 00:36:22,599
Speaker 3: You know. Twenty something years later, here we are.

707
00:36:23,280 --> 00:36:25,000
Speaker 1: It's worring. You were going to chime in on Pearl.

708
00:36:25,719 --> 00:36:28,679
Speaker 5: Oh well, there is a joke there.

709
00:36:28,760 --> 00:36:35,400
Speaker 4: There's like some OCR that converts artwork into text, and

710
00:36:35,519 --> 00:36:37,719
it was like a terrible OCR program, like it did

711
00:36:37,719 --> 00:36:39,559
not do a very good job at all, but if

712
00:36:39,599 --> 00:36:43,320
you threw an image at it, some famous artwork, and

713
00:36:43,719 --> 00:36:46,639
it would generate just arbitrary tax which was like periods

714
00:36:46,639 --> 00:36:50,400
and semicolons and dashes and ad signs and every literally

715
00:36:50,440 --> 00:36:53,000
everything had generated was a valid Pearl program.

716
00:36:55,320 --> 00:36:58,880
Speaker 2: It's great gobbledygook, but Pearl will take it.

717
00:36:59,199 --> 00:37:02,119
Speaker 1: Pearl is the honey Badger of programming languages.

718
00:37:04,320 --> 00:37:06,440
Speaker 2: I still check in on like the Pearl six News

719
00:37:06,440 --> 00:37:10,760
sometimes and I'm like, ah, such hope, such hope, and

720
00:37:10,840 --> 00:37:13,400
such despair and like one small space on the internet,

721
00:37:13,639 --> 00:37:14,519
it's fascinating.

722
00:37:15,880 --> 00:37:22,159
Speaker 1: We'll get that when we get half left three. So

723
00:37:22,239 --> 00:37:27,679
let's talk about artificial intelligence and how that factored into

724
00:37:27,719 --> 00:37:29,000
the Dora metrics this year.

725
00:37:29,000 --> 00:37:32,400
Speaker 2: Because oh so excited for this part, I get to

726
00:37:32,400 --> 00:37:34,960
disagree with the whole thing, the whole one.

727
00:37:35,159 --> 00:37:41,000
Speaker 1: I read, well, never let that lack of knowledge be

728
00:37:41,079 --> 00:37:44,119
a reason that you can't argue a point right.

729
00:37:46,599 --> 00:37:49,320
Speaker 2: Lack of knowledge never form my opinions.

730
00:37:50,280 --> 00:37:54,639
Speaker 1: So in looking at it, this is a card graph

731
00:37:54,679 --> 00:37:57,599
to read, I found the graphs to be challenging.

732
00:37:57,639 --> 00:38:00,440
Speaker 4: So anyone who's actually looking at the report, what you

733
00:38:00,480 --> 00:38:02,599
have to remember is that at the bottom access it's

734
00:38:02,639 --> 00:38:05,800
often like zero point one point two point three, And

735
00:38:05,880 --> 00:38:08,440
what it really means is percentage of the total population

736
00:38:08,760 --> 00:38:13,800
or the actual percentage increase overall comparating the options. So

737
00:38:14,199 --> 00:38:17,440
really it's just often ten to twenty percent for the

738
00:38:17,480 --> 00:38:18,239
different areas.

739
00:38:18,280 --> 00:38:20,039
Speaker 5: There's the elites are you.

740
00:38:20,039 --> 00:38:23,079
Speaker 4: Know, somewhere around ten percent, and the lowest minority is

741
00:38:23,119 --> 00:38:25,800
also ten percent and is about twenty to thirty for

742
00:38:25,840 --> 00:38:27,360
the mid pieces. I don't know if that's the grapher

743
00:38:27,440 --> 00:38:30,960
looking at well, but I found personally the UX on

744
00:38:31,000 --> 00:38:32,159
the graphs not to be the best.

745
00:38:32,760 --> 00:38:35,599
Speaker 1: Yeah, that's that's very confusing. So I'm going to go

746
00:38:35,599 --> 00:38:38,760
with the text up above saying that eighty one percent

747
00:38:38,840 --> 00:38:45,079
reported they're shifting their priorities to increase incorporating AI into

748
00:38:45,119 --> 00:38:51,639
their applications. What's not clear is, well, I guess given

749
00:38:51,679 --> 00:38:54,639
the scope of the audience, we can assume that eighty

750
00:38:54,639 --> 00:38:59,880
one percent of DevOps professionals are incorporating AI into the

751
00:39:00,199 --> 00:39:01,480
develop services.

752
00:39:02,079 --> 00:39:03,599
Speaker 5: I think it's actually businesses.

753
00:39:04,159 --> 00:39:07,960
Speaker 4: Is the total population that they're talking about here, Okay,

754
00:39:08,000 --> 00:39:10,760
eighty one percent of businesses are trying to incorporate AI

755
00:39:10,880 --> 00:39:14,079
into the final product that they're utilizing. I think there's

756
00:39:14,119 --> 00:39:17,599
also a different metric for the amount that are trying

757
00:39:17,599 --> 00:39:19,599
to utilize it as a tool to do some sort

758
00:39:19,639 --> 00:39:23,840
of software development. But I'm not sure that's totally highlighted there.

759
00:39:23,880 --> 00:39:26,239
I don't know which one the eighty one percent was.

760
00:39:26,639 --> 00:39:29,960
I'm so totally curious what the thing was that Jillian

761
00:39:30,039 --> 00:39:33,519
found that also Viamilly disagreed with I am just waiting

762
00:39:33,559 --> 00:39:34,440
in suspense here.

763
00:39:34,960 --> 00:39:35,840
Speaker 1: So well, so now.

764
00:39:36,000 --> 00:39:38,360
Speaker 3: I'm looking at the report and I can't find the sentence.

765
00:39:38,360 --> 00:39:40,440
Speaker 2: But I thought that it was something like, you know,

766
00:39:40,559 --> 00:39:44,719
AI adoption was kind of less than predicted, or AI

767
00:39:44,800 --> 00:39:47,199
significance was less than predicted, And I found that to

768
00:39:47,199 --> 00:39:51,960
be completely opposite, especially in drug discovery companies, where everybody

769
00:39:52,000 --> 00:39:53,920
was like, yeah, whatever, Like we've seen these before, right,

770
00:39:53,960 --> 00:39:56,880
Like Microsofts came out with their AI that was for

771
00:39:57,000 --> 00:39:59,599
healthcare a few years ago that was supposed to revolutionize everything,

772
00:39:59,639 --> 00:40:02,719
and it revolutionize like nothing pretty much, and so on

773
00:40:02,800 --> 00:40:05,800
and so forth, and that has absolutely not been the

774
00:40:05,840 --> 00:40:07,920
case with these kind of with the newer models, with

775
00:40:09,000 --> 00:40:11,840
the chat ChiPT the cloud ones haven't gotten to try

776
00:40:11,880 --> 00:40:16,519
out the new reasoning ones yet. But in particular, the

777
00:40:16,559 --> 00:40:20,199
AI seems to have this kind of scary good knowledge

778
00:40:20,239 --> 00:40:23,559
of how chemistry works. And it's great because my brain

779
00:40:23,639 --> 00:40:27,159
is like equally divided in half on this opinion where

780
00:40:27,199 --> 00:40:29,599
it's like, well, the AI sort of understands language and

781
00:40:29,719 --> 00:40:32,599
human language and that's what it's trained on, and our

782
00:40:32,639 --> 00:40:37,480
representation of chemistry is also humans creating language to represent something.

783
00:40:37,519 --> 00:40:39,800
So this also makes sense, but it shouldn't it shouldn't

784
00:40:39,800 --> 00:40:41,360
make sense the way that it does. And like the

785
00:40:41,400 --> 00:40:43,920
results that you get, everybody you know, just looks at

786
00:40:43,960 --> 00:40:46,519
and says this should like this should not work. This

787
00:40:46,559 --> 00:40:49,599
is so you know, this is so scary good what

788
00:40:49,920 --> 00:40:51,760
the AI can do where you give it like a

789
00:40:51,800 --> 00:40:56,159
string representation of these compounds or these drugs or whatever,

790
00:40:56,400 --> 00:40:58,360
and you can take it and you can like start

791
00:40:58,400 --> 00:41:01,519
asking the AI to generate you Knew drugs, Like you

792
00:41:01,559 --> 00:41:04,039
can say like, hey, here's you know, I ran this

793
00:41:04,119 --> 00:41:06,920
experiment and here's all of these compounds that docked against

794
00:41:06,960 --> 00:41:10,360
this other compound. Make me, you know, make me fifty

795
00:41:10,400 --> 00:41:13,480
new ones that are going to do like you know,

796
00:41:13,519 --> 00:41:17,000
that are gonna be comparative or something like that. And

797
00:41:17,039 --> 00:41:19,599
then you can also with these bigger context windows, you

798
00:41:19,599 --> 00:41:22,760
can throw like all kinds of public data sets at it.

799
00:41:22,760 --> 00:41:25,000
You can take open targets, you can take the therapeutic

800
00:41:25,320 --> 00:41:27,639
like there's just a crazy amount of data that you

801
00:41:27,679 --> 00:41:30,239
can incorporate into it. And it seems to be doing really,

802
00:41:30,280 --> 00:41:33,840
really well. And so that's my very narrow view of AI.

803
00:41:33,920 --> 00:41:35,800
That is the only thing that I actually care about,

804
00:41:35,800 --> 00:41:37,880
which has nothing to do with this report. But I

805
00:41:37,920 --> 00:41:41,079
found the adoption to be much greater than was expected

806
00:41:41,119 --> 00:41:43,480
across the board, Like I had somebody say like, oh,

807
00:41:43,519 --> 00:41:44,800
you know, it was just this toy that I was

808
00:41:44,800 --> 00:41:46,800
playing with a couple of days ago, and now it's

809
00:41:46,840 --> 00:41:47,519
my whole job.

810
00:41:48,280 --> 00:41:50,000
Speaker 3: So I just find this very interesting.

811
00:41:50,559 --> 00:41:52,920
Speaker 5: I think that what is going to.

812
00:41:52,840 --> 00:41:54,559
Speaker 2: Be making our medications and I don't know how I

813
00:41:54,559 --> 00:41:57,239
feel about that it's coming.

814
00:41:57,840 --> 00:42:00,599
Speaker 4: I mean there is like I always had this perspective

815
00:42:00,639 --> 00:42:03,800
of humans make mistakes, and I think we forget that

816
00:42:04,239 --> 00:42:07,079
and we assume that if we design an AI process

817
00:42:07,280 --> 00:42:09,960
to handle something, that it has to be perfect and

818
00:42:10,079 --> 00:42:13,039
realistically it just has to be equal or better than

819
00:42:13,119 --> 00:42:15,119
what humans are doing, or even worse if there are

820
00:42:15,159 --> 00:42:19,840
some other trade offs. And often when we evaluate a process,

821
00:42:19,960 --> 00:42:23,079
there's a very common aspect for engineers who say, oh,

822
00:42:23,119 --> 00:42:25,320
we're looking at the code or the output and it's

823
00:42:25,360 --> 00:42:27,280
not actually one hundred percent.

824
00:42:27,039 --> 00:42:28,840
Speaker 5: Correct, so we should reject it.

825
00:42:28,880 --> 00:42:32,280
Speaker 4: And I don't think that's necessarily the right mentality to

826
00:42:32,360 --> 00:42:35,639
have here. But what you're messaging you're talking about, Jillian,

827
00:42:35,840 --> 00:42:38,719
I totally agree with. I think outside of the software

828
00:42:38,719 --> 00:42:43,760
engineering space, the usage in built in product that are

829
00:42:43,760 --> 00:42:47,519
being built or the end product can be hugely valuable.

830
00:42:49,039 --> 00:42:50,039
Speaker 5: Almost everywhere.

831
00:42:50,280 --> 00:42:52,840
Speaker 4: I think, as far as the report is concerned, we

832
00:42:52,880 --> 00:42:55,639
should look at the impact of the product the company

833
00:42:55,679 --> 00:43:00,320
is creating versus the adoption by the software engineering teams,

834
00:43:00,679 --> 00:43:02,840
and so I think overall, what we are seeing and

835
00:43:02,920 --> 00:43:06,840
what the report says is that everyone's optimistic or most

836
00:43:06,880 --> 00:43:10,679
people are optimistic about the usage of AI for doing

837
00:43:10,679 --> 00:43:13,519
the development, doing software development, using it in the teams.

838
00:43:13,920 --> 00:43:22,159
But when it comes to the actual product delivery, success, performance, durability, reliability,

839
00:43:22,239 --> 00:43:26,639
that actually decreases with the usage of AI. And I

840
00:43:26,719 --> 00:43:30,840
think this actually makes a lot of sense where we're

841
00:43:31,000 --> 00:43:38,239
employing very knowledgeable, inexperienced engineers, a lot of them to

842
00:43:38,519 --> 00:43:41,440
build us our products, and so of course we can.

843
00:43:41,360 --> 00:43:44,000
Speaker 5: Do it faster, but the outcome of what we get

844
00:43:44,199 --> 00:43:46,119
is going to be worse, and it's.

845
00:43:45,920 --> 00:43:47,679
Speaker 4: Going to be the case for I think a very

846
00:43:47,679 --> 00:43:52,639
long time unless we overcome the barrier of making truly

847
00:43:52,840 --> 00:43:56,920
more intelligent lms, and the nature of lms, I think

848
00:43:57,000 --> 00:43:59,079
is incredibly limited there in.

849
00:43:58,960 --> 00:44:02,760
Speaker 2: That regard, I think the lms a little bit they're

850
00:44:02,760 --> 00:44:05,760
going to have the same problem specifically for writing code

851
00:44:05,800 --> 00:44:07,840
that the package managers do, because when I've tried AI

852
00:44:07,960 --> 00:44:11,119
is out for writing code it like it mostly works,

853
00:44:11,119 --> 00:44:16,360
but it'll mix in different syntax versions of different of

854
00:44:16,400 --> 00:44:18,920
like different programs. So like if you're using like PANDAS

855
00:44:18,920 --> 00:44:21,519
for example, that's kind of a common data science library,

856
00:44:21,679 --> 00:44:23,760
and they like to change their syntax quite a bit,

857
00:44:24,000 --> 00:44:26,280
so you'll get like different you know, different changes from

858
00:44:26,280 --> 00:44:28,239
different versions and things like that. So I think that'll

859
00:44:28,280 --> 00:44:30,400
be kind of the next I see that as sort

860
00:44:30,400 --> 00:44:31,960
of the next big change that will have to come

861
00:44:31,960 --> 00:44:34,679
along for the coding AIS to really take off.

862
00:44:35,519 --> 00:44:38,480
Speaker 4: Yeah, I mean what I've seen is the is sort

863
00:44:38,519 --> 00:44:41,800
of two shotting the output where you keep a collective

864
00:44:41,920 --> 00:44:44,559
dock of some of the learnings that you want to

865
00:44:44,599 --> 00:44:48,199
have passed into every code generation you do. So, like,

866
00:44:48,880 --> 00:44:51,119
if you know there are differences in PANDACE, what you

867
00:44:51,119 --> 00:44:53,239
should do is have a doc that says and you

868
00:44:53,280 --> 00:44:56,480
can use your package manager as the source, like you know,

869
00:44:56,559 --> 00:45:00,719
only give me output that matches recommendation based off of

870
00:45:00,719 --> 00:45:03,159
the versions that I have or specifically say Panda's you

871
00:45:03,159 --> 00:45:05,880
know whatever, the version one point three five, and then

872
00:45:06,079 --> 00:45:08,480
it will help to make sure that it generates output

873
00:45:08,559 --> 00:45:10,880
that matches this index from that version. And if you

874
00:45:10,880 --> 00:45:12,840
don't give that, then of course it's going to pull

875
00:45:12,880 --> 00:45:15,519
out the most relevant thing based off of the problem

876
00:45:15,519 --> 00:45:17,800
that you've asked, which may not be in the version

877
00:45:17,800 --> 00:45:18,679
that you're utilizing.

878
00:45:19,320 --> 00:45:20,920
Speaker 2: Well, I think that's going to be interesting with the

879
00:45:20,920 --> 00:45:23,320
newer models having such large context windows.

880
00:45:23,400 --> 00:45:25,119
Speaker 3: Is that I want for PI.

881
00:45:25,159 --> 00:45:26,880
Speaker 2: Charm to reach the point where I set up my

882
00:45:26,960 --> 00:45:29,920
interpreter and then the AI like it's just like this,

883
00:45:29,920 --> 00:45:31,800
this is the interpreter. These are all the versions and

884
00:45:31,880 --> 00:45:33,679
things that we should be using. Just stick with this

885
00:45:33,800 --> 00:45:36,119
the same way that if you have like the code

886
00:45:36,119 --> 00:45:38,239
Complete or anything like that set up, it takes from

887
00:45:38,320 --> 00:45:40,639
the It takes from the dock strings and stuff of

888
00:45:40,679 --> 00:45:43,800
the functions of the libraries that you're using. So I

889
00:45:43,880 --> 00:45:45,360
kind of want for it to do that.

890
00:45:46,480 --> 00:45:50,239
Speaker 1: I think that that's one of the hidden benefits of

891
00:45:50,360 --> 00:45:53,880
using AI for specifically for things like writing code, is

892
00:45:53,880 --> 00:45:58,239
you'll give your chatbot or whatever you're using a prompt

893
00:45:58,599 --> 00:46:00,440
and then it comes back with an am answer and

894
00:46:00,480 --> 00:46:04,039
you find scenarios like that and you're like, oh, okay,

895
00:46:05,039 --> 00:46:07,079
that's one hundred percent my fault because I didn't tell

896
00:46:07,119 --> 00:46:10,800
you all of the constraints. And for me personally, I've

897
00:46:10,880 --> 00:46:14,960
found that it's helping me to become a better communicator

898
00:46:15,400 --> 00:46:18,960
because I would have the same problems interfacing with humans.

899
00:46:19,000 --> 00:46:22,760
I would say, hey, go do this task, and they

900
00:46:22,800 --> 00:46:25,800
come back and I'm like, ah, well, okay, I forgot

901
00:46:25,800 --> 00:46:28,360
to tell you it has to run on you know, Linux,

902
00:46:28,440 --> 00:46:30,239
or I need sixty four giga member. You know, all

903
00:46:30,280 --> 00:46:33,920
these little details that I should have communicated up front,

904
00:46:34,519 --> 00:46:37,880
and so the same mistakes apply. Is just more rapid

905
00:46:37,920 --> 00:46:41,920
feedback for me to understand that I wasn't clear in

906
00:46:42,039 --> 00:46:42,800
my request.

907
00:46:43,159 --> 00:46:44,159
Speaker 5: Yeah, it's my fault.

908
00:46:44,199 --> 00:46:46,000
Speaker 4: I should have mentioned that we were using a sixty

909
00:46:46,079 --> 00:46:49,199
four bit operating system here instead of a thirty two bit.

910
00:46:49,320 --> 00:46:52,320
I mean that's a that's a new invention, I know,

911
00:46:52,400 --> 00:46:53,239
like just last year.

912
00:46:53,360 --> 00:46:53,960
Speaker 5: So I can.

913
00:46:54,440 --> 00:46:56,039
Speaker 1: I'll you know, got.

914
00:46:55,880 --> 00:46:59,400
Speaker 4: Confused here, but yeah, for sure that is exactly what's happening.

915
00:47:00,119 --> 00:47:02,320
Speaker 1: And that was some of the looking at the task

916
00:47:02,360 --> 00:47:05,679
reliance on AI here seventy four point nine percent of

917
00:47:05,800 --> 00:47:12,320
people responding or using it for code writing, sixty two

918
00:47:12,760 --> 00:47:16,280
for code explanation. I use it a lot for that.

919
00:47:18,119 --> 00:47:19,760
Speaker 2: What do you mean do you like highlight your code

920
00:47:19,760 --> 00:47:21,920
and then say write me a doc string or what

921
00:47:22,039 --> 00:47:22,239
do you do?

922
00:47:22,360 --> 00:47:25,159
Speaker 1: No, No, I'll copy some code into it and say

923
00:47:25,159 --> 00:47:28,039
what is this thing doing? And then it comes back

924
00:47:28,039 --> 00:47:32,239
and tells me, especially in like whenever you know it's

925
00:47:32,239 --> 00:47:34,440
a language I don't work with very frequently, you know,

926
00:47:34,480 --> 00:47:37,719
if I'm helping someone debug arrust application or whatever, you know,

927
00:47:37,760 --> 00:47:40,079
I'll drop that in there and say what does this

928
00:47:40,119 --> 00:47:42,199
thing do? And it breaks it down.

929
00:47:42,760 --> 00:47:44,800
Speaker 4: See if you if you get in that situation, I

930
00:47:44,800 --> 00:47:47,239
almost recommend taking the next two steps, which are first

931
00:47:47,239 --> 00:47:50,119
then ask it to generate unit tests for it if

932
00:47:50,119 --> 00:47:52,960
they don't exist. Where the unit tests are the documentation

933
00:47:53,039 --> 00:47:55,280
for the code, and then tell it to rewrite the

934
00:47:55,280 --> 00:48:00,039
code so that it's simpler and more understandable. And you

935
00:48:00,119 --> 00:48:01,679
have the unit tests as well, so that the next

936
00:48:01,760 --> 00:48:03,599
verst netlooks of the code knows.

937
00:48:03,400 --> 00:48:04,639
Speaker 5: That it what it does.

938
00:48:04,760 --> 00:48:05,519
Speaker 1: Oh wow.

939
00:48:06,000 --> 00:48:07,880
Speaker 4: And also you have the unit tests that were generated

940
00:48:07,920 --> 00:48:11,719
and validated regressions for the previous version, so you know

941
00:48:11,760 --> 00:48:14,599
the new version doesn't break either. And yeah, it's extra steps,

942
00:48:14,639 --> 00:48:16,880
but it reduces the loop there.

943
00:48:17,519 --> 00:48:19,880
Speaker 1: Damn level up, Warren, I go, I got.

944
00:48:19,719 --> 00:48:20,480
Speaker 5: Some good ones here.

945
00:48:21,239 --> 00:48:24,000
Speaker 4: So I think the doc where you actually have a

946
00:48:24,039 --> 00:48:26,800
bunch of recommendations in that you keep including its context

947
00:48:26,800 --> 00:48:29,800
over and over again is one of them. Another one

948
00:48:29,880 --> 00:48:31,760
is that we're going to very quickly run into this

949
00:48:31,800 --> 00:48:35,679
place where LMS are trying to generate code in languages

950
00:48:35,719 --> 00:48:38,239
that is not optimized for like the code the software

951
00:48:38,320 --> 00:48:40,760
languages we've created, we're all created by humans to do

952
00:48:41,039 --> 00:48:45,199
human related understanding things, and over time we're going to

953
00:48:45,280 --> 00:48:47,519
if we lean more on LMS, we're going to get

954
00:48:47,559 --> 00:48:49,760
to the point where we actually want an LM focused

955
00:48:49,800 --> 00:48:54,360
language where it will be able to better understand how

956
00:48:54,400 --> 00:48:57,800
to generate code that does the correct thing, understand the

957
00:48:57,840 --> 00:49:01,639
context of what code has been written and documentation around

958
00:49:01,639 --> 00:49:03,719
it in order to do the right thing, Which means

959
00:49:03,719 --> 00:49:06,000
we're going to end up creating software languages which are

960
00:49:06,159 --> 00:49:09,840
more less understandable for human and more understandable for machines,

961
00:49:10,159 --> 00:49:11,960
until we get to a point where what we're writing

962
00:49:12,000 --> 00:49:14,920
is almost completely not understandable by us at all.

963
00:49:14,960 --> 00:49:16,440
Speaker 5: But I think that's still a.

964
00:49:16,400 --> 00:49:18,239
Speaker 4: Little ways out, but we're going to get new software

965
00:49:18,280 --> 00:49:22,199
languages for sure that better incorporate this this concept.

966
00:49:22,639 --> 00:49:25,239
Speaker 1: Yeah, and I think that's the big like, that's the

967
00:49:25,280 --> 00:49:29,719
basis of every sci fi AI takes over the world

968
00:49:29,960 --> 00:49:33,800
story ever, right, is that AI starts improving things until

969
00:49:33,840 --> 00:49:37,039
it's beyond the point of us being able to understand

970
00:49:37,079 --> 00:49:37,599
what it's doing.

971
00:49:38,079 --> 00:49:39,800
Speaker 4: Yeah, but I mean look at like there's very few

972
00:49:39,800 --> 00:49:41,519
of us. I mean I did do this in my

973
00:49:41,599 --> 00:49:44,039
academic career. But you go look at the bytecode that

974
00:49:44,119 --> 00:49:48,519
was generated or the assembly by the assembler, and you

975
00:49:48,559 --> 00:49:50,960
don't like you are looking at that and be like, oh, yeah,

976
00:49:51,000 --> 00:49:54,320
I know. I think shift like three registers here, that's

977
00:49:54,360 --> 00:49:58,880
clearly a crypto you know, hashing algorithm obviously with a

978
00:49:58,920 --> 00:50:01,079
bunch of jumps, and this is making an HDP call.

979
00:50:01,119 --> 00:50:03,719
You see the register here and and the requests in

980
00:50:03,760 --> 00:50:04,519
the socket map.

981
00:50:04,599 --> 00:50:06,400
Speaker 5: Ya of course, like I know exactly what that's.

982
00:50:06,199 --> 00:50:08,840
Speaker 1: Doing, right, binary in your head.

983
00:50:09,599 --> 00:50:15,360
Speaker 4: Already looking at the minified javascriptsss for websites and you're like, oh, yeah,

984
00:50:15,440 --> 00:50:17,840
I know exactly which file this came from and what

985
00:50:17,880 --> 00:50:19,840
the stack trace was. When this is going to no,

986
00:50:19,880 --> 00:50:22,440
it's just like random letters that make no sense, you know,

987
00:50:22,480 --> 00:50:25,039
with parentheses and periods between them. So, I mean we're

988
00:50:25,039 --> 00:50:27,159
already at that stage. I think there's just another one

989
00:50:27,440 --> 00:50:32,199
above here, which is between using LMS as the input.

990
00:50:32,760 --> 00:50:36,199
They're still not the input to software development. It's gonna

991
00:50:36,199 --> 00:50:37,880
but it is going to change. We're going to change

992
00:50:37,880 --> 00:50:41,440
the languages that we're utilizing so that they match better

993
00:50:41,559 --> 00:50:49,840
with our non organic coding partners. I remember, like years

994
00:50:49,840 --> 00:50:53,519
ago when the browser wars were like like at their peak.

995
00:50:53,559 --> 00:50:56,280
We were as a group of people and we were

996
00:50:56,320 --> 00:50:59,599
talking about, you know, different browsers and everybody was watching

997
00:50:59,639 --> 00:51:02,079
for their for a favorite, you know. And one of

998
00:51:02,119 --> 00:51:04,840
the guys in a group he says, a real man

999
00:51:04,920 --> 00:51:07,519
just uses curl and parses the HTML in his head.

1000
00:51:10,599 --> 00:51:14,199
Speaker 2: But it's a fun one. But see that's why, like

1001
00:51:14,239 --> 00:51:18,119
I'm a little bit surprised that I don't like the

1002
00:51:18,119 --> 00:51:21,960
the lms don't do better with code because it is

1003
00:51:22,000 --> 00:51:28,800
still humans like creating models out of language, right, I mean,

1004
00:51:28,840 --> 00:51:30,760
like the code isn't completely that, but it can. You know,

1005
00:51:30,800 --> 00:51:33,559
it's close enough. It's as close as chemistry is anyways.

1006
00:51:33,599 --> 00:51:35,320
So why do we get these really good results in

1007
00:51:35,360 --> 00:51:39,440
some of these fields that have these representations of data

1008
00:51:39,639 --> 00:51:42,440
that at least in part use something that kind of

1009
00:51:42,440 --> 00:51:46,239
looks like a language, and not so much with the code.

1010
00:51:46,800 --> 00:51:48,920
Speaker 4: So someone's going to call me out on this because

1011
00:51:48,960 --> 00:51:51,800
I am for sure not the expert. So there's that caveat,

1012
00:51:51,880 --> 00:51:52,480
and I'm.

1013
00:51:52,320 --> 00:51:55,199
Speaker 2: Sure that's nobody is the expert that's the thing nobody knows.

1014
00:51:55,239 --> 00:51:56,920
Speaker 3: Every meeting I'm in, we're all like, why.

1015
00:51:56,760 --> 00:51:57,320
Speaker 2: Is it doing this?

1016
00:51:57,639 --> 00:52:02,199
Speaker 4: Like that part doesn't get Yeah, I mean, if it

1017
00:52:02,239 --> 00:52:04,280
gets cut out of the podcast, someone will definitely call

1018
00:52:04,400 --> 00:52:13,440
me out. So handling human speech in written words has

1019
00:52:13,519 --> 00:52:18,199
a very single directionality to it, forward directionality as far

1020
00:52:18,239 --> 00:52:21,760
as linear goes in time, you sort of read the context,

1021
00:52:21,800 --> 00:52:26,320
you get the result, whereas software development has a need

1022
00:52:26,400 --> 00:52:31,000
for the output to be sort of understood from a

1023
00:52:31,079 --> 00:52:34,639
huge context of a single class, an object oriented or

1024
00:52:34,760 --> 00:52:37,639
multiple functions, et cetera. And so there's at least a

1025
00:52:37,719 --> 00:52:40,880
bi directionality things going forward and back. Like by the

1026
00:52:41,039 --> 00:52:44,519
end of function or a file of which you have

1027
00:52:44,559 --> 00:52:47,159
some source code, there's maybe some braces at the end, etc.

1028
00:52:47,400 --> 00:52:51,000
Which aren't just based on the previous lines. They do

1029
00:52:51,119 --> 00:52:56,760
impact what should have been written almost and same across files.

1030
00:52:56,800 --> 00:53:00,119
Whereas you could have a many page book where each

1031
00:53:00,159 --> 00:53:02,840
page in the book only follow from the previous one,

1032
00:53:02,880 --> 00:53:05,800
but it's software development is more of like a choose

1033
00:53:05,800 --> 00:53:06,519
your own adventure.

1034
00:53:06,880 --> 00:53:10,199
Speaker 5: Some of those functions are used outside.

1035
00:53:09,679 --> 00:53:12,199
Speaker 4: Of the context of that file, and so there's a

1036
00:53:12,239 --> 00:53:14,199
lot that goes into it that isn't just the same

1037
00:53:14,239 --> 00:53:17,400
as natural language, and so there's actually different models that

1038
00:53:17,440 --> 00:53:20,800
are being used in order, like built up and design

1039
00:53:21,119 --> 00:53:24,400
and how we're actually orchestrating the consumption of that data

1040
00:53:24,440 --> 00:53:27,400
and the neural net that we're creating to process that

1041
00:53:27,480 --> 00:53:30,840
data is fundamentally different depending on the output. It's much

1042
00:53:30,880 --> 00:53:34,400
closer to say, creating an image than it is a

1043
00:53:34,519 --> 00:53:36,360
human language, like a written text.

1044
00:53:37,559 --> 00:53:40,199
Speaker 2: That's interesting, Yeah, you're right, all, but the probability space

1045
00:53:40,639 --> 00:53:42,760
is like much greater with code than it is with

1046
00:53:42,840 --> 00:53:45,920
human language, right, Like if you look at how the

1047
00:53:45,960 --> 00:53:49,440
AI kind of takes thing, it's part of a very

1048
00:53:49,440 --> 00:53:52,440
simplified explanation is that you know, it sort of creates

1049
00:53:52,480 --> 00:53:54,719
like the probability that the next character in the sentence

1050
00:53:54,719 --> 00:53:57,440
will be this thing, and it does that sometimes. And

1051
00:53:57,559 --> 00:54:01,280
with natural language, if people speak king, I mean, I

1052
00:54:01,320 --> 00:54:03,679
don't know, but I don't know that the severb and

1053
00:54:03,760 --> 00:54:06,039
documented anywhere. But I would guess that it's much much

1054
00:54:06,079 --> 00:54:10,440
smaller for natural language than it is for code. You

1055
00:54:10,480 --> 00:54:16,760
can investigate your own adventure.

1056
00:54:14,079 --> 00:54:19,079
Speaker 4: For you can understand a lot more with mistakes i'll

1057
00:54:19,079 --> 00:54:22,519
call them in human natural language, things that don't make sense,

1058
00:54:22,800 --> 00:54:26,280
sentences that run on or curtail or miss the predicate,

1059
00:54:26,559 --> 00:54:29,840
whereas with code it's going to get executed, and so

1060
00:54:30,440 --> 00:54:33,119
there is some fundamental difference here. I don't know how

1061
00:54:33,119 --> 00:54:34,719
it is in chemistry. I always can't speak to that.

1062
00:54:35,239 --> 00:54:38,840
It's possible that the models are highly both fine tuned

1063
00:54:39,840 --> 00:54:42,719
with that in regard, or maybe it is just the

1064
00:54:42,760 --> 00:54:45,480
obvious result of a bunch of natural language as well.

1065
00:54:45,519 --> 00:54:48,199
It lends itself well to that. Whereas but I could

1066
00:54:48,199 --> 00:54:54,119
imagine experiment diagrams or state diagram transfers I forgot they're

1067
00:54:54,119 --> 00:54:56,320
called in chemistry, you know, it can convert from a

1068
00:54:56,360 --> 00:54:59,719
couple of different molecules to different other kind of molecules

1069
00:54:59,719 --> 00:55:03,039
like that. You know, of course, of say, a chemistry

1070
00:55:03,039 --> 00:55:06,159
book could be challenging to get right, like it could

1071
00:55:06,199 --> 00:55:08,920
definitely swap from talking about one kind of molecule to

1072
00:55:09,000 --> 00:55:11,239
a different one later. And where a natural language that

1073
00:55:11,280 --> 00:55:14,360
may not totally matter within the context of running experiment,

1074
00:55:14,400 --> 00:55:18,840
are actually designing a product that someone could use, Uh,

1075
00:55:19,440 --> 00:55:20,559
would not make sense at all.

1076
00:55:21,320 --> 00:55:23,199
Speaker 2: That is always a little bit freaky about the AI

1077
00:55:23,360 --> 00:55:26,440
doing things, especially with like medicine and drug discovery, where

1078
00:55:26,440 --> 00:55:30,199
you're like, listen, one compound is safe for you to

1079
00:55:30,280 --> 00:55:34,239
take the mirror image of that compound is going to

1080
00:55:34,320 --> 00:55:34,880
kill you.

1081
00:55:34,960 --> 00:55:36,800
Speaker 3: Like, and these are you know, they're the same thing.

1082
00:55:36,840 --> 00:55:39,440
Speaker 2: They're just they're just mirror images of each other. We've

1083
00:55:39,440 --> 00:55:41,480
got to make sure that the AI, that the AI

1084
00:55:41,599 --> 00:55:44,079
gets this. Sometimes. I know the ones that I've experimented

1085
00:55:44,079 --> 00:55:46,159
with have just been the general AI models. I know,

1086
00:55:46,800 --> 00:55:48,599
like new ones are coming out all the time, right,

1087
00:55:48,639 --> 00:55:50,039
and I'm sure there are going to be ones that

1088
00:55:50,079 --> 00:55:52,920
are more targeted, you know, specifically to certain fields like

1089
00:55:52,960 --> 00:55:55,639
drug discovery or astronomy or you know, whatever the things are.

1090
00:55:56,320 --> 00:55:59,280
Speaker 4: I have to know, now, does it often misdate getting

1091
00:55:59,280 --> 00:56:02,880
the el chairo and ar chylal molecules mixed up, or

1092
00:56:02,960 --> 00:56:05,239
is it like does it know that it should mostly

1093
00:56:05,239 --> 00:56:06,079
be l chiral.

1094
00:56:06,599 --> 00:56:08,280
Speaker 2: I don't know, because I haven't been running those types

1095
00:56:08,280 --> 00:56:10,360
of experiments. That was just kind of the easiest thing

1096
00:56:10,360 --> 00:56:13,119
that I could pop up with in my head to say,

1097
00:56:13,199 --> 00:56:15,440
these are the stakes for the AI getting things wrong,

1098
00:56:15,480 --> 00:56:17,760
and they can be very simple switches, right, Like this

1099
00:56:18,079 --> 00:56:21,639
has happened in human labs where there was I don't know,

1100
00:56:21,679 --> 00:56:25,239
there was like a type of medication given to women

1101
00:56:25,320 --> 00:56:27,519
during pregnancy for nausea that caused a bunch of birth

1102
00:56:27,559 --> 00:56:30,599
defects and it was just a chiral compound, so your

1103
00:56:30,679 --> 00:56:32,679
chances were, you know, like almost fifty to fifty if

1104
00:56:32,679 --> 00:56:33,920
you've taken it things like that.

1105
00:56:35,519 --> 00:56:38,880
Speaker 3: So I don't know, Sorry, I lost my train of

1106
00:56:38,880 --> 00:56:40,679
thought there. I don't know what I was going on about.

1107
00:56:40,920 --> 00:56:44,559
Speaker 4: I mean, this was always so rivetting for me that

1108
00:56:44,800 --> 00:56:48,880
the molecular compound equations could result in al chiral and

1109
00:56:49,000 --> 00:56:52,320
ar chylal compounds. And in the human world at least

1110
00:56:52,360 --> 00:56:54,920
like ninety nine percent of things that we encounter are

1111
00:56:54,960 --> 00:56:59,320
all l chiral and they're all mostly harmless to us,

1112
00:57:00,360 --> 00:57:03,760
Like the archylo versions of them, which almost don't exist

1113
00:57:03,840 --> 00:57:06,760
in nature, are all like incredibly poisonous for us to

1114
00:57:06,840 --> 00:57:09,639
consume in any regard, causing instant death.

1115
00:57:10,119 --> 00:57:12,880
Speaker 5: And I don't know, this is just always so interesting

1116
00:57:12,920 --> 00:57:13,119
to me.

1117
00:57:13,840 --> 00:57:16,199
Speaker 3: Chemistry is pretty wild. Like chemistry too.

1118
00:57:16,400 --> 00:57:18,320
Speaker 4: I'm trying to think of there's anything left in the

1119
00:57:19,440 --> 00:57:22,119
in the report that we haven't talked about.

1120
00:57:22,320 --> 00:57:24,679
Speaker 1: There's one thing I want to bring up because I

1121
00:57:24,760 --> 00:57:28,840
found this to be really interesting. The drivers of adoption

1122
00:57:29,039 --> 00:57:39,559
for AI number one, marketing number two, for GO bureaucracy

1123
00:57:40,760 --> 00:57:46,239
number three. What if our competitor implements AI before us,

1124
00:57:47,159 --> 00:57:50,039
and I find those interesting because I think those are

1125
00:57:50,880 --> 00:57:56,119
all the exact wrong reasons to implement AI. Like none

1126
00:57:56,159 --> 00:57:59,679
of those say, oh, because it's going to help us

1127
00:57:59,679 --> 00:58:05,280
to live a better product to our customers, which I

1128
00:58:05,280 --> 00:58:09,239
think is should be like the primary goal, well to

1129
00:58:09,320 --> 00:58:14,800
do to do that at a at a profitable margin,

1130
00:58:16,719 --> 00:58:18,480
and that's just kind of the end result.

1131
00:58:18,519 --> 00:58:21,679
Speaker 2: And people tend to, you know, drill down more and

1132
00:58:21,719 --> 00:58:23,800
get more into the particulars of what they're doing on

1133
00:58:23,840 --> 00:58:24,559
their day to day.

1134
00:58:25,039 --> 00:58:28,880
Speaker 3: But I think that like if the.

1135
00:58:27,679 --> 00:58:31,599
Speaker 1: But you can do all three of those things and

1136
00:58:31,679 --> 00:58:35,800
not make a better product for your customer the way

1137
00:58:35,840 --> 00:58:38,440
that they're stated here. Anyways, it could be.

1138
00:58:39,000 --> 00:58:40,639
Speaker 3: It could just be screwing around with AI, and it

1139
00:58:40,679 --> 00:58:42,719
couldn't you know, not mean anything.

1140
00:58:43,679 --> 00:58:46,199
Speaker 5: I think there is right now. I think what's happening

1141
00:58:46,360 --> 00:58:47,559
is it can be.

1142
00:58:47,639 --> 00:58:50,920
Speaker 4: It's always a challenge for you when you're marketing your

1143
00:58:50,920 --> 00:58:55,840
product to explain concisely what the value is that you're

1144
00:58:55,840 --> 00:58:59,199
offering so that you capture your customers to come and

1145
00:58:59,280 --> 00:59:02,800
utilize your things. And because it's such a challenge, I

1146
00:59:02,800 --> 00:59:05,440
think a lot of people are jumping to just saying

1147
00:59:05,519 --> 00:59:08,360
that AI is included in it because it is being

1148
00:59:08,480 --> 00:59:11,000
used as a proxy for all the things in which

1149
00:59:11,039 --> 00:59:14,480
AI could hypothetically be doing to make it better. But

1150
00:59:14,519 --> 00:59:17,119
then there is this sort of death spiral where that

1151
00:59:17,280 --> 00:59:20,039
encourages the use of AI in the product to do

1152
00:59:20,079 --> 00:59:23,039
those things, even though you don't fully understand the value

1153
00:59:23,039 --> 00:59:25,159
it's offering. And then you have to say that it's

1154
00:59:25,239 --> 00:59:27,960
using AI to do that thing, because people are concerned

1155
00:59:27,960 --> 00:59:30,599
that if you're using AI, the output may not actually

1156
00:59:30,639 --> 00:59:33,320
be what it should be. It could be making more

1157
00:59:33,320 --> 00:59:35,679
mistakes than you would expect if it was well coded

1158
00:59:35,719 --> 00:59:38,679
and validated, and so you end up with this death cycle.

1159
00:59:39,199 --> 00:59:41,960
Speaker 5: And I think that is certainly part of the problem

1160
00:59:42,039 --> 00:59:42,400
for sure.

1161
00:59:43,559 --> 00:59:46,440
Speaker 2: Yeah, I think especially on these experiments where they're large

1162
00:59:46,519 --> 00:59:49,719
enough scale that you're not going to want to replicate

1163
00:59:49,760 --> 00:59:52,000
them with people. So I see a lot of literature

1164
00:59:52,000 --> 00:59:54,320
mining projects where it's like, we want to know all

1165
00:59:54,400 --> 00:59:59,840
the literature that's not available on whatever this new cancer,

1166
01:00:00,039 --> 01:00:05,480
okay cancer, And I mean, I don't I wonder how

1167
01:00:05,519 --> 01:00:07,119
much it would cost if I was going to be like,

1168
01:00:07,159 --> 01:00:10,199
I'm going to download three thousand articles dust them. I

1169
01:00:10,280 --> 01:00:11,840
know how much it costs for the AI to do it,

1170
01:00:12,079 --> 01:00:13,920
But you know, just them with the AI, I'm going

1171
01:00:13,960 --> 01:00:15,960
to ask it a bunch of questions and get some results.

1172
01:00:16,239 --> 01:00:19,119
How much would the same experiment costs to be doing

1173
01:00:19,199 --> 01:00:19,719
with people.

1174
01:00:20,519 --> 01:00:21,440
Speaker 3: I don't know, but.

1175
01:00:21,400 --> 01:00:23,360
Speaker 2: I'm going to guess, you know, I don't know how

1176
01:00:23,400 --> 01:00:25,679
many how many papers can you really read before your

1177
01:00:25,679 --> 01:00:28,159
brain turns to mush. I'm going to put it in

1178
01:00:28,239 --> 01:00:30,760
about ten, so you know, you need a good number

1179
01:00:30,760 --> 01:00:31,519
of people to do that.

1180
01:00:31,719 --> 01:00:35,320
Speaker 3: And then I don't know of anybody who's doing that

1181
01:00:35,440 --> 01:00:37,199
kind of experiment, so I don't know.

1182
01:00:37,360 --> 01:00:39,280
Speaker 2: That's always like the interesting thing for me with like

1183
01:00:39,320 --> 01:00:42,840
the marketing and the more like wiki style experiments is

1184
01:00:42,880 --> 01:00:45,920
that I can't really can't really have a control group there,

1185
01:00:46,400 --> 01:00:48,400
not one that I'm paying for anyways, not going to

1186
01:00:48,480 --> 01:00:48,760
do that.

1187
01:00:49,400 --> 01:00:51,000
Speaker 4: I mean, I think it would be and it would

1188
01:00:51,039 --> 01:00:53,239
be interesting if you're talking about trying to come up

1189
01:00:53,280 --> 01:00:58,360
with like real conclusions based off of having downloaded or

1190
01:00:58,360 --> 01:01:00,719
consumed all of those articles. Because if we look at

1191
01:01:00,719 --> 01:01:03,159
what a human would do, like let's say intelligent software

1192
01:01:03,159 --> 01:01:08,480
engineer one of our viewers, right probably write some sort

1193
01:01:08,519 --> 01:01:11,280
of script to go and curl all of the documents,

1194
01:01:11,320 --> 01:01:13,760
get them downloaded, and do some sort of rejex matching

1195
01:01:13,840 --> 01:01:16,280
to find words in the document that you care about

1196
01:01:16,719 --> 01:01:19,920
and then utilize that and investigate each one of those

1197
01:01:20,079 --> 01:01:21,920
more specifically. Right, you know, you start like a very

1198
01:01:22,000 --> 01:01:24,960
high level approach to filter out the information you don't

1199
01:01:24,960 --> 01:01:26,920
care about into only the parts you do.

1200
01:01:27,440 --> 01:01:28,639
Speaker 5: And I think that can help a lot.

1201
01:01:28,639 --> 01:01:31,159
Speaker 4: Now we're actually talking about consuming all of them and

1202
01:01:31,199 --> 01:01:36,440
coming up with a collective, unified epiphany on the topic.

1203
01:01:36,920 --> 01:01:39,320
I'm not sure that the AI would be able to

1204
01:01:39,360 --> 01:01:43,760
do much better job than basically regurgitate some parts of

1205
01:01:43,760 --> 01:01:45,880
the individual documents itself. I don't think it's going to

1206
01:01:45,920 --> 01:01:49,639
be able to pull from a second corpus of like

1207
01:01:49,719 --> 01:01:52,920
sets of epiphanies you can have and like do a match, right,

1208
01:01:52,960 --> 01:01:55,119
you know, I read these documents and then like, you know,

1209
01:01:55,400 --> 01:01:58,599
there's six possible conclusions you can come.

1210
01:01:58,519 --> 01:01:59,559
Speaker 5: To, and you know, pick one of those.

1211
01:01:59,719 --> 01:02:02,719
Speaker 4: I think is where humans are still generating new ideas,

1212
01:02:02,719 --> 01:02:07,079
whereas we've limited the LMS on what they will actually generate.

1213
01:02:07,280 --> 01:02:09,760
If they generate stuff from not the documents, we call

1214
01:02:09,800 --> 01:02:12,280
it wrong and usually it is. And if it generates

1215
01:02:12,360 --> 01:02:14,320
up just from the documents, then I think it's losing

1216
01:02:14,360 --> 01:02:17,119
some sort of insight that we would get even if

1217
01:02:17,159 --> 01:02:19,800
you took the human friend brain approach.

1218
01:02:21,320 --> 01:02:23,880
Speaker 2: I think that's very true. The LLM is not going

1219
01:02:23,880 --> 01:02:26,440
to innovate the way that humans are. Like presumably you

1220
01:02:26,480 --> 01:02:28,440
could get some very smart human who looks at all

1221
01:02:28,440 --> 01:02:31,199
those papers and says, this is what they all missed,

1222
01:02:31,199 --> 01:02:35,239
and then we'd have, you know, some crazy new scientific

1223
01:02:35,280 --> 01:02:39,559
thriller novel that the punch of nerds reading papers or something.

1224
01:02:39,760 --> 01:02:40,159
Speaker 3: I don't know.

1225
01:02:40,679 --> 01:02:44,519
Speaker 1: All right, it feels like time to do some picks. Okay,

1226
01:02:44,800 --> 01:02:47,960
all right, Gillian, welcome back to the show. What's your

1227
01:02:48,000 --> 01:02:48,519
first pick?

1228
01:02:50,559 --> 01:02:50,920
Speaker 2: I don't know.

1229
01:02:51,000 --> 01:02:53,440
Speaker 3: So the show got me thinking about neuroscience again.

1230
01:02:53,519 --> 01:02:56,039
Speaker 2: So I'm going to pick the book The Man who

1231
01:02:56,079 --> 01:03:00,440
Mistook His Wife for a Hat by Oliver Something. But

1232
01:03:00,480 --> 01:03:02,880
that's the title, pretty ditinctive, so you can you know.

1233
01:03:03,800 --> 01:03:07,039
It's on Amazon, I promise. But it's really interesting because

1234
01:03:07,079 --> 01:03:09,800
a lot of the experiments in neuroscience that are done

1235
01:03:09,960 --> 01:03:11,840
you can't you can't just run around doing.

1236
01:03:11,679 --> 01:03:12,320
Speaker 3: Those two people.

1237
01:03:12,440 --> 01:03:15,039
Speaker 2: Okay, we have like ethical committees for stuff like that.

1238
01:03:15,400 --> 01:03:18,760
So instead what happens is that you know, people who

1239
01:03:18,760 --> 01:03:21,960
have suffered certain brain injuries or had strokes or some

1240
01:03:22,159 --> 01:03:24,719
you know, like something like that has happened. They are

1241
01:03:24,840 --> 01:03:27,840
very extensively studied, and then we say, oh this, you know,

1242
01:03:27,960 --> 01:03:30,119
these parts of this person brain got knocked out, and

1243
01:03:30,159 --> 01:03:32,000
then this is this is what happened. So then you

1244
01:03:32,119 --> 01:03:36,760
get very interesting cases like you know, somebody saying, you know,

1245
01:03:36,880 --> 01:03:37,840
this is my.

1246
01:03:37,800 --> 01:03:39,679
Speaker 3: Wife over here is actually a hat, and you.

1247
01:03:39,679 --> 01:03:41,599
Speaker 2: Just you have all kinds of crazy stuff that goes

1248
01:03:41,599 --> 01:03:44,280
on when you start messing with the human brain. So

1249
01:03:44,559 --> 01:03:46,480
that's that, and then you can learn all about why

1250
01:03:46,519 --> 01:03:48,519
you should not be experimenting on people.

1251
01:03:50,000 --> 01:03:51,679
Speaker 1: Seems like a worthy lesson to learn.

1252
01:03:53,440 --> 01:03:56,599
Speaker 3: It's important why mess with the human brain?

1253
01:03:57,280 --> 01:03:59,559
Speaker 1: There may be some data points. We open up a

1254
01:03:59,599 --> 01:04:03,199
couple of history books that provide additional context. There why

1255
01:04:03,280 --> 01:04:04,119
that's a bad idea?

1256
01:04:04,480 --> 01:04:08,880
Speaker 2: There are there definitely are really really maybe I don't know,

1257
01:04:09,199 --> 01:04:10,719
just throwing stuff out here.

1258
01:04:13,519 --> 01:04:14,840
Speaker 1: What about you, Warren? What do you got?

1259
01:04:15,480 --> 01:04:17,400
Speaker 5: Yeah? So I didn't pick a book this week.

1260
01:04:17,440 --> 01:04:20,800
Speaker 4: My pick is going to be QCon San Francisco, which

1261
01:04:20,840 --> 01:04:25,039
should be happening this week. My my CEO of authors

1262
01:04:25,079 --> 01:04:27,639
is actually giving the only security talk I believe at

1263
01:04:27,639 --> 01:04:30,519
the whole conference, which is titled security or Convenience?

1264
01:04:30,880 --> 01:04:31,599
Speaker 5: Why not both?

1265
01:04:32,480 --> 01:04:34,440
Speaker 4: And so if you if you're there and you have

1266
01:04:34,519 --> 01:04:36,719
the opportunity to go and see it, I highly recommend it.

1267
01:04:36,960 --> 01:04:41,159
There's a lot of interesting insights where innovative companies are

1268
01:04:41,559 --> 01:04:45,000
focusing some of their opportunities and what is almost now

1269
01:04:45,000 --> 01:04:46,679
considered legacy.

1270
01:04:47,760 --> 01:04:51,760
Speaker 1: Right on. Nice cool for me. I'm going to pick

1271
01:04:51,760 --> 01:04:55,679
a book that we we were talking about before, just

1272
01:04:55,719 --> 01:05:01,159
before we recorded this episode, and I think it's with

1273
01:05:01,360 --> 01:05:03,760
the conversation we had about AI. I think it ties

1274
01:05:03,840 --> 01:05:08,079
into that and we'll have some relevance there. It's a

1275
01:05:08,079 --> 01:05:11,760
book called Trust Me I'm Lying by Ryan Holiday. You

1276
01:05:11,800 --> 01:05:14,599
may know him from The Daily Stoic, but prior to

1277
01:05:15,119 --> 01:05:19,079
that venture, he was a marketing professional and did a

1278
01:05:19,159 --> 01:05:24,039
bunch of different things in the marketing world that he's

1279
01:05:24,079 --> 01:05:29,159
not proud of, and this book is his admission to

1280
01:05:29,239 --> 01:05:32,840
doing those things. But it's really interesting to read to

1281
01:05:33,039 --> 01:05:38,320
understand how corporate marketing works and the things that they're

1282
01:05:38,880 --> 01:05:42,920
willing to do and will do to promote their brand.

1283
01:05:43,079 --> 01:05:46,760
So yeah, trust Me on Lying by Ryan Holiday. Super

1284
01:05:46,800 --> 01:05:51,719
interesting read. It will forever change every commercial website, ad

1285
01:05:51,760 --> 01:05:55,559
and blog posts that you ever read. And do you

1286
01:05:55,599 --> 01:05:57,360
guys hear that clicking in the background? Is it just me?

1287
01:05:57,920 --> 01:06:01,440
I do Okay, I think that's.

1288
01:06:01,239 --> 01:06:04,800
Speaker 3: My computer, right, n yeah, that's good.

1289
01:06:05,320 --> 01:06:07,840
Speaker 1: Yeah. Well sometimes it's just in my head and then

1290
01:06:07,880 --> 01:06:11,400
I ask and everybody stares at me awkward, going uh no, dude,

1291
01:06:11,679 --> 01:06:13,599
you just go over there in the corner and do

1292
01:06:13,679 --> 01:06:19,719
your thing, and I'm like, okay, all right, Well with that,

1293
01:06:19,760 --> 01:06:22,800
we have an episode. Thank you Warren, thank you Jillian,

1294
01:06:23,039 --> 01:06:25,320
and to all the listeners out there, thank you for

1295
01:06:25,400 --> 01:06:29,400
hanging out listening to our rants and ramblings and misdirection.

1296
01:06:29,559 --> 01:06:31,760
I hope you enjoyed the show. If you did or

1297
01:06:31,760 --> 01:06:33,679
if you didn't, you know how to find us on

1298
01:06:33,840 --> 01:06:36,400
various social media platforms, so give us a shout and

1299
01:06:36,559 --> 01:06:41,360
let us know. We'll see in the next episode.