WEBVTT 1 00:00:06.440 --> 00:00:09.800 Well, hello, hello, and welcome back for another wonderful episode of Jobscript 2 00:00:09.880 --> 00:00:16.239 Jabber. On today's show, we have first our guest Connor books Bronston. 3 00:00:16.359 --> 00:00:18.920 Oh, I'm sorry I spelled your name wrong. I spelled with two wins. 4 00:00:19.160 --> 00:00:22.359 It happens, man, We'll fix it and post. All good. 5 00:00:22.440 --> 00:00:26.440 Not something I'll worry about too much, given that both of my names you 6 00:00:26.440 --> 00:00:29.879 could assume I'm spelling them a little wrong, so peple usually assume double ends 7 00:00:29.920 --> 00:00:33.880 the first name Bronze sun flast name which just happened. Don't worry about all 8 00:00:34.000 --> 00:00:38.799 right, cool, and then you give us the thirty second who we are 9 00:00:38.840 --> 00:00:43.880 and yeah, pardon me here? So I'm the coast the Devin Rupted podcast. 10 00:00:44.039 --> 00:00:48.560 We talk with engineering leaders every week like AJ and Steve and leaders at 11 00:00:48.560 --> 00:00:52.200 you know, Slack, et cetera, about topics like debex. Jen Ai. 12 00:00:52.399 --> 00:00:56.280 I also work at Linear B and one thing we've really done is dived 13 00:00:56.439 --> 00:01:00.079 into data reports around house software engineering and software to the life cycle works. 14 00:01:00.240 --> 00:01:04.159 So I'm excited to share some insights early insights actually from our Jena I report 15 00:01:04.159 --> 00:01:07.560 what's out tomorrow, along with talking debax and some of the impacts we've seen 16 00:01:07.560 --> 00:01:11.439 from things like the Dora study which we worked with Google on this last year, 17 00:01:11.640 --> 00:01:15.719 so it should be incredible conversation and excited to dive in these insights to 18 00:01:15.760 --> 00:01:23.480 you two, all right, And then we also have trusty Rusty Steve Hello, 19 00:01:23.760 --> 00:01:26.079 Hello, Hello a j. We got to work on your posting skills 20 00:01:26.120 --> 00:01:30.200 here. You always do the panelists before the guests say the best for last, 21 00:01:30.280 --> 00:01:34.519 right, so we got to work on this. You're kind to me 22 00:01:34.599 --> 00:01:38.680 on that one. We'll take. And then Connor, I had to keep 23 00:01:38.719 --> 00:01:42.359 a straight face when you put me an engineering leader in the same sentence. 24 00:01:42.920 --> 00:01:47.719 You know my general approach with work and podcasts, I always bring on people 25 00:01:47.719 --> 00:01:53.959 smarter than me, and so my I am the leader in good dad jokes. 26 00:01:53.280 --> 00:01:57.719 But we'll leave it at that, and that's a crucial contribution. So 27 00:01:57.760 --> 00:02:01.599 that's very crucial, Yes, and I will show leadership at the end of 28 00:02:01.599 --> 00:02:07.919 this episode in that area. That's right, leaders joke last, But no, 29 00:02:08.000 --> 00:02:12.400 that's not how it goes now anyway. But with that all the way, 30 00:02:12.439 --> 00:02:16.439 we'll go ahead and introduce today's topic. We're going to be talking about 31 00:02:17.039 --> 00:02:23.000 jen AI, which is like Generation Z except not at all related. It's 32 00:02:23.280 --> 00:02:29.759 just generative AI. And then developer experience and you've got those insights you're talking 33 00:02:29.759 --> 00:02:34.879 about, So where where should we start? Connor, Yes, I think 34 00:02:35.080 --> 00:02:38.000 a great place to start is a definition phase. So you mentioned generative AI. 35 00:02:38.159 --> 00:02:40.360 Let's just make sure we level start with the audience and showcase that. 36 00:02:40.439 --> 00:02:45.080 Yeah, when we say JENAI and software development, we're talking about generating code 37 00:02:46.360 --> 00:02:50.000 through the use of an AI tool like Copilot, like these other tools out 38 00:02:50.000 --> 00:02:53.159 there. And the cool thing that linear B has done is we have done 39 00:02:53.159 --> 00:02:58.280 a study following up to the great work that the Dora team did this last 40 00:02:58.360 --> 00:03:00.520 year and said, Okay, you know the Dora team when I met with 41 00:03:00.599 --> 00:03:05.000 Nathan Harvey and we've partnered with them on their door report in twenty twenty three, 42 00:03:05.439 --> 00:03:08.080 they found that there weren't early efficiency impacts from Jennai that were showing up 43 00:03:08.120 --> 00:03:13.039 in their survey audience. However, what they did see was that teams that 44 00:03:13.080 --> 00:03:17.240 were using Jenni and about fifty percent of the respondents were, saw improved happiness 45 00:03:17.280 --> 00:03:21.719 on their deb teams. And we correlated that with, oh, you know, 46 00:03:21.840 --> 00:03:23.719 maybe we don't have to necessarily spend as much time making tests. We 47 00:03:23.759 --> 00:03:28.439 can have the AI helveles with that generating those or maybe the annoying parts of 48 00:03:28.439 --> 00:03:31.680 the code can get kind of abstracted out, so I can then focus on 49 00:03:31.680 --> 00:03:37.360 these strategic pieces. And that's where I think there's this huge opportunity. Initially 50 00:03:37.479 --> 00:03:38.800 is like, hey, how can we remove these blockers? How can we 51 00:03:38.840 --> 00:03:42.560 improve here? And now what we've done is we've followed this up and we've 52 00:03:42.599 --> 00:03:46.240 done targeted surveys with over one hundred and fifty different development teams across the world 53 00:03:46.560 --> 00:03:51.400 and are now adding in quantitative data as well from our linear By tooling. 54 00:03:52.039 --> 00:03:55.199 Linear By for Context is a platform for software delivery management and software engineering insights. 55 00:03:55.240 --> 00:04:00.639 We helped both provide quantitative and quality of metrics and automation for improvement. 56 00:04:00.240 --> 00:04:02.759 And one of the things that we're going to be rolling out here shortly is 57 00:04:02.759 --> 00:04:09.960 a beta of how you can actually track gen AI pole requests or partially generated 58 00:04:10.080 --> 00:04:15.680 partially GENI generated pole requests and then compare them over time to code that is 59 00:04:15.719 --> 00:04:18.079 not being affected by GENI so you can actually see are we getting efficiency gains? 60 00:04:18.480 --> 00:04:24.040 Is my GENI initiative at the executive level making an impact or is this 61 00:04:24.680 --> 00:04:27.680 kind of more hype than anything else? And we have some really interests, 62 00:04:27.759 --> 00:04:31.399 Yeah, please define GENAI poll requests. Does that mean that the AI wrote 63 00:04:31.560 --> 00:04:36.199 the description or that the AI wrote the code or the code contribution. So 64 00:04:36.600 --> 00:04:42.959 we're we're working on a tool to automatically detect geni's contribution there, so linking 65 00:04:43.000 --> 00:04:46.759 up with Copilot or other different GENAI code generation tools to say, hey, 66 00:04:46.959 --> 00:04:50.000 this pole request is partially written or was written in concert with kind of a 67 00:04:50.079 --> 00:04:54.800 GENAI code assistant, and then we can evaluate code quality down the line and 68 00:04:54.839 --> 00:04:57.759 say, did we have to refractor it more with our issues that were caused 69 00:04:57.759 --> 00:05:00.120 by it? Do we need to maybe add an addition test early on to 70 00:05:00.160 --> 00:05:04.560 make sure that everything that is working. How does this fit? And we're 71 00:05:04.560 --> 00:05:10.079 seeing some fascinating early insights. Okay, so before we get going too much 72 00:05:10.079 --> 00:05:13.720 farther, we got to find some terms here. As a parent, when 73 00:05:13.759 --> 00:05:16.519 I hear Dora, I'm thinking swiper, no swiping, you know, Dora 74 00:05:16.600 --> 00:05:20.240 the explorer? Absolutely, So who is Dora? What is Dora? Why 75 00:05:20.279 --> 00:05:26.279 is Dora important? And et CETERA great question. I am using a lot 76 00:05:26.279 --> 00:05:29.360 of acronyms, and I apologize to anyone who's fallen along and knocketed them. 77 00:05:29.920 --> 00:05:34.120 So Dora is the accelerate state of DevOps team at Google Cloud, and they've 78 00:05:34.120 --> 00:05:39.560 done more than ten years now of seminal research in software development and software development 79 00:05:39.560 --> 00:05:45.680 metrics to understand how productive engineering teams are, how to make teams happier and 80 00:05:45.920 --> 00:05:50.399 help build high performing engineering teams. Okay, so moving on, we've established 81 00:05:50.439 --> 00:05:55.560 dora. So what were you about to ask aj? So you Connor, 82 00:05:55.600 --> 00:06:02.120 you just said something about results or insights that you were kind of summarizing that. 83 00:06:02.279 --> 00:06:05.639 Yeah, it was probing for more detail. Happy, happy to dive 84 00:06:05.639 --> 00:06:10.279 into some more. So I'll say a couple of things. One, across 85 00:06:10.560 --> 00:06:14.759 our survey set, we saw that eighty seven percent of organizations are already planning 86 00:06:14.800 --> 00:06:17.079 to invest or have already invested in the Genai tool in twenty twenty four. 87 00:06:17.480 --> 00:06:23.319 So the idea that this is something that is very broad across the industry is 88 00:06:23.600 --> 00:06:28.319 absolutely true. We're seeing that rapidly accelerating our data set. Compared to fifty 89 00:06:28.319 --> 00:06:31.920 percent in survey respondents last year for the Door survey, We're not seing this 90 00:06:32.040 --> 00:06:36.680 continue to increase, and it's it's a key area of focus, and I 91 00:06:36.680 --> 00:06:40.959 think part of that is that efficiency gains people are starting to see, and 92 00:06:41.040 --> 00:06:45.839 then the happiness gains. I mentioned that are clearly being shown across dev teams 93 00:06:45.839 --> 00:06:47.600 that are labaging this. You know, a great example of a success around 94 00:06:47.600 --> 00:06:53.399 this is Uber, and Uber has actually started to help not only generate code 95 00:06:53.439 --> 00:06:59.240 internally using Genai coding tools, but also to generate comments for codes. So, 96 00:07:00.040 --> 00:07:02.879 what's one of the most annoying things that happens in a software development team's 97 00:07:02.959 --> 00:07:05.560 life cycle? It's like, Oh, I have to go review someone else's 98 00:07:05.560 --> 00:07:08.639 code, right, Like okay, Like I have to, you know, 99 00:07:08.680 --> 00:07:10.920 pause what I'm doing, maybe jump out of a meeting and I'm like, 100 00:07:11.120 --> 00:07:14.279 hey, this, what do I have to understand this PR? Or what's 101 00:07:14.319 --> 00:07:18.240 the context I need? We can start leveraging automations like the guests through automation 102 00:07:18.279 --> 00:07:23.639 that linear B has to provide context, and then Uber's actually saying, hey, 103 00:07:23.680 --> 00:07:27.759 we're going a step farther and we're actually going to help generate comments, 104 00:07:27.759 --> 00:07:31.800 suggested comments that people can then evaluate immediately in that PR to help improve the 105 00:07:31.839 --> 00:07:38.199 PR's quality. So huge opportunities there, And I know Ubers Developer Experience and 106 00:07:38.279 --> 00:07:43.199 Developer Productivity teams have estimated they saved about ten million dollars in time saved for 107 00:07:43.360 --> 00:07:48.920 engineers last year alone. And that doesn't even consider the potential positive benefits on 108 00:07:48.959 --> 00:07:51.759 that happiness piece we talked about earlier. Of like, if I'm a dev 109 00:07:51.759 --> 00:07:56.399 on a team and these annoying tasks are starting to get automated away, or 110 00:07:56.399 --> 00:07:59.720 at least it's getting easier for me to do them, whether that's tests or 111 00:08:00.199 --> 00:08:03.639 reviewing code, We're having better tools to help me that are enabled by generative 112 00:08:03.680 --> 00:08:09.040 AI. This is a huge opportunity to improve retention, improve de have happiness. 113 00:08:09.079 --> 00:08:13.360 And we've already seen a million correlations across various studies with happiness and high 114 00:08:13.399 --> 00:08:18.480 performing teams. So it's a really exciting thing for the industry. Okay, 115 00:08:18.519 --> 00:08:22.639 so we've got a we've got a gambit from there are people who literally do 116 00:08:22.720 --> 00:08:28.199 not start coding without some sort of AI integration. And then on the other 117 00:08:28.279 --> 00:08:39.039 side, we've got people who vehemently refuse or are banned from opening such such 118 00:08:39.120 --> 00:08:46.159 tool. So can you tell us anything about that spectrum and it was any 119 00:08:46.200 --> 00:08:50.240 of that type of data captured in the report? Yeah, And this report 120 00:08:50.240 --> 00:08:52.759 will be out for everyone to read here the next few days, so definitely 121 00:08:52.879 --> 00:08:56.639 go to linear beat on io to check out more information here. It should 122 00:08:56.639 --> 00:09:01.480 be out forty eight hours, is ish from now. But the exciting thing 123 00:09:01.600 --> 00:09:03.120 here, as you mentioned, Aja, is that you know, there's these 124 00:09:03.159 --> 00:09:07.639 opportunities for improvement, but there are risks. And this is one of the 125 00:09:07.720 --> 00:09:11.360 key things that we captured in this report, which is the risks are perceived 126 00:09:11.399 --> 00:09:18.159 as very much compliance quality. IP security is a top concern for devs broadly, 127 00:09:18.639 --> 00:09:22.360 but particularly with generative AI tools, where they can be kind of a 128 00:09:22.399 --> 00:09:26.919 black box of what's this code that's coming out of here, We're you know, 129 00:09:26.080 --> 00:09:30.519 giving access to our code base, what's the risk level here? And 130 00:09:31.279 --> 00:09:35.720 what we saw is that risk levels or perceived risk levels drop across the board 131 00:09:35.919 --> 00:09:41.480 as adoption grows. But IP and compliance are major concerns for frontline managers and 132 00:09:41.519 --> 00:09:46.399 executives. And interestingly, something we saw is that the frontline managers are actually 133 00:09:46.480 --> 00:09:52.519 more concerned about these risks, maybe because they have more in depth understanding of 134 00:09:52.559 --> 00:09:54.960 what the tools are actually doing, maybe simply because they know they feel more 135 00:09:56.000 --> 00:09:58.919 accountable to it than the directors and executives, who I think are feeling a 136 00:09:58.960 --> 00:10:01.559 lot more pressure to go, oh, hey, I have an initiative I 137 00:10:01.600 --> 00:10:05.399 need to do. My board is telling me we have to have an efficiency 138 00:10:05.440 --> 00:10:07.840 initiative, we have to do more with less. I'm sure we've all heard 139 00:10:07.840 --> 00:10:11.879 that terminology thrown around, and so there's this odd change between you know, 140 00:10:11.919 --> 00:10:15.440 the business side which is really pushing this and says, you know, we 141 00:10:15.559 --> 00:10:20.120 have to improve our software development with AI and the managers who are are to 142 00:10:20.159 --> 00:10:24.840 your point at AGA concerned, and I think frontline devs as well in some 143 00:10:24.879 --> 00:10:26.879 cases, who are saying, hey, is this the right thing to be 144 00:10:26.919 --> 00:10:31.120 doing? Does this actually help me? In and AJ Before we started recording, 145 00:10:31.159 --> 00:10:35.080 I know you mentioned that in your experiments with genet of AI code you 146 00:10:35.279 --> 00:10:41.000 felt like there are times where maybe it just wasn't getting things right and you 147 00:10:41.000 --> 00:10:43.960 weren't as tightly aligned to what you actually needed here, and that is definitely 148 00:10:45.159 --> 00:10:46.759 something we saw as well. So there's a huge opportunity for improvement where like 149 00:10:46.799 --> 00:10:50.679 basic tasks, automations, tests, these are things can really be helped. 150 00:10:50.919 --> 00:10:56.600 But as code gets more and more complicated, it's harder to get your GENAI 151 00:10:56.679 --> 00:11:01.879 code assistant to actually match the in depth needs of very particular coding needs. 152 00:11:01.879 --> 00:11:05.919 So there's very much still a need for senior developers to come in and say, 153 00:11:05.919 --> 00:11:09.399 hey, I understand this code based, let me help guide this, 154 00:11:09.679 --> 00:11:13.120 let me make this work. And you know, one of the things that 155 00:11:13.159 --> 00:11:16.759 we're predicting is that that need for senior dev is actually going to really increase, 156 00:11:16.840 --> 00:11:22.080 because, particularly as junior devs start leveraging Genai code to just move faster 157 00:11:22.240 --> 00:11:24.360 deliver more code into the system, we're going to need a lot more senior 158 00:11:24.399 --> 00:11:28.159 devs who can help parse through that and make sure the quality is there. 159 00:11:28.559 --> 00:11:31.279 And there's a huge risk that as we just simply increase the speed which which 160 00:11:31.399 --> 00:11:35.759 we generate code, that there are quality issues in that review cycle, in 161 00:11:35.799 --> 00:11:41.200 that actual delivery cycle, because we designed all of our software development systems to 162 00:11:43.440 --> 00:11:45.720 have a certain amount of code going through them, and now we're saying maybe 163 00:11:45.759 --> 00:11:48.600 we're have twenty percent more code, maybe have fifty percent more code. Are 164 00:11:48.759 --> 00:11:52.639 the different systems of review, systems of tests that we expected to have here 165 00:11:52.000 --> 00:11:54.639 going to work or are we going to have these huge blockages that maybe then 166 00:11:54.679 --> 00:11:58.919 people don't spend as much time reviewing. And this is where there's this need 167 00:11:58.919 --> 00:12:01.200 for us next level out of me like what I mentioned Uber so, I 168 00:12:01.320 --> 00:12:03.000 like what we're doing with our free get stream tool, say okay, how 169 00:12:03.000 --> 00:12:09.679 can we help these blockages in the software development life cycle? These area where 170 00:12:09.720 --> 00:12:16.279 the pipe kind of tightens up, particularly code review move faster. So how 171 00:12:16.440 --> 00:12:22.039 is code review going to be affected by AI? Because to me, like, 172 00:12:22.000 --> 00:12:28.840 okay, so my experience with it, I use both Olama with the 173 00:12:28.360 --> 00:12:33.120 code Up I think it was code up and code Wizard. There's two models 174 00:12:33.159 --> 00:12:39.799 that I've switched between. With Olama and then with GPT chat GPT I use 175 00:12:41.440 --> 00:12:46.519 just the main website. I don't have any VIM integration for code gen in 176 00:12:46.600 --> 00:12:52.559 part because I've from what I've seen and experienced thus far, having it pop 177 00:12:52.679 --> 00:12:58.000 up bad code half the time and me having to do the mental context switch 178 00:12:58.080 --> 00:13:01.039 of like am I writing? Am I correcting code? Back and forth? 179 00:13:01.120 --> 00:13:05.600 That's like that, I'm that's frustrating to me. So I'd rather just take 180 00:13:05.639 --> 00:13:09.360 a few extra seconds to type things out and just type out what I know 181 00:13:09.440 --> 00:13:16.559 I need to do rather than like approve or deny and retry on something that's 182 00:13:16.639 --> 00:13:22.399 relatively you know simple on cogen. So for like little snippets and stuff, 183 00:13:22.440 --> 00:13:26.519 I'm not doing it, but I will. I'll ask a GPT a question. 184 00:13:26.559 --> 00:13:33.759 And again it's like it's like fifty percent of the time it's it's worth 185 00:13:33.799 --> 00:13:35.279 it, I guess, because you know, it'll get it right, it'll 186 00:13:35.279 --> 00:13:37.919 get it wrong. But I'll ask you the question, and it seems like 187 00:13:39.919 --> 00:13:43.080 the closer my question is to Python code to start, the better. It's 188 00:13:43.120 --> 00:13:52.360 going to be a generating go or job. But but there is some value 189 00:13:52.399 --> 00:13:56.000 there because there are some things that are relatively common boilerplate that are you know, 190 00:13:56.080 --> 00:14:00.320 it might be thirty lines of boilerplate and I'll ask it to do something 191 00:14:00.399 --> 00:14:03.720 that I know is maybe in that realm, and it can generate it. 192 00:14:03.720 --> 00:14:05.639 It's very good at that. Yeah, it is. It is. I 193 00:14:05.679 --> 00:14:09.039 also asked it to write a backup routine and it wrote a function that would 194 00:14:09.039 --> 00:14:16.200 delete all files in the folder. Yeah. That tracks as I think we've 195 00:14:16.240 --> 00:14:22.200 both experienced here. The more complicated your use case, the farther away Jenai 196 00:14:22.440 --> 00:14:24.480 is from being ready to really solve it for you without deep fine tuning. 197 00:14:24.559 --> 00:14:28.519 This is where major enterprises have the opportunity to say, hey, we really 198 00:14:28.559 --> 00:14:31.440 want to fine tune this tool internally so that it fits our code. Basic 199 00:14:31.440 --> 00:14:35.840 it fits our needs, but right now it's really good at the basics helping 200 00:14:35.879 --> 00:14:39.840 solve these kind of pain points. And where we're seeing it. The interesting 201 00:14:39.919 --> 00:14:43.559 code review, as you asked, is that we have just more code moving 202 00:14:43.559 --> 00:14:46.120 through the system. Because the basics are all easier now, people can now 203 00:14:46.320 --> 00:14:50.000 just deliver code faster. And so now we're seeing this major pain point which 204 00:14:50.039 --> 00:14:54.679 was frankly already a pain point for many software development teams, which is, 205 00:14:56.120 --> 00:14:58.600 not only are there more code reviews to be done, there's more code to 206 00:14:58.639 --> 00:15:03.519 be reviewed. People are still spending most of their time on generating code. 207 00:15:03.559 --> 00:15:09.240 So what we've seen is this already existent pain point get exacerbated by the increased 208 00:15:09.240 --> 00:15:13.159 speed which code is generated. Yeah. Yeah, but oh so what I 209 00:15:13.440 --> 00:15:16.720 was gonna say is what I see it do is I'll see it add comments 210 00:15:16.720 --> 00:15:20.759 to code and asking it to summarize code. It can it can often do 211 00:15:20.799 --> 00:15:24.320 a really good job at that, even even if it's something pretty niche. 212 00:15:24.360 --> 00:15:28.240 Like there's been cryptographic stuff that I didn't understand and I was working in some 213 00:15:28.240 --> 00:15:31.440 cryptographic library and I paste in some code and I say, you know, 214 00:15:31.759 --> 00:15:35.759 what does what does MKW stand for? Here? And it's like that's the 215 00:15:35.799 --> 00:15:41.840 Mills Buiser kill Awowsen curve. And I'm like, oh, okay, is 216 00:15:41.840 --> 00:15:43.919 that a real thing? And then you know, it turns out it actually 217 00:15:45.039 --> 00:15:46.759 is, and I'm like, oh, okay, that's that. And so 218 00:15:46.919 --> 00:15:50.840 it's done really good at generating comments, but then most of the time it's 219 00:15:50.840 --> 00:15:56.320 pretty dumb. It's you know, because it's this adds two and four to 220 00:15:56.399 --> 00:16:03.000 get six. I'm like, yeah, yeah. And so when I'm when 221 00:16:03.039 --> 00:16:07.799 I'm thinking about in code review, I just most of what it would do. 222 00:16:08.159 --> 00:16:14.000 I you know, if it was something for a cryptographic library, I 223 00:16:14.240 --> 00:16:18.519 would you know, want somebody who's at a really high level to be handling 224 00:16:18.600 --> 00:16:22.120 that because it's something that's really sensitive and we don't want to take a chance 225 00:16:22.200 --> 00:16:27.559 that it's a hallucination, right, And if it's something that's really basic, 226 00:16:29.320 --> 00:16:33.159 and then it's just going to spit out, Okay, this you know, 227 00:16:33.519 --> 00:16:40.320 this really benign or banal description of code or something. Where where does it 228 00:16:40.399 --> 00:16:44.159 Where is it helpful? What is it doing? That's that's really what's the 229 00:16:44.159 --> 00:16:48.320 sweet spot for that. So I actually think you nailed the opportunity here, 230 00:16:48.320 --> 00:16:52.759 which is context and and saying okay, how can we autogenre generate the right 231 00:16:52.799 --> 00:16:56.879 piece of context. So maybe it's things like something that we're doing is PR 232 00:16:56.000 --> 00:17:00.200 labeling. How long is this PR going to take? We can estimate based 233 00:17:00.240 --> 00:17:03.000 off of a little machine learning on the back end and say oh, this 234 00:17:03.000 --> 00:17:07.480 this PR is x number of lines of code long, one hundred four lines 235 00:17:07.480 --> 00:17:08.839 of code long. This will take five minutes, or this will take thirty 236 00:17:08.880 --> 00:17:14.000 minutes to review. And it'll can also give you, hey, here's a 237 00:17:14.039 --> 00:17:15.599 few things that you might not know what this means to your point, you 238 00:17:15.640 --> 00:17:19.079 know, the MKAV or whatever, and give you that context so that you 239 00:17:19.119 --> 00:17:23.200 can hopefully jump in. And it's a lot easier to get acclimated because too 240 00:17:23.319 --> 00:17:27.200 often, and I'm sure you've experienced this someone you know, you get assigned 241 00:17:27.200 --> 00:17:30.799 as a reviewer for a PR for someone else and you used to go and 242 00:17:30.839 --> 00:17:32.960 you're like, Okay, what are you doing here? You know, maybe 243 00:17:32.960 --> 00:17:34.079 they didn't give you a great comment from the start with of like, oh, 244 00:17:34.079 --> 00:17:37.279 here's the context, and that's where I think, Jenny, I can 245 00:17:37.319 --> 00:17:41.079 start start to help us. So it's helping us very much in the Hey, 246 00:17:41.160 --> 00:17:45.960 let's code let's code through the kind of annoying boiler plate faster. Let's 247 00:17:45.039 --> 00:17:49.680 code test faster. Great, and then let's give context faster so that we 248 00:17:49.759 --> 00:17:53.119 can then make more strategic decisions and spend our time on kind of the higher 249 00:17:53.200 --> 00:17:57.359 leverage opportunities of Okay, does this all work. Let me apply my knowledge. 250 00:18:00.119 --> 00:18:07.400 Let's talk about test for a second. My experience in testing code when 251 00:18:07.400 --> 00:18:15.759 I really feel like I need tests is usually one of two situations. It's 252 00:18:15.799 --> 00:18:26.559 usually either a regular expression or it's something where I actually am not certain what 253 00:18:26.920 --> 00:18:30.640 the result is supposed to be. So I'm writing a test kind of to 254 00:18:30.680 --> 00:18:36.079 say I think the result is supposed to be this, and then if I 255 00:18:36.160 --> 00:18:41.079 find a case where it falls outside of those parameters, then I need to 256 00:18:41.200 --> 00:18:47.559 manually review and say, Okay, is it that the code is not handling 257 00:18:47.599 --> 00:18:51.200 an edge case or is it there was an edge case in the input that 258 00:18:51.279 --> 00:19:00.680 needs to be handled in the test I'm typically not doing. I I don't 259 00:19:00.720 --> 00:19:03.839 think that the type of when I'm when I'm reaching for tests, I mean, 260 00:19:03.880 --> 00:19:07.839 I guess the regular expression stuff? Could I mean the test is really 261 00:19:07.880 --> 00:19:11.640 there more to explain what the regular expression does. It's more like a comment 262 00:19:11.880 --> 00:19:14.880 at that point. But then again, so is you know that other thing? 263 00:19:15.000 --> 00:19:21.920 So? So what kind of test generation again? Is is where the 264 00:19:22.039 --> 00:19:26.920 value is being brought to the table. Yeah, so I would say two 265 00:19:26.960 --> 00:19:30.680 things, And I'll admit here that I haven't spent a ton of time generating 266 00:19:30.720 --> 00:19:33.400 test with Jenny Echo, this is very much something that I'm hearing secondhand from 267 00:19:33.400 --> 00:19:36.039 people that I'm seeing in our data. I've heard it. I've heard it. 268 00:19:36.160 --> 00:19:38.119 I just haven't seen it. I haven't seen I totally will say, 269 00:19:38.359 --> 00:19:42.680 oh yeah, I'm generating my tests, but I haven't seen the tests that 270 00:19:42.720 --> 00:19:45.440 were generated to to be like, oh yeah, that brings a lot of 271 00:19:45.480 --> 00:19:48.480 value to the table I should I for two examples, so so one that 272 00:19:48.519 --> 00:19:52.640 regular expression when you mentioned totally a great use case like this is something that's 273 00:19:52.839 --> 00:19:55.720 very parsible. It's I don't want to call it basic, but it's like, 274 00:19:55.720 --> 00:19:56.640 okay, it's boiler plate that we've kind of talked about. They can 275 00:19:56.640 --> 00:20:00.799 get this, let's let's help new things faster. Gave me a little time, 276 00:20:00.880 --> 00:20:03.680 and the edge cases I think are still figured out. Though maybe you'll 277 00:20:03.720 --> 00:20:06.680 have someone who's going to come on next week who's going to be an expert 278 00:20:06.680 --> 00:20:08.559 in edge case testing with Jenna I and they can help help lead us there. 279 00:20:08.880 --> 00:20:15.079 But the other area here is that when you have a large enough database, 280 00:20:15.599 --> 00:20:18.680 So I'm going to keep using this Uber example because it's something that they 281 00:20:18.000 --> 00:20:22.480 looked at and they're saying, Okay, now we have thousands of devs. 282 00:20:22.039 --> 00:20:26.200 We have an idea of basically what our codebaseys we know a functions. Let's 283 00:20:26.279 --> 00:20:30.519 take all of those tests and make them artifacts that help train this AI. 284 00:20:30.559 --> 00:20:34.319 It's particularly saying, okay, let's suggest tests for you right off the bat 285 00:20:34.599 --> 00:20:38.599 for some of those EDU cases. Here's things that we saw as potential educases 286 00:20:38.680 --> 00:20:42.839 that are issues in previous similar areas of the code base. Let's just suggest 287 00:20:42.839 --> 00:20:45.359 some right off the bat for you. And that's an area that I think 288 00:20:45.440 --> 00:20:51.279 is a huge opportunity for companies that and particularly like B to B companies that 289 00:20:51.359 --> 00:20:52.680 are working with a variety of clients. We can say, hey, can 290 00:20:52.720 --> 00:20:57.240 we train a great database to improve testing off of this? And I think 291 00:20:57.279 --> 00:21:00.599 there's a lot to be done there still, but we're hearing great results from 292 00:21:00.640 --> 00:21:04.680 individual devs and small dev teams, and we're seeing some enterprise companies succeeded that. 293 00:21:04.799 --> 00:21:10.200 So I expect there to be an explosion of these tools moving forward. 294 00:21:10.720 --> 00:21:12.200 Yeah. So, I think one really interesting thing we saw on the report 295 00:21:12.519 --> 00:21:18.440 was one of the things we asked Serve of respondents was hey, do you 296 00:21:18.559 --> 00:21:22.759 prefer qualitative metrics or quantitative metrics? Ie, do you want hard data of 297 00:21:22.839 --> 00:21:26.640 like this is ten percent better or do you want Hey, we saw developer 298 00:21:26.640 --> 00:21:32.359 happiness improve. And what we saw is that, particularly with larger companies who 299 00:21:32.359 --> 00:21:36.119 are usually farther along in their adoption journey for AI, they just have the 300 00:21:36.200 --> 00:21:38.839 resources to do it and add it more than the software development process. They 301 00:21:38.839 --> 00:21:41.119 were ready to start saying, hey, we want these hard metrics. We 302 00:21:41.160 --> 00:21:45.720 want to see. Eight percent efficiency gain is the ROI of our Genai initiative, 303 00:21:47.200 --> 00:21:51.680 you know, seven percent of efficiency gain. Whereas earlier on the initial 304 00:21:51.720 --> 00:21:53.400 signals all, hey, our devs are happier, they feel like they're moving 305 00:21:53.400 --> 00:21:57.640 faster. We see this gain in total lines of code created, but those 306 00:21:57.680 --> 00:22:00.480 are all kind of bad mesh. Well, I mean, death happiness isn't 307 00:22:00.519 --> 00:22:03.319 It's an amazing depax measure. We can definitely talk about that. But like 308 00:22:03.400 --> 00:22:07.200 lines of code for example, is I think we all know not necessarily indicative 309 00:22:07.559 --> 00:22:12.799 of great code or you know what you actually want and there are some terrible 310 00:22:14.240 --> 00:22:18.279 favorite there. My favorite prs are more red lines than green lines. Yep. 311 00:22:18.960 --> 00:22:22.279 And this is where I think companies now looking to say, Okay, 312 00:22:22.279 --> 00:22:26.039 what's the actual impact of the code that we generated with gen Ai, And 313 00:22:26.079 --> 00:22:29.359 this is where we're running our beta with linear B we're saying, okay, 314 00:22:29.440 --> 00:22:33.920 let's label every PR that has been code generated with genera of AI or fully 315 00:22:33.920 --> 00:22:37.960 generated with generative AI, and then track its life cycle to see its impact 316 00:22:37.039 --> 00:22:42.039 on the codebase. And we're we're very much just starting to get data there. 317 00:22:42.039 --> 00:22:45.839 We're going to be putting some out in a workshop here in a couple 318 00:22:45.880 --> 00:22:48.680 of weeks. But I'm excited about the long term potential there because I think 319 00:22:48.680 --> 00:22:52.519 we're going to have this transition over the next couple of months from you know, 320 00:22:52.599 --> 00:22:56.359 survey data which is super useful, to survey data backed by these hard 321 00:22:56.440 --> 00:23:00.559 data metrics, these quantitative data metrics that that can really dive into it. 322 00:23:00.599 --> 00:23:03.519 And this is where you also see this in developer experience, which kind of 323 00:23:03.559 --> 00:23:07.799 like was the early indicator here of success. So, as I mentioned in 324 00:23:07.839 --> 00:23:14.319 our work with Google Clouds State of Software Development Report last year, we saw 325 00:23:14.400 --> 00:23:18.279 these efficiency gains for teams because team members were happier, and it was kind 326 00:23:18.279 --> 00:23:22.240 of correlating this positive feedback cycle for dev teams based off of the leveraging gen 327 00:23:22.319 --> 00:23:27.000 AI, whether it's for tests or coding, those basics. And now I 328 00:23:27.039 --> 00:23:30.400 think we're starting to see this wave and are going to continue to see this 329 00:23:30.440 --> 00:23:33.440 wave of here's the actual impact of that org long term, here's where you 330 00:23:33.440 --> 00:23:37.519 know they delivered X number of futures faster. And that's what we're really excited 331 00:23:37.519 --> 00:23:44.119 about, is that next step, and that's where we're building towards. Okay, 332 00:23:44.720 --> 00:23:49.440 and then the developer experience, and I want to allow us to take 333 00:23:49.440 --> 00:23:56.200 this conversation outside of just the AI portion totally, but with the developer experience. 334 00:23:56.799 --> 00:24:00.519 What were the areas in which that happiness is improving or that and well 335 00:24:00.559 --> 00:24:06.519 and also I want to know, I want to know what what tools actually 336 00:24:06.559 --> 00:24:11.839 before I go into the question about the happiness what what tools are people using? 337 00:24:11.960 --> 00:24:15.680 Because I'm I'm familiar with Chad, GBT and Olama those are the ones 338 00:24:15.720 --> 00:24:21.000 that I use the most. And then I know that a lot of people 339 00:24:21.039 --> 00:24:26.400 are using vs code with the Microsoft uh CHATGBT plug in, the co pilot 340 00:24:26.440 --> 00:24:30.680 plug in. What what else is there out there that people are using. 341 00:24:30.039 --> 00:24:33.880 Yeah, so obviously Copilot's the one hundred pound gorilla here. They've got I 342 00:24:33.920 --> 00:24:38.480 think one hundred and fifty thousand organizations across the world using it Chat GPT being 343 00:24:38.519 --> 00:24:44.559 the more broadly available for every individual dev and everyone else, code whispers and 344 00:24:44.599 --> 00:24:47.480 other that's out there that we saw. It was quite popular in our survey. 345 00:24:48.119 --> 00:24:51.359 Codium, dot AI and TAB nine were a few others that we saw. 346 00:24:51.720 --> 00:24:56.759 But you named both of the key ones there, okay, and then 347 00:24:56.799 --> 00:25:02.359 Tab nine man, oh they were doing they were doing a marketing push. 348 00:25:03.200 --> 00:25:10.440 And then I think two weeks later, uh, Microsoft announced the GPT copilot. 349 00:25:11.240 --> 00:25:15.400 I was so sad for them that their timing was tough. Yeah, 350 00:25:15.519 --> 00:25:19.519 yeah, but it's hard to compete with that, a company like Microsoft when 351 00:25:19.519 --> 00:25:22.000 they decided to roll something all like that, well, and I mean, 352 00:25:22.000 --> 00:25:26.240 and you don't know that it's coming right, like, yeah, they they 353 00:25:26.279 --> 00:25:30.680 had they probably they've been working on their platform for however long, you know, 354 00:25:32.039 --> 00:25:37.160 a few years, finally getting out of beta and and then just overnight. 355 00:25:38.759 --> 00:25:42.519 Oh that's so rough. That's so rough. I did use Tab nine 356 00:25:42.720 --> 00:25:47.759 and I kind of had the same you know, feeling feeling about it like 357 00:25:47.799 --> 00:25:51.680 it was. It was. It was cool, it did. It was 358 00:25:51.799 --> 00:25:56.200 so good with refactoring, so good in terms of you know, I need 359 00:25:56.200 --> 00:26:00.920 to go change the next ten lines to be from I don't know grams to 360 00:26:00.039 --> 00:26:03.720 kilograms or whatever, you know, some sort of I need to add sql 361 00:26:03.759 --> 00:26:08.000 tags to the comments I needed whatever, like, it was so good at 362 00:26:08.000 --> 00:26:11.240 that sort of thing. And I imagine code Pilot probably is too, although 363 00:26:11.319 --> 00:26:17.559 or co Pilot, although I haven't used that. And then Codium I think 364 00:26:17.599 --> 00:26:22.559 I've heard of this one. Is this any relation to VS Codeum Yes, 365 00:26:25.680 --> 00:26:27.720 Codeum not AI is its own website, but I do believe it's related. 366 00:26:27.799 --> 00:26:30.400 If I'm not miss speaking here, maybe Steve's gonna correct me here. Well, 367 00:26:30.480 --> 00:26:34.880 no, I just said a panelist co panelist to mine on another podcast 368 00:26:36.079 --> 00:26:38.119 was using Codium and he really liked it. He didn't get into details, 369 00:26:38.160 --> 00:26:44.519 but it was just the way that AI was just sort of built into everything 370 00:26:45.599 --> 00:26:48.200 in the ide was was the way he described it is very helpful. So 371 00:26:48.240 --> 00:26:52.480 I haven't used it myself for seen it, but I just remember his description 372 00:26:52.599 --> 00:26:56.240 of it. I know they have uh well the jet Brains and VS code 373 00:26:56.279 --> 00:27:00.200 Free versions, so I do. I do believe they're related there, Oh, 374 00:27:00.279 --> 00:27:07.000 je brands, I use PHP storm. I checked that out. Well, 375 00:27:07.440 --> 00:27:10.759 you know, I'd be happy to pay for their their their VIM plug 376 00:27:10.799 --> 00:27:17.440 in when that comes out. Okay, is this one more privacy focused or 377 00:27:17.440 --> 00:27:21.359 something? Or what makes it? Do you know what makes it different I'll 378 00:27:21.359 --> 00:27:22.759 be honest, I don't. I don't know a ton about what makes that 379 00:27:22.880 --> 00:27:29.359 makes them different. I've been pretty focused on I'm looking at copy it because 380 00:27:29.359 --> 00:27:33.599 the data sets a lot bigger, and kind of evaluating our metrics across three 381 00:27:33.640 --> 00:27:37.640 areas, So like adoption AKA, are you opening more pole requests? Are 382 00:27:37.640 --> 00:27:40.440 you getting more poll requests merged? Are we just seeing more code flowing through? 383 00:27:40.720 --> 00:27:45.240 And then the benefits piece of are you merging more frequently? Is coding 384 00:27:45.319 --> 00:27:48.720 time going down? Are more stories getting completed? Are you being more accurate 385 00:27:48.759 --> 00:27:52.240 when you plan stories? And then also we're starting to look at the risks 386 00:27:52.279 --> 00:27:56.480 too, like our PR size is bloating, is rework rate up? Is 387 00:27:56.519 --> 00:28:00.519 review depth down? Because people are just kind of skimming our PR has emerged 388 00:28:00.559 --> 00:28:06.079 that review that kind of thing. What about do you have anything in place 389 00:28:06.400 --> 00:28:11.440 for license violations? Because I think that it's it's been shown that in many 390 00:28:11.519 --> 00:28:17.720 cases it's almost a direct copy and paste from the source material, And I 391 00:28:17.759 --> 00:28:22.000 think that I haven't experienced this myself, but I've heard that people have complained 392 00:28:22.200 --> 00:28:30.359 that it's generated GPL license code as like a whole block, which you know 393 00:28:30.440 --> 00:28:33.960 has which is really weird because Microsoft, for the longest time, you know 394 00:28:33.000 --> 00:28:38.440 that they wouldn't even they would not allow their employees to look at GPL code. 395 00:28:40.000 --> 00:28:44.079 And to think that we've got an AI scanning all of GitHub where I 396 00:28:44.079 --> 00:28:47.119 don't know what five percent of the code, I'm just making up a number, 397 00:28:47.839 --> 00:28:51.240 you know, But it's like MIT and Apache. You're probably the highest 398 00:28:51.279 --> 00:28:56.960 BSD, but there's a lot of GPL code out there. Any any thoughts 399 00:28:56.000 --> 00:29:02.200 or observations on the licensing concern, I think it's a huge concern to your 400 00:29:02.240 --> 00:29:06.119 point, like, this is one of the areas where we saw significant serns 401 00:29:06.160 --> 00:29:10.880 this compliance side of things. We don't have a full solve for it yet. 402 00:29:10.960 --> 00:29:12.160 I think this is something well, you can expect us to have an 403 00:29:12.160 --> 00:29:18.000 automation out on our get stream platform within linear b here in the next month 404 00:29:18.079 --> 00:29:22.759 or two that will hopefully help track that. But it's a it's definitely a 405 00:29:22.839 --> 00:29:23.640 risk that I think companies are going to need to solve in it. I 406 00:29:23.640 --> 00:29:27.400 mean, we're seeing lawsuits over this kind of thing, not just in the 407 00:29:27.440 --> 00:29:32.200 code space, but look at the New York Times suing open ai for use 408 00:29:32.279 --> 00:29:36.640 of their content and models. So the legal side of this is going to 409 00:29:36.720 --> 00:29:41.319 be a massive thing. To unfurl over the coming months, no question. 410 00:29:41.640 --> 00:29:44.960 Yeah, I can remember. I can remember the big stink when when Gethub's 411 00:29:44.960 --> 00:29:48.839 copilot first came out that I remember seeing on Hacker News or other places that 412 00:29:48.880 --> 00:29:55.279 there were authors that had private repos you know, with non GPO licenses, 413 00:29:55.359 --> 00:30:00.279 so it should not have been annexed by by copilot, and they would type 414 00:30:00.319 --> 00:30:03.160 in and here comes their code, just you know, straight out of the 415 00:30:03.160 --> 00:30:06.559 way they read it coming up from copilot when it wasn't supposed to be. 416 00:30:07.279 --> 00:30:10.400 So that was I can remember that stink, and I haven't really followed it 417 00:30:10.440 --> 00:30:15.400 since then as to what actually happened with those instances. But it's just struck 418 00:30:15.440 --> 00:30:19.480 me as odd as that I would think that, you know, co pilot 419 00:30:19.480 --> 00:30:23.440 being a GitHub product and GitHub having access to all of the repos, how 420 00:30:23.440 --> 00:30:26.960 you don't have a flag in there says okay, if this isn't private, 421 00:30:26.039 --> 00:30:30.640 don't going into exist. Code didn't seem real complex if you own the whole 422 00:30:30.680 --> 00:30:33.880 back in. But you know, obviously I'm not sure how that works, 423 00:30:33.920 --> 00:30:37.839 but that was a big issue. Yeah, it's a great question, Steeve, 424 00:30:37.839 --> 00:30:40.759 and it's really interesting too because we've talked about these like three use cases 425 00:30:40.799 --> 00:30:44.279 for it right of you know, writing new code, writing tasks, assisting 426 00:30:44.279 --> 00:30:48.359 in code reviews, and then there's this whole other side of i'll call it 427 00:30:48.359 --> 00:30:53.039 like generating documentation, and some of those it's totally fine. Like genering documentation. 428 00:30:53.160 --> 00:30:57.039 Sure, I don't I don't care if I'm copying something probably like maybe 429 00:30:57.039 --> 00:31:00.599 there's some compliance concerns, but that writing new code piece, there are absolute 430 00:31:00.880 --> 00:31:04.440 clients concerns. And and I know Copilot is working on this, but to 431 00:31:04.480 --> 00:31:07.240 your point, it's surprising they're not a little farther along given that it is 432 00:31:07.240 --> 00:31:14.279 a Microsoft for Lata product. Yeah, and I do wonder, Okay, 433 00:31:14.400 --> 00:31:18.759 I've seen one of these reports that came out where it generated like entire paragraph 434 00:31:18.799 --> 00:31:25.799 of paragraphs or text, or it would generate images that looked almost identical to 435 00:31:25.839 --> 00:31:30.640 the original but with basically noise on top of it. And in that case, 436 00:31:30.680 --> 00:31:33.920 I think that that is truly a copyright concern. But I also wonder 437 00:31:33.960 --> 00:31:38.799 how much of this is boy cried wolf, where how many ways are there 438 00:31:38.839 --> 00:31:45.119 to implement the you know, area of a triangle function? And so I 439 00:31:45.720 --> 00:31:48.279 do want to, you know, just put in my own caution there, 440 00:31:48.400 --> 00:31:52.599 even though I'm the one that brought this up. I haven't looked at what 441 00:31:52.680 --> 00:31:57.400 these people are saying to verify, Oh, that's a legitimate copyright problem as 442 00:31:57.400 --> 00:32:04.079 opposed to, well, how many ways in Java can you implement a JWT 443 00:32:04.279 --> 00:32:10.480 siding function? You know, maybe it pulled from some quote unquote pulled from 444 00:32:12.039 --> 00:32:15.880 GPL code, But maybe that's because the GPL code is also the same as 445 00:32:15.920 --> 00:32:20.680 the MIT code and the Apache code for those three lines that everybody has in 446 00:32:20.720 --> 00:32:23.079 common. And I think that that was one of the things in that Oracle 447 00:32:23.160 --> 00:32:30.799 versus Google lawsuit over Java that if I recall correctly, ultimately they ruled in 448 00:32:30.880 --> 00:32:37.240 favor of Google, saying yeah, like for common mathematical functions and you know 449 00:32:38.079 --> 00:32:43.480 inputs and outputs of an API that you can't copyright pat in that stuff because 450 00:32:43.480 --> 00:32:49.920 it's just too basic. So you know, there's there's probably some mix in 451 00:32:49.960 --> 00:32:53.279 there of people crying wolf and making a big della nothing and then and then 452 00:32:53.400 --> 00:32:58.240 some actual you know like what Steve was saying, where you know, there's 453 00:32:58.279 --> 00:33:01.079 some sort of proprietary code but regular to a company, and then boom, 454 00:33:01.119 --> 00:33:04.880 all of a sudden, it's been shared. And actually, that's one question 455 00:33:05.000 --> 00:33:08.359 I had, is that how much of this code because with chat GPT, 456 00:33:09.000 --> 00:33:13.039 the license with chat GPT, and I wish they would make it easier to 457 00:33:13.039 --> 00:33:15.839 get a business account. I'm trying to get a business account and it is 458 00:33:15.279 --> 00:33:21.079 it is really hard to get a business account with OpenAI. But anytime you 459 00:33:21.160 --> 00:33:24.880 use chat GPT, anything that you type in or paste in goes into the 460 00:33:24.920 --> 00:33:29.920 pool for model learning. But on the business account, if you use the 461 00:33:29.960 --> 00:33:37.359 API and then the it doesn't go into the pool for model learning, you 462 00:33:37.440 --> 00:33:42.240 keep it proprietary. Do you know anything about that? Well, I would 463 00:33:42.240 --> 00:33:45.400 just say that I think this is a huge opportunity for open ai and all 464 00:33:45.400 --> 00:33:49.519 these other companies here, and this is why you're seeing them get these crazy 465 00:33:49.559 --> 00:33:53.160 valuations, is because every business is going to want to secure their own code 466 00:33:53.160 --> 00:33:59.680 base and data and they don't want to extend it into the main chat GPT 467 00:33:59.759 --> 00:34:02.480 data. So to your point, AJ like, this is a reason for 468 00:34:02.559 --> 00:34:07.119 every business to look at that business business style version of it and paid that 469 00:34:07.239 --> 00:34:10.280 extra dollar. Is that compliance use case? And it's a classic B two 470 00:34:10.280 --> 00:34:13.880 B one right where it's like, oh, you know, anyone can get 471 00:34:13.880 --> 00:34:16.079 our basic tooling, but you know you have to get our enterprise package in 472 00:34:16.199 --> 00:34:21.039 order to add the compliance piece add you know, role based access control, 473 00:34:21.079 --> 00:34:22.679 these other things, and I think you're absolutely going to see that. I 474 00:34:22.719 --> 00:34:28.360 mean, we already are with all these AI companies, all right, So 475 00:34:28.639 --> 00:34:30.280 now now let's take the turn. Unless, Steve, do you have any 476 00:34:30.280 --> 00:34:35.119 more questions on this topic? Now, let's let's take the turn and go 477 00:34:35.400 --> 00:34:44.519 into the developer experience the developer happiness topic genre. So we go ahead. 478 00:34:44.599 --> 00:34:46.599 Let's start with the AI what what are the biggest improvements, what are people 479 00:34:46.639 --> 00:34:52.840 excited about, What's what's making them happy? Yeah, So I mentioned Dora's 480 00:34:52.880 --> 00:34:55.239 report from last year, and I kind of I'll just say the twenty twenty 481 00:34:55.239 --> 00:34:59.960 three accelerate State of depop Support is what I'm saying when I'm referencing that. 482 00:35:00.079 --> 00:35:05.320 So I interviewed Nathan Harvey, who's the head of Dora's kind of outreach team, 483 00:35:05.559 --> 00:35:08.800 on our dev Rupted podcast last year and we were also partners Lenard View 484 00:35:08.840 --> 00:35:13.880 as partners on the report data. And one of the really interesting things we 485 00:35:13.920 --> 00:35:16.840 saw in that report is set they saw among the three thousand plus organizations they 486 00:35:16.880 --> 00:35:22.760 surveyed that last year that about fifty percent of them were using AI tools already 487 00:35:22.159 --> 00:35:27.719 and most were already considering an adoption, but of those folks, there was 488 00:35:27.719 --> 00:35:31.920 a correlation with improved developer happiness on dev teams, and so this is where 489 00:35:31.960 --> 00:35:37.519 we've started to see these great examples of teams that are saving time on documentation, 490 00:35:37.679 --> 00:35:40.360 saving time on code review, saving time on tests, saving time on 491 00:35:40.400 --> 00:35:46.440 codgeneration, and it's helping devs get rid of these annoying blockers in their time 492 00:35:47.360 --> 00:35:51.599 and spend more time on the strategic inputs. And so it's a really key 493 00:35:51.639 --> 00:35:58.000 piece of how you can potentially make your team both more efficient and happier. 494 00:35:58.039 --> 00:36:01.199 And there's plenty of research, whether from Google or many others, that has 495 00:36:01.239 --> 00:36:06.880 shown that happier teams perform better. There's this like positive feedback loop of hey, 496 00:36:07.039 --> 00:36:09.480 I'm feel good, I have more energy, I maybe feel like I'm 497 00:36:09.519 --> 00:36:13.519 being more impactful on my work. And this is where I think a lot 498 00:36:13.559 --> 00:36:15.880 of companies are starting to wake up to the importance of that developer experience idea. 499 00:36:16.320 --> 00:36:21.000 You can look at LinkedIn, which uses a developer experience index, which 500 00:36:21.039 --> 00:36:24.079 is a specialized metric. LinkedIn provides to teams like aggregate scores based on a 501 00:36:24.159 --> 00:36:30.000 number of different metrics such as you know local build times and you know happiness 502 00:36:30.000 --> 00:36:34.320 scores on surveys and a variety of qualitative and quantitative measures, and they use 503 00:36:34.360 --> 00:36:37.920 that to help kind of calibrate the complexity of developing within their orgon. And 504 00:36:38.800 --> 00:36:43.239 there's plenty of the companies that Google does this, Peloton does this. Obviously 505 00:36:43.239 --> 00:36:45.960 they're different pieces. Like for Peloton, for example, they look at engagement, 506 00:36:46.119 --> 00:36:52.159 velocity, quality, and stability. And it's really important that the organization 507 00:36:52.199 --> 00:36:55.559 starts to think about this in the context of am I making my devs perform 508 00:36:55.639 --> 00:36:59.440 better because we're giving them the right tools, We're giving them the right time, 509 00:36:59.559 --> 00:37:02.199 we're right frameworks, and am I going to be able to recruit and 510 00:37:02.239 --> 00:37:06.920 retain dabs because of that? And so whether whether you're an individual developer or 511 00:37:07.840 --> 00:37:09.119 you know, a major corporation is just looking at some of dollars and cents. 512 00:37:09.119 --> 00:37:13.719 Because STEBAC should be something people are thinking about. I would like to 513 00:37:13.760 --> 00:37:16.639 categorize this under the category of the sky is blue and the grass is green 514 00:37:16.679 --> 00:37:22.679 as well. I mean, if you're you have developers and you want them 515 00:37:22.719 --> 00:37:25.599 to do your work, it would seem logical you're going to do best when 516 00:37:25.599 --> 00:37:30.840 you give them the tools that they need. This is I mean it extends 517 00:37:30.880 --> 00:37:35.599 to more than developer tools. There was a recent article that came out and 518 00:37:35.639 --> 00:37:39.840 this has been along the same lines, although it's not specifically code related. 519 00:37:40.239 --> 00:37:45.920 There's whole studies that have been done about the workspace that you give engineers, 520 00:37:45.480 --> 00:37:52.480 and that you will you find that engineers need an office space instead of being 521 00:37:52.480 --> 00:37:59.519 crammed together in cube forms for various reasons. Has mostly to do with focus 522 00:37:59.559 --> 00:38:01.400 and you know, not being interrupted and so on, and that's one of 523 00:38:01.440 --> 00:38:07.639 those things that that to this day is a no brainer. You know, 524 00:38:07.760 --> 00:38:10.920 anybody who's been an engineer or has worked in environments like that will tell you 525 00:38:10.920 --> 00:38:15.159 you, oh, yeah, that I can focus better, I can concentrate, 526 00:38:15.239 --> 00:38:21.440 I can get more done, and yet you still have people that cram 527 00:38:21.519 --> 00:38:24.840 them all in the queue farms, and there's you know, there's various reasons 528 00:38:24.840 --> 00:38:30.480 for that in terms of office space and having only so much space, you 529 00:38:30.519 --> 00:38:35.039 know, versus being able to give everybody their own individual office. I can 530 00:38:35.079 --> 00:38:38.239 still remember being out at Intel. Intel has a lot of their engineering offices 531 00:38:38.280 --> 00:38:43.119 here on the West side of the Portland area and being out there one time 532 00:38:43.199 --> 00:38:46.840 and holy cow, you literally had to have a map to know where you're 533 00:38:46.880 --> 00:38:50.960 going in this gigantic queue farm, and I was like, how do people 534 00:38:50.960 --> 00:38:54.360 get stuff done in here? So, you know, sorry to go off 535 00:38:54.360 --> 00:38:58.360 on a tangent, but you know what you were saying struck to me? 536 00:38:58.480 --> 00:39:01.800 Is you know it struck me as Yeah, Okay, that goes without saying 537 00:39:01.800 --> 00:39:05.920 it doesn't have to be a dead job. Any job here, Go do 538 00:39:05.960 --> 00:39:07.079 this job, but I'm not going to give you the tools you need to 539 00:39:07.119 --> 00:39:12.360 do the job. Well, I output it. What kind of output are 540 00:39:12.360 --> 00:39:15.119 you going to expect from them? Then? The challenge, I think is 541 00:39:15.159 --> 00:39:20.639 that too often, particularly lately, software development and other R and D teams 542 00:39:20.679 --> 00:39:25.960 are being beholden more to I'll call them CFO metrics without thinking about what's actually 543 00:39:27.000 --> 00:39:29.599 happening on teams. How can we improve it? And so it's saying, 544 00:39:29.639 --> 00:39:31.440 okay, well we can save money by doing X. We can you know, 545 00:39:31.559 --> 00:39:35.840 cut hat count on this team and that'll make us more efficient on our 546 00:39:35.840 --> 00:39:37.840 cap table and we can return more value to shareholders. And you want to 547 00:39:37.880 --> 00:39:43.519 know the classic the poster child for this right now? Everybody knows about Boeing. 548 00:39:44.440 --> 00:39:49.320 Yes, okay, Portland two weeks ago, plane takes off, door 549 00:39:49.440 --> 00:39:53.880 panel comes flying off mid flight, two seats get checked out. A twelve 550 00:39:53.920 --> 00:39:58.760 year old sitting next to them has his shirt ripped off. The plane comes 551 00:39:58.800 --> 00:40:00.800 around and is able to to make a landing. It was an Alaska Airline's 552 00:40:00.800 --> 00:40:05.599 flight, which happens to be an airline I fly quite a bit. And 553 00:40:06.079 --> 00:40:10.719 this has been an ongoing thing with Boeing ever since nineteen ninety seven, especially 554 00:40:10.760 --> 00:40:15.159 with well, especially with the seven thirty seven Max line of planes. Where 555 00:40:15.280 --> 00:40:22.360 it used to be. I've seen articles written ad nauseum where Boeing used to 556 00:40:22.440 --> 00:40:25.880 be an engineer driven company and everything was about quality, didn't matter the cost, 557 00:40:27.760 --> 00:40:30.360 Let's get a good plane made, and that's how they developed reputation. 558 00:40:30.440 --> 00:40:35.079 And then in ninety seven they merged with with it Pratt and Whitney or who 559 00:40:35.119 --> 00:40:37.519 was it that they merged with. There was another giant AERONAYM. Mc donald 560 00:40:37.559 --> 00:40:42.920 douglas. I think it was McDonald douglas. And the joke was that McDonald 561 00:40:42.920 --> 00:40:49.519 douglas bought Boeing with their own money because they came in and pretty soon everything 562 00:40:49.599 --> 00:40:52.920 was driven by cost and cost accounting and let's save money. Let's not do 563 00:40:52.960 --> 00:40:55.760 anything that's going to cost a lot of money. So Boeing decides, hey, 564 00:40:55.760 --> 00:40:59.480 we need to keep up with Airbus. But we don't want to take 565 00:40:59.519 --> 00:41:01.880 ten years to develop a whole new product that's going to actually compete, So 566 00:41:01.960 --> 00:41:06.239 let's just take our seven thirty seven and modify it and tweak it. So 567 00:41:06.320 --> 00:41:12.079 right out of the gate, you've got planes crashing because of software that overrode 568 00:41:12.119 --> 00:41:14.920 the pilots and forced the plane to dive into the ground. That happened at 569 00:41:15.000 --> 00:41:20.719 least two times. And now you've got they started outsourcing production, outsourcing building 570 00:41:20.760 --> 00:41:22.760 of a lot of this, which is where this particular problem was tracked to, 571 00:41:23.679 --> 00:41:29.039 and a lot of QA stuff that happened there all because quality went out 572 00:41:29.039 --> 00:41:32.559 the window. And we've got c suite people who weren't engineers making decisions solely 573 00:41:32.599 --> 00:41:37.199 based on money and not on quality. So sorry, that's my rant, 574 00:41:37.280 --> 00:41:45.320 But no, that comes right to mind as soon as I hear that. 575 00:41:45.320 --> 00:41:47.760 That's an incredible example, Steve, and I think it really correlates to something 576 00:41:47.800 --> 00:41:52.199 we're seeing a lot in software engineering right where we could name layoffs that have 577 00:41:52.199 --> 00:41:55.920 happened in January ad nauseum, and a lot of the reason for them, 578 00:41:57.000 --> 00:42:00.800 you know, whether it's Amazon layoffs, YouTube, just score, whatever else 579 00:42:00.320 --> 00:42:05.280 might be. Because Section one seventy four of the US tax code got changed, 580 00:42:05.320 --> 00:42:09.199 which has now made it so that you cannot expend right off R and 581 00:42:09.280 --> 00:42:13.519 D expenses in the United States as part of tax You have to instead cost 582 00:42:13.559 --> 00:42:15.679 capitalize them over five or fifteen years, making it more expensive to develop. 583 00:42:16.039 --> 00:42:20.239 And companies are saying, oh, well, let's just we can outsource this, 584 00:42:20.480 --> 00:42:22.679 Well, let's not actually pay attention to the quality piece. They're making 585 00:42:22.719 --> 00:42:25.199 those same decisions that Boeing did. To your point, see Eve, and 586 00:42:25.239 --> 00:42:29.239 I think you brought us something else that is really interesting, which is the 587 00:42:29.280 --> 00:42:32.559 idea of the space that we built for developers. And I think it's important 588 00:42:32.599 --> 00:42:37.239 to think not only about you know, the office space, the meat space 589 00:42:37.239 --> 00:42:42.440 i'll call it, but also the digital space. Like platform engineering teams can 590 00:42:42.599 --> 00:42:47.039 significantly enhance developer experience by building a tight information loop that identifies pain points and 591 00:42:47.079 --> 00:42:52.519 gathers DEV feedback. And if they use you know, the right internal developer 592 00:42:52.519 --> 00:42:55.440 portals to do this, you can you know, actively solicit develop or input 593 00:42:55.519 --> 00:43:00.000 via surveys and use quantitative measures like local build times and cycle time to under 594 00:43:00.119 --> 00:43:05.480 standing track blockers and broadly improve developer experience. So like thinking about both how 595 00:43:05.480 --> 00:43:08.840 we're setting up devs in person and setting up devs in digital space is really 596 00:43:08.840 --> 00:43:13.079 crucial, and I think you guys are doing great job highlighting that, because 597 00:43:13.519 --> 00:43:16.079 if we instead spend our time thinking, okay, what's the most cost effective 598 00:43:16.119 --> 00:43:21.559 solution, well, it's probably you know, pushing our dev team off to 599 00:43:21.960 --> 00:43:25.280 some other country and cutting our US team entirely. And there are major risks 600 00:43:25.280 --> 00:43:29.679 that come with that, but they're not usually risks in the moment. They're 601 00:43:29.719 --> 00:43:31.960 usually long term risks. And too often companies are thinking, well, I'm 602 00:43:31.960 --> 00:43:36.119 gonna I can cut costs right now, and I think Wall Street or my 603 00:43:36.159 --> 00:43:38.039 investors are really going to like that. You know, my vcs will love 604 00:43:38.079 --> 00:43:43.639 seeing our cap table look better here because we're spending less money on our R 605 00:43:43.679 --> 00:43:47.800 and D costs. And you have to grapple with the whether or not that's 606 00:43:47.800 --> 00:43:51.079 going to be good for your company and the product you're building, whether you 607 00:43:51.079 --> 00:43:52.480 can actually succeed long term. And this is where I think a lot of 608 00:43:52.480 --> 00:43:58.239 companies are making a major mistake is they aren't tracking developer experience metrics. They 609 00:43:58.280 --> 00:44:00.920 aren't tracking their engineering metrics, and oddly so they could see, hey, 610 00:44:00.960 --> 00:44:04.360 there's an efficiency drop, or you know, the pr quality that is being 611 00:44:04.400 --> 00:44:07.320 merged is lower now. And yeah, this example you gave Steve of the 612 00:44:07.360 --> 00:44:12.199 McDonnell Douglas merger with Boeing, I think is a perfect example of this in 613 00:44:12.239 --> 00:44:15.119 an engineering heavy field. And I'll also say, like I was supposed to 614 00:44:15.159 --> 00:44:19.519 be flying a seven thirty seven Max nine the next morning that got canceled, 615 00:44:19.559 --> 00:44:22.760 so right there with you on the Alaska flights getting canceled and the impact there, 616 00:44:22.920 --> 00:44:27.239 and they then ended up finding like bolts loose on multiple planes. What 617 00:44:27.320 --> 00:44:30.920 could happen? Again, very scary yep. And then there was another one 618 00:44:30.960 --> 00:44:37.960 in Japan where uh, there's four layers apparently to the windshield in the cockpit, 619 00:44:37.079 --> 00:44:40.480 and one of them was cracked as they took off. So they came 620 00:44:40.519 --> 00:44:45.119 back around and said, uh, yeah, no, So as far as 621 00:44:45.199 --> 00:44:47.960 I know, all of the that particular seven thirty seven Max not als like 622 00:44:49.000 --> 00:44:55.480 them seven thirty seven Max nine something I think had been grounded. But anyway, 623 00:44:55.960 --> 00:44:59.599 yeah, I think they're bringing some back into service once they've been like 624 00:44:59.679 --> 00:45:05.199 fully checked over. But definitely it's a super scary thing, and I think 625 00:45:05.239 --> 00:45:10.079 we need to have that same risk concern in software engineering, like sure, 626 00:45:10.119 --> 00:45:15.559 maybe we're not going to have a plane blow out its window. But maybe 627 00:45:15.559 --> 00:45:17.920 the software you're building is impacting you know, whether that plan will dive into 628 00:45:17.960 --> 00:45:22.719 the ground, as you mentioned, Steve, or maybe it's impacting people's bank 629 00:45:22.760 --> 00:45:27.360 account information. They're they are very real world concerns with both the compliance and 630 00:45:27.360 --> 00:45:30.719 security risks that we have to consider and also the quality of our code based 631 00:45:30.800 --> 00:45:35.920 generally, and this is why I think it's so crucial that every software engineering 632 00:45:36.000 --> 00:45:40.199 team above a few people needs to have some basic visibility on how is our 633 00:45:40.199 --> 00:45:45.800 team performing? How is our company getting value out of this? And it 634 00:45:45.840 --> 00:45:50.360 shouldn't be about, hey, is this individual developer delivering X lines of code? 635 00:45:50.360 --> 00:45:52.320 Because it's a terrible metric of success, but it needs to be are 636 00:45:52.320 --> 00:45:55.280 we, you know, delivering on our promises to the business? Are we 637 00:45:55.360 --> 00:45:59.800 having to refactor a bunch or or in fact, are we, you know, 638 00:45:59.800 --> 00:46:04.199 how, helping drive revenue by what we're doing. And I think any 639 00:46:04.239 --> 00:46:07.719 engineering leader today, any dev leader, needs to be stepping up and saying, 640 00:46:07.719 --> 00:46:12.800 hey, like, we have to help influence this discussion at all the 641 00:46:12.840 --> 00:46:15.960 way up to the board level, or else we're going to see more CFOs 642 00:46:15.000 --> 00:46:17.400 make that decision that you mentioned, Steve of saying, hey, let's cut 643 00:46:17.440 --> 00:46:22.639 costs, let's just focus on shareholder value and not necessarily customer value. And 644 00:46:22.519 --> 00:46:27.400 when you start getting away from customer value, that's when these huge risks come 645 00:46:27.400 --> 00:46:31.760 into organizations. All right, So you've you've kind of touched on my favorite 646 00:46:31.800 --> 00:46:38.280 topic there, which is in the world of sas, the customer comes last, 647 00:46:39.280 --> 00:46:45.880 because you're taking money that was almost free printed from the federal reserve that's 648 00:46:45.880 --> 00:46:52.320 given to venture capitalists that are only interested in growth and don't necessarily care about 649 00:46:52.360 --> 00:47:00.000 profitability. So what happens on the black mirror side of things here? Do 650 00:47:00.039 --> 00:47:04.760 you have any thoughts on that? How do we how do we how do 651 00:47:04.840 --> 00:47:10.960 we uh incentivize that we actually care about the customer and the developer or you 652 00:47:12.000 --> 00:47:21.719 know what risks are there with AI being Yeah, Well, so for example, 653 00:47:22.239 --> 00:47:23.760 you know, people for hundreds of years have been saying, oh, 654 00:47:23.840 --> 00:47:30.800 this technological improvement X is going to make everybody's quality of life go up without 655 00:47:30.840 --> 00:47:34.400 having to work more and we're gonna, you know, we're gonna work less 656 00:47:34.440 --> 00:47:37.880 hours. And this is you know, this has been going since you know, 657 00:47:37.000 --> 00:47:44.559 probably the era of the factory. What actually happens every time, well, 658 00:47:45.840 --> 00:47:47.280 oh, by the way, Steve, do you have a thing? 659 00:47:47.519 --> 00:47:51.440 Did you put that thing up on the screen? I can't take that down? 660 00:47:51.519 --> 00:47:55.119 I don't think Okay, yes I did, right? Oh sorry, 661 00:47:55.280 --> 00:48:00.840 anyway, I just got distracted because the the I was expecting comments fade away 662 00:48:00.840 --> 00:48:02.280 after twenty seconds or something, and it didn't. Oh no, I got 663 00:48:02.280 --> 00:48:09.199 to take it down. Yeah. So so the you know what, what's 664 00:48:10.800 --> 00:48:15.719 how do we what what might the abuse situation look like? And how do 665 00:48:15.800 --> 00:48:19.280 we prevent it or curb it? Oh? Man, that's a that's a 666 00:48:19.280 --> 00:48:22.280 big one. So I'll say I think what you're talking about here is that 667 00:48:22.559 --> 00:48:29.039 a lot of the value that gets generated from technological innovations like AI or you 668 00:48:29.079 --> 00:48:34.800 know, improved machinery that helps assembly lines or anything else has happened over the 669 00:48:34.840 --> 00:48:38.159 last one hundred two hundred years, and during industrialization has accrued to capital because 670 00:48:38.760 --> 00:48:43.599 uh, you know, shareholders are are driving this and they accrue most of 671 00:48:43.599 --> 00:48:46.480 the gains, and not most of those gains aren't necessarily accruing to the working 672 00:48:46.480 --> 00:48:49.480 class. I know, I'm swear, I'm not a Marxist, y'all, 673 00:48:49.480 --> 00:48:52.599 I'll tell you, but I'm just this is we're talking about here. To 674 00:48:52.679 --> 00:48:58.000 be fair, the rising Tide raises all ships. Like the total Amazon exists, 675 00:48:58.199 --> 00:49:01.559 I have access to products. I personally have access to products that I 676 00:49:01.599 --> 00:49:07.159 would not be able to get access to otherwise without having much more specialized knowledge 677 00:49:07.360 --> 00:49:13.639 and having to drive much further to get to a specialized store. And as 678 00:49:13.760 --> 00:49:17.800 your points to fake crap, and I can't always distinguish between the product I'm 679 00:49:17.800 --> 00:49:21.719 trying to get in the fake version. But like, I have access to 680 00:49:21.760 --> 00:49:23.920 more products, So it's not it's not like it's all bad. But in 681 00:49:23.920 --> 00:49:27.639 particular, what I was pointing out is that they said, oh, you're 682 00:49:27.639 --> 00:49:30.559 going to work less and you're going to have more. I do think that 683 00:49:30.599 --> 00:49:35.480 we have more. We're certainly not working less. Are we work differently? 684 00:49:36.320 --> 00:49:38.880 Yeah, I don't know. If we're happier, well, I think has 685 00:49:38.880 --> 00:49:43.119 a lot of other impacts, right right, Yeah, sure, I think 686 00:49:43.199 --> 00:49:45.039 we I think we work differently. Is really what it turns out to you 687 00:49:45.119 --> 00:49:49.679 right where it's like, Okay, maybe now with these AI changes, we're 688 00:49:49.679 --> 00:49:52.800 going to spend less time writing tests, We're gonna spend less time generating basic 689 00:49:52.800 --> 00:49:55.599 code. We're gonna spend a lot more time strategizing on the deeper pieces of 690 00:49:55.639 --> 00:50:00.800 code, or how does this all fit together, or understanding making sure these 691 00:50:00.800 --> 00:50:04.639 systems don't break. Maybe like we're seeing an increased need for platform engineering and 692 00:50:04.880 --> 00:50:09.039 internal developer portals and you know dev X teams at major companies because we need 693 00:50:09.079 --> 00:50:15.079 to make sure all these systems we've built to improve development speed work. And 694 00:50:15.159 --> 00:50:16.360 the other piece I think of what you're saying is like, you know, 695 00:50:16.480 --> 00:50:20.440 this kind of goes back to our discussion of hey, dev teams need to 696 00:50:20.559 --> 00:50:23.760 have these conversations. Dev leaders need to have these conversations with their CFOs, 697 00:50:23.760 --> 00:50:27.719 with their partners in finance and their partners in sales and all these other place 698 00:50:27.800 --> 00:50:30.840 parts of the company, and we need to be part of influencing the business 699 00:50:30.880 --> 00:50:32.920 discussion and saying, hey, make sure some of these benefits do accrue to 700 00:50:32.960 --> 00:50:37.719 dev teams, like we're helping provide more efficiency and more value, Like we 701 00:50:37.800 --> 00:50:39.920 need to be able to work on things we're passionate about. We need to 702 00:50:39.960 --> 00:50:44.360 be able to be you know, and sure we're have DEVAX metrics that look 703 00:50:44.360 --> 00:50:49.159 at whether those teams engage and talk to the value of that. So I 704 00:50:49.199 --> 00:50:52.719 do think you're spot on there, and it has absolutely allowed us to create 705 00:50:52.719 --> 00:50:58.320 these incredible technological improvements But what I think we need to be really careful on 706 00:50:58.559 --> 00:51:00.199 is assuming that will just mean we work less. Usually it just means we 707 00:51:00.239 --> 00:51:05.440 work more efficiently, and then we try to increase overall value over all GDP 708 00:51:05.920 --> 00:51:08.679 through that and hopefully you produce more value that then gets you the next pay 709 00:51:08.719 --> 00:51:13.400 raise, the next piece up. And I think as an industry, if 710 00:51:13.440 --> 00:51:17.239 we aren't helping shape that discussion by talking about DEVX, by talking about developer 711 00:51:17.239 --> 00:51:22.400 productivity, by being part of that discussion, we're gonna let, to Steve's 712 00:51:22.440 --> 00:51:27.119 point, the accountants be the ones who make the decisions, and that's where 713 00:51:27.159 --> 00:51:30.199 we get into major trouble. Shout out good accountants. Ever, by the 714 00:51:30.199 --> 00:51:37.800 way, I'm not saying accounts are bad. So one thing that was funny. 715 00:51:37.840 --> 00:51:43.719 You're talking about devs teams. Originally that was called DevOps. The purpose 716 00:51:43.760 --> 00:51:49.400 of DevOps was to make the developer experience better. Then we just rebranded ops 717 00:51:49.599 --> 00:51:54.400 to DevOps without changing anything. I do agree with you. So what's the 718 00:51:54.480 --> 00:52:00.920 new it's gonna happen with THEVAX team stuff? Well, now, platform engineering. 719 00:52:00.119 --> 00:52:05.960 When you heard so, I mean to your point, it's all it's 720 00:52:06.000 --> 00:52:07.199 all a bit of you know, six of one and a half dozen of 721 00:52:07.239 --> 00:52:13.599 another. We kind of use these new terms to speak to new eras of 722 00:52:13.960 --> 00:52:16.679 how we think about soccerlent, but we typically retain it right where it's like, 723 00:52:16.880 --> 00:52:21.239 hey, the agile movement this huge thing, and then DevOps this huge 724 00:52:21.280 --> 00:52:24.000 like layer on top of it. Now we're talking developer experience and platform engineering 725 00:52:24.079 --> 00:52:28.920 and building on this, and really, I'll say my take is that it's 726 00:52:28.960 --> 00:52:32.039 it's basically, how can we improve on the last standardized way we did this 727 00:52:32.800 --> 00:52:37.639 and how can we keep leveling up? But is the role very similar? 728 00:52:37.079 --> 00:52:43.159 Absolutely? Are many of the concepts the exact same? Absolutely? All right, 729 00:52:43.840 --> 00:52:46.079 Well, it is about time for us to wrap up here. Is 730 00:52:46.159 --> 00:52:51.039 there anything else that you wanted to share? Any any topics we didn't discuss. 731 00:52:51.360 --> 00:52:53.320 No, I really enjoyed the conversation. AJ Steve has been super fun. 732 00:52:54.119 --> 00:53:00.519 Thanks so much for having me on. Well we do picks at the 733 00:53:00.599 --> 00:53:04.199 end. Hopefully you were informed, which is, you know, basically can 734 00:53:04.239 --> 00:53:07.599 be anything just stuff that you're interested in, could be tech related, not 735 00:53:07.639 --> 00:53:16.159 tech related. I'll go ahead and start to give you some time to think 736 00:53:16.199 --> 00:53:22.079 about something. If you if you hadn't because I I'm guessing if you if 737 00:53:22.119 --> 00:53:24.199 you got the email with the wrong calendar link. You probably didn't get the 738 00:53:24.239 --> 00:53:31.360 email with the description anyway. So there's a couple of things that I still 739 00:53:31.880 --> 00:53:37.639 that came to my mind a few weeks ago that I still haven't gone through 740 00:53:37.800 --> 00:53:40.039 all of it. So let me see if I can scroll to where this 741 00:53:40.280 --> 00:53:44.320 was. So one thing that I was going to talk about that I mean, 742 00:53:44.360 --> 00:53:46.760 I've mentioned, just like like I did today, this whole issue with 743 00:53:47.239 --> 00:53:52.440 you know, we've got the investors and they're focused on growth and and that's 744 00:53:52.559 --> 00:53:55.719 a lot of what's driving SaaS companies is not profitability but just growth. And 745 00:53:55.760 --> 00:54:01.199 then what happens when you've reached the market that the full market potential of the 746 00:54:01.239 --> 00:54:07.719 growth? You know, what what happens when everybody who wears shoes has a 747 00:54:07.519 --> 00:54:15.559 bluetooth smart shoe. What happens when everybody who needs a calendar link has calendarly? 748 00:54:15.599 --> 00:54:22.039 What happens when everybody who wants to scroll infinitely and starve to death is 749 00:54:22.079 --> 00:54:29.400 on Facebook? Right? And there's a video that let me see what the 750 00:54:29.440 --> 00:54:31.960 title is. The thumbnail was something like ninety eight point nine market saturation. 751 00:54:32.119 --> 00:54:35.840 Now what oh whoa, whoa, whoa whoa? That was really loud? 752 00:54:35.960 --> 00:54:42.239 Sorry about that I I need to reconfigure my don't auto play plug in. 753 00:54:42.360 --> 00:54:45.000 Yeah, okay, So the title of the video is market saturation equals ninety 754 00:54:45.039 --> 00:54:49.199 eight point nine percent? Now what? So? I think this is an 755 00:54:49.199 --> 00:54:57.239 interesting video and it just talks about this issue of these these SaaS companies are 756 00:54:58.039 --> 00:55:01.280 starting to kind of cumulative. We all get to the point where growth is 757 00:55:01.320 --> 00:55:06.440 no longer possible, and many of them still haven't figured out how to be 758 00:55:06.519 --> 00:55:14.119 profitable. So what's going to give as there's no more investment to get and 759 00:55:14.159 --> 00:55:21.000 there's no more people to reach with the product. I'm not it seems like 760 00:55:21.039 --> 00:55:22.519 there should lead to a collapse. I don't think that it will. I 761 00:55:22.559 --> 00:55:27.320 think that something else is going to proxy and take it place. I fear 762 00:55:27.360 --> 00:55:31.639 it might be something government related or you know, I don't know. But 763 00:55:31.639 --> 00:55:36.440 but it's this is a question that I think is I got I don't know. 764 00:55:36.480 --> 00:55:38.119 I guess it's not really important thing about because nobody can do anything about 765 00:55:38.159 --> 00:55:42.360 it. Nobody, you know, unless you're unless you're on the investor capital 766 00:55:42.400 --> 00:55:45.400 side. We're getting really existential today. You're just going to experience it. 767 00:55:46.159 --> 00:55:51.440 But I don't know. It's it's something that that I I wish more people 768 00:55:51.480 --> 00:55:54.599 were aware of and and I really would like to see us get back to 769 00:55:57.159 --> 00:56:04.440 having profitable businesses rather than having growth businesses, because certainly profitable, profitable businesses 770 00:56:04.480 --> 00:56:08.159 are better for the customer by definition, because then, I would argue, 771 00:56:08.239 --> 00:56:12.400 is the one who benefits. They can often be better for the employee too, 772 00:56:12.559 --> 00:56:17.360 because you aren't having these kind of wild growth expectations. Instead, it's 773 00:56:17.440 --> 00:56:21.519 how do we build a strong business. So I've been thinking about as too 774 00:56:21.559 --> 00:56:24.239 lately, actually, AJ, which is you know, I've considered kind of 775 00:56:24.280 --> 00:56:29.480 in the process of starting my own business. I've previously been a founder business 776 00:56:29.480 --> 00:56:34.159 owner, and I'm very much looking at the bootstrap side of things and saying, 777 00:56:34.199 --> 00:56:38.239 I don't want to take VC funding because the pressure that it provides to 778 00:56:38.280 --> 00:56:43.440 me as a founder, to me as an employee is so substantial and some 779 00:56:43.599 --> 00:56:47.320 of the goals that get set by our kind of perverse incentives as you put 780 00:56:47.320 --> 00:56:52.159 out there, and there's a there's I think a big push now for folks 781 00:56:52.199 --> 00:56:53.000 to say, hey, I'm going to bootstrap, I'm or I'm going to 782 00:56:53.039 --> 00:56:57.199 take very minimal funding, or I'll take and I think this is a new 783 00:56:57.239 --> 00:57:00.519 way for starting to see I'll take funding, but you get to share future 784 00:57:00.519 --> 00:57:05.119 profits instead of hey, I just get a percentage of the company. And 785 00:57:05.519 --> 00:57:09.000 I think we're going to see a continued wave of that going forward. Yeah, 786 00:57:09.039 --> 00:57:12.880 I think that that. I hope I hope to see that. I 787 00:57:12.920 --> 00:57:19.800 hope that the dah revolution uh kicks off strong and hard and we get back 788 00:57:19.840 --> 00:57:28.519 to making products that deliver value to customers rather than than deliver poker face uh. 789 00:57:28.639 --> 00:57:30.440 For for investors, I mean, it's it's really a Ponzi scheme. 790 00:57:30.679 --> 00:57:34.920 It's a hot potato hot Ponzi scheme between investors right now and needs to get 791 00:57:34.920 --> 00:57:39.760 back to profitability because because it's just going to lead to government bailouts and anyway, 792 00:57:39.800 --> 00:57:45.440 I'll go further down that route. But other thing really cool keychain, 793 00:57:45.760 --> 00:57:51.480 keychain pin tool that I got that just turns out to be useful all the 794 00:57:51.559 --> 00:57:54.280 time. I don't know if I pull this out real quick, not show 795 00:57:54.360 --> 00:57:58.440 my keys on video where you can easially copy them. But I got a 796 00:57:58.440 --> 00:58:04.920 little keychain tool. It's got little silicon wrapper or the pentle like sim cards, 797 00:58:05.400 --> 00:58:09.519 reset or router, you know, hit the reset button on the game 798 00:58:09.639 --> 00:58:13.800 controller. You name it. There's all sorts of things that have that little 799 00:58:13.840 --> 00:58:15.400 reset button on them. And now I just have this on my keychain. 800 00:58:15.400 --> 00:58:20.159 It's a little and and because it's in a silicon sleeve, it's not poking 801 00:58:20.199 --> 00:58:22.639 into my you know, like ripping my jeans or anything I need one, 802 00:58:22.960 --> 00:58:29.519 or poking in my skin. So yeah, I mean they're they're overpriced only 803 00:58:29.559 --> 00:58:34.039 in the sense that in order to ship it, like the cost of shipping 804 00:58:34.079 --> 00:58:36.679 is more than the cost of the product. But you can get a pack 805 00:58:36.679 --> 00:58:38.880 of three or five of them on Amazon with the silicon sleeves, so I'll 806 00:58:38.920 --> 00:58:44.920 put a link to that. There also open audible. I don't remember when 807 00:58:44.920 --> 00:58:49.400 we were talking about audible last time, but thank Heavens, that's not a 808 00:58:49.440 --> 00:58:53.960 trademarkable word for whatever reason I don't I don't quite know, maybe because it's 809 00:58:53.960 --> 00:58:58.639 a dictionary word, and I don't think you can trademark dictionary words. But 810 00:58:58.719 --> 00:59:06.000 anyway, open audible is you can you can download your audible books and then 811 00:59:06.079 --> 00:59:08.719 you can convert them into a format so that you actually own them. So 812 00:59:08.880 --> 00:59:15.760 rather than being a rental where when the license with that author ends you lose 813 00:59:15.800 --> 00:59:22.400 access to your books, you can give your grandchildren your audio books. And 814 00:59:22.480 --> 00:59:28.880 so I often buy the MP three CDs when they're available, and at a 815 00:59:28.880 --> 00:59:30.880 at a you know, good price, within the same range of you know 816 00:59:30.920 --> 00:59:35.400 what i'd pay on audible, the fifteen dollars, twenty dollars, ten dollars 817 00:59:35.519 --> 00:59:39.760 something like that. But I really I would not be using Audible, and 818 00:59:39.800 --> 00:59:44.559 I would not be listening to as many audiobooks as I do if it weren't 819 00:59:44.599 --> 00:59:49.519 for open audible giving me peace of mind that this stuff isn't just going to 820 00:59:49.559 --> 00:59:52.159 be ripped out from underneath me one day, that I do have it preserved 821 00:59:52.159 --> 00:59:57.480 in a permanent format. I check that. Lastly, Yeah, yeah, 822 00:59:57.480 --> 01:00:00.719 it's it's it's twenty bucks or so, but totally worth it to be able 823 01:00:00.760 --> 01:00:05.440 to have that peace of mind that you know, the stuff you bought you 824 01:00:05.559 --> 01:00:10.800 actually own for you know, for real. And then the last thing was, 825 01:00:10.800 --> 01:00:14.760 because we were talking about the plane stuff, there's this YouTube channel mentor 826 01:00:14.800 --> 01:00:22.760 Pilot, and he he goes over every accident report of every plane incident and 827 01:00:22.840 --> 01:00:28.239 all of history across the globe, and I just love listening to it. 828 01:00:28.239 --> 01:00:30.559 It's one of those things that you can also listen to while you're falling asleep, 829 01:00:30.599 --> 01:00:34.599 you know, kind of half pay attention if if you want to just 830 01:00:34.800 --> 01:00:40.119 need need to hear something. But it's so it's so interesting to learn about 831 01:00:40.199 --> 01:00:45.960 the different plane accidents and the different things that go into play, the different 832 01:00:45.960 --> 01:00:50.760 parts of the report, what people on the ground and the control center reporting, 833 01:00:51.159 --> 01:00:53.079 what was going on in the black box, you know, what other 834 01:00:53.119 --> 01:00:57.320 pilots were seeing on the radar. Whatever. You know, he takes It's 835 01:00:57.360 --> 01:01:00.599 not like that for every single incident, but you know, for all the 836 01:01:00.599 --> 01:01:04.920 incidents that have the external data points, he links it together. He talks 837 01:01:04.920 --> 01:01:09.159 about manufacturing process. He talks about you know that the different grades of lubricant 838 01:01:09.280 --> 01:01:14.400 that are used on a particular shaft and why it you know it failed because 839 01:01:14.400 --> 01:01:17.800 it wasn't they ship that they switch from this brand to this brand, and 840 01:01:17.840 --> 01:01:21.280 now it should have been on a six month rotation, but they kept it 841 01:01:21.320 --> 01:01:24.000 on the eighteenth month rotation. You know, any of that stuff. It's 842 01:01:24.039 --> 01:01:28.239 not too technical, I don't think. I mean like it's enjoy I'm not 843 01:01:28.280 --> 01:01:30.719 a pilot, and it's very enjoyable to me. It is technical, but 844 01:01:30.760 --> 01:01:35.159 it's not like you don't have to be a pilot to appreciate it. But 845 01:01:35.199 --> 01:01:38.280 he does go into detail on things and I really like that. So anyway, 846 01:01:38.360 --> 01:01:42.880 those are those are my picks for the day. Steve Harmley, I 847 01:01:42.920 --> 01:01:47.480 left you a little time for dad jokes. Hopefully I'll take the time whether 848 01:01:47.519 --> 01:01:51.039 you left it or not. That's the most important part of the episode. 849 01:01:51.960 --> 01:01:59.320 No. Along the line of pilots, I'm working on getting my drone pilot 850 01:01:59.400 --> 01:02:05.840 certification and for the FAA. It's called Part one O seven And I've been 851 01:02:05.880 --> 01:02:12.840 taken a class that was recommended to me by somebody else who's a fixed wing 852 01:02:12.880 --> 01:02:15.920 pilot as well. And remember our drone program for our fire district, and 853 01:02:16.039 --> 01:02:23.840 it's called Pilot Institute and really really quite good. The guy's name, I 854 01:02:23.880 --> 01:02:30.559 forget the guy's name is Greg something. He's French that he lives in Prescott, 855 01:02:30.599 --> 01:02:35.119 Arizona and does a lot of teaching and training. But everything I've been 856 01:02:35.159 --> 01:02:39.079 seeing from those who have taken his course and then taken the Part one oh 857 01:02:39.079 --> 01:02:43.119 seven FA tests, he said, you just need to know the course. 858 01:02:43.159 --> 01:02:45.119 Everything you need is in there, and it's been really good. A lot 859 01:02:45.119 --> 01:02:51.119 of a lot of information to cover, for sure. But pilot Institute dot 860 01:02:51.159 --> 01:02:55.000 com is the u r L really quite good. All right. So now 861 01:02:55.039 --> 01:03:01.519 for the the jokes of the day, I got to give Kudo's shout out 862 01:03:01.559 --> 01:03:06.159 to the best stand up communitian of all time, without dispute, and his 863 01:03:06.239 --> 01:03:15.440 name is Stephen Wright, deadpan comedian, was actually actolutely hysterical, and recently 864 01:03:15.519 --> 01:03:22.920 he was on with Stephen Colbert and you know leveluce a couple jokes there that 865 01:03:22.960 --> 01:03:29.880 were that were fantastics. So he thought this one that he told, this 866 01:03:29.920 --> 01:03:31.519 one that had me laughing out loud, and he have to imagine him telling 867 01:03:31.519 --> 01:03:35.880 it in his deadpan style. I'll do my best, he says. My 868 01:03:36.039 --> 01:03:38.320 friend Jimmy brought an electric car and he was very excited about it. Then 869 01:03:38.400 --> 01:03:43.360 he bought an electric blanket, Then he bought an electric guitar, and then 870 01:03:43.400 --> 01:03:45.719 he bought an electric chair. And I haven't heard from him in a while. 871 01:03:46.119 --> 01:03:53.400 Oh that loop got turned on again. Sorry, So right, So 872 01:03:54.320 --> 01:03:58.480 one time when I was in school, I went to my teacher and I 873 01:03:58.559 --> 01:04:00.880 said, would you find a me for something I did not do? She 874 01:04:00.960 --> 01:04:02.800 said no, of course not. I said, okay, I did not 875 01:04:02.880 --> 01:04:12.960 do my home. And then a simple question, what do skunks say at 876 01:04:13.039 --> 01:04:19.519 church? Let us spray? All right, I got to give you prophecy. 877 01:04:19.800 --> 01:04:24.199 This are some solid ones there, thank you. I have high standards 878 01:04:24.199 --> 01:04:27.519 when it comes to the dad jokes I tell, so I'd do my best. 879 01:04:27.760 --> 01:04:29.719 I don't have a good dad joke to share, but I do have 880 01:04:29.719 --> 01:04:31.760 a couple of good carve outs, so AJ shout out for the audiobooks. 881 01:04:31.840 --> 01:04:40.079 Huge fan also recommend, in addition to trying to get your open audible licenses 882 01:04:40.119 --> 01:04:45.760 set up, public libraries have a fantastic audiobook selections. I'm in Seattle area, 883 01:04:45.800 --> 01:04:49.440 Seattle Public Libraries has an amazing connection. Very much recommend it. Most 884 01:04:49.480 --> 01:04:54.119 local libraries, whether it's a county library, city libook library, will be 885 01:04:54.119 --> 01:04:56.639 part of a larger collective that has a ton of audiooks available to you. 886 01:04:56.880 --> 01:05:00.880 So if you want to get an audiobook through your public libraries, fantastic way 887 01:05:00.920 --> 01:05:03.480 to go about it. I'll also show out to Ali Abdell, one of 888 01:05:03.519 --> 01:05:08.559 my favorite YouTubers who I'm currently reading his new book which is called Feel Good 889 01:05:08.599 --> 01:05:12.519 Productivity, How to Do More of what matters to You, Very much enjoying 890 01:05:12.559 --> 01:05:15.599 it, talking about a lot of the concepts that we've kind of brought up 891 01:05:15.599 --> 01:05:19.480 more broadly around de experience and those other pieces of Hey, how can I 892 01:05:19.920 --> 01:05:24.000 be happier in what I'm doing and get to do more things that make me 893 01:05:24.039 --> 01:05:27.599 happy and really enjoying the book to kind of start off my year. And 894 01:05:27.639 --> 01:05:30.880 then I'll close by just saying hey, if you haven't checked out the Devinrupted 895 01:05:30.920 --> 01:05:33.519 Substack or our podcast, would love to hear feedback from everyone on this. 896 01:05:33.719 --> 01:05:39.519 Always great to come on these other shows and really learn from Aj and Steve 897 01:05:39.519 --> 01:05:41.440 here. It has been so much fun. I really appreciate you guys having 898 01:05:41.480 --> 01:05:44.480 me on, and you can check us out at Devenrupted dot substack dot com 899 01:05:44.800 --> 01:05:48.079 and maybe Ajarstevil join us for a guest article this year. I don't know 900 01:05:48.360 --> 01:05:50.880 you talk you guys into it. We'd love to feature something from you guys 901 01:05:50.880 --> 01:05:55.599 if it's interesting later in the year. Sure, I'm up for that. 902 01:05:56.199 --> 01:05:58.880 Like I said, I'm not exactly what you call an engineering leader, but 903 01:05:59.519 --> 01:06:02.159 I could do my best. Well. I'm stoked to continue to collaborate with 904 01:06:02.239 --> 01:06:04.360 you guys, and thanks again for having me on. It's been a ton 905 01:06:04.400 --> 01:06:09.639 of fun. Yeah, thanks for coming. I'm I'm glad that we had 906 01:06:09.679 --> 01:06:12.760 you and we got to talk about these topics and and get a preview of 907 01:06:12.800 --> 01:06:17.039 that report from Linear Body. Yeah, super exciting stuff. It'll be out 908 01:06:17.039 --> 01:06:21.840 here probably by the time this pod goes fully live. On all channels and 909 01:06:23.639 --> 01:06:26.119 lots of cool insights coming. So yeah, linear b dot io to check 910 01:06:26.159 --> 01:06:29.079 out the report, and then Devin op dot substack dot com to check out 911 01:06:29.079 --> 01:06:31.639 our weekly guest articles and the podcast. So yeah, thanks guys, been 912 01:06:31.679 --> 01:06:36.480 a wonderful conversation, really really enjoy all right, well, have a good 913 01:06:36.480 --> 01:06:40.880 one. Audios ideos ideas,