1 00:00:05,239 --> 00:00:07,259 [music] 2 00:00:09,825 --> 00:00:11,845 [music] 3 00:00:15,360 --> 00:00:17,840 Welcome back everybody to Pyon AU in 4 00:00:17,840 --> 00:00:20,960 Brisbane to our afternoon session here 5 00:00:20,960 --> 00:00:23,199 in Ballroom 3. I hope you've all had a 6 00:00:23,199 --> 00:00:27,199 great lunch break and I am going to 7 00:00:27,199 --> 00:00:30,160 introduce you now to Pwell who is going 8 00:00:30,160 --> 00:00:32,320 to talk about breaking the PR review 9 00:00:32,320 --> 00:00:35,040 bottleneck. Please give him a big hand 10 00:00:35,040 --> 00:00:38,200 of applause. 11 00:00:39,955 --> 00:00:41,975 [applause] 12 00:00:48,000 --> 00:00:51,200 Hello everyone. I hope you enjoyed your 13 00:00:51,200 --> 00:00:53,600 lunch break and it was really nice and 14 00:00:53,600 --> 00:00:56,800 nourishing. Um, let's go into my 15 00:00:56,800 --> 00:00:59,840 presentation. So, 30 seconds on me and 16 00:00:59,840 --> 00:01:03,680 then we'll go to slides. Um, I'm working 17 00:01:03,680 --> 00:01:06,000 as a lead AI engineer and well 18 00:01:06,000 --> 00:01:08,000 technically as a forward deployed 19 00:01:08,000 --> 00:01:09,520 engineering consultancy company called 20 00:01:09,520 --> 00:01:11,360 Revity 21 00:01:11,360 --> 00:01:13,760 and I've been in IT for more than 18 22 00:01:13,760 --> 00:01:16,400 years. I've been working with startups, 23 00:01:16,400 --> 00:01:19,360 scaleups, enterprises and what I found 24 00:01:19,360 --> 00:01:22,320 lots of them having very similar issue 25 00:01:22,320 --> 00:01:24,640 and this issue this is what I'm going to 26 00:01:24,640 --> 00:01:27,200 talk about today. 27 00:01:27,200 --> 00:01:30,000 So there was one event that changed my 28 00:01:30,000 --> 00:01:32,720 mind about how software is soft software 29 00:01:32,720 --> 00:01:35,439 development is working and it is speed 30 00:01:35,439 --> 00:01:38,320 isn't about writing code faster it's 31 00:01:38,320 --> 00:01:41,200 about removing baiting and there is a 32 00:01:41,200 --> 00:01:45,360 good example so last year I had a um 33 00:01:45,360 --> 00:01:47,280 customer a large enterprise company 34 00:01:47,280 --> 00:01:48,960 hundreds of developers scattered around 35 00:01:48,960 --> 00:01:53,200 the globe um and they were doing major 36 00:01:53,200 --> 00:01:56,079 migration large migration 37 00:01:56,079 --> 00:01:58,079 and they're facing one particular issue 38 00:01:58,079 --> 00:02:00,000 their delivery cycle was very slow we 39 00:02:00,000 --> 00:02:01,200 are talking not about weeks we're 40 00:02:01,200 --> 00:02:03,360 talking about months 41 00:02:03,360 --> 00:02:07,040 so they approached me and said look we 42 00:02:07,040 --> 00:02:11,200 want to implement AI coding tools codex 43 00:02:11,200 --> 00:02:14,400 copilot corso cloud code whatever it 44 00:02:14,400 --> 00:02:15,680 doesn't really matter so we will be 45 00:02:15,680 --> 00:02:18,400 writing code faster 46 00:02:18,400 --> 00:02:20,959 and presumably delivering faster as well 47 00:02:20,959 --> 00:02:25,200 right no wrong because developers spent 48 00:02:25,200 --> 00:02:28,319 maybe 20 to 30% % of time writing code 49 00:02:28,319 --> 00:02:31,520 and the rest is everything else. It's um 50 00:02:31,520 --> 00:02:34,640 um Jira grooming, meetings, um I don't 51 00:02:34,640 --> 00:02:37,519 know other tasks, emails and most 52 00:02:37,519 --> 00:02:40,720 importantly pull request review and the 53 00:02:40,720 --> 00:02:42,879 pull request review is a bottleneck 54 00:02:42,879 --> 00:02:46,239 quite often. So I had a look into their 55 00:02:46,239 --> 00:02:49,040 processes and realized yes they have a 56 00:02:49,040 --> 00:02:51,680 problem with uh with with pull requests. 57 00:02:51,680 --> 00:02:54,000 So essentially they had features they 58 00:02:54,000 --> 00:02:56,480 were sitting in a in a queue well fully 59 00:02:56,480 --> 00:02:59,440 developed features sometimes weeks and 60 00:02:59,440 --> 00:03:02,239 there were and were dozens of them some 61 00:03:02,239 --> 00:03:05,120 of them were approved but never merged 62 00:03:05,120 --> 00:03:07,920 and I said look if you implement AI 63 00:03:07,920 --> 00:03:10,800 coding tool right now you're going to 64 00:03:10,800 --> 00:03:13,840 explode your queue from weeks you'll go 65 00:03:13,840 --> 00:03:16,239 to months and from dozens of pull 66 00:03:16,239 --> 00:03:18,159 requests you will have hundreds of pull 67 00:03:18,159 --> 00:03:20,959 requests so they listened to me they 68 00:03:20,959 --> 00:03:23,840 started working in that process and um 69 00:03:23,840 --> 00:03:26,319 eventually their queue reduced from 70 00:03:26,319 --> 00:03:29,200 dozens to single digits and then they 71 00:03:29,200 --> 00:03:31,280 started implementing coding tools and 72 00:03:31,280 --> 00:03:33,440 then magic happened. They start 73 00:03:33,440 --> 00:03:35,680 delivering features much faster not 74 00:03:35,680 --> 00:03:38,239 weeks but sometimes they could deliver 75 00:03:38,239 --> 00:03:40,799 within a week. 76 00:03:40,799 --> 00:03:45,200 So um the problem is a bottleneck and 77 00:03:45,200 --> 00:03:48,159 code review p request review quite often 78 00:03:48,159 --> 00:03:50,400 is a bottleneck 79 00:03:50,400 --> 00:03:52,879 similar to uh traffic gems. You cannot 80 00:03:52,879 --> 00:03:55,760 really put more cars in a um congested 81 00:03:55,760 --> 00:03:57,760 road. If you put more it will just slow 82 00:03:57,760 --> 00:04:00,959 down. Similar with code review. If you 83 00:04:00,959 --> 00:04:05,840 push more pull requests and you have the 84 00:04:05,840 --> 00:04:07,439 same amount of people with the same 85 00:04:07,439 --> 00:04:09,200 cadence, it's going to slow down 86 00:04:09,200 --> 00:04:11,920 everything. 87 00:04:12,080 --> 00:04:14,560 So how do we measure poll requests 88 00:04:14,560 --> 00:04:17,280 review specifically? So we have three 89 00:04:17,280 --> 00:04:19,919 key metrics. The first one is time to 90 00:04:19,919 --> 00:04:22,639 the first review. It's B it's it's not 91 00:04:22,639 --> 00:04:24,240 when your review been reviewed but 92 00:04:24,240 --> 00:04:26,160 essentially the time when someone has 93 00:04:26,160 --> 00:04:28,240 attended your review and said yes I got 94 00:04:28,240 --> 00:04:31,520 it. I'll have a look. There is a um 95 00:04:31,520 --> 00:04:34,400 Google research shows that if could uh 96 00:04:34,400 --> 00:04:36,080 if pull request review has been attended 97 00:04:36,080 --> 00:04:38,240 within an hour quite often you will 98 00:04:38,240 --> 00:04:40,560 close this pull request within a day. If 99 00:04:40,560 --> 00:04:42,639 it's next day then we're talking about a 100 00:04:42,639 --> 00:04:45,199 week. You see the pattern? So as longer 101 00:04:45,199 --> 00:04:48,560 it takes uh to um attend it. So longer 102 00:04:48,560 --> 00:04:51,360 it takes to review. Second is a median 103 00:04:51,360 --> 00:04:53,199 time to merge. Well it's obvious. So 104 00:04:53,199 --> 00:04:55,840 it's time from end to end to median. So 105 00:04:55,840 --> 00:04:58,080 nothing to explain. 106 00:04:58,080 --> 00:05:00,880 And last but not least, change fail 107 00:05:00,880 --> 00:05:03,280 change failure rate. We cannot we can 108 00:05:03,280 --> 00:05:05,600 ship faster, but if we ship bugs, it 109 00:05:05,600 --> 00:05:07,759 doesn't count, right? We need to deliver 110 00:05:07,759 --> 00:05:10,160 high quality code. 111 00:05:10,160 --> 00:05:12,320 Well, we we talk about metrics. Now talk 112 00:05:12,320 --> 00:05:14,960 about problems. And these problems, they 113 00:05:14,960 --> 00:05:16,479 are not human problems. They are system 114 00:05:16,479 --> 00:05:19,199 problems. And system problems require 115 00:05:19,199 --> 00:05:21,680 system solutions. So let's let's talk 116 00:05:21,680 --> 00:05:23,919 about eight major problems. Here is a 117 00:05:23,919 --> 00:05:26,479 road. Imagine [sighs and gasps] 118 00:05:26,479 --> 00:05:29,120 that you have spent a few days in 119 00:05:29,120 --> 00:05:33,120 writing code um for some feature. 120 00:05:33,120 --> 00:05:36,240 You made it, you open pull request and 121 00:05:36,240 --> 00:05:38,720 the first comment is hm why do you use 122 00:05:38,720 --> 00:05:41,039 this pattern? Why haven't you used this 123 00:05:41,039 --> 00:05:44,000 library and so on so forth. So it's a 124 00:05:44,000 --> 00:05:45,759 late debate. It's a late decision that's 125 00:05:45,759 --> 00:05:47,520 already been made when you have shipped 126 00:05:47,520 --> 00:05:49,280 well not shipped but de delivered the 127 00:05:49,280 --> 00:05:50,800 code 128 00:05:50,800 --> 00:05:54,560 deployed developed the code and quite 129 00:05:54,560 --> 00:05:56,160 often 130 00:05:56,160 --> 00:05:58,240 re-engineering solution takes more time 131 00:05:58,240 --> 00:06:01,520 than making it from the scratch okay you 132 00:06:01,520 --> 00:06:03,280 agreed on these comments because they 133 00:06:03,280 --> 00:06:06,000 were valid and you decided right I'll 134 00:06:06,000 --> 00:06:08,880 fix them you fixed and now you have mega 135 00:06:08,880 --> 00:06:12,080 monster more than 2,000 lines of code 136 00:06:12,080 --> 00:06:16,720 hundreds of well 30 plus files 137 00:06:16,720 --> 00:06:19,840 and who's going to review that? It's 138 00:06:19,840 --> 00:06:21,919 very difficult to keep in mind keep in 139 00:06:21,919 --> 00:06:25,199 the head this context and um the review 140 00:06:25,199 --> 00:06:27,919 might take sometimes days or sometimes 141 00:06:27,919 --> 00:06:30,000 you know it just skimmed looks good to 142 00:06:30,000 --> 00:06:31,919 me and 143 00:06:31,919 --> 00:06:34,400 ready to production. So you decided to 144 00:06:34,400 --> 00:06:37,680 help your peers and split it on several 145 00:06:37,680 --> 00:06:40,400 smaller PRs and now you have domino 146 00:06:40,400 --> 00:06:42,160 effect because all of them depends on 147 00:06:42,160 --> 00:06:45,039 the previous one and if let's say pull 148 00:06:45,039 --> 00:06:47,199 request number one stuck because there 149 00:06:47,199 --> 00:06:49,919 are some issues needs to be resolved you 150 00:06:49,919 --> 00:06:51,919 cannot chip further. 151 00:06:51,919 --> 00:06:54,960 Well that's in a queue. 152 00:06:54,960 --> 00:06:55,919 Well, [sighs] 153 00:06:55,919 --> 00:06:57,600 let's say you're still stuck with this 154 00:06:57,600 --> 00:07:00,639 solution and you decide that okay, I 155 00:07:00,639 --> 00:07:02,160 will 156 00:07:02,160 --> 00:07:04,960 put it in um open this pull request 157 00:07:04,960 --> 00:07:07,199 review and somebody will remove them. 158 00:07:07,199 --> 00:07:09,440 But who's going to review? Let's assume 159 00:07:09,440 --> 00:07:12,000 your first pull request had touched API 160 00:07:12,000 --> 00:07:14,720 and authentication and well that's two 161 00:07:14,720 --> 00:07:17,199 is enough. And there are three people in 162 00:07:17,199 --> 00:07:20,000 the team. Ellis who is um working on 163 00:07:20,000 --> 00:07:23,759 APIs, Bob who is um security specialist 164 00:07:23,759 --> 00:07:27,039 and working with OS let's say and Carlos 165 00:07:27,039 --> 00:07:29,199 who spent months in this module and 166 00:07:29,199 --> 00:07:31,440 knows all ins and outs who is going to 167 00:07:31,440 --> 00:07:34,639 review everyone think oh he will do they 168 00:07:34,639 --> 00:07:37,120 will do she will do and eventually 169 00:07:37,120 --> 00:07:39,360 nobody's reviewing [clears throat] 170 00:07:39,360 --> 00:07:43,120 so you decided to 171 00:07:43,120 --> 00:07:46,960 um assign to everyone in the team to all 172 00:07:46,960 --> 00:07:48,800 all at least three plus the rest of the 173 00:07:48,800 --> 00:07:52,160 team. And then we think uh so somebody 174 00:07:52,160 --> 00:07:54,800 in the team definitely will review that 175 00:07:54,800 --> 00:07:56,560 or perhaps they open the pull request, 176 00:07:56,560 --> 00:07:58,639 they just skimmed it, said looks good to 177 00:07:58,639 --> 00:08:00,960 me. Uh someone will definitely review it 178 00:08:00,960 --> 00:08:03,680 properly and what happens 179 00:08:03,680 --> 00:08:05,840 usually everyone just skimmed never 180 00:08:05,840 --> 00:08:07,680 looks through and you get a green light 181 00:08:07,680 --> 00:08:10,479 but there are some bugs. 182 00:08:10,479 --> 00:08:13,520 So you realize that 183 00:08:13,520 --> 00:08:15,840 and you decided okay I'll assign it only 184 00:08:15,840 --> 00:08:19,120 to Ellis because she's in a specialist 185 00:08:19,120 --> 00:08:22,319 but what happened Ellis suddenly leave 186 00:08:22,319 --> 00:08:24,800 so or perhaps she received a 187 00:08:24,800 --> 00:08:26,400 notification 188 00:08:26,400 --> 00:08:28,240 open the pull request and then she's got 189 00:08:28,240 --> 00:08:31,360 with um other tasks and um because she's 190 00:08:31,360 --> 00:08:34,320 also busy days passed nothing happened 191 00:08:34,320 --> 00:08:37,279 with your pull request 192 00:08:37,279 --> 00:08:39,440 that's quite common situation isn't it 193 00:08:39,440 --> 00:08:42,000 well you chase here finally and she 194 00:08:42,000 --> 00:08:44,399 reviewed your pull request and you've 195 00:08:44,399 --> 00:08:47,760 get you've got 50 comments 196 00:08:47,760 --> 00:08:50,720 48 of them are quite trivial. So naming 197 00:08:50,720 --> 00:08:54,720 conventions uh structure I don't know um 198 00:08:54,720 --> 00:08:57,519 uh spaces versus stubs or whatever else 199 00:08:57,519 --> 00:09:00,480 and two of them are real let's say race 200 00:09:00,480 --> 00:09:03,440 condition and n potential null pointer 201 00:09:03,440 --> 00:09:06,000 exception and it's quite easy to miss 202 00:09:06,000 --> 00:09:07,920 these two because they're buried in 203 00:09:07,920 --> 00:09:10,240 these 50 other comments. Well, 204 00:09:10,240 --> 00:09:12,399 eventually you close all of them. You're 205 00:09:12,399 --> 00:09:15,200 ready to merge this code. Finally, 206 00:09:15,200 --> 00:09:17,920 almost to press the button, but somebody 207 00:09:17,920 --> 00:09:20,000 just merged two and other two other pull 208 00:09:20,000 --> 00:09:22,080 requests in front of you. And now you 209 00:09:22,080 --> 00:09:24,160 have merge conflicts. 210 00:09:24,160 --> 00:09:26,640 Voila. You can start the cycle again. 211 00:09:26,640 --> 00:09:28,640 And you can got into this cycle several 212 00:09:28,640 --> 00:09:30,160 times until you finally merge the pull 213 00:09:30,160 --> 00:09:32,800 requests. So you see the problems and 214 00:09:32,800 --> 00:09:34,399 you might think, what is the most 215 00:09:34,399 --> 00:09:36,480 important problem? I guess the most 216 00:09:36,480 --> 00:09:38,240 import most most important problem for 217 00:09:38,240 --> 00:09:40,320 you is what you're facing right now 218 00:09:40,320 --> 00:09:41,600 because every team is different. Every 219 00:09:41,600 --> 00:09:44,160 situation is different as well. 220 00:09:44,160 --> 00:09:46,720 But we have solutions 221 00:09:46,720 --> 00:09:48,880 and large enterprise companies. Well, 222 00:09:48,880 --> 00:09:50,880 I'm specifically talking about open 223 00:09:50,880 --> 00:09:53,440 source projects. They solve these 224 00:09:53,440 --> 00:09:55,120 problems on a large scale because they 225 00:09:55,120 --> 00:09:56,800 have hundreds of or sometimes thousands 226 00:09:56,800 --> 00:10:00,240 of contributors and they deal with these 227 00:10:00,240 --> 00:10:03,360 problems on a scale. So let's learn from 228 00:10:03,360 --> 00:10:05,226 their experience. 229 00:10:05,226 --> 00:10:06,080 [snorts] 230 00:10:06,080 --> 00:10:08,959 First of all, uh we can talk about what 231 00:10:08,959 --> 00:10:12,080 we can do before we start coding. 232 00:10:12,080 --> 00:10:15,040 If you uh doing some important feature 233 00:10:15,040 --> 00:10:17,200 and it's quite complex, first of all, 234 00:10:17,200 --> 00:10:19,839 you can do ADRs or RFCS. It's a simple 235 00:10:19,839 --> 00:10:22,320 document maybe one or two pages. You 236 00:10:22,320 --> 00:10:24,560 spend maybe I don't know an hour to 237 00:10:24,560 --> 00:10:26,560 write it, maybe a day to discuss it with 238 00:10:26,560 --> 00:10:28,959 the team. You find all the solutions. Uh 239 00:10:28,959 --> 00:10:31,279 you discuss all the problems. And 240 00:10:31,279 --> 00:10:32,480 essentially the structure of this 241 00:10:32,480 --> 00:10:34,640 document is basically what you need to 242 00:10:34,640 --> 00:10:38,800 build uh what alternatives you have um 243 00:10:38,800 --> 00:10:40,880 thought about and what you decided and 244 00:10:40,880 --> 00:10:42,480 why and you discuss it with the team. 245 00:10:42,480 --> 00:10:44,720 Finally you agreed on a solution and 246 00:10:44,720 --> 00:10:46,880 then you can start implementing. Well we 247 00:10:46,880 --> 00:10:49,760 can there is a good example. So Linux is 248 00:10:49,760 --> 00:10:52,800 doing it for u Linux kernel for ages. So 249 00:10:52,800 --> 00:10:54,959 and they do it doing it successfully and 250 00:10:54,959 --> 00:10:57,920 other organizations do it too. 251 00:10:57,920 --> 00:11:01,200 The other one is called owners. It's a 252 00:11:01,200 --> 00:11:03,360 very simple file that basically maps who 253 00:11:03,360 --> 00:11:05,360 is who is responsible for what part of 254 00:11:05,360 --> 00:11:07,279 the repository. 255 00:11:07,279 --> 00:11:11,200 Um it's it's clear ownership and instant 256 00:11:11,200 --> 00:11:13,440 assignment as soon as you uh push a pull 257 00:11:13,440 --> 00:11:16,959 request. So um CI or system picks it up 258 00:11:16,959 --> 00:11:19,920 and assign it to the right person. 259 00:11:19,920 --> 00:11:22,560 Um 260 00:11:22,560 --> 00:11:25,040 GitHub has their their standard code 261 00:11:25,040 --> 00:11:27,680 owners. Kubernetes has their um other 262 00:11:27,680 --> 00:11:30,160 standard called owners but fundamentally 263 00:11:30,160 --> 00:11:33,360 it's the same thing 264 00:11:33,360 --> 00:11:35,760 then because we all using computers so 265 00:11:35,760 --> 00:11:37,360 let's computers do what computers do 266 00:11:37,360 --> 00:11:42,079 well automation we can automate merge 267 00:11:42,079 --> 00:11:44,160 cues so essentially instead of pressing 268 00:11:44,160 --> 00:11:46,399 the button to merge your solution you 269 00:11:46,399 --> 00:11:48,560 leverage to your computer and it does it 270 00:11:48,560 --> 00:11:50,160 well so it rebates it checks with the 271 00:11:50,160 --> 00:11:51,760 current main if there are any mistakes 272 00:11:51,760 --> 00:11:54,079 you get it immediately which is good 273 00:11:54,079 --> 00:11:55,440 because you have immediate feedback back 274 00:11:55,440 --> 00:11:59,040 and you can respond to to it faster. 275 00:11:59,040 --> 00:12:02,240 Um, GitHub, GitLab and other products 276 00:12:02,240 --> 00:12:04,800 have these solutions um in place. So, 277 00:12:04,800 --> 00:12:07,360 just pick whatever works for you. Then 278 00:12:07,360 --> 00:12:09,519 auto linking. 279 00:12:09,519 --> 00:12:11,839 So, you just remove all these nitpicking 280 00:12:11,839 --> 00:12:13,440 um issues and comments that you might 281 00:12:13,440 --> 00:12:16,399 get in a um in a pull request review and 282 00:12:16,399 --> 00:12:18,240 you just get it straight away when you 283 00:12:18,240 --> 00:12:20,959 do it on your laptop or presumably you 284 00:12:20,959 --> 00:12:23,920 um deployed it to CI/CD pipeline and you 285 00:12:23,920 --> 00:12:25,440 get it from there. So essentially you 286 00:12:25,440 --> 00:12:28,800 just ask computer to review your code 287 00:12:28,800 --> 00:12:31,440 and there are many different tools rough 288 00:12:31,440 --> 00:12:33,600 my pi and plenty others. So I personally 289 00:12:33,600 --> 00:12:35,519 prefer rough because it's written in 290 00:12:35,519 --> 00:12:38,079 rust so and rust are cool tools. They 291 00:12:38,079 --> 00:12:39,519 are much faster than everything else. So 292 00:12:39,519 --> 00:12:41,839 that's why I picked that but you know 293 00:12:41,839 --> 00:12:44,639 pick your own. 294 00:12:44,639 --> 00:12:48,560 Um then let's talk about humans. We all 295 00:12:48,560 --> 00:12:50,639 busy. We all have we all forget about 296 00:12:50,639 --> 00:12:52,639 things. We all have mistakes. So let's 297 00:12:52,639 --> 00:12:55,680 admit it. There is a role called PR 298 00:12:55,680 --> 00:12:58,000 wrangler specifically in Kubernetes what 299 00:12:58,000 --> 00:13:00,560 they do. So they have 3,000 active cube 300 00:13:00,560 --> 00:13:03,760 3,000 active comput 301 00:13:03,760 --> 00:13:05,920 uh contributors. 302 00:13:05,920 --> 00:13:09,839 So you know to to uh manage this amount 303 00:13:09,839 --> 00:13:11,760 of pull requests you need to account 304 00:13:11,760 --> 00:13:13,920 them and as I said people forget about 305 00:13:13,920 --> 00:13:16,560 things people are busy. So if pull 306 00:13:16,560 --> 00:13:19,040 request is stuck in more than 24 hours 307 00:13:19,040 --> 00:13:22,800 that's the SLA that PR wrangler is going 308 00:13:22,800 --> 00:13:24,560 to notch the right person who is 309 00:13:24,560 --> 00:13:28,000 assigned in owners file and eventually 310 00:13:28,000 --> 00:13:32,399 so they um notch once twice thrice 311 00:13:32,399 --> 00:13:34,560 um and they you know get a pull request 312 00:13:34,560 --> 00:13:36,800 review done 313 00:13:36,800 --> 00:13:38,880 we can also help people to review pull 314 00:13:38,880 --> 00:13:41,360 requests small PRs and stacking when 315 00:13:41,360 --> 00:13:44,079 it's done well it actually helps uh 316 00:13:44,079 --> 00:13:45,839 there is another Google study that shows 317 00:13:45,839 --> 00:13:49,279 that if you have 400 lines or less then 318 00:13:49,279 --> 00:13:50,959 these pull requests most likely will be 319 00:13:50,959 --> 00:13:52,800 reviewed within a day and as we 320 00:13:52,800 --> 00:13:54,480 discussed if it's been reviewed within a 321 00:13:54,480 --> 00:13:56,480 day so most likely you can ship it 322 00:13:56,480 --> 00:13:58,880 within a day as well. Well, of course, 323 00:13:58,880 --> 00:14:01,199 if you have um all the automations done 324 00:14:01,199 --> 00:14:03,199 and all the other things. So, but you 325 00:14:03,199 --> 00:14:05,519 know, faster review, faster you faster 326 00:14:05,519 --> 00:14:08,560 you deploy, 327 00:14:08,560 --> 00:14:10,880 then there is a tool called canban 328 00:14:10,880 --> 00:14:13,920 board. Well, it's um helps to visualize 329 00:14:13,920 --> 00:14:16,160 the problem, helps to visualize the the 330 00:14:16,160 --> 00:14:18,399 bottleneck. You have a few columns. 331 00:14:18,399 --> 00:14:20,240 Well, you design it your own on your own 332 00:14:20,240 --> 00:14:22,720 way. most importantly most importantly 333 00:14:22,720 --> 00:14:25,199 with whip limits and essentially you 334 00:14:25,199 --> 00:14:28,639 pull tickets from the right uh to the 335 00:14:28,639 --> 00:14:32,000 left. So you you try to move whatever is 336 00:14:32,000 --> 00:14:35,519 closer to production and if you if you 337 00:14:35,519 --> 00:14:37,120 have a bottleneck you'll see it straight 338 00:14:37,120 --> 00:14:40,800 away. It helps you to um um um make 339 00:14:40,800 --> 00:14:43,040 accountable actions on what to do and 340 00:14:43,040 --> 00:14:45,040 presumably talk to your stakeholders and 341 00:14:45,040 --> 00:14:46,959 and saying look we have a problem here 342 00:14:46,959 --> 00:14:49,279 so let's invest some uh effort to 343 00:14:49,279 --> 00:14:51,839 resolve it. 344 00:14:51,839 --> 00:14:54,800 Also because we're living in AI era so 345 00:14:54,800 --> 00:14:57,040 we can use help of AI to review your 346 00:14:57,040 --> 00:15:01,040 code you can um once you finish 347 00:15:01,040 --> 00:15:02,880 development so you can actually ask AI 348 00:15:02,880 --> 00:15:05,440 agent to review code check some 349 00:15:05,440 --> 00:15:07,040 solutions see how it works under the 350 00:15:07,040 --> 00:15:10,000 load see if um um ask what happens if 351 00:15:10,000 --> 00:15:12,240 null been assigned to that variable and 352 00:15:12,240 --> 00:15:15,360 so on so forth and that can be done not 353 00:15:15,360 --> 00:15:18,880 only on your laptop but also in on CI/CD 354 00:15:18,880 --> 00:15:20,639 pipeline there uh hundreds of different 355 00:15:20,639 --> 00:15:22,000 well not hundreds but few different 356 00:15:22,000 --> 00:15:24,720 tools copilot code cloud and pick your 357 00:15:24,720 --> 00:15:27,680 own again so fundamentally they're doing 358 00:15:27,680 --> 00:15:31,360 very similar job um there is two caveats 359 00:15:31,360 --> 00:15:34,959 the first one um is you need to tune it 360 00:15:34,959 --> 00:15:36,800 well because if you don't you'll get 361 00:15:36,800 --> 00:15:40,079 lots of nitpicks and you don't want to 362 00:15:40,079 --> 00:15:42,800 um and the second caveat so you actually 363 00:15:42,800 --> 00:15:45,839 want to do linting on CICD before you do 364 00:15:45,839 --> 00:15:48,240 PR automated review by agents because 365 00:15:48,240 --> 00:15:50,079 they will most likely pick these issues 366 00:15:50,079 --> 00:15:53,839 by that should be picked by linting 367 00:15:53,839 --> 00:15:56,160 and the benefit of that. So you get 368 00:15:56,160 --> 00:15:57,920 almost immediate feedback. Usually takes 369 00:15:57,920 --> 00:16:00,240 a few minutes for agent to review your 370 00:16:00,240 --> 00:16:03,440 pull change pull request change. Um well 371 00:16:03,440 --> 00:16:06,240 of of course on dependency sorry on on 372 00:16:06,240 --> 00:16:09,120 the size of the pull request and the 373 00:16:09,120 --> 00:16:10,800 um 374 00:16:10,800 --> 00:16:12,720 um 375 00:16:12,720 --> 00:16:15,199 and the complexity of that. But anyway, 376 00:16:15,199 --> 00:16:18,320 so you get feedback within minutes while 377 00:16:18,320 --> 00:16:20,639 you're waiting for the real people to 378 00:16:20,639 --> 00:16:24,279 review your pull request. 379 00:16:24,320 --> 00:16:25,759 Well, let's admit [sighs and gasps] 380 00:16:25,759 --> 00:16:29,279 elephant in the room. We have AI agents. 381 00:16:29,279 --> 00:16:31,920 They're writing lots of code for us and 382 00:16:31,920 --> 00:16:35,040 they usually produce much more. So we we 383 00:16:35,040 --> 00:16:36,720 are talking about hundred not hundreds 384 00:16:36,720 --> 00:16:38,720 of lines but thousands of lines of code 385 00:16:38,720 --> 00:16:40,959 within a day and they ship very 386 00:16:40,959 --> 00:16:43,519 different side of set of um issues and 387 00:16:43,519 --> 00:16:45,759 bugs uh because they don't know anything 388 00:16:45,759 --> 00:16:47,199 about your system. They don't know 389 00:16:47,199 --> 00:16:48,800 anything about your product about her 390 00:16:48,800 --> 00:16:52,160 knowledge about u um microservices that 391 00:16:52,160 --> 00:16:53,600 you have and APIs and everything else. 392 00:16:53,600 --> 00:16:56,240 So you can try to fit it in into this 393 00:16:56,240 --> 00:16:58,240 solution but they have limited context 394 00:16:58,240 --> 00:17:01,120 so they can only comprehend as much well 395 00:17:01,120 --> 00:17:03,600 skills helping to the some to some 396 00:17:03,600 --> 00:17:05,600 extent but they can't fully resolve the 397 00:17:05,600 --> 00:17:07,439 problem 398 00:17:07,439 --> 00:17:12,400 and also um um um oh yeah I slipped the 399 00:17:12,400 --> 00:17:15,600 slide but anyway um 400 00:17:15,600 --> 00:17:18,880 and also um you are the one who is 401 00:17:18,880 --> 00:17:21,600 responsible for the code even if AI has 402 00:17:21,600 --> 00:17:24,400 written it for you if your if the system 403 00:17:24,400 --> 00:17:26,799 that been deployed uh refunded $5 404 00:17:26,799 --> 00:17:28,799 billion who is going to be accountable 405 00:17:28,799 --> 00:17:30,799 for that 406 00:17:30,799 --> 00:17:33,679 most likely you 407 00:17:33,679 --> 00:17:36,480 anyway let's talk about AI generated 408 00:17:36,480 --> 00:17:39,520 code and how to review it 409 00:17:39,520 --> 00:17:42,080 first of all challenge the solution ask 410 00:17:42,080 --> 00:17:44,480 AI why have you did this why have you 411 00:17:44,480 --> 00:17:45,919 done that have you tried different 412 00:17:45,919 --> 00:17:48,480 solution most likely the first approach 413 00:17:48,480 --> 00:17:50,720 is not the optimal one sometimes you get 414 00:17:50,720 --> 00:17:53,200 a really good really good outcome on the 415 00:17:53,200 --> 00:17:54,960 second third I don't know fourth 416 00:17:54,960 --> 00:17:57,520 approach but challenge it it never it 417 00:17:57,520 --> 00:18:00,080 never it never get tired of your you 418 00:18:00,080 --> 00:18:01,919 know try this try that and so on so 419 00:18:01,919 --> 00:18:03,520 forth 420 00:18:03,520 --> 00:18:06,640 secondly verify it because what AI does 421 00:18:06,640 --> 00:18:09,679 well it does uh write a generic solution 422 00:18:09,679 --> 00:18:12,240 for the generic problem but your problem 423 00:18:12,240 --> 00:18:13,919 could be very specific for this 424 00:18:13,919 --> 00:18:16,320 environment for this product and so on 425 00:18:16,320 --> 00:18:18,000 so you need to check that actually 426 00:18:18,000 --> 00:18:21,280 optimal solution for your 427 00:18:22,799 --> 00:18:25,600 Next one is check for hallucinations. We 428 00:18:25,600 --> 00:18:28,240 know that AI try to invent APIs every 429 00:18:28,240 --> 00:18:30,720 now and then. Try to pull um um 430 00:18:30,720 --> 00:18:32,640 dependencies that doesn't exist and so 431 00:18:32,640 --> 00:18:34,799 on so forth. 432 00:18:34,799 --> 00:18:38,400 So um just run your lintis try to 433 00:18:38,400 --> 00:18:41,039 compile see what happens and if you if 434 00:18:41,039 --> 00:18:42,880 it breaks something so just fix it well 435 00:18:42,880 --> 00:18:46,320 or instruct AI how to fix it. And also 436 00:18:46,320 --> 00:18:48,000 you must have mandatory tests. they are 437 00:18:48,000 --> 00:18:50,160 non-negotiable nowadays. However, if you 438 00:18:50,160 --> 00:18:52,960 ask AI, the same agent that wrote your 439 00:18:52,960 --> 00:18:55,919 code to write tests for this to cover 440 00:18:55,919 --> 00:18:58,720 this code, most likely you get probably 441 00:18:58,720 --> 00:19:00,880 not optimal solution. So, you either 442 00:19:00,880 --> 00:19:02,480 need to clear the context or use another 443 00:19:02,480 --> 00:19:05,120 agent or use presumably another model as 444 00:19:05,120 --> 00:19:06,720 well. 445 00:19:06,720 --> 00:19:08,240 Um, 446 00:19:08,240 --> 00:19:11,440 and even if it writes the tests, you 447 00:19:11,440 --> 00:19:13,120 need to really make sure that these 448 00:19:13,120 --> 00:19:15,120 tests are definitely working because 449 00:19:15,120 --> 00:19:17,120 I've seen several times that AI creates 450 00:19:17,120 --> 00:19:19,600 a test that always return true, always 451 00:19:19,600 --> 00:19:22,400 succeed. It doesn't helpful. It's not 452 00:19:22,400 --> 00:19:25,440 helpful, isn't it? 453 00:19:25,440 --> 00:19:29,600 Um, next, because AI reads lots of code, 454 00:19:29,600 --> 00:19:32,640 we need to split it. instruct your model 455 00:19:32,640 --> 00:19:36,720 to write to um plan it writing in 456 00:19:36,720 --> 00:19:38,480 smaller chunks that are just digestible 457 00:19:38,480 --> 00:19:40,240 by people because then at the end of the 458 00:19:40,240 --> 00:19:42,480 day um most likely it's going to be 459 00:19:42,480 --> 00:19:44,799 viewed by a human. So let's let's help 460 00:19:44,799 --> 00:19:47,280 them and um when it's creates a small 461 00:19:47,280 --> 00:19:50,720 chunk we can ship it well not ship it 462 00:19:50,720 --> 00:19:54,320 but um um open the pull request review 463 00:19:54,320 --> 00:19:55,919 you know all the just happened that we 464 00:19:55,919 --> 00:19:58,640 discussed and people can start reviewing 465 00:19:58,640 --> 00:20:01,280 it while AI is still working in the um 466 00:20:01,280 --> 00:20:03,200 solution 467 00:20:03,200 --> 00:20:06,320 and also watch for over complications. 468 00:20:06,320 --> 00:20:08,720 If solution looks too complex to you, 469 00:20:08,720 --> 00:20:11,360 most likely it is. Ask AI to simplify 470 00:20:11,360 --> 00:20:15,200 it. Ask to make a lean, clear and 471 00:20:15,200 --> 00:20:17,600 elegant solution. There are actions that 472 00:20:17,600 --> 00:20:21,760 let's say um um um action simplify that 473 00:20:21,760 --> 00:20:24,720 looks into your um code and try to 474 00:20:24,720 --> 00:20:26,960 reduce it and make it less noisy, but 475 00:20:26,960 --> 00:20:31,080 fundamentally still check it. 476 00:20:31,280 --> 00:20:34,559 So where do we start? 477 00:20:34,559 --> 00:20:37,120 There is no silver bullet. Every 478 00:20:37,120 --> 00:20:39,200 organization is different. Every team is 479 00:20:39,200 --> 00:20:41,360 different as well. Team A might have 480 00:20:41,360 --> 00:20:44,400 different problems than team B. So I'm 481 00:20:44,400 --> 00:20:46,640 not advocating that you know it's your 482 00:20:46,640 --> 00:20:49,520 your problem, but there are a few 483 00:20:49,520 --> 00:20:51,280 lowhanging fruits that you can implement 484 00:20:51,280 --> 00:20:54,159 next week. First one is auto linting. 485 00:20:54,159 --> 00:20:55,600 Easy to implement. You can do it within 486 00:20:55,600 --> 00:20:59,919 a day. Pick one uh ler agree with the 487 00:20:59,919 --> 00:21:03,200 team what kind of syntax you prefer and 488 00:21:03,200 --> 00:21:05,919 so on. and so forth and do it let's say 489 00:21:05,919 --> 00:21:09,919 on your laptop or even better if you um 490 00:21:09,919 --> 00:21:12,799 um uh do it on CI/CD so everyone has the 491 00:21:12,799 --> 00:21:15,840 same outcome. 492 00:21:15,840 --> 00:21:18,559 The next one is called owners that takes 493 00:21:18,559 --> 00:21:20,400 a little longer. It might take a week 494 00:21:20,400 --> 00:21:21,600 because you need to discuss with the 495 00:21:21,600 --> 00:21:23,919 team who's responsible for what for that 496 00:21:23,919 --> 00:21:26,240 folder that module and so on so forth. 497 00:21:26,240 --> 00:21:28,720 And the uh trick is you never want to 498 00:21:28,720 --> 00:21:31,200 have uh more than two people responsible 499 00:21:31,200 --> 00:21:33,919 for one folder. The reason is clear. So 500 00:21:33,919 --> 00:21:35,840 if you have more than two there is a 501 00:21:35,840 --> 00:21:37,440 responsibility diffusion and nobody 502 00:21:37,440 --> 00:21:40,640 knows who is going to uh review usually 503 00:21:40,640 --> 00:21:42,720 it's one maxi max maximum two people for 504 00:21:42,720 --> 00:21:47,440 redundancy but you know talk to the team 505 00:21:47,440 --> 00:21:50,080 and the last one is canban flow there 506 00:21:50,080 --> 00:21:52,720 are lots of different options you can do 507 00:21:52,720 --> 00:21:56,240 trailer jirro uh and many others but 508 00:21:56,240 --> 00:21:58,240 what what I prefer personally a 509 00:21:58,240 --> 00:22:02,240 hardboard tickets on a wall I personally 510 00:22:02,240 --> 00:22:06,559 feel very um um uh nice. It's like um 511 00:22:06,559 --> 00:22:08,480 you know popping bubble wraps, instant 512 00:22:08,480 --> 00:22:10,480 satisfaction when you move to the done 513 00:22:10,480 --> 00:22:14,000 column. That's what I like. But again, 514 00:22:14,000 --> 00:22:18,559 you know, try see how it works for you. 515 00:22:18,559 --> 00:22:22,320 And as I said, [sighs] 516 00:22:22,320 --> 00:22:24,480 there is no perfect solution, but we 517 00:22:24,480 --> 00:22:27,520 know that quality lives in a guard rails 518 00:22:27,520 --> 00:22:32,000 and speed leaves in removing waiting. 519 00:22:32,000 --> 00:22:35,520 Thank you very much for having me. 520 00:22:35,520 --> 00:22:37,830 Do you have any questions? 521 00:22:37,830 --> 00:22:39,850 [applause] 522 00:22:44,000 --> 00:22:46,080 Thank you very much for that, PL. All 523 00:22:46,080 --> 00:22:50,600 right. Do we have any questions? 524 00:22:56,400 --> 00:22:58,960 Can you tell me more about code owners 525 00:22:58,960 --> 00:23:01,200 and not ending up in a world where one 526 00:23:01,200 --> 00:23:03,360 person owns a piece of code forever and 527 00:23:03,360 --> 00:23:05,360 no one else ever has the skill to take 528 00:23:05,360 --> 00:23:07,200 over as code owner when that person 529 00:23:07,200 --> 00:23:09,600 moves on or is on vacation or something 530 00:23:09,600 --> 00:23:11,679 like that cuz I guess that's kind of my 531 00:23:11,679 --> 00:23:13,280 big worry that's prevented me from ever 532 00:23:13,280 --> 00:23:16,080 using code owners is you just kind of 533 00:23:16,080 --> 00:23:18,960 away about kind of siloing that skill 534 00:23:18,960 --> 00:23:21,440 siloing that responsibility. 535 00:23:21,440 --> 00:23:24,159 Yeah, that's a great question. Look, um, 536 00:23:24,159 --> 00:23:25,760 as I said, you probably need to start 537 00:23:25,760 --> 00:23:27,280 with a problem. Do you have the problem 538 00:23:27,280 --> 00:23:28,799 when you don't know who is going to 539 00:23:28,799 --> 00:23:30,400 review your code? If you don't have this 540 00:23:30,400 --> 00:23:31,679 problem, you probably don't need code 541 00:23:31,679 --> 00:23:34,559 owners. But if you need, so discuss it 542 00:23:34,559 --> 00:23:36,240 with a team. Discuss if you want a 543 00:23:36,240 --> 00:23:40,159 rotation. Discuss if you want uh some um 544 00:23:40,159 --> 00:23:42,000 redundancy. So, not one person, but two, 545 00:23:42,000 --> 00:23:43,520 because everyone gets sick, everyone got 546 00:23:43,520 --> 00:23:45,760 some leave, everyone move to, I don't 547 00:23:45,760 --> 00:23:48,480 know, other jobs and so on so forth. Um 548 00:23:48,480 --> 00:23:50,640 maybe you want a I don't know a team 549 00:23:50,640 --> 00:23:53,760 lead who is going to cover if let's say 550 00:23:53,760 --> 00:23:56,159 there is nobody to review. You might 551 00:23:56,159 --> 00:23:58,559 have also a PR wrangler who can help 552 00:23:58,559 --> 00:24:01,840 with that. So a PR wrangler is a role in 553 00:24:01,840 --> 00:24:04,000 rotation as well because you don't want 554 00:24:04,000 --> 00:24:06,240 some person to constantly stuck to this 555 00:24:06,240 --> 00:24:09,600 pro to to this role. Um who is going to 556 00:24:09,600 --> 00:24:11,919 check if the PR view has been attendant? 557 00:24:11,919 --> 00:24:14,559 If it hasn't been who is responsible for 558 00:24:14,559 --> 00:24:17,200 that there's a code owners. Okay. Uh 559 00:24:17,200 --> 00:24:19,679 it's Bob. Okay, Bob is on leave. Who is 560 00:24:19,679 --> 00:24:21,520 going to review it? Let's let's let's 561 00:24:21,520 --> 00:24:22,960 check with the team lead. He might 562 00:24:22,960 --> 00:24:26,559 assign someone, you know, it's a um it's 563 00:24:26,559 --> 00:24:30,720 not a um universal solution, but again, 564 00:24:30,720 --> 00:24:34,919 whatever works for your team. 565 00:24:37,600 --> 00:24:40,799 Hi, you mentioned about how open-source 566 00:24:40,799 --> 00:24:43,279 software um how they're able to use some 567 00:24:43,279 --> 00:24:45,679 ways in order to allow for the number of 568 00:24:45,679 --> 00:24:47,840 contributors. Um would you be able to 569 00:24:47,840 --> 00:24:50,400 elaborate more on which of the specific 570 00:24:50,400 --> 00:24:53,840 solutions you mentioned are most useful 571 00:24:53,840 --> 00:24:56,720 in those andor any uh examples within 572 00:24:56,720 --> 00:24:58,320 open source. Thank you. 573 00:24:58,320 --> 00:25:00,159 Oh, thank you very much. Um probably 574 00:25:00,159 --> 00:25:02,480 missed a couple of examples. So I was 575 00:25:02,480 --> 00:25:04,559 talking about let's say code owners uh 576 00:25:04,559 --> 00:25:07,600 let's say kubernetes use them uh I was 577 00:25:07,600 --> 00:25:12,080 talking about a um what was it um uh 578 00:25:12,080 --> 00:25:16,559 RFC's and ADRs so Linux use them rust 579 00:25:16,559 --> 00:25:19,360 having them as well uh in some way or 580 00:25:19,360 --> 00:25:22,960 form um what else PR wrangler is 581 00:25:22,960 --> 00:25:25,520 implemented by 582 00:25:25,520 --> 00:25:27,760 Kubernetes as well so they have this 583 00:25:27,760 --> 00:25:30,400 role on rotation specifically so 24 584 00:25:30,400 --> 00:25:33,440 hours SLA Okay. [snorts] Um what else? 585 00:25:33,440 --> 00:25:35,120 Um 586 00:25:35,120 --> 00:25:37,679 well that's on off the top of my head. 587 00:25:37,679 --> 00:25:40,159 Uh but fundamentally so um as I as I 588 00:25:40,159 --> 00:25:42,080 mentioned there is no silver bullet. So 589 00:25:42,080 --> 00:25:43,840 you need to find what's your problem 590 00:25:43,840 --> 00:25:47,120 first. What is the slowest part in your 591 00:25:47,120 --> 00:25:49,600 system and then find the solution and 592 00:25:49,600 --> 00:25:51,679 once you fix this problem then you can 593 00:25:51,679 --> 00:25:54,080 go to another solution uh to to the next 594 00:25:54,080 --> 00:25:56,400 problem and try the next solution. So 595 00:25:56,400 --> 00:25:57,760 sometimes you know you might think this 596 00:25:57,760 --> 00:26:00,559 is a problem we'll try that with a team 597 00:26:00,559 --> 00:26:02,400 but it doesn't work let's try a 598 00:26:02,400 --> 00:26:04,960 different option. So yeah you need to 599 00:26:04,960 --> 00:26:07,360 you need to work with a team. 600 00:26:07,360 --> 00:26:10,240 Thank you Pavl. Any other questions? Yes 601 00:26:10,240 --> 00:26:13,799 one over there. 602 00:26:18,559 --> 00:26:22,480 Hi. Um you mentioned 24 hours as the SLA 603 00:26:22,480 --> 00:26:24,080 that's used for pull request review 604 00:26:24,080 --> 00:26:27,200 times. Would you say that that time 605 00:26:27,200 --> 00:26:29,679 should vary with the size of the team or 606 00:26:29,679 --> 00:26:32,080 is that pretty universal? 607 00:26:32,080 --> 00:26:33,919 Again, there is no silver bullet. So, 608 00:26:33,919 --> 00:26:35,679 every every team is different. If you 609 00:26:35,679 --> 00:26:37,919 have let's say three people in the team, 610 00:26:37,919 --> 00:26:41,760 that's one number. If you have uh 10 611 00:26:41,760 --> 00:26:42,880 people in the team, it's a different 612 00:26:42,880 --> 00:26:45,360 number. Again, uh it depends on how 613 00:26:45,360 --> 00:26:48,080 critical is um um the work that you're 614 00:26:48,080 --> 00:26:49,760 doing. If it's I don't know some kind of 615 00:26:49,760 --> 00:26:52,240 a um 616 00:26:52,240 --> 00:26:55,120 major issues and you're working as um 617 00:26:55,120 --> 00:26:56,799 let's say as a support engineer that's a 618 00:26:56,799 --> 00:26:58,720 different SLA. If you're working in a 619 00:26:58,720 --> 00:27:02,000 features that you know might take two 620 00:27:02,000 --> 00:27:04,159 weeks to implement that might be a 621 00:27:04,159 --> 00:27:07,360 different SLA. So um again work with a 622 00:27:07,360 --> 00:27:10,559 team work with a team lead um with 623 00:27:10,559 --> 00:27:12,799 stakeholders to identify what is an 624 00:27:12,799 --> 00:27:15,919 optimal solution for you. All right, one 625 00:27:15,919 --> 00:27:19,760 short question and a short answer. 626 00:27:19,760 --> 00:27:21,200 Say you're asked to work with a team 627 00:27:21,200 --> 00:27:22,640 that didn't have a culture of peer 628 00:27:22,640 --> 00:27:25,360 reviews at all. How would you seed the 629 00:27:25,360 --> 00:27:27,120 culture and teach them what a good peer 630 00:27:27,120 --> 00:27:28,480 review is? 631 00:27:28,480 --> 00:27:30,159 That's a great question and you know 632 00:27:30,159 --> 00:27:32,960 it's a more cultural question I guess. 633 00:27:32,960 --> 00:27:37,279 So um well first of all we need to in in 634 00:27:37,279 --> 00:27:40,240 my case it's good to visualize it. In 635 00:27:40,240 --> 00:27:42,320 that case canvan board helps a lot. So 636 00:27:42,320 --> 00:27:45,600 you have uh a board that says this is 637 00:27:45,600 --> 00:27:48,720 what you're going to sorry this side uh 638 00:27:48,720 --> 00:27:49,840 what you're going to deliver in 639 00:27:49,840 --> 00:27:52,000 production. So it's a done column. Uh 640 00:27:52,000 --> 00:27:54,399 then you have your QA. Then you have 641 00:27:54,399 --> 00:27:56,480 pull requests. Then you have uh 642 00:27:56,480 --> 00:27:58,640 development and perhaps some I don't 643 00:27:58,640 --> 00:28:01,120 know system design and so on so forth. 644 00:28:01,120 --> 00:28:03,440 And you must have limits. So you you 645 00:28:03,440 --> 00:28:04,880 must start with very generous rep 646 00:28:04,880 --> 00:28:07,440 limits. Let's say 10 tickets or whatever 647 00:28:07,440 --> 00:28:10,480 number of your teams in your team. And 648 00:28:10,480 --> 00:28:13,120 then you see how cards are moving. So if 649 00:28:13,120 --> 00:28:16,240 you have a column that overgrowth and 10 650 00:28:16,240 --> 00:28:18,640 uh tickets. So that means you know you 651 00:28:18,640 --> 00:28:22,880 have a bottleneck there. Um and you find 652 00:28:22,880 --> 00:28:24,480 and you look for the solution for that. 653 00:28:24,480 --> 00:28:25,919 So it might be you know the column might 654 00:28:25,919 --> 00:28:28,320 be coding, the column might be QA, the 655 00:28:28,320 --> 00:28:30,480 column might be pull request review. So 656 00:28:30,480 --> 00:28:33,520 you work with a problem and um 657 00:28:33,520 --> 00:28:35,360 eventually you get into the pull 658 00:28:35,360 --> 00:28:37,120 requests I assume because you you 659 00:28:37,120 --> 00:28:38,880 mentioned that's a problem. So and then 660 00:28:38,880 --> 00:28:40,640 you start looking what exactly the 661 00:28:40,640 --> 00:28:42,480 problem is. Is it people don't want to 662 00:28:42,480 --> 00:28:44,880 review? Is it it's too hard to review 663 00:28:44,880 --> 00:28:47,760 and nobody wants to? Is it uh people 664 00:28:47,760 --> 00:28:49,760 just you know I have other hundreds of 665 00:28:49,760 --> 00:28:52,399 tasks so I just don't have time or 666 00:28:52,399 --> 00:28:54,960 something else. So uh find the problem 667 00:28:54,960 --> 00:28:56,559 and then start looking into the 668 00:28:56,559 --> 00:28:58,880 solutions. 669 00:28:58,880 --> 00:29:01,279 All right, another round of applause for 670 00:29:01,279 --> 00:29:03,520 our great speaker Puzzle. [applause] 671 00:29:03,520 --> 00:29:06,640 And I'm honored to present you a 672 00:29:06,640 --> 00:29:09,360 wonderful Python AU mug. Thank you. And 673 00:29:09,360 --> 00:29:10,399 thank you again. 674 00:29:10,399 --> 00:29:13,510 Thank you. [applause]