1
00:00:00,420 --> 00:00:05,910
[Music]

2
00:00:10,320 --> 00:00:15,599
Welcome back everyone. Um our next

3
00:00:12,559 --> 00:00:17,440
speaker is Michael Gribbon. Um Michael

4
00:00:15,599 --> 00:00:19,920
Gribbon is going to be talking us

5
00:00:17,440 --> 00:00:22,560
talking to us about uh a talk called

6
00:00:19,920 --> 00:00:25,600
Beyond the Hype using AI coding agents

7
00:00:22,560 --> 00:00:26,960
today. Uh over to you Michael.

8
00:00:25,600 --> 00:00:29,960
Thank you. Free round of applause

9
00:00:26,960 --> 00:00:29,960
please.

10
00:00:32,239 --> 00:00:37,840
Uh hello, I'm Michael and I have about

11
00:00:36,320 --> 00:00:40,879
four years of experience as a software

12
00:00:37,840 --> 00:00:42,640
engineer and for

13
00:00:40,879 --> 00:00:44,879
slightly less than one of those years,

14
00:00:42,640 --> 00:00:46,800
coding agents have been a thing and

15
00:00:44,879 --> 00:00:49,680
that's what I want to talk about today.

16
00:00:46,800 --> 00:00:51,440
Um I want to make you slightly more

17
00:00:49,680 --> 00:00:54,239
productive at your job using coding

18
00:00:51,440 --> 00:00:56,640
agents. Now, as I've learned uh

19
00:00:54,239 --> 00:00:59,039
yesterday watching the Dal and I track,

20
00:00:56,640 --> 00:01:01,440
every talk mentioning AI has to mention

21
00:00:59,039 --> 00:01:03,840
the elephant in the room. There was a

22
00:01:01,440 --> 00:01:06,400
lot of hype about AI and about coding

23
00:01:03,840 --> 00:01:10,000
agents. And by a lot of hype, I mean

24
00:01:06,400 --> 00:01:12,960
like a lot of hype. Like some people

25
00:01:10,000 --> 00:01:15,520
think there will be uh no more software

26
00:01:12,960 --> 00:01:19,360
engineers by 2026,

27
00:01:15,520 --> 00:01:25,360
which is a bold bet to make given 2026

28
00:01:19,360 --> 00:01:28,240
is like five, no four months away. Um so

29
00:01:25,360 --> 00:01:30,400
I don't know about you though, but I've

30
00:01:28,240 --> 00:01:34,400
seen this type of hype before. Does

31
00:01:30,400 --> 00:01:37,920
anyone remember GitHub copilots?

32
00:01:34,400 --> 00:01:40,320
uh quite a few people. Yep. So I do and

33
00:01:37,920 --> 00:01:42,400
I want to tell you a story. So back in

34
00:01:40,320 --> 00:01:44,799
2021, I was in my last year of

35
00:01:42,400 --> 00:01:47,600
university and I was uh in on a meeting

36
00:01:44,799 --> 00:01:49,119
with a bunch of academics and we watched

37
00:01:47,600 --> 00:01:52,159
uh GitHub co-pilot which had been

38
00:01:49,119 --> 00:01:56,079
released that October ace a first year

39
00:01:52,159 --> 00:01:57,439
programming assignment in one shot. Um

40
00:01:56,079 --> 00:01:59,439
there was a lot of like fear,

41
00:01:57,439 --> 00:02:02,000
uncertainty and doubts and a lot of hype

42
00:01:59,439 --> 00:02:05,439
back then. um like what could GitHub

43
00:02:02,000 --> 00:02:09,840
pilot GitHub copilot not do and it turns

44
00:02:05,439 --> 00:02:11,599
out like actually quite a lot. Um I have

45
00:02:09,840 --> 00:02:13,200
no doubt that it helped like a few

46
00:02:11,599 --> 00:02:16,239
students cheat on assignments that was a

47
00:02:13,200 --> 00:02:18,720
problem but it didn't radically change

48
00:02:16,239 --> 00:02:22,400
coding like didn't radically change how

49
00:02:18,720 --> 00:02:25,120
I coded and I haven't heard of like mass

50
00:02:22,400 --> 00:02:26,640
job losses thanks to GitHub copilots.

51
00:02:25,120 --> 00:02:30,319
Um,

52
00:02:26,640 --> 00:02:32,640
but it did change coding. Like 20

53
00:02:30,319 --> 00:02:35,519
million developers use GitHub copilot.

54
00:02:32,640 --> 00:02:38,080
That's a lot of developers. Um, I don't

55
00:02:35,519 --> 00:02:40,319
think there's like any going back. The

56
00:02:38,080 --> 00:02:42,560
generation of developers nowadays coming

57
00:02:40,319 --> 00:02:44,480
out of university just expect to tab to

58
00:02:42,560 --> 00:02:48,319
the end of every like line of code. They

59
00:02:44,480 --> 00:02:49,840
just they love that tab button. Um, and

60
00:02:48,319 --> 00:02:52,239
what lesson can we learn from all of

61
00:02:49,840 --> 00:02:55,440
this? Well, we're going through a hype

62
00:02:52,239 --> 00:02:58,000
cycle. And I think the lesson is coding

63
00:02:55,440 --> 00:02:59,360
agents will change coding just like

64
00:02:58,000 --> 00:03:01,599
somewhere in the middle of all those

65
00:02:59,360 --> 00:03:04,879
predictions like from it's a fad that

66
00:03:01,599 --> 00:03:06,640
will go nowhere to uh by early 2026

67
00:03:04,879 --> 00:03:09,760
there will be no more software engineers

68
00:03:06,640 --> 00:03:11,599
somewhere in between because no one

69
00:03:09,760 --> 00:03:14,480
really knows what they're talking about.

70
00:03:11,599 --> 00:03:17,040
Um, Claude Code started this explosion

71
00:03:14,480 --> 00:03:19,200
of command line based coding agents I'll

72
00:03:17,040 --> 00:03:22,640
be speaking about today. And it was

73
00:03:19,200 --> 00:03:26,000
released in February, which is 7 months

74
00:03:22,640 --> 00:03:27,599
ago. So if someone says they know what

75
00:03:26,000 --> 00:03:30,080
they're talking about, then they have at

76
00:03:27,599 --> 00:03:32,480
most 7 months more experience than you.

77
00:03:30,080 --> 00:03:34,400
And I don't think seven someone with

78
00:03:32,480 --> 00:03:37,440
like seven months of experience is an

79
00:03:34,400 --> 00:03:39,440
expert at anything. Um, but that also

80
00:03:37,440 --> 00:03:41,280
means I have to give you a disclaimer.

81
00:03:39,440 --> 00:03:42,400
Um, I don't know what I'm talking about.

82
00:03:41,280 --> 00:03:45,120
I have less than seven months of

83
00:03:42,400 --> 00:03:47,280
experience on this topic. Um, anything

84
00:03:45,120 --> 00:03:49,120
and everything I say in this talk might

85
00:03:47,280 --> 00:03:51,360
be out of date within a few months. You

86
00:03:49,120 --> 00:03:53,920
have been warned. Uh, don't take me too

87
00:03:51,360 --> 00:03:59,040
seriously, but this is where the fun

88
00:03:53,920 --> 00:04:01,599
begins. Um, most of you like might write

89
00:03:59,040 --> 00:04:03,680
code at your job. Um, and the most

90
00:04:01,599 --> 00:04:05,360
obvious way to boost your productivity

91
00:04:03,680 --> 00:04:09,760
if you write code is to get coding

92
00:04:05,360 --> 00:04:11,200
agents to write code. Um,

93
00:04:09,760 --> 00:04:12,640
if you just heard the other talk, you

94
00:04:11,200 --> 00:04:16,400
know what I'm talking about. It's vibe

95
00:04:12,640 --> 00:04:18,400
coding. Um, and I found from experience

96
00:04:16,400 --> 00:04:20,479
that like talking really loud about vibe

97
00:04:18,400 --> 00:04:24,080
coding in front of developers generates

98
00:04:20,479 --> 00:04:26,800
a mix of reactions. Um, I have a theory

99
00:04:24,080 --> 00:04:28,800
on why. I think part of it is like the

100
00:04:26,800 --> 00:04:32,720
people who talk about vibe coding talk

101
00:04:28,800 --> 00:04:35,040
like really big about vibe coding. Um,

102
00:04:32,720 --> 00:04:37,360
but my experience is like it's like a

103
00:04:35,040 --> 00:04:38,560
New Year's resolution. Um, everyone

104
00:04:37,360 --> 00:04:41,440
likes to talk about their New Year's

105
00:04:38,560 --> 00:04:43,199
resolutions, but by February, like, you

106
00:04:41,440 --> 00:04:45,120
know, you made a big promise, you just

107
00:04:43,199 --> 00:04:48,479
don't talk about it that much anymore.

108
00:04:45,120 --> 00:04:51,759
Um, don't ask me. Um, I tried out vibe

109
00:04:48,479 --> 00:04:54,240
coding. It was it was pretty fun. It was

110
00:04:51,759 --> 00:04:57,040
even a little useful, but I don't vibe

111
00:04:54,240 --> 00:05:02,479
code that much at my job. I mostly just

112
00:04:57,040 --> 00:05:06,160
push artisal handwritten code um,

113
00:05:02,479 --> 00:05:07,680
like a craftsman. And it turns out most

114
00:05:06,160 --> 00:05:10,000
software engineers are like me. This is

115
00:05:07,680 --> 00:05:12,479
from the Stack Overflow. It's still a

116
00:05:10,000 --> 00:05:15,360
thing apparently. Um the Stack Overflow

117
00:05:12,479 --> 00:05:18,720
developer survey from this year. Um most

118
00:05:15,360 --> 00:05:21,280
developers don't vibe code at their job.

119
00:05:18,720 --> 00:05:23,440
But there's something going on here

120
00:05:21,280 --> 00:05:26,800
because vibe coding is really really

121
00:05:23,440 --> 00:05:28,800
popular. Like it's a these vibe coding

122
00:05:26,800 --> 00:05:31,919
startups were apparently the fastest

123
00:05:28,800 --> 00:05:33,520
growing businesses in human history. Um,

124
00:05:31,919 --> 00:05:35,600
and

125
00:05:33,520 --> 00:05:38,720
it seems to be really popular amongst

126
00:05:35,600 --> 00:05:40,880
like the general population. Um, and to

127
00:05:38,720 --> 00:05:42,800
be clear, that's a great thing. I think,

128
00:05:40,880 --> 00:05:44,400
uh, bit of a side note, but like the

129
00:05:42,800 --> 00:05:47,759
vast majority of the population can't

130
00:05:44,400 --> 00:05:49,919
code. Um, I think this is like roughly

131
00:05:47,759 --> 00:05:51,120
50 million developers, so like assuming

132
00:05:49,919 --> 00:05:53,759
the rest of the code can't code, the

133
00:05:51,120 --> 00:05:56,320
rest of the world can't code. Um, I wish

134
00:05:53,759 --> 00:05:58,400
more people could code. That's my

135
00:05:56,320 --> 00:06:01,680
opinion. I like I think it's a really

136
00:05:58,400 --> 00:06:03,600
empowering skill. Um, but we've been

137
00:06:01,680 --> 00:06:06,560
trying to teach people to code for years

138
00:06:03,600 --> 00:06:08,319
and some people just the reality is some

139
00:06:06,560 --> 00:06:10,479
people aren't going to learn to code.

140
00:06:08,319 --> 00:06:13,199
And a few years ago, well, not a few

141
00:06:10,479 --> 00:06:15,360
years, one year ago, it was kind of

142
00:06:13,199 --> 00:06:16,960
impossible to write a few hundred line

143
00:06:15,360 --> 00:06:20,240
like Python script to automate part of

144
00:06:16,960 --> 00:06:22,240
your job. Um, or to write like a

145
00:06:20,240 --> 00:06:25,039
thousand line JS game to play with your

146
00:06:22,240 --> 00:06:27,120
kid. But now that's possible for

147
00:06:25,039 --> 00:06:30,479
basically everyone. I think that's a

148
00:06:27,120 --> 00:06:33,440
really good thing. But we're left in the

149
00:06:30,479 --> 00:06:36,319
weird situation where the people who

150
00:06:33,440 --> 00:06:38,960
can't really code just love vibe coding

151
00:06:36,319 --> 00:06:41,759
and the people who code for their job

152
00:06:38,960 --> 00:06:44,960
like don't love vibe coding. Um

153
00:06:41,759 --> 00:06:48,639
something's going on here and now I have

154
00:06:44,960 --> 00:06:51,759
to explain large language models the

155
00:06:48,639 --> 00:06:54,720
type of um AI model that powers uh

156
00:06:51,759 --> 00:06:58,160
today's coding agents. Um, I don't want

157
00:06:54,720 --> 00:07:00,560
to talk about Transformers and attention

158
00:06:58,160 --> 00:07:01,919
and just like say all these other jargon

159
00:07:00,560 --> 00:07:03,919
words in front of you. So, I'm going to

160
00:07:01,919 --> 00:07:07,120
do what everyone else does and just like

161
00:07:03,919 --> 00:07:09,120
add sparkles um and also dumb things

162
00:07:07,120 --> 00:07:11,919
down a bit. So, here here's how I view

163
00:07:09,120 --> 00:07:16,160
it. Um, you give a large language model

164
00:07:11,919 --> 00:07:17,840
some text, you get back more text. Um,

165
00:07:16,160 --> 00:07:19,440
okay, fine. Let's take away the

166
00:07:17,840 --> 00:07:21,680
sparkles.

167
00:07:19,440 --> 00:07:24,000
We actually get tokens. like you give a

168
00:07:21,680 --> 00:07:25,840
large language model some tokens, you

169
00:07:24,000 --> 00:07:28,800
get back more tokens, just one token at

170
00:07:25,840 --> 00:07:32,880
a time. Um, and like what are these

171
00:07:28,800 --> 00:07:36,319
tokens? Um, this is FSBuzz going through

172
00:07:32,880 --> 00:07:40,240
OpenAI's token tokenizer. Um, tokens

173
00:07:36,319 --> 00:07:42,240
look like this. Um, and large language

174
00:07:40,240 --> 00:07:45,599
models generally allow imports of up to

175
00:07:42,240 --> 00:07:47,039
200,000 tokens. Um, that's called the

176
00:07:45,599 --> 00:07:49,280
context window. The input is called the

177
00:07:47,039 --> 00:07:51,759
context window. Now, as you can see

178
00:07:49,280 --> 00:07:55,840
here, a token is about like one word

179
00:07:51,759 --> 00:07:57,440
roughly. Um, as you can see from the

180
00:07:55,840 --> 00:08:00,720
symbols, they kind of mess things up a

181
00:07:57,440 --> 00:08:03,440
bit, but one line of code is about 10

182
00:08:00,720 --> 00:08:04,960
tokens, give or take. And in practice

183
00:08:03,440 --> 00:08:06,240
though, like larger language bottles

184
00:08:04,960 --> 00:08:08,240
have trouble as you fill up more of the

185
00:08:06,240 --> 00:08:11,680
context window. Uh, this is something

186
00:08:08,240 --> 00:08:15,440
called context rot. So, 200,000 tokens

187
00:08:11,680 --> 00:08:17,360
is a bit generous. However, you know the

188
00:08:15,440 --> 00:08:19,599
every single day apparently there's a

189
00:08:17,360 --> 00:08:22,080
new model like the tech gets better all

190
00:08:19,599 --> 00:08:24,560
the time and a state-of-the-art model

191
00:08:22,080 --> 00:08:28,400
these days can have a context window of

192
00:08:24,560 --> 00:08:30,960
a million tokens but that's about

193
00:08:28,400 --> 00:08:33,760
100,000 lines of code um using the

194
00:08:30,960 --> 00:08:37,120
heristic I said earlier and that's just

195
00:08:33,760 --> 00:08:40,240
like not enough um like most software

196
00:08:37,120 --> 00:08:43,440
engineers work in large code bases with

197
00:08:40,240 --> 00:08:45,200
more than 100,000 lines of code Um, I

198
00:08:43,440 --> 00:08:47,920
know Wikipedia says these are the

199
00:08:45,200 --> 00:08:52,839
notably large code bases, but hey,

200
00:08:47,920 --> 00:08:52,839
Google, that's not good.

201
00:08:53,279 --> 00:08:59,440
Can I Ah, we're back. Um, Google also

202
00:08:56,800 --> 00:09:02,240
employs a lot of software engineers. So,

203
00:08:59,440 --> 00:09:04,320
what ends up happening is, uh, companies

204
00:09:02,240 --> 00:09:07,680
with large code bases employ most

205
00:09:04,320 --> 00:09:09,760
software engineers probably. And my

206
00:09:07,680 --> 00:09:12,320
experience is writing a few thousand

207
00:09:09,760 --> 00:09:14,800
lines of code from scratch is a very

208
00:09:12,320 --> 00:09:16,959
different problem from writing code in a

209
00:09:14,800 --> 00:09:20,160
very large code base with millions of

210
00:09:16,959 --> 00:09:22,320
lines of code. Um where shipping too

211
00:09:20,160 --> 00:09:24,560
many bugs too frequently will lose

212
00:09:22,320 --> 00:09:28,640
someone a lot of money.

213
00:09:24,560 --> 00:09:30,399
So I'm going to make a bold claim. Uh

214
00:09:28,640 --> 00:09:33,440
the biggest productivity boost you'll

215
00:09:30,399 --> 00:09:37,360
get is probably not from writing code.

216
00:09:33,440 --> 00:09:39,519
Um, few software engineers

217
00:09:37,360 --> 00:09:40,720
uh actually use AI to write code. And

218
00:09:39,519 --> 00:09:44,080
this is just the software engineers who

219
00:09:40,720 --> 00:09:46,720
use AI a lot. Okay, I didn't actually

220
00:09:44,080 --> 00:09:49,440
give you a productivity tip, but here's

221
00:09:46,720 --> 00:09:51,120
where the fun actually begins. Um,

222
00:09:49,440 --> 00:09:52,880
here's how I personally use coding

223
00:09:51,120 --> 00:09:54,480
agents to be like slightly more

224
00:09:52,880 --> 00:09:56,640
productive.

225
00:09:54,480 --> 00:09:59,839
Uh, number one, searching the codebase.

226
00:09:56,640 --> 00:10:01,360
So, I know I just claim that um these

227
00:09:59,839 --> 00:10:04,080
large language models can't handle more

228
00:10:01,360 --> 00:10:06,399
than 100,000 lines of code, but

229
00:10:04,080 --> 00:10:09,680
neither can I. Um I work in a large

230
00:10:06,399 --> 00:10:13,839
codebase. Um I can't name every file off

231
00:10:09,680 --> 00:10:16,640
by off by heart. What I do is I use the

232
00:10:13,839 --> 00:10:17,920
search bar.

233
00:10:16,640 --> 00:10:20,399
Something's really going off with the

234
00:10:17,920 --> 00:10:22,000
internet. Let me let me go back. Can I

235
00:10:20,399 --> 00:10:24,320
continue?

236
00:10:22,000 --> 00:10:25,519
Oh, no.

237
00:10:24,320 --> 00:10:27,760
the right hand side.

238
00:10:25,519 --> 00:10:30,079
Right hand side.

239
00:10:27,760 --> 00:10:34,079
There we go.

240
00:10:30,079 --> 00:10:35,920
Continue. There we are. Um,

241
00:10:34,079 --> 00:10:38,640
yeah, I use the search bar and that's

242
00:10:35,920 --> 00:10:41,680
also pretty much what they do as well.

243
00:10:38,640 --> 00:10:43,440
Um, Claude was one of the first large

244
00:10:41,680 --> 00:10:46,640
language models get really good at

245
00:10:43,440 --> 00:10:49,920
coding and they got really good by

246
00:10:46,640 --> 00:10:54,079
calling tools. tools like uh search a

247
00:10:49,920 --> 00:10:57,680
directory or read a file or edit a file.

248
00:10:54,079 --> 00:10:59,760
Now I'm going to give you an example. I

249
00:10:57,680 --> 00:11:02,160
know very little about Django, the

250
00:10:59,760 --> 00:11:05,040
Python web framework. I've used Flask

251
00:11:02,160 --> 00:11:06,720
more, but I wanted to I just really

252
00:11:05,040 --> 00:11:09,760
wanted to know how select for update is

253
00:11:06,720 --> 00:11:11,519
implemented in the Django. Um no reason

254
00:11:09,760 --> 00:11:15,680
other than I needed an example to

255
00:11:11,519 --> 00:11:17,760
motivate this. Um this is Gemini code. I

256
00:11:15,680 --> 00:11:19,279
recommend Gemini Code to beginners just

257
00:11:17,760 --> 00:11:22,079
because it has like a super generous

258
00:11:19,279 --> 00:11:23,760
free tier. Um, and if you're cheap like

259
00:11:22,079 --> 00:11:26,320
I was when I was preparing this, it

260
00:11:23,760 --> 00:11:27,839
might help as a good like cost-free way

261
00:11:26,320 --> 00:11:31,920
to introduce yourself to this

262
00:11:27,839 --> 00:11:34,640
technology. So I asked Django, how is

263
00:11:31,920 --> 00:11:37,360
select for update implemented? And just

264
00:11:34,640 --> 00:11:40,160
look at the go. Now I'm not expecting

265
00:11:37,360 --> 00:11:41,519
you to read all this slop, but make a

266
00:11:40,160 --> 00:11:43,200
look make a take a note of the tool

267
00:11:41,519 --> 00:11:45,440
cause. So it's searching text and it's

268
00:11:43,200 --> 00:11:47,680
reading a file. searching text again and

269
00:11:45,440 --> 00:11:50,160
it's reading another file and so on and

270
00:11:47,680 --> 00:11:52,880
so on. If you enroll that it kind of

271
00:11:50,160 --> 00:11:55,519
looks like this. Um I give it a short

272
00:11:52,880 --> 00:11:57,200
prompt. Gemini starts running. It

273
00:11:55,519 --> 00:12:00,720
decides to call the tool to search for

274
00:11:57,200 --> 00:12:02,320
some text. Um it pauses some someone

275
00:12:00,720 --> 00:12:05,200
else something else goes and searches

276
00:12:02,320 --> 00:12:07,680
for the text. Um it goes back to Gemini.

277
00:12:05,200 --> 00:12:09,760
Gemini is like I found a file. I want to

278
00:12:07,680 --> 00:12:12,560
read that file. Um, it calls another

279
00:12:09,760 --> 00:12:14,720
tool and then Gemini starts running it

280
00:12:12,560 --> 00:12:17,920
again and then eventually decides to

281
00:12:14,720 --> 00:12:20,160
just return an answer. Um, yes, I

282
00:12:17,920 --> 00:12:22,480
skipped a few like steps like there's a

283
00:12:20,160 --> 00:12:25,279
lot of tool calls sometimes, but that's

284
00:12:22,480 --> 00:12:27,440
the gist of it. And it eventually gave

285
00:12:25,279 --> 00:12:29,120
me an answer. I did have to tell it to

286
00:12:27,440 --> 00:12:31,120
like be more concise so I could fit on

287
00:12:29,120 --> 00:12:32,560
the slide, but it found the answer in

288
00:12:31,120 --> 00:12:36,720
the end.

289
00:12:32,560 --> 00:12:39,279
Now, Gemini Code and Claude Code are AI

290
00:12:36,720 --> 00:12:40,880
agents, you might call them. Um, you

291
00:12:39,279 --> 00:12:43,519
might have heard of that term before,

292
00:12:40,880 --> 00:12:47,360
like AI agents. Um, you might be

293
00:12:43,519 --> 00:12:50,000
wondering, what is an AI agent? Um, it's

294
00:12:47,360 --> 00:12:52,399
probably one of the most illdefined like

295
00:12:50,000 --> 00:12:55,040
terms around nowadays. Like, anything

296
00:12:52,399 --> 00:12:57,839
that has like a sprinkle of AI is called

297
00:12:55,040 --> 00:12:59,760
an AI agent. Um, I've heard dozens of

298
00:12:57,839 --> 00:13:02,639
definitions, but there's a definition I

299
00:12:59,760 --> 00:13:04,880
really like, which is an AI agent is

300
00:13:02,639 --> 00:13:09,200
just a large language model calling

301
00:13:04,880 --> 00:13:13,839
tools in a loop. A tool like search code

302
00:13:09,200 --> 00:13:16,720
or write code to a file or search the

303
00:13:13,839 --> 00:13:19,600
web or you know whatever, but they just

304
00:13:16,720 --> 00:13:23,040
calling tools in a loop.

305
00:13:19,600 --> 00:13:26,240
Um, cool. Now, the second thing I like

306
00:13:23,040 --> 00:13:28,079
to do is debugging stack traces.

307
00:13:26,240 --> 00:13:30,240
Maybe someone here will surprise me, but

308
00:13:28,079 --> 00:13:33,680
I haven't met that many people who just

309
00:13:30,240 --> 00:13:35,920
love debugging stack traces. Um,

310
00:13:33,680 --> 00:13:39,519
thankfully, you don't really need to

311
00:13:35,920 --> 00:13:42,480
that much anymore because, well, here's

312
00:13:39,519 --> 00:13:44,240
a slight digression, but as a general

313
00:13:42,480 --> 00:13:46,880
huristic with large language models, one

314
00:13:44,240 --> 00:13:49,600
that I use a lot is that going from many

315
00:13:46,880 --> 00:13:52,959
words to a few words generally pretty

316
00:13:49,600 --> 00:13:56,800
good. Going from a few words to many

317
00:13:52,959 --> 00:13:58,399
words is a bit more sketchy. Um,

318
00:13:56,800 --> 00:14:01,440
this is like a pretty common joke about

319
00:13:58,399 --> 00:14:03,600
AI. Like, I'm going to write a single

320
00:14:01,440 --> 00:14:06,880
bullet point. I'm going to turn that

321
00:14:03,600 --> 00:14:08,560
into a long email and then you're going

322
00:14:06,880 --> 00:14:11,519
to take my long email and turn it back

323
00:14:08,560 --> 00:14:13,680
into that single bullet point. Um, I

324
00:14:11,519 --> 00:14:15,279
think that's a bit unfair because

325
00:14:13,680 --> 00:14:17,440
turning a single bullet point into a

326
00:14:15,279 --> 00:14:19,360
long email is begging the model just to

327
00:14:17,440 --> 00:14:23,600
make stuff up. Like, you're telling it

328
00:14:19,360 --> 00:14:25,360
to write slop. Um, but when you go from

329
00:14:23,600 --> 00:14:27,920
a long email to like a single bullet

330
00:14:25,360 --> 00:14:29,519
points, um, it might miss a crucial

331
00:14:27,920 --> 00:14:31,760
detail, but it's compressing a lot of

332
00:14:29,519 --> 00:14:33,199
text, like it's it's not it doesn't need

333
00:14:31,760 --> 00:14:34,800
to make stuff up because it has a long

334
00:14:33,199 --> 00:14:36,800
email.

335
00:14:34,800 --> 00:14:39,920
I think that debugging a stack trace is

336
00:14:36,800 --> 00:14:42,720
fairly similar. So, um, a stack trace

337
00:14:39,920 --> 00:14:45,600
has a lot of lines. Generally, like one

338
00:14:42,720 --> 00:14:47,920
or two lines actually matter, and you

339
00:14:45,600 --> 00:14:50,480
can tell fairly quickly if you found the

340
00:14:47,920 --> 00:14:53,199
right line. Um, that just saves you the

341
00:14:50,480 --> 00:14:56,320
time of reading the entire stack trace.

342
00:14:53,199 --> 00:14:57,519
Um, side note, this is not important.

343
00:14:56,320 --> 00:14:58,639
Um, I didn't know what to put on these

344
00:14:57,519 --> 00:15:01,279
slides because who would actually read

345
00:14:58,639 --> 00:15:03,440
the stack trace? Um, this is just some

346
00:15:01,279 --> 00:15:05,600
software go I found. Um, here's some

347
00:15:03,440 --> 00:15:08,000
more software go this time with a

348
00:15:05,600 --> 00:15:11,519
password being leaked in a stack trace.

349
00:15:08,000 --> 00:15:13,920
Um, just for fun. But yeah, the the cool

350
00:15:11,519 --> 00:15:16,480
thing I found with debugging stack

351
00:15:13,920 --> 00:15:19,360
traces using coding agents is you might

352
00:15:16,480 --> 00:15:23,440
genuinely be 10x more productive. Like

353
00:15:19,360 --> 00:15:25,680
10x programming is a thing. Um that

354
00:15:23,440 --> 00:15:27,440
doesn't really work out because like

355
00:15:25,680 --> 00:15:29,760
most of your time right now is probably

356
00:15:27,440 --> 00:15:32,320
not debugging stack traces. It's like

357
00:15:29,760 --> 00:15:34,639
you know 2% of your time. Now it's 0.2%

358
00:15:32,320 --> 00:15:37,440
of your time. Um but one survey found

359
00:15:34,639 --> 00:15:39,600
that just debugging stack traces was the

360
00:15:37,440 --> 00:15:42,480
single um biggest time saving from using

361
00:15:39,600 --> 00:15:45,600
AI tools um just because it's so

362
00:15:42,480 --> 00:15:48,959
radically um efficient.

363
00:15:45,600 --> 00:15:50,880
But there is a caveat um it helps to

364
00:15:48,959 --> 00:15:52,480
choose boring technology. Some people

365
00:15:50,880 --> 00:15:54,800
might have heard of this motto before

366
00:15:52,480 --> 00:15:56,959
basically you know prefer popular

367
00:15:54,800 --> 00:16:00,240
frameworks with lots of support rather

368
00:15:56,959 --> 00:16:02,480
than um a cool new library I published

369
00:16:00,240 --> 00:16:04,480
yesterday that no one knows about. Um,

370
00:16:02,480 --> 00:16:07,839
coding agents kind of turbocharge that a

371
00:16:04,480 --> 00:16:09,279
bit. Like more code in the wild means uh

372
00:16:07,839 --> 00:16:11,120
more training for the large language

373
00:16:09,279 --> 00:16:13,920
models. That means they understand your

374
00:16:11,120 --> 00:16:16,800
tech bit a bit better. Um, you'll have

375
00:16:13,920 --> 00:16:19,040
an easier time with a popular framework

376
00:16:16,800 --> 00:16:21,600
like Django than a library published

377
00:16:19,040 --> 00:16:23,199
like this year. It does make me sad

378
00:16:21,600 --> 00:16:25,120
sometimes knowing that if I ever wanted

379
00:16:23,199 --> 00:16:27,120
to, you know, make a cool new Python web

380
00:16:25,120 --> 00:16:29,040
framework, I'll have a hard time

381
00:16:27,120 --> 00:16:31,199
competing against Django. But I think

382
00:16:29,040 --> 00:16:33,680
the ship has sailed.

383
00:16:31,199 --> 00:16:34,720
Um and of course you know luckily for

384
00:16:33,680 --> 00:16:36,959
everyone here this is a Python

385
00:16:34,720 --> 00:16:39,519
conference. Uh Python is a boring

386
00:16:36,959 --> 00:16:42,320
technology these days. Um last year it

387
00:16:39,519 --> 00:16:45,040
took overtook JavaScript as the most

388
00:16:42,320 --> 00:16:46,880
used language in public GitHub repos. Um

389
00:16:45,040 --> 00:16:48,639
Python is generally the most

390
00:16:46,880 --> 00:16:50,240
wellsupported language by um coding

391
00:16:48,639 --> 00:16:53,440
agents.

392
00:16:50,240 --> 00:16:56,160
Okay. Number three. You want to fix some

393
00:16:53,440 --> 00:16:57,920
bugs. Um, this seems to be like the

394
00:16:56,160 --> 00:17:01,759
bread and butter for casual coding agent

395
00:16:57,920 --> 00:17:05,679
users, like just um fix my bugs for me.

396
00:17:01,759 --> 00:17:09,600
But I suspect most casual users aren't

397
00:17:05,679 --> 00:17:12,480
doing it super well. Um, like picture

398
00:17:09,600 --> 00:17:13,760
this. It's the first day in your job. As

399
00:17:12,480 --> 00:17:16,640
part of your on boarding, you just get

400
00:17:13,760 --> 00:17:18,799
the few assigned a few bug fixes and

401
00:17:16,640 --> 00:17:22,559
your boss comes over and just says two

402
00:17:18,799 --> 00:17:26,319
words, fix this. That's all they say.

403
00:17:22,559 --> 00:17:29,520
they just say fix this. Um, personally

404
00:17:26,319 --> 00:17:32,080
that would suck. Um, but I've seen a lot

405
00:17:29,520 --> 00:17:35,760
of developers and just like try

406
00:17:32,080 --> 00:17:37,919
debugging by just saying fix this and

407
00:17:35,760 --> 00:17:39,360
you would not say that to a new hire on

408
00:17:37,919 --> 00:17:42,640
the first day of their job with a few

409
00:17:39,360 --> 00:17:46,400
new like bug fix tickets because they

410
00:17:42,640 --> 00:17:49,120
have no context. Um, don't really expect

411
00:17:46,400 --> 00:17:52,080
coding agents to be any different. Like

412
00:17:49,120 --> 00:17:54,880
a newbie on their first day needs

413
00:17:52,080 --> 00:17:56,799
context just like a coding agent or as I

414
00:17:54,880 --> 00:17:58,720
like to say context is king. The

415
00:17:56,799 --> 00:18:00,080
huristic I use is just give as much

416
00:17:58,720 --> 00:18:02,640
context as you would to a newbie on

417
00:18:00,080 --> 00:18:05,039
their first day. Um I have seen coding

418
00:18:02,640 --> 00:18:07,520
agents fix very complex bugs from

419
00:18:05,039 --> 00:18:09,600
detailed bug reports even with images.

420
00:18:07,520 --> 00:18:12,640
They can read images these days but I

421
00:18:09,600 --> 00:18:15,520
have not seen many um coding agents fix

422
00:18:12,640 --> 00:18:19,440
many bugs from like a one or two word

423
00:18:15,520 --> 00:18:21,360
bug report. Uh, two tips I recommend to

424
00:18:19,440 --> 00:18:23,440
ensure you have enough context is just

425
00:18:21,360 --> 00:18:24,960
like prompted to ask clarifying

426
00:18:23,440 --> 00:18:27,600
questions. They sometimes won't by

427
00:18:24,960 --> 00:18:30,320
default, but literally just add like if

428
00:18:27,600 --> 00:18:33,120
anything's unclear, ask me a few

429
00:18:30,320 --> 00:18:34,960
questions before going ahead. Um, it's

430
00:18:33,120 --> 00:18:37,679
the same way that, you know, I'veed

431
00:18:34,960 --> 00:18:39,200
newbies at work a few times and the

432
00:18:37,679 --> 00:18:41,600
newbies that ask more questions

433
00:18:39,200 --> 00:18:44,640
generally on board a bit faster. The

434
00:18:41,600 --> 00:18:47,760
other thing is, this might sound a bit

435
00:18:44,640 --> 00:18:50,799
crazy, but literally just talk to it.

436
00:18:47,760 --> 00:18:54,240
Um, people usually speak faster than

437
00:18:50,799 --> 00:18:56,240
they type. So, use your microphone and

438
00:18:54,240 --> 00:18:58,880
use a transcription tool and literally

439
00:18:56,240 --> 00:19:00,559
talk to your computer. Um, large

440
00:18:58,880 --> 00:19:03,600
language models understand language

441
00:19:00,559 --> 00:19:06,240
really well. They'll get past like, you

442
00:19:03,600 --> 00:19:08,400
know, grammar mistakes or transcription

443
00:19:06,240 --> 00:19:11,520
errors. They won't get past a lack of

444
00:19:08,400 --> 00:19:13,280
context. Uh just a warning though that

445
00:19:11,520 --> 00:19:15,919
if you do it in an office you might get

446
00:19:13,280 --> 00:19:20,160
some weird looks but

447
00:19:15,919 --> 00:19:22,160
um I have I've done it for you. Uh and

448
00:19:20,160 --> 00:19:24,480
last thing is you know avoid context

449
00:19:22,160 --> 00:19:26,160
rot. So as I said earlier like these

450
00:19:24,480 --> 00:19:29,039
large language models struggle a bit as

451
00:19:26,160 --> 00:19:30,720
you fill up more that context window and

452
00:19:29,039 --> 00:19:33,840
if you fill up entirely it's kind of

453
00:19:30,720 --> 00:19:35,280
game over. Uh yes cing agents can search

454
00:19:33,840 --> 00:19:37,280
really well. they can call tools and so

455
00:19:35,280 --> 00:19:39,679
on, but if you're working in a

456
00:19:37,280 --> 00:19:41,840
sufficiently large codebase, um my

457
00:19:39,679 --> 00:19:45,360
experience is like 100,000 lines of

458
00:19:41,840 --> 00:19:47,600
code, you know, from with no context, um

459
00:19:45,360 --> 00:19:49,120
is surprisingly small. So, you do want

460
00:19:47,600 --> 00:19:52,000
to protect your context window if you

461
00:19:49,120 --> 00:19:53,840
can. The most obvious way to do that is

462
00:19:52,000 --> 00:19:56,720
these agents are configured to read from

463
00:19:53,840 --> 00:20:00,320
like a markdown file. In Gemini, it's uh

464
00:19:56,720 --> 00:20:02,160
gemini.md. In Claude, it's claw.md. um

465
00:20:00,320 --> 00:20:04,400
it's just the place the go-to place to

466
00:20:02,160 --> 00:20:07,760
dump tip tips and tricks about your

467
00:20:04,400 --> 00:20:09,440
codebase and often um the coding agent

468
00:20:07,760 --> 00:20:12,720
will like offer to do it for you if it's

469
00:20:09,440 --> 00:20:15,760
the first time but you know might be hit

470
00:20:12,720 --> 00:20:18,240
and miss. Um this is like Gemini code is

471
00:20:15,760 --> 00:20:22,320
open source and so is that like

472
00:20:18,240 --> 00:20:24,880
gemini.md they use to work on Gemini.

473
00:20:22,320 --> 00:20:27,280
um it's a bit meta what you'll mostly

474
00:20:24,880 --> 00:20:31,919
see it's about uh running command line

475
00:20:27,280 --> 00:20:35,440
tools and style conventions. Okay,

476
00:20:31,919 --> 00:20:37,039
number four writing code. I know I said

477
00:20:35,440 --> 00:20:40,480
earlier that the biggest productivity

478
00:20:37,039 --> 00:20:42,720
boost won't be from writing code but I

479
00:20:40,480 --> 00:20:45,200
confess I have used coding agents to

480
00:20:42,720 --> 00:20:48,240
write code. I have committed that code.

481
00:20:45,200 --> 00:20:50,720
Um, but you might have heard people

482
00:20:48,240 --> 00:20:54,559
complaining very loudly that Claude Code

483
00:20:50,720 --> 00:20:56,159
just loves deleting unit tests. Um,

484
00:20:54,559 --> 00:20:58,960
I'm one of these people, by the way.

485
00:20:56,159 --> 00:21:01,120
I've seen it firsthand or that they

486
00:20:58,960 --> 00:21:04,000
might just delete your database um once

487
00:21:01,120 --> 00:21:07,120
in a while just for fun. Uh, and that's

488
00:21:04,000 --> 00:21:08,799
why I have trust issues. Um, I have

489
00:21:07,120 --> 00:21:11,200
trust issues with these coding agents.

490
00:21:08,799 --> 00:21:13,039
Like maybe I'll trust them in the future

491
00:21:11,200 --> 00:21:15,200
when they get better, but right now the

492
00:21:13,039 --> 00:21:18,159
future isn't here. Um, I've learned that

493
00:21:15,200 --> 00:21:21,679
the hard way too many times. And I'm not

494
00:21:18,159 --> 00:21:24,480
alone. Like, very few developers seem to

495
00:21:21,679 --> 00:21:27,760
highly trust AI. Like 3% seems like a

496
00:21:24,480 --> 00:21:29,919
small number to me. Um, but why do

497
00:21:27,760 --> 00:21:34,799
coding agents just love deleting my unit

498
00:21:29,919 --> 00:21:36,320
tests? Um, okay. So, now I have to talk

499
00:21:34,799 --> 00:21:38,320
about how large language models are

500
00:21:36,320 --> 00:21:40,000
trained. This is going to be a massive

501
00:21:38,320 --> 00:21:42,320
oversimplification because there's so

502
00:21:40,000 --> 00:21:44,640
much you can fill into 30 minutes. But

503
00:21:42,320 --> 00:21:47,679
one way of thinking about it is large

504
00:21:44,640 --> 00:21:50,159
language models are trained to maximize

505
00:21:47,679 --> 00:21:52,559
a reward function. A reward function

506
00:21:50,159 --> 00:21:55,200
like write code that passes the unit

507
00:21:52,559 --> 00:21:58,000
tests. Um you might see like the flaw in

508
00:21:55,200 --> 00:22:00,159
this but we'll get there. Um the success

509
00:21:58,000 --> 00:22:03,679
of chatbt for example is credited with

510
00:22:00,159 --> 00:22:05,280
just like um you know getting trained at

511
00:22:03,679 --> 00:22:06,799
getting really good at writing text that

512
00:22:05,280 --> 00:22:09,919
humans like as measured by that little

513
00:22:06,799 --> 00:22:12,880
thumbs up or thumbs down button.

514
00:22:09,919 --> 00:22:15,360
Um, but in practice writing like

515
00:22:12,880 --> 00:22:17,840
ungameable reward functions is really

516
00:22:15,360 --> 00:22:22,080
hard and sometimes large language models

517
00:22:17,840 --> 00:22:24,159
find an easier way to achieve the reward

518
00:22:22,080 --> 00:22:27,039
um without doing the hard work. That's

519
00:22:24,159 --> 00:22:28,720
called reward hacking. Um, and sometimes

520
00:22:27,039 --> 00:22:31,120
coding agents just figure out like why

521
00:22:28,720 --> 00:22:33,679
do I have to fix the tests when I can

522
00:22:31,120 --> 00:22:36,400
just delete the tests? Um, it's obvious,

523
00:22:33,679 --> 00:22:39,200
right? Just that zero tests pass

524
00:22:36,400 --> 00:22:41,760
successfully. Um, and that's why my

525
00:22:39,200 --> 00:22:43,600
experience is, you know, you won't save

526
00:22:41,760 --> 00:22:46,960
that much time generating a thousand

527
00:22:43,600 --> 00:22:49,600
lines of untested slop to a critical

528
00:22:46,960 --> 00:22:51,600
system that you can't go down because

529
00:22:49,600 --> 00:22:54,480
you're on call and you like to sleep.

530
00:22:51,600 --> 00:22:56,159
Um, I have tried. It was not fun

531
00:22:54,480 --> 00:23:00,720
reviewing every single line and fixing

532
00:22:56,159 --> 00:23:03,679
all that code. Um my general rule is the

533
00:23:00,720 --> 00:23:06,240
time saved with coding agents is sort of

534
00:23:03,679 --> 00:23:09,520
inversely proportional to the time spent

535
00:23:06,240 --> 00:23:11,840
verifying like um that's why I really

536
00:23:09,520 --> 00:23:14,799
like using coding agents to debug code

537
00:23:11,840 --> 00:23:17,679
because bugs are usually one or two

538
00:23:14,799 --> 00:23:19,120
lines and usually obvious in hindsight.

539
00:23:17,679 --> 00:23:22,240
It's just finding those two lines is

540
00:23:19,120 --> 00:23:24,720
really hard. Um it's just easy to verify

541
00:23:22,240 --> 00:23:28,159
so you save a lot of time.

542
00:23:24,720 --> 00:23:30,960
Having said that, you can um save time

543
00:23:28,159 --> 00:23:33,679
using coding agents to write code. Uh

544
00:23:30,960 --> 00:23:36,400
for example, you'll save more time

545
00:23:33,679 --> 00:23:39,120
refactoring code if there already exists

546
00:23:36,400 --> 00:23:41,600
a comprehensive set of unit tests. Just

547
00:23:39,120 --> 00:23:43,200
make sure they don't get deleted. Um if

548
00:23:41,600 --> 00:23:45,600
you don't have a comprehensive set of

549
00:23:43,200 --> 00:23:48,880
unit tests, just be prepared to review

550
00:23:45,600 --> 00:23:51,360
every line of code very carefully.

551
00:23:48,880 --> 00:23:53,679
You can try to ask your coding agents to

552
00:23:51,360 --> 00:23:56,400
write some tests beforehands. that can

553
00:23:53,679 --> 00:23:59,200
work, but in my experience, it's still a

554
00:23:56,400 --> 00:24:00,480
bit of a slog than, you know, a set of

555
00:23:59,200 --> 00:24:02,559
coding agents that you really, really

556
00:24:00,480 --> 00:24:05,840
trust, set of tests you really, really

557
00:24:02,559 --> 00:24:07,520
trust. Um, and coding agents are also

558
00:24:05,840 --> 00:24:10,640
really good at writing shell scripts.

559
00:24:07,520 --> 00:24:13,360
Like remember five coding from earlier?

560
00:24:10,640 --> 00:24:14,720
Um, running a few hundred lines of code

561
00:24:13,360 --> 00:24:17,279
from scratch is bread and butter for

562
00:24:14,720 --> 00:24:20,480
these things. And most shell scripts

563
00:24:17,279 --> 00:24:23,279
don't do anything too spicy. Um, most of

564
00:24:20,480 --> 00:24:26,720
my shell scripts just transform a CSV

565
00:24:23,279 --> 00:24:28,960
file to a different CSV file. Uh, maybe

566
00:24:26,720 --> 00:24:31,760
like add a common or something. Um,

567
00:24:28,960 --> 00:24:33,440
nothing too spicy. And the neat thing

568
00:24:31,760 --> 00:24:35,200
about those types of shell scripts is

569
00:24:33,440 --> 00:24:37,440
you can just look at the output. Does

570
00:24:35,200 --> 00:24:41,279
the output look same? If so, you don't

571
00:24:37,440 --> 00:24:44,080
even need to look at the code. Um,

572
00:24:41,279 --> 00:24:46,640
yes. Number five, generating synthetic

573
00:24:44,080 --> 00:24:48,880
data. So large language models are

574
00:24:46,640 --> 00:24:53,120
really good at making stuff up. stuff

575
00:24:48,880 --> 00:24:54,960
that is realistic but not real. And a

576
00:24:53,120 --> 00:24:57,120
couple years ago I was working at Canva

577
00:24:54,960 --> 00:24:58,880
on autocomplete.

578
00:24:57,120 --> 00:25:02,559
And we were gearing up to announce a

579
00:24:58,880 --> 00:25:04,720
design generation tool called uh magic

580
00:25:02,559 --> 00:25:06,480
design. Given the prompts, it will

581
00:25:04,720 --> 00:25:08,400
create a presentation or social media

582
00:25:06,480 --> 00:25:10,880
post. But there was a problem we had in

583
00:25:08,400 --> 00:25:12,880
autocomplete which is the vast majority

584
00:25:10,880 --> 00:25:16,799
of search queries are fewer than five

585
00:25:12,880 --> 00:25:18,559
words like usually one or two words. Um,

586
00:25:16,799 --> 00:25:21,520
magic design prompts are more than five

587
00:25:18,559 --> 00:25:24,720
words. So, how do we get autocomplete to

588
00:25:21,520 --> 00:25:28,000
work on magic design? Uh, we literally

589
00:25:24,720 --> 00:25:29,520
just made stuff up. We got one human,

590
00:25:28,000 --> 00:25:32,480
uh, not an engineer, by the way, to

591
00:25:29,520 --> 00:25:35,360
write down like several hundred prompts

592
00:25:32,480 --> 00:25:37,840
that were realistic and real and then

593
00:25:35,360 --> 00:25:39,520
used um, chatbt to generate like

594
00:25:37,840 --> 00:25:41,760
thousands of more prompts from that like

595
00:25:39,520 --> 00:25:44,400
seed.

596
00:25:41,760 --> 00:25:47,520
I'm skipping out over a lot of hard work

597
00:25:44,400 --> 00:25:50,240
here, but it kind of worked. Um, and we

598
00:25:47,520 --> 00:25:52,400
had solved the cold start problem in in

599
00:25:50,240 --> 00:25:55,520
like these recommendation systems where

600
00:25:52,400 --> 00:25:57,600
autocomplete is almost magical if you

601
00:25:55,520 --> 00:26:00,080
have a lot of data, but if you don't

602
00:25:57,600 --> 00:26:01,840
have any data, you're in a hard spot.

603
00:26:00,080 --> 00:26:04,720
Large language models can help bootstrap

604
00:26:01,840 --> 00:26:06,720
across that like bridge by, you know,

605
00:26:04,720 --> 00:26:09,600
generating realms and realms of of fake

606
00:26:06,720 --> 00:26:11,760
but realistic data. Yes, real data is

607
00:26:09,600 --> 00:26:13,919
better than fake data, but fake data was

608
00:26:11,760 --> 00:26:15,520
better than no data.

609
00:26:13,919 --> 00:26:17,520
Okay,

610
00:26:15,520 --> 00:26:21,520
those are my five big tips and I want to

611
00:26:17,520 --> 00:26:23,200
ask, you know, what did we learn? Um, my

612
00:26:21,520 --> 00:26:24,880
takeaways are, you know, coding agents

613
00:26:23,200 --> 00:26:27,600
are really good at some things, but

614
00:26:24,880 --> 00:26:31,520
they're also really bad at other things.

615
00:26:27,600 --> 00:26:33,840
Um, but you only find out that coding

616
00:26:31,520 --> 00:26:36,880
agent is good at something by wasting

617
00:26:33,840 --> 00:26:41,200
time. um productivity doesn't only go

618
00:26:36,880 --> 00:26:43,679
up. Um so you know you will waste time

619
00:26:41,200 --> 00:26:45,360
figuring that figuring out that putting

620
00:26:43,679 --> 00:26:47,279
coding agent up to a task was just a

621
00:26:45,360 --> 00:26:49,200
really bad idea. But then you'll save a

622
00:26:47,279 --> 00:26:51,919
lot of time like figuring out it's

623
00:26:49,200 --> 00:26:54,000
really good at something. Um and that's

624
00:26:51,919 --> 00:26:57,400
it. If there's still time I'm happy to

625
00:26:54,000 --> 00:26:57,400
take any questions.

626
00:27:01,600 --> 00:27:05,520
Thank you very much Michael. Um before

627
00:27:03,600 --> 00:27:07,200
we have any questions, if Tom Ellen is

628
00:27:05,520 --> 00:27:08,799
in this room, could they please head to

629
00:27:07,200 --> 00:27:11,360
the next to next door for their talk?

630
00:27:08,799 --> 00:27:13,520
Thank you. Um we have a couple of

631
00:27:11,360 --> 00:27:16,520
minutes for questions. Uh do we have any

632
00:27:13,520 --> 00:27:16,520
questions?

633
00:27:16,640 --> 00:27:21,960
We have a question over here about

634
00:27:18,559 --> 00:27:21,960
halfway down.

635
00:27:25,679 --> 00:27:30,240
Thank you very much. Um hia, I'm Renee.

636
00:27:28,000 --> 00:27:34,159
I work at Microsoft the wonderful world

637
00:27:30,240 --> 00:27:37,039
of co-pilot. Um, and I've been recently

638
00:27:34,159 --> 00:27:39,120
using uh, as you showed with Gemini, the

639
00:27:37,039 --> 00:27:42,000
um, Gemini don't mark, we've got

640
00:27:39,120 --> 00:27:44,559
co-pilot instructions.mmd

641
00:27:42,000 --> 00:27:46,400
going on both at a global like level for

642
00:27:44,559 --> 00:27:48,960
all of your repos and at a repo level. I

643
00:27:46,400 --> 00:27:50,799
like including things like uh, you know,

644
00:27:48,960 --> 00:27:53,200
make all my variable names in Australian

645
00:27:50,799 --> 00:27:55,760
spelling um, and things like that or

646
00:27:53,200 --> 00:27:58,080
like this project is using fast API. I

647
00:27:55,760 --> 00:28:00,320
was wondering if you have any overall

648
00:27:58,080 --> 00:28:02,960
tips that you would include in those

649
00:28:00,320 --> 00:28:03,440
kind of instruction data sets, markdown

650
00:28:02,960 --> 00:28:06,960
files.

651
00:28:03,440 --> 00:28:09,679
Uh yeah, sure. I think the best tip is

652
00:28:06,960 --> 00:28:12,399
generally like get it to develop a plan

653
00:28:09,679 --> 00:28:16,320
and then get you to approve that plan.

654
00:28:12,399 --> 00:28:19,120
Um yeah, that's my biggest tip. Um I've

655
00:28:16,320 --> 00:28:20,640
sometimes you like I I have a lot of

656
00:28:19,120 --> 00:28:22,080
command lines which are really really

657
00:28:20,640 --> 00:28:24,720
slow and sometimes it takes a long time

658
00:28:22,080 --> 00:28:26,559
and I realize oh it's cuz I said run all

659
00:28:24,720 --> 00:28:28,559
the tests and then runs all the tests

660
00:28:26,559 --> 00:28:31,600
and I have to wait 5 minutes. So you

661
00:28:28,559 --> 00:28:34,320
know it differs a lot depending on your

662
00:28:31,600 --> 00:28:36,960
project. Um but main one get to make a

663
00:28:34,320 --> 00:28:39,840
plan.

664
00:28:36,960 --> 00:28:44,039
Uh, one more question.

665
00:28:39,840 --> 00:28:44,039
No, just up here at the front.

666
00:28:48,240 --> 00:28:53,279
Great talk, Michael. I don't think you

667
00:28:50,320 --> 00:28:54,960
mentioned anything about using AI for

668
00:28:53,279 --> 00:28:55,919
code reviews. Do you have a take on

669
00:28:54,960 --> 00:28:59,679
that?

670
00:28:55,919 --> 00:29:02,240
Um, my experience is they are really

671
00:28:59,679 --> 00:29:07,200
good at nitpicks. Really, really good at

672
00:29:02,240 --> 00:29:09,279
nitpicks. Um but you again the biggest

673
00:29:07,200 --> 00:29:10,960
mistake is like people just expect it to

674
00:29:09,279 --> 00:29:12,880
be good but you have to kind of refine

675
00:29:10,960 --> 00:29:15,200
it with a lot of explanation of like

676
00:29:12,880 --> 00:29:16,880
what do you want it to review exactly um

677
00:29:15,200 --> 00:29:18,880
to get good results otherwise it'll just

678
00:29:16,880 --> 00:29:20,960
be like you know in Python we expect

679
00:29:18,880 --> 00:29:23,520
variables to be like this and you did

680
00:29:20,960 --> 00:29:26,320
not do that you should do that um that's

681
00:29:23,520 --> 00:29:28,399
my experience at least

682
00:29:26,320 --> 00:29:30,480
cool uh that's all the time we have

683
00:29:28,399 --> 00:29:31,520
thank you very much Michael here is your

684
00:29:30,480 --> 00:29:32,559
mug

685
00:29:31,520 --> 00:29:32,960
are you around for the rest of the

686
00:29:32,559 --> 00:29:37,399
weekend

687
00:29:32,960 --> 00:29:37,399
yes I Um, thank you, Michael.

688
00:29:39,520 --> 00:29:42,799
Cool. We've got