1
00:00:00,480 --> 00:00:03,480
foreign

2
00:00:08,400 --> 00:00:13,620
welcome back everyone I'd like to

3
00:00:10,679 --> 00:00:15,420
introduce Jeremy Rothstein Jeremy is a

4
00:00:13,620 --> 00:00:17,880
systems engineer with a background in

5
00:00:15,420 --> 00:00:20,160
industrial Control Systems dowels in

6
00:00:17,880 --> 00:00:23,660
micropython and had the displeasure of

7
00:00:20,160 --> 00:00:23,660
being my colleague for several years

8
00:00:23,699 --> 00:00:27,040
take it away Jeremy now it was fun it

9
00:00:26,220 --> 00:00:31,560
was fun

10
00:00:27,040 --> 00:00:34,500
[Applause]

11
00:00:31,560 --> 00:00:37,800
g'day um yeah thanks a lot Chris

12
00:00:34,500 --> 00:00:39,660
um yeah whenever I go to pycon I've

13
00:00:37,800 --> 00:00:41,879
always thought it'd be nice to know you

14
00:00:39,660 --> 00:00:44,100
know ahead of time when a Talk's going

15
00:00:41,879 --> 00:00:46,680
to get to the really juicy python

16
00:00:44,100 --> 00:00:48,360
content so it's coming I promise you

17
00:00:46,680 --> 00:00:52,039
we've just got to get through some

18
00:00:48,360 --> 00:00:52,039
conceptual stuff first

19
00:00:52,559 --> 00:00:58,620
so in one word this talks really about

20
00:00:55,980 --> 00:01:02,579
the notion of reliability

21
00:00:58,620 --> 00:01:05,100
saw and giving attention to an aspect of

22
00:01:02,579 --> 00:01:07,260
Apple liability that's often an

23
00:01:05,100 --> 00:01:10,619
afterthought or sometimes even

24
00:01:07,260 --> 00:01:12,360
overlooked entirely and by then it's too

25
00:01:10,619 --> 00:01:15,659
late

26
00:01:12,360 --> 00:01:18,240
so when people say you know this app's

27
00:01:15,659 --> 00:01:20,100
Rock Solid

28
00:01:18,240 --> 00:01:23,100
yep

29
00:01:20,100 --> 00:01:24,240
this app's Rock Solid it runs like a

30
00:01:23,100 --> 00:01:26,220
dream

31
00:01:24,240 --> 00:01:28,320
what they're really saying is that it

32
00:01:26,220 --> 00:01:30,960
handles all the edge cases well it's

33
00:01:28,320 --> 00:01:33,299
engineered to be fault tolerant has good

34
00:01:30,960 --> 00:01:34,439
performance it does what it needs to do

35
00:01:33,299 --> 00:01:37,320
well

36
00:01:34,439 --> 00:01:40,700
what they're talking about is robustness

37
00:01:37,320 --> 00:01:40,700
the app is robust

38
00:01:40,799 --> 00:01:45,360
and it's robustness that what we tend to

39
00:01:43,439 --> 00:01:47,340
think about and what we put all our

40
00:01:45,360 --> 00:01:49,579
energy into achieving all those coding

41
00:01:47,340 --> 00:01:53,280
standards code reviews design meetings

42
00:01:49,579 --> 00:01:55,020
refactoring sessions testing all that

43
00:01:53,280 --> 00:01:57,360
it's all designed to achieve a

44
00:01:55,020 --> 00:01:59,579
robustness

45
00:01:57,360 --> 00:02:01,619
but as you know if there's something we

46
00:01:59,579 --> 00:02:04,079
all know it's that despite our best

47
00:02:01,619 --> 00:02:06,299
efforts our applications can fail if

48
00:02:04,079 --> 00:02:09,599
they can fail they will fail at some

49
00:02:06,299 --> 00:02:12,540
point and entirely unexpected ways

50
00:02:09,599 --> 00:02:14,879
so shouldn't we be prepared for that in

51
00:02:12,540 --> 00:02:17,400
some way

52
00:02:14,879 --> 00:02:19,379
so when it does enter that failed state

53
00:02:17,400 --> 00:02:22,980
state in production

54
00:02:19,379 --> 00:02:24,959
isn't reliability more of a Time measure

55
00:02:22,980 --> 00:02:26,760
how long it takes to get the system

56
00:02:24,959 --> 00:02:28,739
working again

57
00:02:26,760 --> 00:02:30,959
so that's the repairability of your

58
00:02:28,739 --> 00:02:32,099
application now that becomes the crucial

59
00:02:30,959 --> 00:02:35,160
Factor

60
00:02:32,099 --> 00:02:39,500
it's not just how robust your app is

61
00:02:35,160 --> 00:02:39,500
that ship has sailed and I guess sank

62
00:02:42,000 --> 00:02:45,180
so if you're a developer and a

63
00:02:43,739 --> 00:02:46,920
maintainer so you've got Skinner that's

64
00:02:45,180 --> 00:02:50,060
in the game you're developing and you're

65
00:02:46,920 --> 00:02:50,060
maintaining your own product

66
00:02:50,360 --> 00:02:55,019
beard can I just adjust the mic a little

67
00:02:52,620 --> 00:02:56,520
bit yep

68
00:02:55,019 --> 00:02:59,599
oh

69
00:02:56,520 --> 00:02:59,599
[Music]

70
00:03:00,959 --> 00:03:06,060
yeah cool so yeah if you're if you're a

71
00:03:04,860 --> 00:03:07,860
developer and maintainer you've got skin

72
00:03:06,060 --> 00:03:08,819
in the game you're developing what

73
00:03:07,860 --> 00:03:10,580
you've

74
00:03:08,819 --> 00:03:13,440
you're maintaining what you've developed

75
00:03:10,580 --> 00:03:15,659
it comes down to doing all the right

76
00:03:13,440 --> 00:03:18,120
software craftsmanship things to extend

77
00:03:15,659 --> 00:03:20,280
out that time between production bugs so

78
00:03:18,120 --> 00:03:23,760
that's the developer side

79
00:03:20,280 --> 00:03:25,560
but on the maintenance side you also

80
00:03:23,760 --> 00:03:28,080
need a knuckle down and have some tools

81
00:03:25,560 --> 00:03:30,780
and techniques to keep the time taken to

82
00:03:28,080 --> 00:03:33,200
fix these production bugs as short as

83
00:03:30,780 --> 00:03:33,200
possible

84
00:03:33,840 --> 00:03:38,459
and let's not forget what the real goal

85
00:03:36,300 --> 00:03:40,319
is here it's all about us being able to

86
00:03:38,459 --> 00:03:43,700
sleep well at night

87
00:03:40,319 --> 00:03:43,700
and they'll get woken up

88
00:03:43,739 --> 00:03:49,200
so I'm sure most of you have seen this

89
00:03:46,560 --> 00:03:51,480
type of chart before it's normally used

90
00:03:49,200 --> 00:03:52,799
as a lesson to try and catch bugs as

91
00:03:51,480 --> 00:03:54,959
early as possible

92
00:03:52,799 --> 00:03:58,620
in the process because later and later

93
00:03:54,959 --> 00:04:00,720
it gets the more expensive it is to fix

94
00:03:58,620 --> 00:04:03,420
not just in engineering time of you but

95
00:04:00,720 --> 00:04:05,760
in costs to the End customer from the

96
00:04:03,420 --> 00:04:08,819
downtime and and the defects of their

97
00:04:05,760 --> 00:04:11,400
products or services

98
00:04:08,819 --> 00:04:13,739
so if you improve repairability what

99
00:04:11,400 --> 00:04:16,019
you're really doing is driving down the

100
00:04:13,739 --> 00:04:19,199
cost at the last stage where a bug is

101
00:04:16,019 --> 00:04:20,639
most costly there's other advantages for

102
00:04:19,199 --> 00:04:22,560
the other stages too

103
00:04:20,639 --> 00:04:27,259
but the main one where repair ability

104
00:04:22,560 --> 00:04:27,259
has impact is at that last stage there

105
00:04:28,500 --> 00:04:33,720
and if you if you break down what goes

106
00:04:30,900 --> 00:04:36,240
into fixing a production bug if you've

107
00:04:33,720 --> 00:04:38,460
first got to find the Clause so you're

108
00:04:36,240 --> 00:04:40,740
in what I call a plumber situation a

109
00:04:38,460 --> 00:04:42,479
plumber can't fix the leak let alone

110
00:04:40,740 --> 00:04:44,160
even estimate how long it's going to

111
00:04:42,479 --> 00:04:45,660
take unless they know exactly where the

112
00:04:44,160 --> 00:04:47,759
leak is

113
00:04:45,660 --> 00:04:50,580
so then you then you've got to apply the

114
00:04:47,759 --> 00:04:53,400
fix both in terms of changing your code

115
00:04:50,580 --> 00:04:54,540
and deploying it and it's that first

116
00:04:53,400 --> 00:04:56,880
part

117
00:04:54,540 --> 00:04:58,500
the time to find the cause that can be

118
00:04:56,880 --> 00:05:00,360
the really really hard part and that's

119
00:04:58,500 --> 00:05:03,380
where having Diagnostics

120
00:05:00,360 --> 00:05:03,380
helps the most

121
00:05:03,960 --> 00:05:08,520
so when we're talking about Diagnostics

122
00:05:05,759 --> 00:05:11,220
it's its definition is is kind of

123
00:05:08,520 --> 00:05:13,800
contained within the word itself it's a

124
00:05:11,220 --> 00:05:17,280
tool for diagnosing it's a tool to help

125
00:05:13,800 --> 00:05:20,759
you know the diagnosis it's for knowing

126
00:05:17,280 --> 00:05:22,800
knowing the cause of the problem

127
00:05:20,759 --> 00:05:25,080
or even finding out whether there is a

128
00:05:22,800 --> 00:05:27,419
problem or not with your code

129
00:05:25,080 --> 00:05:29,160
which is still really useful because so

130
00:05:27,419 --> 00:05:31,800
much of troubleshooting complicated

131
00:05:29,160 --> 00:05:34,080
systems with many moving parts

132
00:05:31,800 --> 00:05:36,120
is about eliminating those potential

133
00:05:34,080 --> 00:05:38,460
root causes you know Sherlock Holmes

134
00:05:36,120 --> 00:05:42,800
stuff until you arrive at the truth

135
00:05:38,460 --> 00:05:42,800
however improbable it is

136
00:05:46,400 --> 00:05:52,800
and how many times does it come up

137
00:05:49,740 --> 00:05:55,680
whether you know you or someone Upstream

138
00:05:52,800 --> 00:05:57,900
says oh it's working for me sorry can't

139
00:05:55,680 --> 00:06:00,900
replicate the problem

140
00:05:57,900 --> 00:06:02,699
what they really mean is they can't find

141
00:06:00,900 --> 00:06:04,800
the cause of the problem and if they

142
00:06:02,699 --> 00:06:06,360
can't find the cause of the problem then

143
00:06:04,800 --> 00:06:10,139
you're out of luck thinking they can

144
00:06:06,360 --> 00:06:14,940
even fix the problem sucks to be you

145
00:06:10,139 --> 00:06:17,600
now to me this indicates a failure

146
00:06:14,940 --> 00:06:17,600
yep

147
00:06:21,560 --> 00:06:26,840
I'm doing this tomorrow by the way I'm

148
00:06:24,060 --> 00:06:26,840
doing his job tomorrow

149
00:06:27,600 --> 00:06:33,600
yeah so to me that type of situation

150
00:06:30,900 --> 00:06:35,460
indicates a failure to have enough good

151
00:06:33,600 --> 00:06:37,620
diagnostic tools at hand to find the

152
00:06:35,460 --> 00:06:38,819
root cause when you're inspecting the

153
00:06:37,620 --> 00:06:40,620
so-called

154
00:06:38,819 --> 00:06:44,060
crime scene

155
00:06:40,620 --> 00:06:44,060
of a failed application

156
00:06:57,000 --> 00:07:01,100
all right let's go

157
00:07:00,300 --> 00:07:03,240
um

158
00:07:01,100 --> 00:07:05,699
yeah so you don't have the right

159
00:07:03,240 --> 00:07:08,160
diagnostic tools it's not so much to do

160
00:07:05,699 --> 00:07:10,680
with having perfect replicas

161
00:07:08,160 --> 00:07:13,139
of the real world installation setups

162
00:07:10,680 --> 00:07:15,479
lying around on your desk to probe in

163
00:07:13,139 --> 00:07:18,000
practice it's really hard to do that

164
00:07:15,479 --> 00:07:19,259
there's just so many possibilities

165
00:07:18,000 --> 00:07:22,380
in

166
00:07:19,259 --> 00:07:25,699
on site and although obviously that

167
00:07:22,380 --> 00:07:25,699
would help a lot too as well

168
00:07:25,979 --> 00:07:32,220
so without much data or tools if in that

169
00:07:29,819 --> 00:07:34,560
situation you're forced to resort to the

170
00:07:32,220 --> 00:07:37,319
software equivalent of physicist Richard

171
00:07:34,560 --> 00:07:39,060
feyman's favorite famous problem solving

172
00:07:37,319 --> 00:07:40,440
algorithm at least according to his

173
00:07:39,060 --> 00:07:43,139
colleagues

174
00:07:40,440 --> 00:07:45,240
just think really hard and then come up

175
00:07:43,139 --> 00:07:47,099
with a solution

176
00:07:45,240 --> 00:07:50,280
that's that's not a situation you want

177
00:07:47,099 --> 00:07:53,340
to be in when times are the essence

178
00:07:50,280 --> 00:07:55,199
I'm told also in Oppenheimer is

179
00:07:53,340 --> 00:07:57,539
basically relegated to a bongo drum

180
00:07:55,199 --> 00:08:00,720
player so I'm not happy not too happy

181
00:07:57,539 --> 00:08:01,440
with that Mr Nolan

182
00:08:00,720 --> 00:08:04,139
um

183
00:08:01,440 --> 00:08:06,599
so let's uh look at some of the

184
00:08:04,139 --> 00:08:09,539
attributes of software-based diagnostic

185
00:08:06,599 --> 00:08:11,520
tools before diving into some concrete

186
00:08:09,539 --> 00:08:13,319
examples

187
00:08:11,520 --> 00:08:15,360
the firstly

188
00:08:13,319 --> 00:08:17,759
it's got to be on demand that is we

189
00:08:15,360 --> 00:08:19,979
don't normally require the tool it's not

190
00:08:17,759 --> 00:08:22,139
part of the core intended function of

191
00:08:19,979 --> 00:08:24,479
the application everything works fine

192
00:08:22,139 --> 00:08:26,879
without it it's only glanced at when

193
00:08:24,479 --> 00:08:28,680
needed and ideally the data is only

194
00:08:26,879 --> 00:08:30,780
generated and displayed when it's needed

195
00:08:28,680 --> 00:08:32,039
as well making it really light on

196
00:08:30,780 --> 00:08:34,620
resources

197
00:08:32,039 --> 00:08:37,560
so being light on resources including

198
00:08:34,620 --> 00:08:39,360
cost is really important especially on

199
00:08:37,560 --> 00:08:42,120
constrained you know Internet of Things

200
00:08:39,360 --> 00:08:43,860
iot devices where there's thousands in

201
00:08:42,120 --> 00:08:47,240
the field

202
00:08:43,860 --> 00:08:47,240
and they're charging per device

203
00:08:47,640 --> 00:08:51,180
um

204
00:08:48,360 --> 00:08:53,760
it's permanently attached or probably

205
00:08:51,180 --> 00:08:55,320
better stated embedded so what I mean by

206
00:08:53,760 --> 00:08:57,060
that is it's part of your application

207
00:08:55,320 --> 00:08:59,279
the diagnostic tool is part of your

208
00:08:57,060 --> 00:09:02,160
application available for use at any

209
00:08:59,279 --> 00:09:03,899
time it's not some separate tool that

210
00:09:02,160 --> 00:09:05,100
you need to connect or spin up when the

211
00:09:03,899 --> 00:09:07,260
time comes

212
00:09:05,100 --> 00:09:11,399
and it's also part of your code too like

213
00:09:07,260 --> 00:09:13,680
like tests except yeah this this tool is

214
00:09:11,399 --> 00:09:15,240
designed to be run in production when

215
00:09:13,680 --> 00:09:16,980
you need it most

216
00:09:15,240 --> 00:09:19,380
and you don't really want it to be a

217
00:09:16,980 --> 00:09:21,660
separate tool because it's going to be

218
00:09:19,380 --> 00:09:24,000
just another tool in another repo you've

219
00:09:21,660 --> 00:09:25,560
got to maintain and potentially keep in

220
00:09:24,000 --> 00:09:27,540
sync with your app

221
00:09:25,560 --> 00:09:30,360
and if you have to stop the application

222
00:09:27,540 --> 00:09:32,160
attach the diagnostic tool then restart

223
00:09:30,360 --> 00:09:35,100
the application you're not not

224
00:09:32,160 --> 00:09:38,459
preserving that crime scene so to speak

225
00:09:35,100 --> 00:09:43,100
this this huge value and seeing Insider

226
00:09:38,459 --> 00:09:43,100
partially failed but running application

227
00:09:44,100 --> 00:09:48,480
I really hate that I've used this term

228
00:09:45,959 --> 00:09:51,240
but what we're talking about here is

229
00:09:48,480 --> 00:09:54,060
Diagnostics Being Human readable it

230
00:09:51,240 --> 00:09:56,399
means you can't just provide a Json dump

231
00:09:54,060 --> 00:09:59,160
of your App State

232
00:09:56,399 --> 00:10:01,800
accessible via some API and call it a

233
00:09:59,160 --> 00:10:05,339
day you can't just debug log everything

234
00:10:01,800 --> 00:10:07,920
at higher resolution and spew out pages

235
00:10:05,339 --> 00:10:09,540
and pages and logs and color today it's

236
00:10:07,920 --> 00:10:13,200
more about being infrared information

237
00:10:09,540 --> 00:10:15,839
Rich having having a high signal sort of

238
00:10:13,200 --> 00:10:16,800
noise ratio if if you want to call it

239
00:10:15,839 --> 00:10:18,360
that

240
00:10:16,800 --> 00:10:20,279
and I guess it implies that the

241
00:10:18,360 --> 00:10:22,380
Diagnostics you create are highly

242
00:10:20,279 --> 00:10:24,779
customized for your application to spit

243
00:10:22,380 --> 00:10:27,720
out exactly what you need to find

244
00:10:24,779 --> 00:10:28,860
to Define where the issue is and not

245
00:10:27,720 --> 00:10:31,500
much else

246
00:10:28,860 --> 00:10:33,180
otherwise it's going to take too long

247
00:10:31,500 --> 00:10:35,220
when you need it to track down the

248
00:10:33,180 --> 00:10:37,640
course

249
00:10:35,220 --> 00:10:39,779
and finally this one's about

250
00:10:37,640 --> 00:10:42,120
being able to see inside your

251
00:10:39,779 --> 00:10:44,760
application while it's running so all

252
00:10:42,120 --> 00:10:48,240
the necessary Nuts and Bolts

253
00:10:44,760 --> 00:10:51,060
the so-called System state the App State

254
00:10:48,240 --> 00:10:53,760
it's all in service to answer that basic

255
00:10:51,060 --> 00:10:55,740
question is it working and if not where

256
00:10:53,760 --> 00:10:59,399
has it failed so this is that preserving

257
00:10:55,740 --> 00:11:01,800
of the crime scene aspect I talked about

258
00:10:59,399 --> 00:11:04,320
and this attribute really determines

259
00:11:01,800 --> 00:11:08,220
whether the diagnostic tool is useful at

260
00:11:04,320 --> 00:11:10,440
a more fundamental level

261
00:11:08,220 --> 00:11:13,459
so let's look at some real world

262
00:11:10,440 --> 00:11:16,620
examples of Diagnostics

263
00:11:13,459 --> 00:11:19,500
here's your classic car dashboard car

264
00:11:16,620 --> 00:11:21,540
dashboard it's uh it's pretty good I'd

265
00:11:19,500 --> 00:11:23,060
say when your car acts up it tells you

266
00:11:21,540 --> 00:11:25,800
where to go and look

267
00:11:23,060 --> 00:11:28,560
it's information Rich lots of symbols

268
00:11:25,800 --> 00:11:31,680
and icons packed into a small area

269
00:11:28,560 --> 00:11:33,660
I just subtract points here for being on

270
00:11:31,680 --> 00:11:34,940
demand because you do kind of need to

271
00:11:33,660 --> 00:11:37,860
know the speed

272
00:11:34,940 --> 00:11:39,060
you're traveling to drive legally and

273
00:11:37,860 --> 00:11:42,000
you've got to know whether you've got

274
00:11:39,060 --> 00:11:43,800
enough petrol so that's it's still kind

275
00:11:42,000 --> 00:11:46,260
of still part of the core functionality

276
00:11:43,800 --> 00:11:49,200
of the car

277
00:11:46,260 --> 00:11:51,180
um here's a so-called bed of nails set

278
00:11:49,200 --> 00:11:52,740
up for testing voltages and signals at

279
00:11:51,180 --> 00:11:55,220
various points on a printed circuit

280
00:11:52,740 --> 00:11:58,100
board you can see the Pogo pins

281
00:11:55,220 --> 00:12:00,779
underneath probing the circuit

282
00:11:58,100 --> 00:12:03,839
traces of that upper board

283
00:12:00,779 --> 00:12:06,959
and uh yeah I I subtract points there

284
00:12:03,839 --> 00:12:08,519
because it's not embedded you've got to

285
00:12:06,959 --> 00:12:11,279
attach it

286
00:12:08,519 --> 00:12:13,620
um and unless it comes with a really

287
00:12:11,279 --> 00:12:15,959
intuitive UI all it's doing is spitting

288
00:12:13,620 --> 00:12:18,060
out raw data that would be hard to

289
00:12:15,959 --> 00:12:21,660
comprehend

290
00:12:18,060 --> 00:12:24,600
this is github's operational status page

291
00:12:21,660 --> 00:12:27,060
pretty pretty good I reckon you can

292
00:12:24,600 --> 00:12:29,399
clearly see where the issue lies if

293
00:12:27,060 --> 00:12:31,920
you're a customer it condenses a lot of

294
00:12:29,399 --> 00:12:33,899
services behind those little green dots

295
00:12:31,920 --> 00:12:35,880
which is also a little bit of a

296
00:12:33,899 --> 00:12:38,279
shortcoming I reckon

297
00:12:35,880 --> 00:12:39,720
and the incident history button is

298
00:12:38,279 --> 00:12:41,820
really useful for tracking down

299
00:12:39,720 --> 00:12:44,180
something you're waiting for resolution

300
00:12:41,820 --> 00:12:44,180
on

301
00:12:44,519 --> 00:12:49,440
now let's look at your tickle logging

302
00:12:47,040 --> 00:12:51,920
high-res logging it's certainly on

303
00:12:49,440 --> 00:12:55,139
demand and permanently attached

304
00:12:51,920 --> 00:12:57,540
but it's normally quite costly in terms

305
00:12:55,139 --> 00:13:00,120
of the time to trawl through the pages

306
00:12:57,540 --> 00:13:02,480
to trace an issue and if you're only

307
00:13:00,120 --> 00:13:06,660
logging errors so low res

308
00:13:02,480 --> 00:13:08,940
logging it's often too too terse to

309
00:13:06,660 --> 00:13:11,519
really know how your app's functioning

310
00:13:08,940 --> 00:13:13,700
at any point in time and you might not

311
00:13:11,519 --> 00:13:15,240
get that full

312
00:13:13,700 --> 00:13:20,040
breadcrumbs

313
00:13:15,240 --> 00:13:22,740
as Sentry calls it to track down

314
00:13:20,040 --> 00:13:25,940
all the steps that led up to that that

315
00:13:22,740 --> 00:13:25,940
unhandled exception

316
00:13:27,240 --> 00:13:32,399
um which which leads us to our standard

317
00:13:29,880 --> 00:13:34,079
python debugger again not firmly

318
00:13:32,399 --> 00:13:36,360
attached and

319
00:13:34,079 --> 00:13:38,940
way harder to use than logging otherwise

320
00:13:36,360 --> 00:13:40,380
everyone would use it all the time but

321
00:13:38,940 --> 00:13:43,079
does a really good job on telling you

322
00:13:40,380 --> 00:13:45,060
the internal State of Affairs so if this

323
00:13:43,079 --> 00:13:47,160
was somehow running all the time in our

324
00:13:45,060 --> 00:13:49,260
app with minimal overhead and customized

325
00:13:47,160 --> 00:13:51,899
just for the information we need

326
00:13:49,260 --> 00:13:55,139
presented in a really easy to digest way

327
00:13:51,899 --> 00:13:57,860
I reckon that would be amazing so let's

328
00:13:55,139 --> 00:13:57,860
keep that in mind

329
00:13:57,959 --> 00:14:02,459
now for Internet of Things industrial

330
00:14:01,079 --> 00:14:06,360
control

331
00:14:02,459 --> 00:14:08,660
even our connected universe

332
00:14:06,360 --> 00:14:11,660
okay

333
00:14:08,660 --> 00:14:14,519
that's where Diagnostics really shines

334
00:14:11,660 --> 00:14:17,399
because you don't have that comfortable

335
00:14:14,519 --> 00:14:18,839
environment of the cloud

336
00:14:17,399 --> 00:14:21,180
so your app could be in a factory

337
00:14:18,839 --> 00:14:23,639
controlling an important piece of

338
00:14:21,180 --> 00:14:25,260
equipment on the line could be on a farm

339
00:14:23,639 --> 00:14:27,660
in the middle of nowhere collating

340
00:14:25,260 --> 00:14:31,320
sensor data and using it to control

341
00:14:27,660 --> 00:14:34,019
Machinery at irrigation time

342
00:14:31,320 --> 00:14:36,300
it could be in someone's house and they

343
00:14:34,019 --> 00:14:38,820
certainly did not in consent to sending

344
00:14:36,300 --> 00:14:41,459
you logs and rightly so given all the

345
00:14:38,820 --> 00:14:43,920
privacy issues that come with that

346
00:14:41,459 --> 00:14:46,320
um and yeah it also could be located in

347
00:14:43,920 --> 00:14:49,079
the highly secure inaccessible private

348
00:14:46,320 --> 00:14:51,600
Network self-hosted by the customer for

349
00:14:49,079 --> 00:14:54,240
security reasons

350
00:14:51,600 --> 00:14:56,579
so in all those examples it's quite

351
00:14:54,240 --> 00:14:59,100
likely either you've got no direct comms

352
00:14:56,579 --> 00:15:00,000
or you've got very very limited cons if

353
00:14:59,100 --> 00:15:02,399
that

354
00:15:00,000 --> 00:15:04,680
your standard high-res heavyweight and

355
00:15:02,399 --> 00:15:08,639
costly cloud-based locking setup is just

356
00:15:04,680 --> 00:15:11,639
not an off option in a lot of cases

357
00:15:08,639 --> 00:15:13,800
so unlike in the cloud your application

358
00:15:11,639 --> 00:15:16,380
in those sorts of scenarios can't just

359
00:15:13,800 --> 00:15:18,360
be designed to exit on unhandled error

360
00:15:16,380 --> 00:15:20,639
and rely on your horizontal scaling

361
00:15:18,360 --> 00:15:22,860
behind a load balancer to pick up the

362
00:15:20,639 --> 00:15:24,540
workload of the falter unit there's no

363
00:15:22,860 --> 00:15:26,820
horizontal scaling

364
00:15:24,540 --> 00:15:28,260
there's no kubernetes there's no

365
00:15:26,820 --> 00:15:31,260
failover until there's a disaster

366
00:15:28,260 --> 00:15:33,660
recovery site there's only one unit

367
00:15:31,260 --> 00:15:35,399
and in this situation you're often

368
00:15:33,660 --> 00:15:37,620
likely not you're relying on the

369
00:15:35,399 --> 00:15:40,560
customer to tell you what's wrong

370
00:15:37,620 --> 00:15:42,000
but normally they aren't as tech savvy

371
00:15:40,560 --> 00:15:44,760
as you need them to be

372
00:15:42,000 --> 00:15:46,920
they're you know certainly highly

373
00:15:44,760 --> 00:15:48,540
motivated to help you but they're often

374
00:15:46,920 --> 00:15:50,339
unreliable Witnesses and a lot of what

375
00:15:48,540 --> 00:15:52,459
they observe end up just being red

376
00:15:50,339 --> 00:15:54,779
herrings for you

377
00:15:52,459 --> 00:15:57,060
and I'm sure you've been thinking this

378
00:15:54,779 --> 00:15:58,800
whole talk like why don't you just roll

379
00:15:57,060 --> 00:16:00,480
back to an earlier known good version

380
00:15:58,800 --> 00:16:02,880
when this happens

381
00:16:00,480 --> 00:16:04,680
but the problem is that they need this

382
00:16:02,880 --> 00:16:06,060
feature right now it's a new feature

383
00:16:04,680 --> 00:16:08,279
this whole they've bought this whole

384
00:16:06,060 --> 00:16:09,779
product on the basis of you having of

385
00:16:08,279 --> 00:16:12,180
having this feature and it's not working

386
00:16:09,779 --> 00:16:13,920
so you're going to get into situations

387
00:16:12,180 --> 00:16:16,399
where you just need to fix it you can't

388
00:16:13,920 --> 00:16:16,399
roll back

389
00:16:16,860 --> 00:16:22,500
side let's Now get stuck into how to

390
00:16:20,220 --> 00:16:24,180
embed Diagnostics into your python

391
00:16:22,500 --> 00:16:26,820
application there's there's many many

392
00:16:24,180 --> 00:16:29,779
fancy ways but I'm going to show you

393
00:16:26,820 --> 00:16:29,779
what works for me

394
00:16:30,360 --> 00:16:37,380
um I use a pretty simple

395
00:16:33,660 --> 00:16:39,540
UDP setup so the way it works is you

396
00:16:37,380 --> 00:16:42,180
connect your production app

397
00:16:39,540 --> 00:16:42,959
over UDP

398
00:16:42,180 --> 00:16:46,259
um

399
00:16:42,959 --> 00:16:49,440
via a client to ask it and you ask it to

400
00:16:46,259 --> 00:16:52,680
send you diagnostic information

401
00:16:49,440 --> 00:16:55,139
so there's no fancy web front-end no

402
00:16:52,680 --> 00:16:57,779
structured data all it is is the udb

403
00:16:55,139 --> 00:17:00,480
client run from the command line that

404
00:16:57,779 --> 00:17:03,779
sends a number a page number if you will

405
00:17:00,480 --> 00:17:06,900
to that production app and all that gets

406
00:17:03,779 --> 00:17:09,959
returned is a bunch of text

407
00:17:06,900 --> 00:17:13,199
that's all so there's no nested Pages no

408
00:17:09,959 --> 00:17:14,699
hyperlinks no charts no ability to send

409
00:17:13,199 --> 00:17:16,980
anything but a number and get nicely

410
00:17:14,699 --> 00:17:18,600
formed our text back

411
00:17:16,980 --> 00:17:21,360
the advantage of this is really simple

412
00:17:18,600 --> 00:17:24,480
it's easy to lock down or open up if you

413
00:17:21,360 --> 00:17:27,839
need to or UDP or you can turn it into a

414
00:17:24,480 --> 00:17:31,280
fancy UI if you need to later on

415
00:17:27,839 --> 00:17:31,280
the fundamentals are there

416
00:17:31,799 --> 00:17:35,880
um I'm not going to go through this

417
00:17:33,299 --> 00:17:37,500
whole thing but this is the client side

418
00:17:35,880 --> 00:17:40,380
so

419
00:17:37,500 --> 00:17:42,000
that's it in entirety it's basically 40

420
00:17:40,380 --> 00:17:44,039
lines of code

421
00:17:42,000 --> 00:17:46,280
lifted pretty much from the python docks

422
00:17:44,039 --> 00:17:49,980
with a bit of exception handling

423
00:17:46,280 --> 00:17:52,020
there's a section there at the top you

424
00:17:49,980 --> 00:17:54,260
you retrieve a number from the user

425
00:17:52,020 --> 00:17:56,880
somehow I'm just using input

426
00:17:54,260 --> 00:17:59,580
you send it across

427
00:17:56,880 --> 00:18:02,220
over UDP and then you wait for the

428
00:17:59,580 --> 00:18:05,220
response in this case block with no

429
00:18:02,220 --> 00:18:06,900
timeout again really simple and then

430
00:18:05,220 --> 00:18:11,539
just print out

431
00:18:06,900 --> 00:18:11,539
print out the message you get back

432
00:18:11,580 --> 00:18:17,880
and on the python application side so

433
00:18:14,520 --> 00:18:20,460
your Handler again all it is is a UDP

434
00:18:17,880 --> 00:18:21,980
server listening on a known diagnostic

435
00:18:20,460 --> 00:18:24,840
port

436
00:18:21,980 --> 00:18:27,140
again straight out of the straight out

437
00:18:24,840 --> 00:18:31,860
of the Python docks

438
00:18:27,140 --> 00:18:33,120
it reads that page number requested by

439
00:18:31,860 --> 00:18:35,940
the client

440
00:18:33,120 --> 00:18:39,179
and has sub handlers to process each

441
00:18:35,940 --> 00:18:41,700
specific page number request you could

442
00:18:39,179 --> 00:18:43,620
use async IO you could put it in a

443
00:18:41,700 --> 00:18:45,900
separate thread make it blocking and use

444
00:18:43,620 --> 00:18:48,179
this built-in socket server module which

445
00:18:45,900 --> 00:18:50,400
I have here

446
00:18:48,179 --> 00:18:52,620
um in many cases the handling part

447
00:18:50,400 --> 00:18:55,140
happens so rarely that it's just

448
00:18:52,620 --> 00:18:57,000
fetching pre-computed info

449
00:18:55,140 --> 00:18:58,740
that you can easily put it in the main

450
00:18:57,000 --> 00:19:00,960
thread of your app without impacting

451
00:18:58,740 --> 00:19:05,100
anything providing the listening

452
00:19:00,960 --> 00:19:07,760
component of the UDP Port is made to be

453
00:19:05,100 --> 00:19:07,760
non-blocking

454
00:19:08,820 --> 00:19:12,179
um

455
00:19:09,480 --> 00:19:14,460
but just as an aside I use a python

456
00:19:12,179 --> 00:19:16,740
package called blessed which I want to

457
00:19:14,460 --> 00:19:19,160
give a little bit of love to to make the

458
00:19:16,740 --> 00:19:22,799
text output all pretty

459
00:19:19,160 --> 00:19:24,480
in my Diagnostics so it just provides a

460
00:19:22,799 --> 00:19:27,360
nice interface for adding in those

461
00:19:24,480 --> 00:19:29,840
really complicated antsy color Escape

462
00:19:27,360 --> 00:19:29,840
codes

463
00:19:30,660 --> 00:19:36,179
and

464
00:19:32,280 --> 00:19:38,520
you can see in the bottom picture I'm

465
00:19:36,179 --> 00:19:40,860
basically printing it out

466
00:19:38,520 --> 00:19:42,600
from the terminal and it works exactly

467
00:19:40,860 --> 00:19:46,340
the same as if you're printing it out in

468
00:19:42,600 --> 00:19:46,340
Python it's all color formatted

469
00:19:48,360 --> 00:19:53,280
okay so the first thing you're going to

470
00:19:51,299 --> 00:19:55,200
want to do with Diagnostics is have a

471
00:19:53,280 --> 00:19:59,220
table of contents page at a known

472
00:19:55,200 --> 00:20:01,559
location so in this case 255

473
00:19:59,220 --> 00:20:04,020
you remember that promise with the rest

474
00:20:01,559 --> 00:20:06,179
you know the apis would be

475
00:20:04,020 --> 00:20:07,140
self-revealing self-documenting you

476
00:20:06,179 --> 00:20:09,480
started

477
00:20:07,140 --> 00:20:11,940
at initial end point and discover all

478
00:20:09,480 --> 00:20:14,220
the rest all that never happened but um

479
00:20:11,940 --> 00:20:17,720
table contents has been around for ages

480
00:20:14,220 --> 00:20:17,720
it achieves the same thing

481
00:20:17,940 --> 00:20:23,400
um and yeah you can easily

482
00:20:20,700 --> 00:20:26,460
use a human to human interface where you

483
00:20:23,400 --> 00:20:28,500
just shout I want diagnostic 230 to

484
00:20:26,460 --> 00:20:32,940
someone on a phone in the field and they

485
00:20:28,500 --> 00:20:36,740
go and get it copy it and paste it into

486
00:20:32,940 --> 00:20:36,740
a chat message or an email for you

487
00:20:38,100 --> 00:20:44,340
um so yeah the subtitle of my talk was

488
00:20:41,580 --> 00:20:46,559
you know doing it without logging but

489
00:20:44,340 --> 00:20:48,660
I've got an Asterix there so it's not

490
00:20:46,559 --> 00:20:52,500
technically lying

491
00:20:48,660 --> 00:20:54,840
um but it's still it's still really nice

492
00:20:52,500 --> 00:20:55,980
to have high-res debug logging to see if

493
00:20:54,840 --> 00:20:57,539
your app is

494
00:20:55,980 --> 00:21:00,179
ticking along

495
00:20:57,539 --> 00:21:02,160
but with Diagnostics

496
00:21:00,179 --> 00:21:04,919
good just to have a snapshot like last

497
00:21:02,160 --> 00:21:06,900
109 lines or so is all you need

498
00:21:04,919 --> 00:21:07,860
every time you hit that

499
00:21:06,900 --> 00:21:08,580
that

500
00:21:07,860 --> 00:21:11,280
um

501
00:21:08,580 --> 00:21:12,720
page 245 it just gets the last hundred

502
00:21:11,280 --> 00:21:16,559
lines

503
00:21:12,720 --> 00:21:19,679
and to achieve that you can create your

504
00:21:16,559 --> 00:21:21,660
own circular buffer logger Handler if

505
00:21:19,679 --> 00:21:23,520
you're using standard python or create

506
00:21:21,660 --> 00:21:26,520
your own logging classes if you're using

507
00:21:23,520 --> 00:21:29,400
something like micro python and all that

508
00:21:26,520 --> 00:21:32,280
would do is output to standard output

509
00:21:29,400 --> 00:21:35,480
but also at the same time storing in

510
00:21:32,280 --> 00:21:38,280
memory in a list that you can unroll

511
00:21:35,480 --> 00:21:40,940
when time comes to to view that

512
00:21:38,280 --> 00:21:40,940
diagnostic

513
00:21:42,000 --> 00:21:47,400
um but the quickest win you can have

514
00:21:44,780 --> 00:21:49,500
with Diagnostics is reporting on I've

515
00:21:47,400 --> 00:21:51,620
handled exceptions and crashes

516
00:21:49,500 --> 00:21:55,260
yes

517
00:21:51,620 --> 00:21:59,460
and you know even if you are

518
00:21:55,260 --> 00:22:02,280
trapping it and you know you don't care

519
00:21:59,460 --> 00:22:05,460
if it happens you still want to know

520
00:22:02,280 --> 00:22:08,940
about it because you might want to make

521
00:22:05,460 --> 00:22:11,880
a specific error handling for it

522
00:22:08,940 --> 00:22:14,700
so normally you want to write that to a

523
00:22:11,880 --> 00:22:18,559
file so it persists across uh reboots

524
00:22:14,700 --> 00:22:21,780
and to carefully limit that file in size

525
00:22:18,559 --> 00:22:24,960
via rotation or similar to prevent

526
00:22:21,780 --> 00:22:25,620
running out of disk space

527
00:22:24,960 --> 00:22:27,780
um

528
00:22:25,620 --> 00:22:31,380
now

529
00:22:27,780 --> 00:22:33,659
I use a decorator

530
00:22:31,380 --> 00:22:36,419
um so the left is the implementation

531
00:22:33,659 --> 00:22:39,900
it's it's one of those fancy ones that's

532
00:22:36,419 --> 00:22:40,860
an actual class that takes arguments

533
00:22:39,900 --> 00:22:43,380
um

534
00:22:40,860 --> 00:22:46,740
so the way you use it is you just

535
00:22:43,380 --> 00:22:49,140
whatever function you're using you just

536
00:22:46,740 --> 00:22:52,679
decorate it saying you want to consume

537
00:22:49,140 --> 00:22:54,740
the unhandled error and it will still

538
00:22:52,679 --> 00:22:59,240
report on that

539
00:22:54,740 --> 00:22:59,240
unhandled error to your Diagnostics

540
00:23:02,760 --> 00:23:08,340
with getting uh crash data that's also

541
00:23:05,820 --> 00:23:11,400
really really useful you've got a few

542
00:23:08,340 --> 00:23:13,740
options not so elegant you can take a

543
00:23:11,400 --> 00:23:16,020
look at sys dot accept hook in the

544
00:23:13,740 --> 00:23:18,840
python docs you basically just register

545
00:23:16,020 --> 00:23:21,120
a function to use to run when any

546
00:23:18,840 --> 00:23:23,460
exception is hit including unhandled

547
00:23:21,120 --> 00:23:26,640
ones I don't really use it too much

548
00:23:23,460 --> 00:23:28,080
myself you can trap signals but you've

549
00:23:26,640 --> 00:23:29,580
got to you've got to be really careful

550
00:23:28,080 --> 00:23:33,140
with signals with

551
00:23:29,580 --> 00:23:33,140
not doing too much

552
00:23:33,480 --> 00:23:36,480
and

553
00:23:36,720 --> 00:23:41,460
third option is the one I tend to use is

554
00:23:39,480 --> 00:23:45,240
making sure every function call is

555
00:23:41,460 --> 00:23:49,500
within some sort of try accept

556
00:23:45,240 --> 00:23:52,500
ion uh Clause so it's catching

557
00:23:49,500 --> 00:23:55,980
all the unhandled exceptions as well and

558
00:23:52,500 --> 00:23:58,159
probably re-raising them

559
00:23:55,980 --> 00:23:58,159
um

560
00:23:58,320 --> 00:24:03,840
so in my crash file I'll go and dump the

561
00:24:01,679 --> 00:24:06,179
entire contents of that circular buffer

562
00:24:03,840 --> 00:24:08,100
to provide at least 100 lines of context

563
00:24:06,179 --> 00:24:11,100
leading up to that crash

564
00:24:08,100 --> 00:24:13,080
and then yeah using using these these

565
00:24:11,100 --> 00:24:17,059
little tricks to to get it into that

566
00:24:13,080 --> 00:24:17,059
file before your application exits

567
00:24:18,480 --> 00:24:23,580
um so another really good use for

568
00:24:21,419 --> 00:24:26,039
Diagnostics is if you've got some

569
00:24:23,580 --> 00:24:28,980
complicated state or calculations you

570
00:24:26,039 --> 00:24:30,840
can put it in an easy to digest format

571
00:24:28,980 --> 00:24:33,780
including the results of all the

572
00:24:30,840 --> 00:24:36,059
intermediate computations that went into

573
00:24:33,780 --> 00:24:38,340
to generating the output

574
00:24:36,059 --> 00:24:40,679
so it's kind of like a spreadsheet you

575
00:24:38,340 --> 00:24:43,260
know that that shows all the steps in a

576
00:24:40,679 --> 00:24:44,940
really Visual and intuitive way

577
00:24:43,260 --> 00:24:47,700
so this kind of information would never

578
00:24:44,940 --> 00:24:50,400
be available in a black box you know

579
00:24:47,700 --> 00:24:52,559
product to the user and it really allows

580
00:24:50,400 --> 00:24:55,320
you to potentially track down some some

581
00:24:52,559 --> 00:24:56,340
pretty nasty edge cases while they're

582
00:24:55,320 --> 00:24:59,179
happening

583
00:24:56,340 --> 00:24:59,179
the crime scene

584
00:24:59,940 --> 00:25:04,679
here you can see I'm showing on the left

585
00:25:02,039 --> 00:25:06,539
the evolution of each variable that went

586
00:25:04,679 --> 00:25:09,360
into a calculation and how it was

587
00:25:06,539 --> 00:25:11,460
adjusted before it got used so that's

588
00:25:09,360 --> 00:25:13,799
that's very valuable so why does it have

589
00:25:11,460 --> 00:25:14,460
that value I don't understand

590
00:25:13,799 --> 00:25:16,020
um

591
00:25:14,460 --> 00:25:18,179
and yeah it's part of any

592
00:25:16,020 --> 00:25:20,400
troubleshooting exercise particularly in

593
00:25:18,179 --> 00:25:22,260
the field your technicians are going to

594
00:25:20,400 --> 00:25:25,020
want to know if the sensors are working

595
00:25:22,260 --> 00:25:27,140
if there's Communications back to base

596
00:25:25,020 --> 00:25:27,140
Etc

597
00:25:28,440 --> 00:25:33,659
um I'm showing some really simple

598
00:25:30,380 --> 00:25:36,600
diagnostic page handlers this one here

599
00:25:33,659 --> 00:25:38,760
you can print out the contents of a

600
00:25:36,600 --> 00:25:40,860
config file that users wouldn't

601
00:25:38,760 --> 00:25:43,440
ordinarily see but still still very

602
00:25:40,860 --> 00:25:45,500
useful to see what's happening

603
00:25:43,440 --> 00:25:45,500
um

604
00:25:46,200 --> 00:25:51,480
it's useful to know what's on disk even

605
00:25:49,799 --> 00:25:54,539
how much space is left even though I

606
00:25:51,480 --> 00:25:57,960
didn't show that there and you can

607
00:25:54,539 --> 00:26:03,559
you can run you know system CTL if

608
00:25:57,960 --> 00:26:03,559
you're using system D to oh or traversal

609
00:26:04,320 --> 00:26:11,400
um yeah to view app status as well and

610
00:26:08,100 --> 00:26:13,919
yeah again you're condensing down that

611
00:26:11,400 --> 00:26:16,440
information just to the the bare minimum

612
00:26:13,919 --> 00:26:19,140
you need to provide that value during

613
00:26:16,440 --> 00:26:23,520
fault finding

614
00:26:19,140 --> 00:26:25,799
uh more app meter data knowing when

615
00:26:23,520 --> 00:26:27,559
your application has started and stopped

616
00:26:25,799 --> 00:26:31,020
and why

617
00:26:27,559 --> 00:26:32,880
it may not be possible to work out you

618
00:26:31,020 --> 00:26:35,520
know when it stopped you were to reboot

619
00:26:32,880 --> 00:26:36,600
but I've put it there

620
00:26:35,520 --> 00:26:39,600
um

621
00:26:36,600 --> 00:26:41,520
and yeah on the right hand side you do

622
00:26:39,600 --> 00:26:43,740
want to know all that nice stuff about

623
00:26:41,520 --> 00:26:45,840
the commit ID and the build number so

624
00:26:43,740 --> 00:26:48,299
you can check out the right version of

625
00:26:45,840 --> 00:26:51,480
code to see if you can

626
00:26:48,299 --> 00:26:54,000
find the issue by inspecting your code

627
00:26:51,480 --> 00:26:55,620
and if if you want to

628
00:26:54,000 --> 00:26:58,159
if you want to Branch off from that with

629
00:26:55,620 --> 00:26:58,159
a hotfix

630
00:26:58,380 --> 00:27:04,980
finally I'll admit it's a bit of an

631
00:27:01,620 --> 00:27:08,640
abuse of the the page number system to

632
00:27:04,980 --> 00:27:11,700
get it to actually do stuff but you may

633
00:27:08,640 --> 00:27:13,380
want to consider providing common Quick

634
00:27:11,700 --> 00:27:15,960
Fix operations

635
00:27:13,380 --> 00:27:18,059
to users and to field technicians

636
00:27:15,960 --> 00:27:22,279
directly through this diagnostic

637
00:27:18,059 --> 00:27:25,880
interface so stuff like reboot rebooting

638
00:27:22,279 --> 00:27:29,400
restarting Services clearing out files

639
00:27:25,880 --> 00:27:32,000
that sort of thing and yeah when when

640
00:27:29,400 --> 00:27:35,940
you do Implement a reboot

641
00:27:32,000 --> 00:27:38,279
the device diagnostic you want to report

642
00:27:35,940 --> 00:27:39,480
that you're going to reboot wait a

643
00:27:38,279 --> 00:27:43,080
little bit

644
00:27:39,480 --> 00:27:45,720
there that's the micro python example

645
00:27:43,080 --> 00:27:48,120
um and uh and so they get the message

646
00:27:45,720 --> 00:27:50,760
and then reboot afterwards otherwise

647
00:27:48,120 --> 00:27:52,620
it's just going to hang

648
00:27:50,760 --> 00:27:54,539
um yeah so that's that's about all I had

649
00:27:52,620 --> 00:27:57,179
to speak about

650
00:27:54,539 --> 00:27:59,340
um as I hope you can appreciate I mean

651
00:27:57,179 --> 00:28:01,200
embedded Diagnostics they're really

652
00:27:59,340 --> 00:28:05,059
powerful and they're you know worth

653
00:28:01,200 --> 00:28:05,059
checking out so give it a try

654
00:28:08,240 --> 00:28:15,720
[Applause]

655
00:28:12,779 --> 00:28:18,659
thank you Jeremy we have time for a

656
00:28:15,720 --> 00:28:21,740
question or two uh so signal Jack who

657
00:28:18,659 --> 00:28:21,740
can see past the lights

658
00:28:21,900 --> 00:28:24,919
and make him run

659
00:28:25,970 --> 00:28:31,080
[Music]

660
00:28:28,740 --> 00:28:33,240
um yeah so I had a question about um and

661
00:28:31,080 --> 00:28:36,240
maybe this is like a detail or something

662
00:28:33,240 --> 00:28:37,740
but I wanted to know more about the page

663
00:28:36,240 --> 00:28:39,779
number system and if there's an

664
00:28:37,740 --> 00:28:42,020
advantage to doing it with page numbers

665
00:28:39,779 --> 00:28:46,620
as opposed to just like topic numbers

666
00:28:42,020 --> 00:28:50,820
yeah you could use you know words or

667
00:28:46,620 --> 00:28:51,539
mnemonics to describe what you want

668
00:28:50,820 --> 00:28:54,000
um

669
00:28:51,539 --> 00:28:55,559
I've I've taken this approach from

670
00:28:54,000 --> 00:28:59,760
what's used

671
00:28:55,559 --> 00:29:01,740
in Industrial Field devices they tend to

672
00:28:59,760 --> 00:29:04,620
use this type of system and

673
00:29:01,740 --> 00:29:07,500
if it ain't broke don't fix it but there

674
00:29:04,620 --> 00:29:11,760
is also text-based

675
00:29:07,500 --> 00:29:15,080
um yeah uh lookups as well yeah you

676
00:29:11,760 --> 00:29:15,080
could there's there's no issue

677
00:29:16,440 --> 00:29:21,899
hello I got really excited when I you

678
00:29:18,899 --> 00:29:23,039
brought up the the pdb debugger and you

679
00:29:21,899 --> 00:29:25,200
said I wouldn't be nice if we could have

680
00:29:23,039 --> 00:29:26,940
that all the time sorry which what were

681
00:29:25,200 --> 00:29:28,140
you talking about oh the the python

682
00:29:26,940 --> 00:29:29,399
debugger and you said wouldn't it be

683
00:29:28,140 --> 00:29:31,620
nice if we could have the python

684
00:29:29,399 --> 00:29:33,480
debugger all the time yeah and I thought

685
00:29:31,620 --> 00:29:35,940
oh yeah that I would love that too and

686
00:29:33,480 --> 00:29:38,159
it reminds me of attaching a remote

687
00:29:35,940 --> 00:29:40,440
Rebel enclosure and I saw your python

688
00:29:38,159 --> 00:29:43,320
Ripple there as well yeah

689
00:29:40,440 --> 00:29:45,659
do you know of a way we can just do that

690
00:29:43,320 --> 00:29:47,159
can we get a remote Rebel in Python if

691
00:29:45,659 --> 00:29:49,500
you've seen something like that that we

692
00:29:47,159 --> 00:29:52,559
could do and then we could find anything

693
00:29:49,500 --> 00:29:53,220
whether we'd instrumented it or not

694
00:29:52,559 --> 00:29:55,460
um

695
00:29:53,220 --> 00:30:00,360
apart from just running GDP and

696
00:29:55,460 --> 00:30:01,020
production I don't really know the way

697
00:30:00,360 --> 00:30:02,340
um

698
00:30:01,020 --> 00:30:04,559
yeah

699
00:30:02,340 --> 00:30:06,179
it's it's a tough one no thank you I

700
00:30:04,559 --> 00:30:08,520
just I feel like performance here as

701
00:30:06,179 --> 00:30:09,899
well obviously I should because I should

702
00:30:08,520 --> 00:30:11,580
have I should have instrumented the

703
00:30:09,899 --> 00:30:12,919
commit hash but I forgot if only I'd

704
00:30:11,580 --> 00:30:15,899
remember to instrument the camera

705
00:30:12,919 --> 00:30:18,600
there's something what you can do is run

706
00:30:15,899 --> 00:30:21,539
GDP over a network connection

707
00:30:18,600 --> 00:30:23,399
so I've seen that being done okay as

708
00:30:21,539 --> 00:30:25,140
well which is quite interesting so cool

709
00:30:23,399 --> 00:30:27,679
take a look at that it's a bit more

710
00:30:25,140 --> 00:30:27,679
lightweight

711
00:30:30,539 --> 00:30:34,020
I I know a way this isn't a question

712
00:30:33,000 --> 00:30:34,740
okay

713
00:30:34,020 --> 00:30:37,200
um

714
00:30:34,740 --> 00:30:40,320
the Twisted framework has a module

715
00:30:37,200 --> 00:30:42,240
called Twisted manhole which uh is I

716
00:30:40,320 --> 00:30:44,700
think it lets you tone it into your

717
00:30:42,240 --> 00:30:46,860
process or SSH or actually like SSH into

718
00:30:44,700 --> 00:30:49,080
your process and from there get a repo

719
00:30:46,860 --> 00:30:51,360
and do whatever you want

720
00:30:49,080 --> 00:30:53,640
um I remember a presentation given by

721
00:30:51,360 --> 00:30:55,799
someone at the very first kiwi pycon in

722
00:30:53,640 --> 00:30:58,140
2009 on the subject

723
00:30:55,799 --> 00:30:59,460
so I'm not too sure how up to date that

724
00:30:58,140 --> 00:31:01,620
is

725
00:30:59,460 --> 00:31:03,000
um but yes there's there's a simple

726
00:31:01,620 --> 00:31:04,679
library that you can add to your process

727
00:31:03,000 --> 00:31:06,120
which will just let you SSH into your

728
00:31:04,679 --> 00:31:07,559
process and then do whatever you want

729
00:31:06,120 --> 00:31:09,980
yep

730
00:31:07,559 --> 00:31:12,539
yeah not sure I recommend it

731
00:31:09,980 --> 00:31:14,880
yeah I don't know if you want to be

732
00:31:12,539 --> 00:31:16,740
modifying variables of a production

733
00:31:14,880 --> 00:31:19,080
application that's controlling a you

734
00:31:16,740 --> 00:31:21,539
know really expensive machine

735
00:31:19,080 --> 00:31:25,080
while it's running yeah

736
00:31:21,539 --> 00:31:28,740
well thank you everyone uh

737
00:31:25,080 --> 00:31:32,100
sorry oh wait track

738
00:31:28,740 --> 00:31:33,980
thank you Jeremy please enjoy your mug

739
00:31:32,100 --> 00:31:37,159
thank you very much

740
00:31:33,980 --> 00:31:37,159
thank you

741
00:31:38,970 --> 00:31:42,009
[Applause]