1
00:00:13,120 --> 00:00:16,160
hello welcome back

2
00:00:15,200 --> 00:00:18,720
opening

3
00:00:16,160 --> 00:00:19,760
opening data science and analytics track

4
00:00:18,720 --> 00:00:21,680
this morning

5
00:00:19,760 --> 00:00:23,680
laura summers who is a

6
00:00:21,680 --> 00:00:27,119
multi-disciplinary designer researching

7
00:00:23,680 --> 00:00:28,480
technology ethics and technology ethics

8
00:00:27,119 --> 00:00:30,800
and building tools to promote fair

9
00:00:28,480 --> 00:00:32,719
machine learning laura is passionate

10
00:00:30,800 --> 00:00:34,800
about feminism digital rights and

11
00:00:32,719 --> 00:00:35,920
designing for privacy which feels like

12
00:00:34,800 --> 00:00:38,000
something that should be kind of

13
00:00:35,920 --> 00:00:40,079
baseline for anyone practicing machine

14
00:00:38,000 --> 00:00:41,920
learning and artificial intelligence but

15
00:00:40,079 --> 00:00:44,719
unfortunately isn't

16
00:00:41,920 --> 00:00:47,200
laura is the founder of dubai sai and

17
00:00:44,719 --> 00:00:49,600
also the human beside behind open source

18
00:00:47,200 --> 00:00:52,000
and community projects like the ethics

19
00:00:49,600 --> 00:00:54,160
limits test fair archive and the

20
00:00:52,000 --> 00:00:55,760
melbourne fair machine learning reading

21
00:00:54,160 --> 00:00:57,760
group

22
00:00:55,760 --> 00:01:00,160
laura can be found on twitter her handle

23
00:00:57,760 --> 00:01:03,039
is at summerscope and she can also be

24
00:01:00,160 --> 00:01:04,960
found on the venuless platform where she

25
00:01:03,039 --> 00:01:07,360
will be answering your questions during

26
00:01:04,960 --> 00:01:09,439
her talk which has been pre-recorded she

27
00:01:07,360 --> 00:01:11,040
will also be available for questions at

28
00:01:09,439 --> 00:01:12,560
the end so please

29
00:01:11,040 --> 00:01:15,680
ask your questions and some of them will

30
00:01:12,560 --> 00:01:17,759
be selected for a live answer

31
00:01:15,680 --> 00:01:20,080
so laura starts with important questions

32
00:01:17,759 --> 00:01:22,159
about ethics but also about practicality

33
00:01:20,080 --> 00:01:24,000
because we all want to be ethical or at

34
00:01:22,159 --> 00:01:26,479
least we all say we don't want to be

35
00:01:24,000 --> 00:01:29,360
ethical but how

36
00:01:26,479 --> 00:01:32,479
so drawing on her experience as the

37
00:01:29,360 --> 00:01:34,479
ethics ops consultant embedded with a

38
00:01:32,479 --> 00:01:35,600
small fraud detection team in a big

39
00:01:34,479 --> 00:01:38,400
telco

40
00:01:35,600 --> 00:01:39,119
laura gives us tips on how to move past

41
00:01:38,400 --> 00:01:41,119
the

42
00:01:39,119 --> 00:01:43,200
ai ethics hype

43
00:01:41,119 --> 00:01:45,920
and how we can start trying testing and

44
00:01:43,200 --> 00:01:47,920
implementing practical approaches

45
00:01:45,920 --> 00:01:51,040
so please give a virtual hand of

46
00:01:47,920 --> 00:01:53,119
applause for our virtual keynote speaker

47
00:01:51,040 --> 00:01:57,680
laura summers and her talk

48
00:01:53,119 --> 00:01:57,680
game the ethics of telco fraud

49
00:01:59,920 --> 00:02:03,040
okay let's get started

50
00:02:03,680 --> 00:02:07,759
i've put together a google docs um

51
00:02:05,840 --> 00:02:09,280
document online you can go to it here at

52
00:02:07,759 --> 00:02:11,680
this tiny url

53
00:02:09,280 --> 00:02:14,480
you can see the links to the slides

54
00:02:11,680 --> 00:02:16,959
extra resources and also any q a you

55
00:02:14,480 --> 00:02:19,120
want to pop in there go for it

56
00:02:16,959 --> 00:02:20,879
so just to quickly point to what i'm

57
00:02:19,120 --> 00:02:22,800
going to discuss today

58
00:02:20,879 --> 00:02:24,239
the lineup is we start with deconstruct

59
00:02:22,800 --> 00:02:27,520
your proxy

60
00:02:24,239 --> 00:02:29,120
we then go to explicit data design

61
00:02:27,520 --> 00:02:31,440
assuming good faith

62
00:02:29,120 --> 00:02:33,040
and failure first design and hopefully

63
00:02:31,440 --> 00:02:35,360
all those will make sense by the time we

64
00:02:33,040 --> 00:02:37,040
get to the end of the talk

65
00:02:35,360 --> 00:02:40,080
so let me set the scene

66
00:02:37,040 --> 00:02:42,560
starting in early 2020 i was embedded in

67
00:02:40,080 --> 00:02:44,800
a small fraud detection team as an

68
00:02:42,560 --> 00:02:47,120
ethics ops consultant

69
00:02:44,800 --> 00:02:48,720
and as part of a larger telco

70
00:02:47,120 --> 00:02:50,800
it was really a pretty small team there

71
00:02:48,720 --> 00:02:53,200
was one data scientist one project

72
00:02:50,800 --> 00:02:55,200
manager one ops person who was like

73
00:02:53,200 --> 00:02:57,440
communicating with customers and

74
00:02:55,200 --> 00:02:59,280
researching stuff with us

75
00:02:57,440 --> 00:03:01,200
we sort of had half a data engineer and

76
00:02:59,280 --> 00:03:03,760
a few other devs kind of floating in and

77
00:03:01,200 --> 00:03:05,360
out a bit but mostly that was the team

78
00:03:03,760 --> 00:03:07,680
we were using a combination of internal

79
00:03:05,360 --> 00:03:09,519
tools and detection mechanisms and a

80
00:03:07,680 --> 00:03:11,760
third-party tool that was doing payment

81
00:03:09,519 --> 00:03:13,519
gateway fraud detection um and that was

82
00:03:11,760 --> 00:03:15,680
using ml but none of our internal stuff

83
00:03:13,519 --> 00:03:17,360
was like ml it was all kind of pretty

84
00:03:15,680 --> 00:03:19,200
straightforward deterministic and

85
00:03:17,360 --> 00:03:21,760
business logic type stuff

86
00:03:19,200 --> 00:03:23,120
um i can't share specifics on the

87
00:03:21,760 --> 00:03:24,640
features or the thresholds we were

88
00:03:23,120 --> 00:03:26,959
working with but um you'll get an idea

89
00:03:24,640 --> 00:03:28,799
through the talk um and we were looking

90
00:03:26,959 --> 00:03:30,879
primarily for a kind of fraud called

91
00:03:28,799 --> 00:03:33,840
international revenue sharing which is

92
00:03:30,879 --> 00:03:36,319
essentially jacking up the price of the

93
00:03:33,840 --> 00:03:38,640
call of a call for an international call

94
00:03:36,319 --> 00:03:40,480
um by sending it to a mobile or sending

95
00:03:38,640 --> 00:03:43,360
it to a premium number where you can

96
00:03:40,480 --> 00:03:45,680
charge a higher rate and then sort of

97
00:03:43,360 --> 00:03:47,840
taking a margin taking that higher rate

98
00:03:45,680 --> 00:03:49,599
um slicing it off the top and um

99
00:03:47,840 --> 00:03:51,599
pocketing it as

100
00:03:49,599 --> 00:03:53,200
as revenue essentially

101
00:03:51,599 --> 00:03:54,720
so instead of you actually getting a

102
00:03:53,200 --> 00:03:56,879
service or talking to someone they're

103
00:03:54,720 --> 00:03:58,480
just making those calls and letting them

104
00:03:56,879 --> 00:04:00,560
sit there in a crew minutes and then

105
00:03:58,480 --> 00:04:03,200
charging them back to the telco

106
00:04:00,560 --> 00:04:04,879
but we also had other kinds of um

107
00:04:03,200 --> 00:04:07,680
issues we were dealing with like um

108
00:04:04,879 --> 00:04:08,959
service overuse and promotions abuse or

109
00:04:07,680 --> 00:04:10,480
people were sort of pushing the

110
00:04:08,959 --> 00:04:11,840
friendship or trying to use like

111
00:04:10,480 --> 00:04:14,080
promotions more than they were

112
00:04:11,840 --> 00:04:16,799
technically allowed to um so you know

113
00:04:14,080 --> 00:04:18,959
there's quite a quite a mixed scene um

114
00:04:16,799 --> 00:04:20,479
yeah and if you don't know what ethics

115
00:04:18,959 --> 00:04:22,079
ops is or if you haven't heard that term

116
00:04:20,479 --> 00:04:24,479
before it's um something i've heard

117
00:04:22,079 --> 00:04:27,120
around the space it's it's i think of it

118
00:04:24,479 --> 00:04:29,600
sort of as like devops or sec ops but

119
00:04:27,120 --> 00:04:31,360
for ethics with the vu to try and

120
00:04:29,600 --> 00:04:34,160
improve process

121
00:04:31,360 --> 00:04:35,919
and give ourselves tools and deployment

122
00:04:34,160 --> 00:04:38,000
sort of monitoring

123
00:04:35,919 --> 00:04:41,040
approaches that will help us get better

124
00:04:38,000 --> 00:04:42,960
at improving the ethics of our system

125
00:04:41,040 --> 00:04:44,639
another metaphor i like to use is with

126
00:04:42,960 --> 00:04:46,960
sre is like you thinking about this

127
00:04:44,639 --> 00:04:48,880
question of how good is your uptime and

128
00:04:46,960 --> 00:04:50,400
you know that you never get to 100 but

129
00:04:48,880 --> 00:04:53,199
you're sort of aiming to get better all

130
00:04:50,400 --> 00:04:54,560
the time similarly with um ethics ops

131
00:04:53,199 --> 00:04:56,320
you don't assume that it's going to be

132
00:04:54,560 --> 00:04:58,320
done and you're never going to have 100

133
00:04:56,320 --> 00:05:00,000
perfect ethics but rather you're trying

134
00:04:58,320 --> 00:05:02,479
to observe the state of the world now

135
00:05:00,000 --> 00:05:04,400
and get to better

136
00:05:02,479 --> 00:05:07,280
so yeah fraud is an interesting space in

137
00:05:04,400 --> 00:05:09,199
which to discuss ethics not only are you

138
00:05:07,280 --> 00:05:11,919
the company sort of capable of

139
00:05:09,199 --> 00:05:13,840
inflicting harm on your customers you're

140
00:05:11,919 --> 00:05:16,160
also under attack by a subset of

141
00:05:13,840 --> 00:05:17,919
customers who mean you harm so you sort

142
00:05:16,160 --> 00:05:20,479
of have to balance the real and often

143
00:05:17,919 --> 00:05:23,280
urgent costs of identifying bad behavior

144
00:05:20,479 --> 00:05:25,280
and sort of mitigating those costs that

145
00:05:23,280 --> 00:05:27,039
they're they're charging to you while

146
00:05:25,280 --> 00:05:29,199
also acknowledging the possibility of

147
00:05:27,039 --> 00:05:31,199
false positives and considering how you

148
00:05:29,199 --> 00:05:33,759
might identify and support people who

149
00:05:31,199 --> 00:05:35,759
get caught in the crossfire

150
00:05:33,759 --> 00:05:36,720
and i think an important way to do that

151
00:05:35,759 --> 00:05:38,880
is to

152
00:05:36,720 --> 00:05:40,960
develop this gut feeling this sense

153
00:05:38,880 --> 00:05:43,280
track that we mostly i think already

154
00:05:40,960 --> 00:05:45,680
have to be honest which is keeping that

155
00:05:43,280 --> 00:05:47,280
healthy cynicism about our data sources

156
00:05:45,680 --> 00:05:48,800
um and certainly in the team i was

157
00:05:47,280 --> 00:05:50,400
embedded in like the thing i observed

158
00:05:48,800 --> 00:05:52,240
was it was only when we felt really

159
00:05:50,400 --> 00:05:53,600
comfortable and confident in what we

160
00:05:52,240 --> 00:05:54,639
thought was going on that something

161
00:05:53,600 --> 00:05:56,319
would

162
00:05:54,639 --> 00:05:58,960
be really surprising and like kind of

163
00:05:56,319 --> 00:06:01,360
turn us on our heads

164
00:05:58,960 --> 00:06:02,400
so yeah this idea that like we have to

165
00:06:01,360 --> 00:06:05,360
always

166
00:06:02,400 --> 00:06:07,600
be sensitive to the distance between

167
00:06:05,360 --> 00:06:10,080
what it is we can observe and how we

168
00:06:07,600 --> 00:06:12,000
draw causality to events in the world or

169
00:06:10,080 --> 00:06:14,000
behavior in the world or why people are

170
00:06:12,000 --> 00:06:15,360
doing things even is something i think

171
00:06:14,000 --> 00:06:18,000
is important to keep in the back of our

172
00:06:15,360 --> 00:06:20,000
heads at all times um and a good example

173
00:06:18,000 --> 00:06:21,919
of this was we were observing a

174
00:06:20,000 --> 00:06:24,479
particular customer who was sending a

175
00:06:21,919 --> 00:06:26,080
whole bunch of sms's overseas um and it

176
00:06:24,479 --> 00:06:28,240
was it was a lot it was a very high

177
00:06:26,080 --> 00:06:29,600
volume and we were all kind of pretty

178
00:06:28,240 --> 00:06:32,160
confident that you couldn't be sending

179
00:06:29,600 --> 00:06:33,840
this many sms's just like manually so we

180
00:06:32,160 --> 00:06:36,319
thought maybe there was some kind of

181
00:06:33,840 --> 00:06:38,639
script or bot or automation platform

182
00:06:36,319 --> 00:06:40,160
that they were using to broadcast sms's

183
00:06:38,639 --> 00:06:42,000
um and sort of the

184
00:06:40,160 --> 00:06:43,680
sort of extension of that was we thought

185
00:06:42,000 --> 00:06:45,520
probably they're doing some kind of

186
00:06:43,680 --> 00:06:47,680
marketing push so maybe it's you know

187
00:06:45,520 --> 00:06:49,680
spam or maybe they're trying to

188
00:06:47,680 --> 00:06:51,039
get visibility somewhere but you know we

189
00:06:49,680 --> 00:06:53,360
thought it was basically like them

190
00:06:51,039 --> 00:06:56,160
trying to use our our personal services

191
00:06:53,360 --> 00:06:56,960
essentially a marketing platform

192
00:06:56,160 --> 00:06:58,240
um

193
00:06:56,960 --> 00:07:00,319
and that really

194
00:06:58,240 --> 00:07:02,960
that really seemed obvious but then when

195
00:07:00,319 --> 00:07:05,520
we called this person up and had a chat

196
00:07:02,960 --> 00:07:07,039
with them we found out that actually

197
00:07:05,520 --> 00:07:09,360
they were sitting there with their

198
00:07:07,039 --> 00:07:11,360
thumbs messaging just heaps of people

199
00:07:09,360 --> 00:07:13,440
they had a ton of group messages set up

200
00:07:11,360 --> 00:07:14,960
and it was essentially um you know a

201
00:07:13,440 --> 00:07:17,280
communication

202
00:07:14,960 --> 00:07:18,880
uh communication and talking to people

203
00:07:17,280 --> 00:07:21,599
was still like the sort of underlying

204
00:07:18,880 --> 00:07:23,199
reason behind their behavior um so that

205
00:07:21,599 --> 00:07:24,960
really like flipped everything we

206
00:07:23,199 --> 00:07:27,280
thought we knew on its head and it made

207
00:07:24,960 --> 00:07:29,120
us worried that we were um gonna have

208
00:07:27,280 --> 00:07:30,720
difficulty working out

209
00:07:29,120 --> 00:07:33,280
what was the malicious behavior from

210
00:07:30,720 --> 00:07:35,199
what was just like overuse

211
00:07:33,280 --> 00:07:37,039
um and i think one way to think about

212
00:07:35,199 --> 00:07:39,039
this is to like take some time and

213
00:07:37,039 --> 00:07:41,440
deconstruct your proxy and that's sort

214
00:07:39,039 --> 00:07:43,520
of assuming that you acknowledge that

215
00:07:41,440 --> 00:07:45,599
pretty much every data that we might

216
00:07:43,520 --> 00:07:47,360
collect as a proxy it's it's an

217
00:07:45,599 --> 00:07:49,120
approximation for the thing we really

218
00:07:47,360 --> 00:07:51,680
want to know but it's never the actual

219
00:07:49,120 --> 00:07:53,440
thing we really want to know

220
00:07:51,680 --> 00:07:55,599
so i'd like to offer a concept framework

221
00:07:53,440 --> 00:07:56,960
for this and we can use the sms person

222
00:07:55,599 --> 00:07:58,800
as an example

223
00:07:56,960 --> 00:08:00,879
so we start with the signal so in this

224
00:07:58,800 --> 00:08:03,120
case we were observing how many sms's

225
00:08:00,879 --> 00:08:05,120
the service was sending and at what time

226
00:08:03,120 --> 00:08:06,960
and looking at you know like the sort of

227
00:08:05,120 --> 00:08:08,240
um frequency of their sends and that

228
00:08:06,960 --> 00:08:10,960
kind of thing

229
00:08:08,240 --> 00:08:13,280
um from there we were

230
00:08:10,960 --> 00:08:15,680
then inferring an activity that we

231
00:08:13,280 --> 00:08:18,000
thought would be causing the the data

232
00:08:15,680 --> 00:08:20,080
that we could observe um so on the top

233
00:08:18,000 --> 00:08:21,840
you can see a mobile and that sort of

234
00:08:20,080 --> 00:08:24,240
someone sitting on their phone typing

235
00:08:21,840 --> 00:08:25,520
out sms's and on the bottom was what we

236
00:08:24,240 --> 00:08:27,360
thought it was which was some kind of

237
00:08:25,520 --> 00:08:29,440
bot or script

238
00:08:27,360 --> 00:08:31,280
and from there we can then make another

239
00:08:29,440 --> 00:08:34,320
logic leap which is inferring some kind

240
00:08:31,280 --> 00:08:35,519
of persona or motive behind the activity

241
00:08:34,320 --> 00:08:37,200
that we think

242
00:08:35,519 --> 00:08:38,240
is causing the signal that we're

243
00:08:37,200 --> 00:08:39,839
observing

244
00:08:38,240 --> 00:08:41,440
so you can see here how i'm constructing

245
00:08:39,839 --> 00:08:44,399
this like it's kind of forcing us to

246
00:08:41,440 --> 00:08:45,360
acknowledge that we're taking one or two

247
00:08:44,399 --> 00:08:46,880
or more

248
00:08:45,360 --> 00:08:49,040
leaps of logic

249
00:08:46,880 --> 00:08:50,560
in getting from what it is we can see to

250
00:08:49,040 --> 00:08:52,959
what it is we think is happening in the

251
00:08:50,560 --> 00:08:56,080
world and that helps us keep a really

252
00:08:52,959 --> 00:08:57,839
healthy um humility and nuance and

253
00:08:56,080 --> 00:08:59,839
understanding what it is we can observe

254
00:08:57,839 --> 00:09:01,360
and not getting too confident and saying

255
00:08:59,839 --> 00:09:02,880
like this equals this and there's no

256
00:09:01,360 --> 00:09:05,120
other interpretation

257
00:09:02,880 --> 00:09:06,399
um so when we look at persona like again

258
00:09:05,120 --> 00:09:08,399
we went down the bad route and we

259
00:09:06,399 --> 00:09:10,160
thought oh you know they're sending spam

260
00:09:08,399 --> 00:09:12,800
or sms or something but in fact it was

261
00:09:10,160 --> 00:09:14,399
just a really large number of people

262
00:09:12,800 --> 00:09:16,880
talking on group chats and like

263
00:09:14,399 --> 00:09:18,480
communicating and sharing information um

264
00:09:16,880 --> 00:09:20,160
and you can think of other ways that

265
00:09:18,480 --> 00:09:22,080
people might generate that kind of data

266
00:09:20,160 --> 00:09:24,640
as well for instance you could be a

267
00:09:22,080 --> 00:09:26,720
developer testing out a twilio api and

268
00:09:24,640 --> 00:09:30,800
that's also not a malicious use of your

269
00:09:26,720 --> 00:09:30,800
phone but it might be unexpected

270
00:09:31,279 --> 00:09:34,959
so the next thing i want to talk about

271
00:09:33,440 --> 00:09:37,920
is this idea that

272
00:09:34,959 --> 00:09:39,200
once we really get into our heads that

273
00:09:37,920 --> 00:09:41,360
we don't know

274
00:09:39,200 --> 00:09:44,160
everything perfectly and that we have to

275
00:09:41,360 --> 00:09:46,640
like allow for uncertainty we also have

276
00:09:44,160 --> 00:09:49,839
to try and give ourselves the biggest

277
00:09:46,640 --> 00:09:52,640
leg up we can to interpreting and

278
00:09:49,839 --> 00:09:55,040
handling information we do think we know

279
00:09:52,640 --> 00:09:57,040
especially under stress so i want to

280
00:09:55,040 --> 00:09:58,800
make a distinction here um between what

281
00:09:57,040 --> 00:10:00,640
you might think of explanatory data

282
00:09:58,800 --> 00:10:02,000
science where you're trying to describe

283
00:10:00,640 --> 00:10:03,839
the world you're trying to understand

284
00:10:02,000 --> 00:10:05,360
what's happening you may not be sure

285
00:10:03,839 --> 00:10:06,000
what the right answer is and you're sort

286
00:10:05,360 --> 00:10:07,760
of

287
00:10:06,000 --> 00:10:09,440
you know going at it from different

288
00:10:07,760 --> 00:10:11,360
lenses or cutting the data in different

289
00:10:09,440 --> 00:10:12,959
ways and that's that's a really

290
00:10:11,360 --> 00:10:14,480
important and useful thing to do but

291
00:10:12,959 --> 00:10:16,880
that's not really what i'm talking to

292
00:10:14,480 --> 00:10:19,120
right now so what i mean when i talk

293
00:10:16,880 --> 00:10:21,519
about load-bearing design or

294
00:10:19,120 --> 00:10:24,640
you know data science under pressure is

295
00:10:21,519 --> 00:10:27,440
when you have an alert set up and

296
00:10:24,640 --> 00:10:29,360
you know like that alert triggers some

297
00:10:27,440 --> 00:10:31,040
activity or it means that you have to go

298
00:10:29,360 --> 00:10:32,720
check out a service or you have to go

299
00:10:31,040 --> 00:10:34,320
like have a look at a customer and see

300
00:10:32,720 --> 00:10:35,519
if something unexpected is happening and

301
00:10:34,320 --> 00:10:37,120
it usually means that you have to

302
00:10:35,519 --> 00:10:38,800
respond

303
00:10:37,120 --> 00:10:40,000
in the moment whether it's you know

304
00:10:38,800 --> 00:10:41,200
eight in the morning or two in the

305
00:10:40,000 --> 00:10:43,040
morning

306
00:10:41,200 --> 00:10:45,040
um so yeah i'm really talking

307
00:10:43,040 --> 00:10:46,800
specifically to load bearing in high

308
00:10:45,040 --> 00:10:48,640
pressure situations for the rest of this

309
00:10:46,800 --> 00:10:50,640
section

310
00:10:48,640 --> 00:10:51,680
and with that in mind i think it's

311
00:10:50,640 --> 00:10:53,600
really

312
00:10:51,680 --> 00:10:56,560
important to emphasize that data and

313
00:10:53,600 --> 00:10:58,480
misinterpretation is not just easy but

314
00:10:56,560 --> 00:11:00,880
likely under pressure

315
00:10:58,480 --> 00:11:02,959
we're putting like or putting our data

316
00:11:00,880 --> 00:11:05,360
science outputs through a ringer or like

317
00:11:02,959 --> 00:11:08,480
into a crucible and like really seeing

318
00:11:05,360 --> 00:11:10,880
how well they stand up um so if we want

319
00:11:08,480 --> 00:11:13,040
them to survive that environment we have

320
00:11:10,880 --> 00:11:15,279
to design for a cognitive load that

321
00:11:13,040 --> 00:11:17,839
means not putting too much information

322
00:11:15,279 --> 00:11:19,760
in the view and that means being really

323
00:11:17,839 --> 00:11:23,120
cautious to make everything as

324
00:11:19,760 --> 00:11:24,800
interpretable and explicit as possible

325
00:11:23,120 --> 00:11:26,079
and i i think it's important to remember

326
00:11:24,800 --> 00:11:28,399
that simple

327
00:11:26,079 --> 00:11:30,240
design doesn't imply that you are stupid

328
00:11:28,399 --> 00:11:32,079
the person receiving that design or

329
00:11:30,240 --> 00:11:34,560
maybe even your friend or your colleague

330
00:11:32,079 --> 00:11:36,480
or the junior or your project manager

331
00:11:34,560 --> 00:11:39,040
it just means that you're recognizing

332
00:11:36,480 --> 00:11:41,120
how important it is for this to go right

333
00:11:39,040 --> 00:11:43,600
um and another anecdote from my time in

334
00:11:41,120 --> 00:11:46,079
fraud detection was that we had these

335
00:11:43,600 --> 00:11:48,000
alerts coming through to slack and the

336
00:11:46,079 --> 00:11:50,079
alerts were

337
00:11:48,000 --> 00:11:51,760
coming from data that would be ingested

338
00:11:50,079 --> 00:11:53,440
then a batch would run and a feature

339
00:11:51,760 --> 00:11:55,279
would get calculated and then depending

340
00:11:53,440 --> 00:11:56,480
on the outcome of that

341
00:11:55,279 --> 00:11:57,760
there was another delay and then they

342
00:11:56,480 --> 00:11:59,440
would come into slack so there's sort of

343
00:11:57,760 --> 00:12:01,839
a time frame between when the data would

344
00:11:59,440 --> 00:12:04,639
be ingested and when we would get this

345
00:12:01,839 --> 00:12:06,720
alert and whenever the alert was bad and

346
00:12:04,639 --> 00:12:08,320
we were worried about things we would go

347
00:12:06,720 --> 00:12:10,480
check our logs and have this like

348
00:12:08,320 --> 00:12:12,639
momentary panic because we would forget

349
00:12:10,480 --> 00:12:14,720
about the delay between when the data

350
00:12:12,639 --> 00:12:15,760
was ingested and when this alert was

351
00:12:14,720 --> 00:12:17,920
coming out

352
00:12:15,760 --> 00:12:18,959
and we would be like oh [ __ ] oh the log

353
00:12:17,920 --> 00:12:21,040
is showing something different to the

354
00:12:18,959 --> 00:12:23,120
alert oh no something's so we'd think

355
00:12:21,040 --> 00:12:25,200
not only that something was going wrong

356
00:12:23,120 --> 00:12:27,120
as in we were under a fraud attack but

357
00:12:25,200 --> 00:12:30,240
also that our systems had fallen down in

358
00:12:27,120 --> 00:12:31,920
some way and kind of doubly panic um so

359
00:12:30,240 --> 00:12:34,079
i really wish i could go back in time

360
00:12:31,920 --> 00:12:36,560
and tell laura from the past to like

361
00:12:34,079 --> 00:12:39,600
take the time to make that as explicit

362
00:12:36,560 --> 00:12:41,680
as possible so what are other pointers

363
00:12:39,600 --> 00:12:42,880
for making your data as interpretable as

364
00:12:41,680 --> 00:12:44,880
possible

365
00:12:42,880 --> 00:12:47,440
this is probably an obvious one but it's

366
00:12:44,880 --> 00:12:50,320
bears repeating avoid dsl avoid

367
00:12:47,440 --> 00:12:52,399
domain-specific language avoid tlas the

368
00:12:50,320 --> 00:12:54,399
three-letter acronyms um and

369
00:12:52,399 --> 00:12:56,720
particularly don't put more than one of

370
00:12:54,399 --> 00:12:58,399
them in a line or in a sentence right

371
00:12:56,720 --> 00:13:00,959
like the more that you have the more

372
00:12:58,399 --> 00:13:03,600
opportunity for people to get confused

373
00:13:00,959 --> 00:13:06,079
um so an example here of this like going

374
00:13:03,600 --> 00:13:08,399
wrong on the left is where we see usn's

375
00:13:06,079 --> 00:13:10,480
breach tier one what's the usn

376
00:13:08,399 --> 00:13:11,839
um on the right you can see that it's

377
00:13:10,480 --> 00:13:14,800
actually unique service numbers that

378
00:13:11,839 --> 00:13:17,279
makes that a bit more clear

379
00:13:14,800 --> 00:13:21,040
another again probably obvious one but

380
00:13:17,279 --> 00:13:23,600
one that people easily miss is add units

381
00:13:21,040 --> 00:13:26,800
we can see on the left i've got a tier

382
00:13:23,600 --> 00:13:28,480
one is a breach of over 30 something in

383
00:13:26,800 --> 00:13:31,519
24 hours but what does that mean it

384
00:13:28,480 --> 00:13:33,680
could be sms's it could be hours on call

385
00:13:31,519 --> 00:13:34,720
it could be number of calls made in

386
00:13:33,680 --> 00:13:36,639
total

387
00:13:34,720 --> 00:13:38,320
on the right we can see it says 30 calls

388
00:13:36,639 --> 00:13:39,600
in 24 hours so

389
00:13:38,320 --> 00:13:41,360
now i know what it is we're talking

390
00:13:39,600 --> 00:13:43,120
about

391
00:13:41,360 --> 00:13:45,279
this is a bit of a bug bear of mine if

392
00:13:43,120 --> 00:13:47,040
you are talking about things like delta

393
00:13:45,279 --> 00:13:49,600
or percentiles and you're looking at

394
00:13:47,040 --> 00:13:51,600
change up or down say over a rolling 24

395
00:13:49,600 --> 00:13:53,760
hour period or over a week

396
00:13:51,600 --> 00:13:54,639
that's a perfectly valid way of looking

397
00:13:53,760 --> 00:13:57,360
for

398
00:13:54,639 --> 00:13:59,120
spikes in behavior or changes but it's

399
00:13:57,360 --> 00:14:01,279
really important to ground that with

400
00:13:59,120 --> 00:14:02,399
your absolute or total count of the

401
00:14:01,279 --> 00:14:04,240
thing

402
00:14:02,399 --> 00:14:07,360
so for example here on the left we see

403
00:14:04,240 --> 00:14:09,600
12 and 24 increases in tier one and tier

404
00:14:07,360 --> 00:14:11,760
two that seems like tier two is really a

405
00:14:09,600 --> 00:14:13,839
problem but then if you ground that in

406
00:14:11,760 --> 00:14:16,959
well it's only three unique surface

407
00:14:13,839 --> 00:14:18,880
numbers for for the top tier and then

408
00:14:16,959 --> 00:14:21,279
two unique service numbers for the

409
00:14:18,880 --> 00:14:22,639
middle tier like they're about the same

410
00:14:21,279 --> 00:14:24,959
they don't actually have that much

411
00:14:22,639 --> 00:14:26,959
difference it just it's just because the

412
00:14:24,959 --> 00:14:29,519
volume of the initial

413
00:14:26,959 --> 00:14:31,680
breaches wasn't was quite different so

414
00:14:29,519 --> 00:14:33,839
yes that kind of getting sucked into

415
00:14:31,680 --> 00:14:35,279
percentiles without knowing how many in

416
00:14:33,839 --> 00:14:37,839
total we're thinking about really

417
00:14:35,279 --> 00:14:40,160
matters for deciding what needs action

418
00:14:37,839 --> 00:14:42,160
for instance

419
00:14:40,160 --> 00:14:44,160
um and then again to the anecdote i was

420
00:14:42,160 --> 00:14:46,560
sharing at the beginning of this section

421
00:14:44,160 --> 00:14:48,560
if your data currency is like

422
00:14:46,560 --> 00:14:50,800
different to the time you see the alert

423
00:14:48,560 --> 00:14:52,800
or see the dashboard or whatever it is

424
00:14:50,800 --> 00:14:55,120
you're looking at make that as clear as

425
00:14:52,800 --> 00:14:57,120
possible and you know don't assume that

426
00:14:55,120 --> 00:14:59,040
people will remember because you're just

427
00:14:57,120 --> 00:15:01,519
adding more stress into their heads at a

428
00:14:59,040 --> 00:15:03,279
period when they're already under stress

429
00:15:01,519 --> 00:15:05,199
i think it's important to remember when

430
00:15:03,279 --> 00:15:07,839
we work with fraud but also when we

431
00:15:05,199 --> 00:15:09,279
think about um you know fairness and

432
00:15:07,839 --> 00:15:11,600
ethics

433
00:15:09,279 --> 00:15:12,880
it's easy to kind of get into this

434
00:15:11,600 --> 00:15:15,199
mindset of

435
00:15:12,880 --> 00:15:16,800
being a bit spy versus bi and seeing

436
00:15:15,199 --> 00:15:18,800
seeing like kind of bad actors and

437
00:15:16,800 --> 00:15:20,160
malfeasance everywhere

438
00:15:18,800 --> 00:15:22,399
um and i think

439
00:15:20,160 --> 00:15:24,399
not not that we don't want to be able to

440
00:15:22,399 --> 00:15:26,480
imagine what can go wrong

441
00:15:24,399 --> 00:15:27,839
but i think it's important that we start

442
00:15:26,480 --> 00:15:30,320
from the assumption of good faith

443
00:15:27,839 --> 00:15:32,160
particularly for our comms and then work

444
00:15:30,320 --> 00:15:34,320
our way down from there so we don't want

445
00:15:32,160 --> 00:15:35,920
to start it like you know a 10 for

446
00:15:34,320 --> 00:15:38,160
suspicion we want to start at a 1 and

447
00:15:35,920 --> 00:15:40,399
like kind of add points of evidence and

448
00:15:38,160 --> 00:15:42,160
then you know if someone's really acting

449
00:15:40,399 --> 00:15:44,560
in bad faith then we can

450
00:15:42,160 --> 00:15:46,880
up the ante on the tone

451
00:15:44,560 --> 00:15:49,360
and especially remembering that there's

452
00:15:46,880 --> 00:15:51,680
no need to suspect bad intentions when

453
00:15:49,360 --> 00:15:54,720
laziness or incompetence explains the

454
00:15:51,680 --> 00:15:57,279
same thing just as well

455
00:15:54,720 --> 00:15:59,519
so i think it's important to

456
00:15:57,279 --> 00:16:01,600
avoid falling into that trap of being

457
00:15:59,519 --> 00:16:03,440
ultra suspicious and seeing bad stuff

458
00:16:01,600 --> 00:16:07,040
everywhere and i think a good way to

459
00:16:03,440 --> 00:16:08,399
avoid that trap is to be explicitly

460
00:16:07,040 --> 00:16:10,399
deliberately

461
00:16:08,399 --> 00:16:12,240
kind and respectful in all of our

462
00:16:10,399 --> 00:16:14,639
communications and when i say

463
00:16:12,240 --> 00:16:16,880
communications i mean not only how we

464
00:16:14,639 --> 00:16:19,519
talk to customers whether it's actually

465
00:16:16,880 --> 00:16:20,480
a communication like a phone call or an

466
00:16:19,519 --> 00:16:23,040
email

467
00:16:20,480 --> 00:16:24,800
or whether it's a email template or a

468
00:16:23,040 --> 00:16:27,360
notification template that we're setting

469
00:16:24,800 --> 00:16:29,920
up or if it's how we talk about them

470
00:16:27,360 --> 00:16:32,160
inside our teams about a customer or an

471
00:16:29,920 --> 00:16:34,000
end user like the more we can set this

472
00:16:32,160 --> 00:16:37,360
intention and stick to it i think the

473
00:16:34,000 --> 00:16:39,759
safer we are and you might be saying to

474
00:16:37,360 --> 00:16:40,800
me well laura what if it's a bad actor

475
00:16:39,759 --> 00:16:43,360
and

476
00:16:40,800 --> 00:16:46,480
to be totally blunt my answer is so what

477
00:16:43,360 --> 00:16:48,959
um i don't think it matters if you speak

478
00:16:46,480 --> 00:16:51,279
respectfully to a bad actor but i do

479
00:16:48,959 --> 00:16:54,560
think you can cause a lot of harm if you

480
00:16:51,279 --> 00:16:57,120
come at you know a confused actor or

481
00:16:54,560 --> 00:16:58,639
actor who's just like doesn't understand

482
00:16:57,120 --> 00:17:00,480
what's what they're doing that's wrong

483
00:16:58,639 --> 00:17:03,040
with like a lot of aggression or a lot

484
00:17:00,480 --> 00:17:05,679
of suspicion or sort of interrogative

485
00:17:03,040 --> 00:17:07,280
tone so if you can

486
00:17:05,679 --> 00:17:08,880
bring it all back and start from the

487
00:17:07,280 --> 00:17:10,319
assumption that they may just not know

488
00:17:08,880 --> 00:17:12,480
that they're not allowed to do the thing

489
00:17:10,319 --> 00:17:15,120
or that it's not appropriate

490
00:17:12,480 --> 00:17:17,520
and try and give them as much sort of

491
00:17:15,120 --> 00:17:19,760
principled support as you can and then

492
00:17:17,520 --> 00:17:21,520
work from there you're much less likely

493
00:17:19,760 --> 00:17:24,000
to cause brand damage you're much less

494
00:17:21,520 --> 00:17:25,839
likely to lose customers because you

495
00:17:24,000 --> 00:17:28,480
know you've alienated them

496
00:17:25,839 --> 00:17:30,000
um and i think if you if you simply take

497
00:17:28,480 --> 00:17:32,720
it down to the bottom line you're much

498
00:17:30,000 --> 00:17:36,080
less likely to lose money and on top of

499
00:17:32,720 --> 00:17:38,880
that if you do talk to a bad actor in

500
00:17:36,080 --> 00:17:40,320
good faith you sometimes catch them out

501
00:17:38,880 --> 00:17:43,120
right like they kind of think oh they

502
00:17:40,320 --> 00:17:44,880
don't suspect me and as a result they'll

503
00:17:43,120 --> 00:17:46,960
tell you things that confirm your

504
00:17:44,880 --> 00:17:49,600
suspicions about them so there's this

505
00:17:46,960 --> 00:17:52,240
like additional value in coming at them

506
00:17:49,600 --> 00:17:55,440
with real like you know clean neutral

507
00:17:52,240 --> 00:17:55,440
respectful language

508
00:17:55,679 --> 00:17:58,240
and

509
00:17:56,400 --> 00:18:00,160
to add to all of that

510
00:17:58,240 --> 00:18:02,160
we're in the middle of a pandemic people

511
00:18:00,160 --> 00:18:04,720
are having a bad time you don't know

512
00:18:02,160 --> 00:18:06,240
their circumstances you don't know how

513
00:18:04,720 --> 00:18:07,600
much money they're earning or not

514
00:18:06,240 --> 00:18:09,039
earning and you don't know certainly

515
00:18:07,600 --> 00:18:10,880
what's driving them to do what they're

516
00:18:09,039 --> 00:18:13,039
doing so i think it's important to

517
00:18:10,880 --> 00:18:15,600
remember also that we don't want to be

518
00:18:13,039 --> 00:18:18,000
moralizing about this or judgmental so

519
00:18:15,600 --> 00:18:20,240
it's better to try and keep that aspect

520
00:18:18,000 --> 00:18:22,160
of it out of it and just focus on what

521
00:18:20,240 --> 00:18:25,280
it is you need to do so if it's

522
00:18:22,160 --> 00:18:27,840
preventing behaviors if it's um you know

523
00:18:25,280 --> 00:18:30,160
nudging people along the right path like

524
00:18:27,840 --> 00:18:33,120
keep those as your as your guiding star

525
00:18:30,160 --> 00:18:34,640
and get away from this issue of i was

526
00:18:33,120 --> 00:18:37,200
right and they were wrong or i was

527
00:18:34,640 --> 00:18:38,559
better than them

528
00:18:37,200 --> 00:18:40,799
another tip for trying to get this

529
00:18:38,559 --> 00:18:43,679
better is to to focus on describing

530
00:18:40,799 --> 00:18:46,720
behaviors and not people or activity

531
00:18:43,679 --> 00:18:48,320
that you can observe and not people so

532
00:18:46,720 --> 00:18:50,320
for example in the fraud team we were

533
00:18:48,320 --> 00:18:52,080
calling people fraudsters and abusers a

534
00:18:50,320 --> 00:18:54,240
lot and it was something that kind of

535
00:18:52,080 --> 00:18:56,080
didn't sit right in my gut and i was

536
00:18:54,240 --> 00:18:59,120
sort of interrogating why do i not like

537
00:18:56,080 --> 00:19:01,039
this this um use of language it was like

538
00:18:59,120 --> 00:19:02,160
oh you know we're

539
00:19:01,039 --> 00:19:03,280
doing the thing of making the

540
00:19:02,160 --> 00:19:04,880
assumptions saying that's definitely

541
00:19:03,280 --> 00:19:07,039
what's happening but also it's got this

542
00:19:04,880 --> 00:19:08,960
kind of judgmental tone

543
00:19:07,039 --> 00:19:10,559
and we're adding a bunch of like

544
00:19:08,960 --> 00:19:14,240
probably unnecessary

545
00:19:10,559 --> 00:19:16,160
uh like um judgmental and moralizing

546
00:19:14,240 --> 00:19:18,160
attitude to to something that can be

547
00:19:16,160 --> 00:19:20,320
quite simple so we just started talking

548
00:19:18,160 --> 00:19:22,160
instead about you know

549
00:19:20,320 --> 00:19:24,960
i can observe like this person's sending

550
00:19:22,160 --> 00:19:26,640
a lot of sms's to country x or oh

551
00:19:24,960 --> 00:19:28,799
they've had a spike in phone

552
00:19:26,640 --> 00:19:31,200
phone call behavior and when you start

553
00:19:28,799 --> 00:19:33,440
doing that it it again like

554
00:19:31,200 --> 00:19:35,919
it focuses you on what you actually know

555
00:19:33,440 --> 00:19:37,919
and doesn't take you past that into like

556
00:19:35,919 --> 00:19:39,679
the assumptions that you make from that

557
00:19:37,919 --> 00:19:41,520
or the inferences that you make from

558
00:19:39,679 --> 00:19:42,799
that

559
00:19:41,520 --> 00:19:44,799
don't mistake

560
00:19:42,799 --> 00:19:47,280
this this whole rant for saying that i

561
00:19:44,799 --> 00:19:49,520
don't think you should intervene on

562
00:19:47,280 --> 00:19:51,679
people who are behaving inappropriately

563
00:19:49,520 --> 00:19:53,840
in your systems i absolutely do think

564
00:19:51,679 --> 00:19:55,919
that that's important and in fact often

565
00:19:53,840 --> 00:19:58,000
i think we can solve these problems

566
00:19:55,919 --> 00:19:59,440
better upstream and product by like

567
00:19:58,000 --> 00:20:01,200
designing in

568
00:19:59,440 --> 00:20:04,000
clearer like boundaries for what's

569
00:20:01,200 --> 00:20:06,559
acceptable or making the product offer

570
00:20:04,000 --> 00:20:08,159
more clear or you know otherwise adding

571
00:20:06,559 --> 00:20:10,240
in those guardrails

572
00:20:08,159 --> 00:20:12,000
but i'm just saying you can do that work

573
00:20:10,240 --> 00:20:14,799
and still stay respectful in your

574
00:20:12,000 --> 00:20:17,200
communication style

575
00:20:14,799 --> 00:20:19,039
so the last finding i want to talk to is

576
00:20:17,200 --> 00:20:21,440
this thing i like to call failure first

577
00:20:19,039 --> 00:20:23,840
design and that stated simply is is

578
00:20:21,440 --> 00:20:26,400
saying that we start from the assumption

579
00:20:23,840 --> 00:20:29,520
that the system will sometimes fail and

580
00:20:26,400 --> 00:20:29,520
then we plan for it

581
00:20:29,840 --> 00:20:34,640
this quote i love from stuart russell

582
00:20:32,159 --> 00:20:36,159
and he says assuming perfect knowledge

583
00:20:34,640 --> 00:20:38,720
of the objective

584
00:20:36,159 --> 00:20:41,120
decouples the machine from the human

585
00:20:38,720 --> 00:20:43,120
what the human does no longer matters

586
00:20:41,120 --> 00:20:44,400
because the machine knows the goal and

587
00:20:43,120 --> 00:20:46,080
pursues it

588
00:20:44,400 --> 00:20:47,840
and i think that's a really important

589
00:20:46,080 --> 00:20:49,919
way of like helping us think about this

590
00:20:47,840 --> 00:20:52,080
like we don't even know our own

591
00:20:49,919 --> 00:20:54,240
objectives perfectly let alone people's

592
00:20:52,080 --> 00:20:56,960
objectives we often do a really poor job

593
00:20:54,240 --> 00:20:59,280
of capturing preferences for instance

594
00:20:56,960 --> 00:21:01,919
and we live in a world where there's no

595
00:20:59,280 --> 00:21:04,799
bug-free software and when you add in

596
00:21:01,919 --> 00:21:06,880
stochastic noise and non-deterministic

597
00:21:04,799 --> 00:21:09,200
systems you have all of these

598
00:21:06,880 --> 00:21:11,679
abstraction layers and um

599
00:21:09,200 --> 00:21:13,360
aspects of our tech that can go wrong so

600
00:21:11,679 --> 00:21:15,919
just accepting that that's the world

601
00:21:13,360 --> 00:21:18,480
we're living in and trying to say well

602
00:21:15,919 --> 00:21:20,640
we'll inject some humility into our

603
00:21:18,480 --> 00:21:22,480
planning and into our design and we'll

604
00:21:20,640 --> 00:21:24,640
try and continue to do these little

605
00:21:22,480 --> 00:21:27,679
micro adjustments based on what people

606
00:21:24,640 --> 00:21:30,400
tell us about their preferences um you

607
00:21:27,679 --> 00:21:32,159
know their experiences etc the more

608
00:21:30,400 --> 00:21:34,559
likely we are to

609
00:21:32,159 --> 00:21:36,559
not get down the path of having you know

610
00:21:34,559 --> 00:21:38,080
done something horrifically bad to a lot

611
00:21:36,559 --> 00:21:40,960
of people and not even known about it

612
00:21:38,080 --> 00:21:42,720
until something terrible happens

613
00:21:40,960 --> 00:21:45,039
so i think about this work as like

614
00:21:42,720 --> 00:21:47,840
designing an escape hatch or adding

615
00:21:45,039 --> 00:21:49,919
lifeboats to a big ship right like

616
00:21:47,840 --> 00:21:52,080
it's not necessarily something that we

617
00:21:49,919 --> 00:21:54,480
like to think about it's not necessarily

618
00:21:52,080 --> 00:21:56,880
fun work and in fact you may not you may

619
00:21:54,480 --> 00:21:58,480
think that success looks like never

620
00:21:56,880 --> 00:22:00,159
seeing it get used

621
00:21:58,480 --> 00:22:02,559
but you still have to make sure it's

622
00:22:00,159 --> 00:22:04,880
there and it's functioning and if people

623
00:22:02,559 --> 00:22:07,840
do need to use it you want to make sure

624
00:22:04,880 --> 00:22:09,919
that it's actually usable

625
00:22:07,840 --> 00:22:11,520
so feedback loops um in order to

626
00:22:09,919 --> 00:22:13,520
discover things about people's

627
00:22:11,520 --> 00:22:16,960
experience in the moment need to be

628
00:22:13,520 --> 00:22:20,000
intuitive contextual and timely and what

629
00:22:16,960 --> 00:22:22,159
i mean by that is we really can't just

630
00:22:20,000 --> 00:22:23,679
throw some support channel at the bottom

631
00:22:22,159 --> 00:22:26,320
of our page and assume that people will

632
00:22:23,679 --> 00:22:29,120
talk to us right we have to actually ask

633
00:22:26,320 --> 00:22:31,039
them questions about the the specific

634
00:22:29,120 --> 00:22:33,120
contextual thing they're doing in our

635
00:22:31,039 --> 00:22:35,679
tool at the moment

636
00:22:33,120 --> 00:22:37,760
so some little examples of that

637
00:22:35,679 --> 00:22:39,760
we might for instance on the left we're

638
00:22:37,760 --> 00:22:41,520
looking at botify which is a music

639
00:22:39,760 --> 00:22:43,120
streaming service and it's showing me

640
00:22:41,520 --> 00:22:45,360
like the different genres that i listen

641
00:22:43,120 --> 00:22:47,120
to the most now i might know that i

642
00:22:45,360 --> 00:22:48,559
listen to a lot of lo-fi hip-hop but

643
00:22:47,120 --> 00:22:50,960
actually prefer to listen to a bit more

644
00:22:48,559 --> 00:22:53,440
classical so i've got a link which lets

645
00:22:50,960 --> 00:22:55,280
me adjust my preferences

646
00:22:53,440 --> 00:22:57,679
on the right a little bit more of a dire

647
00:22:55,280 --> 00:23:00,480
example i've applied for a loan of 85

648
00:22:57,679 --> 00:23:02,000
000 and been rejected and what i've done

649
00:23:00,480 --> 00:23:03,679
is added a couple of buttons that give

650
00:23:02,000 --> 00:23:05,760
me ways to kind of

651
00:23:03,679 --> 00:23:08,000
further explore that explanation i've

652
00:23:05,760 --> 00:23:11,679
seen and maybe like let me talk to

653
00:23:08,000 --> 00:23:13,520
someone or let me kind of break down

654
00:23:11,679 --> 00:23:15,440
how that algorithmic decision worked so

655
00:23:13,520 --> 00:23:17,520
that i can actually understand if i

656
00:23:15,440 --> 00:23:19,840
agree with it and see if i need to take

657
00:23:17,520 --> 00:23:21,600
more action or not

658
00:23:19,840 --> 00:23:24,320
another big one with image

659
00:23:21,600 --> 00:23:26,000
classification is just letting people

660
00:23:24,320 --> 00:23:27,600
tell you when you're wrong especially if

661
00:23:26,000 --> 00:23:28,960
it's their images

662
00:23:27,600 --> 00:23:31,679
in this example we can see it's

663
00:23:28,960 --> 00:23:33,280
incorrectly classified as a cute puppy

664
00:23:31,679 --> 00:23:34,799
as a cute kitten

665
00:23:33,280 --> 00:23:37,039
another one that i like to talk about a

666
00:23:34,799 --> 00:23:39,360
lot is adding in the level of confidence

667
00:23:37,039 --> 00:23:40,799
of your system um just kind of letting

668
00:23:39,360 --> 00:23:43,039
people know that you don't think you're

669
00:23:40,799 --> 00:23:46,000
perfect is much more encouraging for

670
00:23:43,039 --> 00:23:47,440
gathering feedback as well

671
00:23:46,000 --> 00:23:49,520
this is a pattern you've probably seen a

672
00:23:47,440 --> 00:23:50,559
lot it's getting a lot more popular now

673
00:23:49,520 --> 00:23:52,880
where we see

674
00:23:50,559 --> 00:23:54,960
spikes of activity or things changing on

675
00:23:52,880 --> 00:23:57,360
your account unexpected logins and new

676
00:23:54,960 --> 00:23:59,120
machines um and we'll sort of send

677
00:23:57,360 --> 00:24:01,120
someone a message immediately in the

678
00:23:59,120 --> 00:24:03,360
moment and just check and see it was

679
00:24:01,120 --> 00:24:05,279
them and most of the time it is but when

680
00:24:03,360 --> 00:24:07,600
it's not it lets us sort of

681
00:24:05,279 --> 00:24:10,000
smell out suspicious activity very

682
00:24:07,600 --> 00:24:10,000
quickly

683
00:24:11,360 --> 00:24:14,400
and the last one which i think is

684
00:24:13,360 --> 00:24:16,240
pretty

685
00:24:14,400 --> 00:24:18,960
easy to do and important but easy to

686
00:24:16,240 --> 00:24:20,400
forget is if you're working in a domain

687
00:24:18,960 --> 00:24:22,080
where you're expecting someone to

688
00:24:20,400 --> 00:24:25,039
experience harm or you're worried that

689
00:24:22,080 --> 00:24:28,159
that's a likely outcome for them you can

690
00:24:25,039 --> 00:24:29,840
just ask you can ask them you know

691
00:24:28,159 --> 00:24:32,000
in the moment that they're doing a thing

692
00:24:29,840 --> 00:24:34,000
like filling out the form or using your

693
00:24:32,000 --> 00:24:36,400
tool or you can ask them like as a

694
00:24:34,000 --> 00:24:38,400
screening question before they join um

695
00:24:36,400 --> 00:24:40,000
but asking these questions of if they've

696
00:24:38,400 --> 00:24:42,400
experienced harm before or if they're

697
00:24:40,000 --> 00:24:45,039
experiencing harm currently in your tool

698
00:24:42,400 --> 00:24:48,960
can be a surprisingly fruitful way of

699
00:24:45,039 --> 00:24:48,960
eliciting what it is that's happening

700
00:24:49,360 --> 00:24:52,960
so it's really important that if we do

701
00:24:51,039 --> 00:24:54,960
do these things we don't just want them

702
00:24:52,960 --> 00:24:57,440
to fall in a heap right so we do have to

703
00:24:54,960 --> 00:24:59,039
plan and do a bit more work

704
00:24:57,440 --> 00:25:01,120
and that can look like customer support

705
00:24:59,039 --> 00:25:02,240
or writing a support script or thinking

706
00:25:01,120 --> 00:25:05,200
through

707
00:25:02,240 --> 00:25:06,799
policies for the use of your tool

708
00:25:05,200 --> 00:25:08,240
it also means you have to be really

709
00:25:06,799 --> 00:25:10,000
meticulous about capturing your

710
00:25:08,240 --> 00:25:11,520
learnings you don't want them to just

711
00:25:10,000 --> 00:25:14,080
sit in a bunch of support tickets you

712
00:25:11,520 --> 00:25:15,760
need to have some kind of data schema

713
00:25:14,080 --> 00:25:17,120
you know it can just be a spreadsheet

714
00:25:15,760 --> 00:25:19,760
somewhere but you still want to make

715
00:25:17,120 --> 00:25:21,200
sure you know what it is someone did and

716
00:25:19,760 --> 00:25:22,400
what you classified them as and if

717
00:25:21,200 --> 00:25:24,400
something changed how that

718
00:25:22,400 --> 00:25:27,039
classification changed

719
00:25:24,400 --> 00:25:28,559
and um importantly if you do find out

720
00:25:27,039 --> 00:25:30,400
that your model isn't great or if

721
00:25:28,559 --> 00:25:32,960
something's going wrong in your logic

722
00:25:30,400 --> 00:25:35,919
that you can leave time for product or

723
00:25:32,960 --> 00:25:35,919
model improvements

724
00:25:36,720 --> 00:25:40,559
and also

725
00:25:37,840 --> 00:25:42,400
preparing for pushback um this is a

726
00:25:40,559 --> 00:25:43,679
little bit of a sad story part of the of

727
00:25:42,400 --> 00:25:45,520
the talk but

728
00:25:43,679 --> 00:25:48,480
people don't like it when you make bad

729
00:25:45,520 --> 00:25:49,840
assumptions about them and that's okay

730
00:25:48,480 --> 00:25:52,240
you just have to prepare yourself

731
00:25:49,840 --> 00:25:53,440
mentally for that experience

732
00:25:52,240 --> 00:25:55,520
um

733
00:25:53,440 --> 00:25:57,600
if you think to yourself oh well i'm i'm

734
00:25:55,520 --> 00:25:59,760
making something visible that previously

735
00:25:57,600 --> 00:26:01,440
wasn't visible and that means that some

736
00:25:59,760 --> 00:26:03,840
people are going to say oh yeah that

737
00:26:01,440 --> 00:26:05,679
makes sense to me and i agree with it

738
00:26:03,840 --> 00:26:07,360
and some people are going to be like oh

739
00:26:05,679 --> 00:26:08,480
okay i don't like that anymore or i

740
00:26:07,360 --> 00:26:09,520
think this company doesn't know what

741
00:26:08,480 --> 00:26:12,080
they're doing

742
00:26:09,520 --> 00:26:14,000
you know you're inviting the opportunity

743
00:26:12,080 --> 00:26:15,760
for people to disagree with you or to

744
00:26:14,000 --> 00:26:16,960
think that you are doing something a

745
00:26:15,760 --> 00:26:20,159
little bit different than they

746
00:26:16,960 --> 00:26:22,240
originally expected um but that said

747
00:26:20,159 --> 00:26:24,400
you're still going to find out much

748
00:26:22,240 --> 00:26:26,480
sooner if something is going wrong

749
00:26:24,400 --> 00:26:28,400
than you do if you just have no feedback

750
00:26:26,480 --> 00:26:30,240
loops baked in in the first place

751
00:26:28,400 --> 00:26:31,200
um and certainly in my experience in

752
00:26:30,240 --> 00:26:33,039
fraud

753
00:26:31,200 --> 00:26:35,200
we had a few instances with these kinds

754
00:26:33,039 --> 00:26:37,840
of feedback loops where we had some very

755
00:26:35,200 --> 00:26:40,000
angry customers and some very like clear

756
00:26:37,840 --> 00:26:41,200
mismatches of their mental model of what

757
00:26:40,000 --> 00:26:43,520
they thought they were buying and what

758
00:26:41,200 --> 00:26:45,919
we were telling them they had bought

759
00:26:43,520 --> 00:26:49,279
and it was uncomfortable but it was also

760
00:26:45,919 --> 00:26:49,279
really important for us to know

761
00:26:49,440 --> 00:26:53,760
so just to review we've talked to

762
00:26:51,600 --> 00:26:57,200
deconstructing your proxy

763
00:26:53,760 --> 00:27:00,480
explicit data design assuming good faith

764
00:26:57,200 --> 00:27:03,279
and failure first design and hopefully

765
00:27:00,480 --> 00:27:05,440
those are useful little like flag posts

766
00:27:03,279 --> 00:27:08,240
for you to remember and think about how

767
00:27:05,440 --> 00:27:10,640
to do this work a little bit better

768
00:27:08,240 --> 00:27:11,760
and um just to close up with a metaphor

769
00:27:10,640 --> 00:27:14,240
i think of

770
00:27:11,760 --> 00:27:17,200
making tech as us working together

771
00:27:14,240 --> 00:27:19,679
sitting on a a big ship a cruise liner

772
00:27:17,200 --> 00:27:21,600
and when you go below decks it's quiet

773
00:27:19,679 --> 00:27:22,799
and safe it's maybe even boring like you

774
00:27:21,600 --> 00:27:24,159
don't even have a sense that you're

775
00:27:22,799 --> 00:27:26,159
going anywhere

776
00:27:24,159 --> 00:27:28,320
but when you come back out on deck you

777
00:27:26,159 --> 00:27:30,080
have a sense of the speed and the mass

778
00:27:28,320 --> 00:27:33,360
of water displaced

779
00:27:30,080 --> 00:27:35,279
and i think as technologists and as

780
00:27:33,360 --> 00:27:38,000
people working you know with data

781
00:27:35,279 --> 00:27:40,000
systems with real humans in the world

782
00:27:38,000 --> 00:27:42,799
we don't want to insulate ourselves from

783
00:27:40,000 --> 00:27:44,559
that sensation we want to feel the wind

784
00:27:42,799 --> 00:27:47,039
rushing through our hair

785
00:27:44,559 --> 00:27:50,159
we want to have a little frissona fear

786
00:27:47,039 --> 00:27:52,720
and not be too comfortable

787
00:27:50,159 --> 00:27:55,440
because i do think the real game is

788
00:27:52,720 --> 00:27:57,679
trying to catch fraud yet not defraud

789
00:27:55,440 --> 00:27:59,279
ourselves

790
00:27:57,679 --> 00:28:01,840
thank you so much for your attention i

791
00:27:59,279 --> 00:28:03,760
hope you enjoyed that um i just want to

792
00:28:01,840 --> 00:28:05,679
add one little note which is i am

793
00:28:03,760 --> 00:28:08,399
working on a tool which is coming soon

794
00:28:05,679 --> 00:28:10,960
which is called sweet summer child score

795
00:28:08,399 --> 00:28:13,440
um it's going to be a

796
00:28:10,960 --> 00:28:15,120
test which scans for harm to people and

797
00:28:13,440 --> 00:28:17,279
communities by data driven systems

798
00:28:15,120 --> 00:28:18,960
that's going to be free and open source

799
00:28:17,279 --> 00:28:21,200
and i'm looking for testers to help me

800
00:28:18,960 --> 00:28:23,760
kick the tires so please do reach out to

801
00:28:21,200 --> 00:28:24,960
me at laura depis dot ai if you think

802
00:28:23,760 --> 00:28:27,520
you could help

803
00:28:24,960 --> 00:28:27,520
thanks again

804
00:28:29,440 --> 00:28:34,000
welcome back thank you laura that has

805
00:28:31,279 --> 00:28:36,159
been an amazing talk and you know as

806
00:28:34,000 --> 00:28:37,760
someone who already knew the outline and

807
00:28:36,159 --> 00:28:39,360
had learned something from it i've

808
00:28:37,760 --> 00:28:41,760
learned so much from every one of your

809
00:28:39,360 --> 00:28:45,120
slides there are questions the first one

810
00:28:41,760 --> 00:28:47,760
is um what have you found helpful for

811
00:28:45,120 --> 00:28:50,000
getting people who might be you know i

812
00:28:47,760 --> 00:28:52,240
have some resistance some friction to

813
00:28:50,000 --> 00:28:53,840
changing the ways they operate they have

814
00:28:52,240 --> 00:28:56,240
operated traditionally like traditional

815
00:28:53,840 --> 00:28:57,679
fraud teams at you know this kind of

816
00:28:56,240 --> 00:28:59,360
institutions

817
00:28:57,679 --> 00:29:01,279
yeah look it's it's a great question

818
00:28:59,360 --> 00:29:03,440
it's one i ran into a lot in this

819
00:29:01,279 --> 00:29:05,120
experience um

820
00:29:03,440 --> 00:29:07,279
the the short answer is it's hard and

821
00:29:05,120 --> 00:29:09,279
you can't fix it straight away um the

822
00:29:07,279 --> 00:29:11,120
the longer answer is trying to come up

823
00:29:09,279 --> 00:29:13,039
with champions people who like genuinely

824
00:29:11,120 --> 00:29:14,399
care about the work and being really

825
00:29:13,039 --> 00:29:16,799
visible about

826
00:29:14,399 --> 00:29:18,640
you know awarding them applauding them

827
00:29:16,799 --> 00:29:19,760
um you know like recognizing their work

828
00:29:18,640 --> 00:29:22,159
when they do

829
00:29:19,760 --> 00:29:24,880
um they they do kind of model these

830
00:29:22,159 --> 00:29:26,960
behaviors of um holding themselves back

831
00:29:24,880 --> 00:29:29,120
from being too judgmental or sort of

832
00:29:26,960 --> 00:29:31,200
focusing on behaviors and outcomes and

833
00:29:29,120 --> 00:29:34,320
not focusing on you know moralizing or

834
00:29:31,200 --> 00:29:35,760
judging people um it can be a long slog

835
00:29:34,320 --> 00:29:37,200
because you're really like changing

836
00:29:35,760 --> 00:29:39,600
mental models and you're changing like

837
00:29:37,200 --> 00:29:41,440
these very ingrained behaviors

838
00:29:39,600 --> 00:29:43,279
so also being patient with yourself that

839
00:29:41,440 --> 00:29:46,279
it's not going to happen overnight i

840
00:29:43,279 --> 00:29:46,279
think