1 00:00:00,480 --> 00:00:03,480 foreign 2 00:00:09,800 --> 00:00:14,460 I would like to introduce your speaker 3 00:00:11,940 --> 00:00:17,100 this morning uh Tom Eastman 4 00:00:14,460 --> 00:00:19,619 Tom works for the Kraken sponsor that we 5 00:00:17,100 --> 00:00:22,080 have and he'll be he has attended every 6 00:00:19,619 --> 00:00:23,939 Pike on Australia we've had so far 7 00:00:22,080 --> 00:00:25,980 I think that's 10 years worth which is 8 00:00:23,939 --> 00:00:28,080 great 9 00:00:25,980 --> 00:00:30,300 um Tom loves to talk about many many 10 00:00:28,080 --> 00:00:31,800 things his great person to have 11 00:00:30,300 --> 00:00:33,300 conversations with so if you do want to 12 00:00:31,800 --> 00:00:35,640 talk more about this talk afterwards 13 00:00:33,300 --> 00:00:38,219 find him at the Kraken booth at 14 00:00:35,640 --> 00:00:39,180 lunchtime and he will happily talk your 15 00:00:38,219 --> 00:00:41,820 ear off 16 00:00:39,180 --> 00:00:44,520 I'll let you go if you need to 17 00:00:41,820 --> 00:00:46,100 um please provide a warm welcoming 18 00:00:44,520 --> 00:00:49,459 Applause for Tom 19 00:00:46,100 --> 00:00:49,459 [Applause] 20 00:00:49,860 --> 00:00:55,620 hey I'm Tom um who has heard me talk 21 00:00:52,860 --> 00:00:57,360 about secateur before in this room 22 00:00:55,620 --> 00:00:59,399 quite a few of you cool cool because 23 00:00:57,360 --> 00:01:01,920 it's this is the third talk that I've 24 00:00:59,399 --> 00:01:03,960 given on this what was just like a silly 25 00:01:01,920 --> 00:01:05,519 little Django side project 26 00:01:03,960 --> 00:01:06,540 um that kind of grew a little bit out of 27 00:01:05,519 --> 00:01:09,000 control 28 00:01:06,540 --> 00:01:10,560 and so there's a lot of detail that I 29 00:01:09,000 --> 00:01:11,880 might skip over and I'm more than happy 30 00:01:10,560 --> 00:01:13,439 to talk about that stuff or you can 31 00:01:11,880 --> 00:01:14,700 check out my previous two talks on the 32 00:01:13,439 --> 00:01:17,520 subject 33 00:01:14,700 --> 00:01:18,960 um just a bit of a Content warning this 34 00:01:17,520 --> 00:01:20,400 is a talk about a tool to protect people 35 00:01:18,960 --> 00:01:22,320 from social media harassment so I'm 36 00:01:20,400 --> 00:01:24,479 probably going to be mentioning 37 00:01:22,320 --> 00:01:25,979 some of the things some of the sorts of 38 00:01:24,479 --> 00:01:26,759 harassment that kind of happen on 39 00:01:25,979 --> 00:01:28,500 Twitter 40 00:01:26,759 --> 00:01:29,460 um I'm not going to dwell on any of it 41 00:01:28,500 --> 00:01:31,380 but that means that there might be 42 00:01:29,460 --> 00:01:32,939 mentions of transphobia and homophobia 43 00:01:31,380 --> 00:01:34,560 and stuff like that just because that's 44 00:01:32,939 --> 00:01:36,060 the kind of awful [ __ ] that happens on 45 00:01:34,560 --> 00:01:36,659 Twitter 46 00:01:36,060 --> 00:01:38,400 um 47 00:01:36,659 --> 00:01:39,659 the Project's dead now and we're going 48 00:01:38,400 --> 00:01:41,280 to go into detail we're going to go into 49 00:01:39,659 --> 00:01:44,040 a little bit of detail on that so this 50 00:01:41,280 --> 00:01:45,479 is kind of just a recap of why I built 51 00:01:44,040 --> 00:01:49,079 it what I learned along the way it's 52 00:01:45,479 --> 00:01:51,540 kind of a grab bag of of anecdotes and 53 00:01:49,079 --> 00:01:53,520 things that happened maybe some of this 54 00:01:51,540 --> 00:01:55,380 will be useful lessons 55 00:01:53,520 --> 00:01:56,640 um maybe it won't be some of it will 56 00:01:55,380 --> 00:01:58,220 hopefully be a little bit entertaining 57 00:01:56,640 --> 00:02:00,960 some of it will be a little bit surreal 58 00:01:58,220 --> 00:02:03,540 I'm going to try 59 00:02:00,960 --> 00:02:06,780 to play a tiny bit of a question that I 60 00:02:03,540 --> 00:02:10,319 received at 2019 when I first launched 61 00:02:06,780 --> 00:02:12,780 this uh tool 62 00:02:10,319 --> 00:02:14,760 you there we go have you done any threat 63 00:02:12,780 --> 00:02:17,280 modeling on sort of the impact that this 64 00:02:14,760 --> 00:02:18,720 could directly Place Onto You by being 65 00:02:17,280 --> 00:02:20,640 the creator of this app with like 66 00:02:18,720 --> 00:02:22,860 because your GitHub name is out there 67 00:02:20,640 --> 00:02:24,599 how like your DNS creates that kind of 68 00:02:22,860 --> 00:02:26,760 stuff have you done anything to sort of 69 00:02:24,599 --> 00:02:29,040 cover yourself when it comes to building 70 00:02:26,760 --> 00:02:30,900 a tool like this that you know [ __ ] 71 00:02:29,040 --> 00:02:33,300 with too much time on their hands can 72 00:02:30,900 --> 00:02:37,260 just start and play with 73 00:02:33,300 --> 00:02:39,239 a little but only in so far as my GitHub 74 00:02:37,260 --> 00:02:41,220 account has also got 2fa on it most of 75 00:02:39,239 --> 00:02:44,700 most of what's important to me has 76 00:02:41,220 --> 00:02:46,620 reasonable protections on it my real 77 00:02:44,700 --> 00:02:48,480 name is of course everywhere 78 00:02:46,620 --> 00:02:49,860 um I am Tom Eastman that is actually on 79 00:02:48,480 --> 00:02:51,900 my birth certificate I'm findable 80 00:02:49,860 --> 00:02:54,000 someone could hack my run keeper and 81 00:02:51,900 --> 00:02:56,400 possibly find out where I lived there's 82 00:02:54,000 --> 00:02:58,800 there's interesting things once you 83 00:02:56,400 --> 00:03:01,739 become a Target yourself 84 00:02:58,800 --> 00:03:02,760 um I again get to okay I'm going to stop 85 00:03:01,739 --> 00:03:04,019 there but that's a little bit of 86 00:03:02,760 --> 00:03:04,860 foreshadowing we'll come we'll come back 87 00:03:04,019 --> 00:03:06,180 to that 88 00:03:04,860 --> 00:03:08,519 a little bit 89 00:03:06,180 --> 00:03:10,739 um don't don't worry about it it's cool 90 00:03:08,519 --> 00:03:13,140 um so what what is secretary secature is 91 00:03:10,739 --> 00:03:16,920 a little tiny web app it uses the 92 00:03:13,140 --> 00:03:18,900 Twitter API to allow someone to block a 93 00:03:16,920 --> 00:03:20,459 person who's harassed on Twitter and all 94 00:03:18,900 --> 00:03:23,940 of their followers for a set period of 95 00:03:20,459 --> 00:03:25,260 time so if someone is attacking you on 96 00:03:23,940 --> 00:03:26,159 Twitter and they have like 20 000 97 00:03:25,260 --> 00:03:27,780 followers and they're sending their 98 00:03:26,159 --> 00:03:29,340 followers after you 99 00:03:27,780 --> 00:03:30,780 um you can hit one button and you can 100 00:03:29,340 --> 00:03:32,060 block them and you can block all of 101 00:03:30,780 --> 00:03:34,080 their followers 102 00:03:32,060 --> 00:03:36,659 uh the 103 00:03:34,080 --> 00:03:38,760 it gave them it and it did it for a set 104 00:03:36,659 --> 00:03:40,739 period of time so um it kind of gave you 105 00:03:38,760 --> 00:03:42,659 two advantages over the usual sort of 106 00:03:40,739 --> 00:03:44,400 Twitter protection mechanism that they 107 00:03:42,659 --> 00:03:46,799 had it gave you the ability to 108 00:03:44,400 --> 00:03:48,299 neutralize a large group of people who 109 00:03:46,799 --> 00:03:49,860 are sort of trying to Dogpile you which 110 00:03:48,299 --> 00:03:52,620 is one of Twitter's 111 00:03:49,860 --> 00:03:54,599 great Innovations and harassment 112 00:03:52,620 --> 00:03:56,519 um and it gave you the freedom to do it 113 00:03:54,599 --> 00:03:57,959 fairly ruthlessly because you knew that 114 00:03:56,519 --> 00:03:59,640 like because you could block people 115 00:03:57,959 --> 00:04:00,480 temporarily 116 00:03:59,640 --> 00:04:01,799 um 117 00:04:00,480 --> 00:04:03,959 you knew that if you accidentally 118 00:04:01,799 --> 00:04:05,519 blocked nice people while you were doing 119 00:04:03,959 --> 00:04:07,019 it they would automatically be unblocked 120 00:04:05,519 --> 00:04:08,459 eventually and I found that to be a 121 00:04:07,019 --> 00:04:10,319 really useful tool 122 00:04:08,459 --> 00:04:12,659 um in my original talk on the subject I 123 00:04:10,319 --> 00:04:16,199 mentioned that Twitter allows you to 124 00:04:12,659 --> 00:04:18,540 mute topics for a set period of time 125 00:04:16,199 --> 00:04:19,620 um so they like have mechanisms inbuilt 126 00:04:18,540 --> 00:04:21,120 they understand the value of being able 127 00:04:19,620 --> 00:04:23,600 to protect yourself temporarily from 128 00:04:21,120 --> 00:04:26,520 like Star Wars spoilers but not from 129 00:04:23,600 --> 00:04:27,479 Nazis I don't know 130 00:04:26,520 --> 00:04:30,720 um 131 00:04:27,479 --> 00:04:33,000 it was a really unsophisticated UI this 132 00:04:30,720 --> 00:04:35,580 is basically the only uh interactive 133 00:04:33,000 --> 00:04:37,020 piece of the entire app uh this is what 134 00:04:35,580 --> 00:04:38,520 it looks like because I'm a pretty good 135 00:04:37,020 --> 00:04:40,020 python programmer I'm pretty good 136 00:04:38,520 --> 00:04:42,479 back-end programmer 137 00:04:40,020 --> 00:04:46,560 and I am just I'm just useless at HTML 138 00:04:42,479 --> 00:04:49,860 and CSS and ux it's just not what I do 139 00:04:46,560 --> 00:04:51,660 so for the python developers here 140 00:04:49,860 --> 00:04:54,180 this app wasn't a lot to write home 141 00:04:51,660 --> 00:04:57,660 about okay four Django models sort of 142 00:04:54,180 --> 00:05:00,180 mirroring some of what um Twitter's data 143 00:04:57,660 --> 00:05:02,040 model looks like a couple a half a dozen 144 00:05:00,180 --> 00:05:05,340 or a dozen generic Django views you know 145 00:05:02,040 --> 00:05:07,979 a list detail model detail uh literally 146 00:05:05,340 --> 00:05:09,960 one form as which you just saw 147 00:05:07,979 --> 00:05:12,300 um a dozen or so celery tasks and that's 148 00:05:09,960 --> 00:05:15,000 where some of the clever stuff happened 149 00:05:12,300 --> 00:05:19,620 um a crappy bootstrap UI 150 00:05:15,000 --> 00:05:21,840 um not a lot to it uh but it worked 151 00:05:19,620 --> 00:05:23,639 pretty hard and the salary tasks were 152 00:05:21,840 --> 00:05:27,120 probably the more interesting part I'm 153 00:05:23,639 --> 00:05:31,680 gonna run through uh an encapsulated 154 00:05:27,120 --> 00:05:33,900 timeline of how this came about uh I 155 00:05:31,680 --> 00:05:37,380 first had the idea 156 00:05:33,900 --> 00:05:40,560 um in 2016 at LCA actually 157 00:05:37,380 --> 00:05:42,660 um when I was talking to a 158 00:05:40,560 --> 00:05:45,360 online activist who had been involved in 159 00:05:42,660 --> 00:05:48,060 protecting people from gamergate 160 00:05:45,360 --> 00:05:49,979 um and I had the idea of like maybe this 161 00:05:48,060 --> 00:05:52,199 would be something that would be useful 162 00:05:49,979 --> 00:05:53,639 for people 163 00:05:52,199 --> 00:05:55,919 and then I didn't do anything with it 164 00:05:53,639 --> 00:05:59,100 for two years because ideas are easy and 165 00:05:55,919 --> 00:06:00,780 implementation is hard uh in 2018 I 166 00:05:59,100 --> 00:06:03,840 finally started writing code and I wrote 167 00:06:00,780 --> 00:06:06,300 it very much because I wanted it 168 00:06:03,840 --> 00:06:08,460 um Twitter in 2018 was just post 169 00:06:06,300 --> 00:06:10,560 Charlotte charlottesburg there was like 170 00:06:08,460 --> 00:06:12,660 a lot of ugliness there 171 00:06:10,560 --> 00:06:13,919 um and yet it was a place where because 172 00:06:12,660 --> 00:06:16,020 of my toxic relationship with social 173 00:06:13,919 --> 00:06:18,780 media I spent a lot of time there 174 00:06:16,020 --> 00:06:20,220 um I wrote it in Django but I wrote it 175 00:06:18,780 --> 00:06:22,319 for myself and that meant that it was 176 00:06:20,220 --> 00:06:23,940 largely command line interface it was 177 00:06:22,319 --> 00:06:25,740 really just local database and stuff but 178 00:06:23,940 --> 00:06:27,419 I knew that it would be a benefit for 179 00:06:25,740 --> 00:06:28,500 other people so I knew eventually I was 180 00:06:27,419 --> 00:06:30,000 going to get it online but that would 181 00:06:28,500 --> 00:06:32,220 involve 182 00:06:30,000 --> 00:06:33,419 writing HTML and writing CSS and you 183 00:06:32,220 --> 00:06:34,680 know the stuff that I don't like doing 184 00:06:33,419 --> 00:06:35,940 that much 185 00:06:34,680 --> 00:06:39,060 um 186 00:06:35,940 --> 00:06:42,060 most of the development for secatura 187 00:06:39,060 --> 00:06:44,699 happened in fits and starts and like do 188 00:06:42,060 --> 00:06:46,560 nothing on it for months and then like a 189 00:06:44,699 --> 00:06:48,720 Sprint of like a day or a weekend's 190 00:06:46,560 --> 00:06:51,120 hyper focused work or or a week's worth 191 00:06:48,720 --> 00:06:52,259 of work a lot of it was conference 192 00:06:51,120 --> 00:06:55,259 driven development which I'm sure 193 00:06:52,259 --> 00:06:57,000 several of you are familiar with so it's 194 00:06:55,259 --> 00:06:59,340 not a coincidence that it only finally 195 00:06:57,000 --> 00:07:02,220 got launched like a couple days before 196 00:06:59,340 --> 00:07:03,660 Picone Au 2019 because I was giving a 197 00:07:02,220 --> 00:07:05,520 talk on it and that's like oh crap I 198 00:07:03,660 --> 00:07:08,039 better actually finish this thing 199 00:07:05,520 --> 00:07:09,840 um it had its first couple users who 200 00:07:08,039 --> 00:07:11,819 weren't me by then a couple people who 201 00:07:09,840 --> 00:07:13,919 needed to protect themselves from uh 202 00:07:11,819 --> 00:07:14,880 transphobic attacks 203 00:07:13,919 --> 00:07:16,639 um 204 00:07:14,880 --> 00:07:19,259 and 205 00:07:16,639 --> 00:07:22,860 effectively people first started signing 206 00:07:19,259 --> 00:07:25,139 into it during my talk in uh at pikon EU 207 00:07:22,860 --> 00:07:27,199 in 2019. by the end of the year it had a 208 00:07:25,139 --> 00:07:30,360 couple hundred users 209 00:07:27,199 --> 00:07:34,919 in 2020 things went a little bit more 210 00:07:30,360 --> 00:07:38,400 nuts uh lockdown happened uh everyone 211 00:07:34,919 --> 00:07:41,900 was spending more time on Twitter I 212 00:07:38,400 --> 00:07:45,620 in about let's see is my mouse on screen 213 00:07:41,900 --> 00:07:48,720 in about May 214 00:07:45,620 --> 00:07:49,979 I stopped it requiring an invitation to 215 00:07:48,720 --> 00:07:51,120 join like you could sign up but then I 216 00:07:49,979 --> 00:07:52,919 would have to push a button to enable 217 00:07:51,120 --> 00:07:54,240 your account and I just let people sign 218 00:07:52,919 --> 00:07:56,520 up freely 219 00:07:54,240 --> 00:07:59,099 um and I also 220 00:07:56,520 --> 00:08:02,099 changed one feature which was if I go 221 00:07:59,099 --> 00:08:03,780 back to that form I added the forever 222 00:08:02,099 --> 00:08:05,819 button there because I thought that the 223 00:08:03,780 --> 00:08:08,099 key feature was the temporary blocking 224 00:08:05,819 --> 00:08:09,419 but I added the ability to let someone 225 00:08:08,099 --> 00:08:11,479 block forever 226 00:08:09,419 --> 00:08:11,479 um 227 00:08:13,259 --> 00:08:17,880 usage started to spread and started to 228 00:08:15,960 --> 00:08:20,460 get popular and 229 00:08:17,880 --> 00:08:21,780 the server that I was running it on 230 00:08:20,460 --> 00:08:23,879 um started falling over all the time 231 00:08:21,780 --> 00:08:26,720 because I was running it on a T2 tiny 232 00:08:23,879 --> 00:08:29,520 then it teaches small then a T2 medium 233 00:08:26,720 --> 00:08:31,800 I had I I go into a lot of detail on 234 00:08:29,520 --> 00:08:34,440 this in a in a different talk but um I 235 00:08:31,800 --> 00:08:37,140 spent a lot of time 236 00:08:34,440 --> 00:08:41,279 working on keeping it running on one 237 00:08:37,140 --> 00:08:43,620 instance of one Amazon device because I 238 00:08:41,279 --> 00:08:45,000 wanted the cost to be predictable when 239 00:08:43,620 --> 00:08:46,860 you think about a technical platform 240 00:08:45,000 --> 00:08:48,180 like this you can easily design it in 241 00:08:46,860 --> 00:08:49,500 your head such that it would be like I 242 00:08:48,180 --> 00:08:52,500 don't know you'd use lambdas to do the 243 00:08:49,500 --> 00:08:53,760 API calls and you'd use dynamodb and you 244 00:08:52,500 --> 00:08:55,080 could make it infinitely horizontally 245 00:08:53,760 --> 00:08:56,700 scalable and therefore infinitely 246 00:08:55,080 --> 00:08:59,120 expensive which is a bad idea for a 247 00:08:56,700 --> 00:08:59,120 hobby project 248 00:08:59,459 --> 00:09:05,160 um it crashed a lot in 2020 and so the 249 00:09:02,279 --> 00:09:06,360 talk that I gave uh in 2020 at paikon EU 250 00:09:05,160 --> 00:09:08,100 is probably the most technical one on 251 00:09:06,360 --> 00:09:12,480 the subject because I I learned a lot 252 00:09:08,100 --> 00:09:14,100 about tuning postgres and AWS gp2 iops 253 00:09:12,480 --> 00:09:15,899 exhaustion caused by disk rights from 254 00:09:14,100 --> 00:09:17,220 the database and RAM exhaustion swapping 255 00:09:15,899 --> 00:09:19,279 to the disk 256 00:09:17,220 --> 00:09:19,279 um 257 00:09:19,320 --> 00:09:22,440 but yeah like I thought I had solved the 258 00:09:21,060 --> 00:09:23,220 scaling problems I thought I thought I 259 00:09:22,440 --> 00:09:25,320 didn't think it was going to get too 260 00:09:23,220 --> 00:09:26,700 much more popular than that I'm going to 261 00:09:25,320 --> 00:09:29,220 just zoom the graph out a little bit for 262 00:09:26,700 --> 00:09:31,260 the context of where we were 263 00:09:29,220 --> 00:09:34,560 um 264 00:09:31,260 --> 00:09:38,880 in by the end of 2021 it had 30 000 265 00:09:34,560 --> 00:09:41,040 users so it it it kind of it's it's 266 00:09:38,880 --> 00:09:41,820 spread entirely by like word of mouth I 267 00:09:41,040 --> 00:09:46,620 think 268 00:09:41,820 --> 00:09:47,700 um and it was basically just doing quite 269 00:09:46,620 --> 00:09:48,899 a lot of work 270 00:09:47,700 --> 00:09:51,720 um I didn't do pretty much any 271 00:09:48,899 --> 00:09:54,480 development on it in 2021 I was too busy 272 00:09:51,720 --> 00:09:56,519 um by the end of 2022 it had 60 000 273 00:09:54,480 --> 00:09:59,700 users and it was making roughly six 274 00:09:56,519 --> 00:10:01,620 million API calls uh to Twitter per day 275 00:09:59,700 --> 00:10:03,060 uh by this stage it was running on the 276 00:10:01,620 --> 00:10:05,100 T4 medium you know the budget was 277 00:10:03,060 --> 00:10:07,560 starting to creep up a little bit 278 00:10:05,100 --> 00:10:09,000 um in April of that year I did like one 279 00:10:07,560 --> 00:10:10,920 of those hyper focusing Sprint things I 280 00:10:09,000 --> 00:10:12,060 took a week off work ostensibly to have 281 00:10:10,920 --> 00:10:13,980 a holiday and then I spent the whole 282 00:10:12,060 --> 00:10:15,660 damn week programming 283 00:10:13,980 --> 00:10:17,519 um you know I upgraded at the Django 4 I 284 00:10:15,660 --> 00:10:20,339 upgraded to postgres 14 I reworked the 285 00:10:17,519 --> 00:10:21,660 database code to make it more efficient 286 00:10:20,339 --> 00:10:23,100 um I bootstrapped the whole thing into 287 00:10:21,660 --> 00:10:24,240 open Telemetry and started sending data 288 00:10:23,100 --> 00:10:25,500 to Honeycomb which I'll talk about more 289 00:10:24,240 --> 00:10:27,240 in a moment 290 00:10:25,500 --> 00:10:28,620 um I finally realized that I was an 291 00:10:27,240 --> 00:10:30,720 idiot for running this thing in Amazon 292 00:10:28,620 --> 00:10:32,880 Sydney because like you know I live near 293 00:10:30,720 --> 00:10:34,920 Sydney well in Wellington 294 00:10:32,880 --> 00:10:36,300 um I should run things in Sydney it's 295 00:10:34,920 --> 00:10:39,000 spending all of its time talking to the 296 00:10:36,300 --> 00:10:41,820 Twitter API where's the Twitter API 297 00:10:39,000 --> 00:10:43,500 California so I finally realized that 298 00:10:41,820 --> 00:10:45,120 actually if you're if you're gonna build 299 00:10:43,500 --> 00:10:47,640 something you might as well put it 300 00:10:45,120 --> 00:10:49,260 anyway you cut a lot of costs by moving 301 00:10:47,640 --> 00:10:50,220 it to a graviton instance and putting it 302 00:10:49,260 --> 00:10:50,820 in Oregon 303 00:10:50,220 --> 00:10:52,920 um 304 00:10:50,820 --> 00:10:54,000 and I set up Django Waffle and I set up 305 00:10:52,920 --> 00:10:55,740 a patreon 306 00:10:54,000 --> 00:10:58,560 um 307 00:10:55,740 --> 00:10:59,820 and then finally in 2023 Twitter began 308 00:10:58,560 --> 00:11:01,320 if those of those of you who use Twitter 309 00:10:59,820 --> 00:11:02,940 probably know it seems to be in 310 00:11:01,320 --> 00:11:05,459 basically a cultural relevance death 311 00:11:02,940 --> 00:11:06,959 cycle uh it started letting all of the 312 00:11:05,459 --> 00:11:08,940 extremists back on Twitter while at the 313 00:11:06,959 --> 00:11:11,220 same time sort of ending its ability to 314 00:11:08,940 --> 00:11:12,480 deal with abuse uh and they announced 315 00:11:11,220 --> 00:11:15,240 that they'd be deprecating all of their 316 00:11:12,480 --> 00:11:17,339 free apis in a attempt to sort of get 317 00:11:15,240 --> 00:11:20,279 more money from developers 318 00:11:17,339 --> 00:11:22,800 um the free API was cut off I was 319 00:11:20,279 --> 00:11:25,140 suspended from Twitter on a sorry my 320 00:11:22,800 --> 00:11:27,740 developer credentials were suspended uh 321 00:11:25,140 --> 00:11:29,880 on Twitter 322 00:11:27,740 --> 00:11:30,839 mid-april one week after I started my 323 00:11:29,880 --> 00:11:31,800 new job 324 00:11:30,839 --> 00:11:33,360 um 325 00:11:31,800 --> 00:11:35,220 and 326 00:11:33,360 --> 00:11:36,839 every single login attempt an API call 327 00:11:35,220 --> 00:11:39,540 started returning 500s it was 328 00:11:36,839 --> 00:11:43,320 effectively dead so that was basically 329 00:11:39,540 --> 00:11:49,079 the life of the app from 2019 sorry 2019 330 00:11:43,320 --> 00:11:51,180 to 2023 it died anonymously at night uh 331 00:11:49,079 --> 00:11:51,899 woke him in the morning it was broken 332 00:11:51,180 --> 00:11:54,000 um 333 00:11:51,899 --> 00:11:56,120 end of talk not quite 334 00:11:54,000 --> 00:11:56,120 um 335 00:11:56,399 --> 00:12:03,000 that's the gist of it 336 00:11:58,860 --> 00:12:04,140 um I'm gonna give you sort of a grab bag 337 00:12:03,000 --> 00:12:06,240 of things that kind of happened along 338 00:12:04,140 --> 00:12:08,100 the way first of all 339 00:12:06,240 --> 00:12:10,320 just some numbers for a side project 340 00:12:08,100 --> 00:12:13,560 like this that got kind of popular it 341 00:12:10,320 --> 00:12:15,540 cost me about 342 00:12:13,560 --> 00:12:17,820 not sure if that number is right it cost 343 00:12:15,540 --> 00:12:20,040 me about six thousand dollars to run 344 00:12:17,820 --> 00:12:23,640 for its entire lifespan 345 00:12:20,040 --> 00:12:26,579 um and about a year into it like in 2020 346 00:12:23,640 --> 00:12:27,140 I started accepting donations 347 00:12:26,579 --> 00:12:29,959 um 348 00:12:27,140 --> 00:12:32,760 and the donations 349 00:12:29,959 --> 00:12:35,160 added up to just over six thousand 350 00:12:32,760 --> 00:12:36,540 dollars so I just barely didn't lose any 351 00:12:35,160 --> 00:12:37,260 money on it 352 00:12:36,540 --> 00:12:40,440 um 353 00:12:37,260 --> 00:12:43,800 it brought in yeah so over three years 354 00:12:40,440 --> 00:12:45,959 it grew to 70 000 users as you saw uh 355 00:12:43,800 --> 00:12:48,660 since early 2020 when I started logging 356 00:12:45,959 --> 00:12:50,820 these it made about four billion Twitter 357 00:12:48,660 --> 00:12:51,540 API calls total 358 00:12:50,820 --> 00:12:52,980 um 359 00:12:51,540 --> 00:12:55,740 according to their current pricing plan 360 00:12:52,980 --> 00:12:57,180 that would be pretty expensive 361 00:12:55,740 --> 00:12:58,620 um that'd be in the millions of dollars 362 00:12:57,180 --> 00:12:59,880 so it's one of the reasons why it's not 363 00:12:58,620 --> 00:13:03,060 really 364 00:12:59,880 --> 00:13:05,279 um if you're wondering the most blocked 365 00:13:03,060 --> 00:13:07,260 person on the planet using the tool is 366 00:13:05,279 --> 00:13:09,420 someone you've never heard of 367 00:13:07,260 --> 00:13:11,519 um it's someone who it's probably like a 368 00:13:09,420 --> 00:13:13,200 bot account because they follow like 200 369 00:13:11,519 --> 00:13:14,700 000 people so it stands to reason that 370 00:13:13,200 --> 00:13:16,500 like the person who got blocked the most 371 00:13:14,700 --> 00:13:18,180 by a tool that blocks the followers of 372 00:13:16,500 --> 00:13:20,040 people is someone who just seems to 373 00:13:18,180 --> 00:13:21,120 follow everybody on Twitter 374 00:13:20,040 --> 00:13:24,060 um 375 00:13:21,120 --> 00:13:24,959 one user on secretary used the site 8 376 00:13:24,060 --> 00:13:28,880 000 times 377 00:13:24,959 --> 00:13:31,680 so in their in their usage of the site 378 00:13:28,880 --> 00:13:34,620 they asked to block the followers of 379 00:13:31,680 --> 00:13:36,540 people 8 000 times total another user 380 00:13:34,620 --> 00:13:37,980 triggered 10 million API calls on their 381 00:13:36,540 --> 00:13:41,339 own so they they probably blocked a 382 00:13:37,980 --> 00:13:42,180 total of 10 million people on Twitter 383 00:13:41,339 --> 00:13:45,480 um 384 00:13:42,180 --> 00:13:48,540 so I'm going to talk a little bit about 385 00:13:45,480 --> 00:13:51,240 why this was good for me I guess 386 00:13:48,540 --> 00:13:52,680 um I learned a lot of stuff doing this 387 00:13:51,240 --> 00:13:54,540 um some of it I was already pretty good 388 00:13:52,680 --> 00:13:56,519 at and some of it I absolutely wasn't um 389 00:13:54,540 --> 00:13:58,200 I got a lot better at using redis for 390 00:13:56,519 --> 00:14:02,459 celery and quiche I learned a lot more 391 00:13:58,200 --> 00:14:03,899 about tuning postgres for performance 392 00:14:02,459 --> 00:14:05,519 um some pretty cool Advanced celery 393 00:14:03,899 --> 00:14:07,980 patterns because the hard parts of the 394 00:14:05,519 --> 00:14:11,880 celery work was um 395 00:14:07,980 --> 00:14:13,740 uh handling the rate limiting and the 396 00:14:11,880 --> 00:14:15,660 back off algorithms and just sort of the 397 00:14:13,740 --> 00:14:18,120 Fanning out of 398 00:14:15,660 --> 00:14:19,980 this call triggers 100 calls to get 399 00:14:18,120 --> 00:14:22,260 pages of followers which then trigger 400 00:14:19,980 --> 00:14:23,279 5000 tasks each to do the block the 401 00:14:22,260 --> 00:14:24,899 blocks of the followers and sort of 402 00:14:23,279 --> 00:14:27,000 tuning all that stuff 403 00:14:24,899 --> 00:14:29,399 um I got a lot better at Docker and 404 00:14:27,000 --> 00:14:31,139 Docker compose uh because that was the 405 00:14:29,399 --> 00:14:32,519 production platform was just a Docker 406 00:14:31,139 --> 00:14:35,880 compose file 407 00:14:32,519 --> 00:14:38,220 um running on an ec2 instance uh my pie 408 00:14:35,880 --> 00:14:39,839 for Django stuff struct log a lot of 409 00:14:38,220 --> 00:14:41,220 these things I then brought into my 410 00:14:39,839 --> 00:14:42,660 workplace 411 00:14:41,220 --> 00:14:44,100 um which was really handy it was like oh 412 00:14:42,660 --> 00:14:46,860 how did you know about this cool tool 413 00:14:44,100 --> 00:14:48,959 well I built it over here struct log 414 00:14:46,860 --> 00:14:51,300 open Telemetry honeycomb 415 00:14:48,959 --> 00:14:53,519 um psycho pg2 instrumentation and 416 00:14:51,300 --> 00:14:56,699 traffic for sort of load loading um 417 00:14:53,519 --> 00:14:58,500 sorry front-end load balancing stuff 418 00:14:56,699 --> 00:15:00,360 um okay 419 00:14:58,500 --> 00:15:01,380 I'm going to jump topics a little bit 420 00:15:00,360 --> 00:15:02,339 for the next little bit because I'm just 421 00:15:01,380 --> 00:15:04,500 going to talk about you know my favorite 422 00:15:02,339 --> 00:15:06,480 parts of what kind of happened here 423 00:15:04,500 --> 00:15:09,800 um the reason it stopped falling over 424 00:15:06,480 --> 00:15:14,160 dead was because I finally put user 425 00:15:09,800 --> 00:15:16,260 Centric rate limits onto secature um by 426 00:15:14,160 --> 00:15:18,480 far the biggest early mistake I made was 427 00:15:16,260 --> 00:15:20,820 not setting per user usage controls so 428 00:15:18,480 --> 00:15:23,519 that like this person over here 429 00:15:20,820 --> 00:15:25,800 who really needs the app because they're 430 00:15:23,519 --> 00:15:27,120 being attacked can't use it because this 431 00:15:25,800 --> 00:15:29,579 person over here 432 00:15:27,120 --> 00:15:31,260 has triggered the blocking of 15 million 433 00:15:29,579 --> 00:15:33,120 people and so it's going to take six 434 00:15:31,260 --> 00:15:34,740 days for that backlog to drain before 435 00:15:33,120 --> 00:15:35,579 coming over here 436 00:15:34,740 --> 00:15:37,740 um 437 00:15:35,579 --> 00:15:38,820 this is not interesting code but it's my 438 00:15:37,740 --> 00:15:40,260 favorite code on the whole thing because 439 00:15:38,820 --> 00:15:41,579 it was sort of simple and elegant and I 440 00:15:40,260 --> 00:15:43,019 got to use high school math 441 00:15:41,579 --> 00:15:44,459 who here has ever used high school math 442 00:15:43,019 --> 00:15:45,779 it's amazing 443 00:15:44,459 --> 00:15:46,800 okay some of you have used high school 444 00:15:45,779 --> 00:15:48,000 math fine 445 00:15:46,800 --> 00:15:50,459 [Applause] 446 00:15:48,000 --> 00:15:51,899 I I don't get to use high school math I 447 00:15:50,459 --> 00:15:54,120 was very excited it's like it's got a 448 00:15:51,899 --> 00:15:57,000 gradient you know like the the gradient 449 00:15:54,120 --> 00:15:58,620 thing and like anyway 450 00:15:57,000 --> 00:16:00,959 um the original the original account 451 00:15:58,620 --> 00:16:02,639 controls on the site where 452 00:16:00,959 --> 00:16:03,779 Tom has to enable your account first 453 00:16:02,639 --> 00:16:05,220 that's the thing that I got rid of 454 00:16:03,779 --> 00:16:06,959 pretty early you're not allowed to block 455 00:16:05,220 --> 00:16:09,420 all of your own followers because that 456 00:16:06,959 --> 00:16:12,660 would be a recipe for some pain 457 00:16:09,420 --> 00:16:14,699 um and I had to have a limit on like 458 00:16:12,660 --> 00:16:16,079 you can't block someone you can't block 459 00:16:14,699 --> 00:16:18,660 all the followers of someone with like 460 00:16:16,079 --> 00:16:19,980 over 500 000 followers because once you 461 00:16:18,660 --> 00:16:21,180 get into the millions it's just not 462 00:16:19,980 --> 00:16:22,440 practical with the tools that Twitter 463 00:16:21,180 --> 00:16:23,940 gives you 464 00:16:22,440 --> 00:16:25,740 um for example to block all of Donald 465 00:16:23,940 --> 00:16:27,000 Trump's followers you'd have to with 466 00:16:25,740 --> 00:16:29,220 Twitter's rate limits it would take 467 00:16:27,000 --> 00:16:31,920 about seven weeks just to download the 468 00:16:29,220 --> 00:16:33,360 list so it's just not really practical 469 00:16:31,920 --> 00:16:36,120 um 470 00:16:33,360 --> 00:16:37,920 but once I built this mechanism I was 471 00:16:36,120 --> 00:16:39,660 able to keep the site online by tuning 472 00:16:37,920 --> 00:16:42,000 this and that's far better than like 473 00:16:39,660 --> 00:16:44,519 trying to work out oh if I add more 474 00:16:42,000 --> 00:16:47,160 threads or if I switch to G event or um 475 00:16:44,519 --> 00:16:49,980 or or is it finally time to do the 476 00:16:47,160 --> 00:16:51,500 dynamodb Lambda thing 477 00:16:49,980 --> 00:16:54,360 um 478 00:16:51,500 --> 00:16:56,160 eventually with a lot of experimentation 479 00:16:54,360 --> 00:16:58,259 on these rate limits what I settled for 480 00:16:56,160 --> 00:17:00,480 was um 481 00:16:58,259 --> 00:17:03,779 if you first signed up to secretary you 482 00:17:00,480 --> 00:17:05,339 had a bucket and you and your bucket was 483 00:17:03,779 --> 00:17:07,559 full and you could block a lot of people 484 00:17:05,339 --> 00:17:09,720 with the tokens in that bucket and then 485 00:17:07,559 --> 00:17:11,760 the tokens refilled quite slowly so 486 00:17:09,720 --> 00:17:13,620 maybe you could block 200 000 people 487 00:17:11,760 --> 00:17:16,020 when you first join 488 00:17:13,620 --> 00:17:17,459 um but then from then on it only refills 489 00:17:16,020 --> 00:17:21,199 at the rate of like five thousand a day 490 00:17:17,459 --> 00:17:22,760 and that seemed to be a very 491 00:17:21,199 --> 00:17:25,500 sustainable 492 00:17:22,760 --> 00:17:26,760 use model for this 493 00:17:25,500 --> 00:17:28,319 um but all of that just comes from 494 00:17:26,760 --> 00:17:30,780 experimentation and what was really 495 00:17:28,319 --> 00:17:32,340 valuable was just having the ability to 496 00:17:30,780 --> 00:17:33,660 if someone came to me and said hey I'm 497 00:17:32,340 --> 00:17:34,559 being attacked by this person over here 498 00:17:33,660 --> 00:17:36,780 and they actually have a ton of 499 00:17:34,559 --> 00:17:38,280 followers can you please help 500 00:17:36,780 --> 00:17:40,080 knowing that there was capacity in the 501 00:17:38,280 --> 00:17:41,580 system for me to just say like 502 00:17:40,080 --> 00:17:43,380 anyone who ever asked me that I was able 503 00:17:41,580 --> 00:17:46,080 to go yep go for it I've just refilled 504 00:17:43,380 --> 00:17:47,460 your your rate limit just go nuts um 505 00:17:46,080 --> 00:17:49,980 and it meant that all the sort of 506 00:17:47,460 --> 00:17:52,940 drive-by background usage uh didn't 507 00:17:49,980 --> 00:17:52,940 bring the whole thing down 508 00:17:53,940 --> 00:17:58,919 I told you that I didn't do basically 509 00:17:55,919 --> 00:18:00,960 any work on the in 2021 and that was 510 00:17:58,919 --> 00:18:03,299 pretty much because 511 00:18:00,960 --> 00:18:05,400 it got really unwieldy and scary to do 512 00:18:03,299 --> 00:18:06,840 because the database by then was really 513 00:18:05,400 --> 00:18:09,660 big and it was on a really small 514 00:18:06,840 --> 00:18:11,039 instance and it was very difficult to 515 00:18:09,660 --> 00:18:12,419 test production conditions it was 516 00:18:11,039 --> 00:18:14,700 impossible to test production conditions 517 00:18:12,419 --> 00:18:15,720 and I thought that that meant that I was 518 00:18:14,700 --> 00:18:16,860 failing as a developer right because 519 00:18:15,720 --> 00:18:18,299 you're supposed to be able to do good 520 00:18:16,860 --> 00:18:20,640 testing do good unit testing do good 521 00:18:18,299 --> 00:18:22,200 integration testing do good load testing 522 00:18:20,640 --> 00:18:24,000 in practice 523 00:18:22,200 --> 00:18:26,039 that's really hard 524 00:18:24,000 --> 00:18:28,200 um and when I was trying to build new 525 00:18:26,039 --> 00:18:29,880 features or things for this a single 526 00:18:28,200 --> 00:18:31,740 sequential scan would basically bring 527 00:18:29,880 --> 00:18:33,240 down the whole server because the disk 528 00:18:31,740 --> 00:18:35,039 IO would be way too much for the tiny 529 00:18:33,240 --> 00:18:37,140 instance it was running on 530 00:18:35,039 --> 00:18:39,299 um and if you're using postgres as your 531 00:18:37,140 --> 00:18:40,620 backing store you don't necessarily know 532 00:18:39,299 --> 00:18:42,440 if you've built something that's not 533 00:18:40,620 --> 00:18:44,580 going to use a sequential scan because 534 00:18:42,440 --> 00:18:45,780 postgres's query plan or behavior 535 00:18:44,580 --> 00:18:50,120 changes depending on the size of your 536 00:18:45,780 --> 00:18:50,120 tables so when you are working from home 537 00:18:53,000 --> 00:18:58,020 when you're working on your local laptop 538 00:18:56,400 --> 00:18:59,820 with a couple hundred rows or a couple 539 00:18:58,020 --> 00:19:02,700 thousand rows postgres is going to 540 00:18:59,820 --> 00:19:04,140 behave differently to when you have 150 541 00:19:02,700 --> 00:19:07,440 million or 2 billion rows in your 542 00:19:04,140 --> 00:19:09,179 database uh so it's just very hard to 543 00:19:07,440 --> 00:19:13,559 deal with 544 00:19:09,179 --> 00:19:15,240 at LCA last year at the online LCA uh 545 00:19:13,559 --> 00:19:16,980 Liz Fong Jones gave a keynote address on 546 00:19:15,240 --> 00:19:19,620 observability engineering and it really 547 00:19:16,980 --> 00:19:21,360 opened my eyes to some cool new stuff 548 00:19:19,620 --> 00:19:22,380 um the first one was open Telemetry 549 00:19:21,360 --> 00:19:24,840 which I'm going to be giving a talk 550 00:19:22,380 --> 00:19:27,000 about at kiwi pycon next month which you 551 00:19:24,840 --> 00:19:29,039 guys should come to it'll be really cool 552 00:19:27,000 --> 00:19:29,760 um 553 00:19:29,039 --> 00:19:30,900 um 554 00:19:29,760 --> 00:19:33,660 I'm going to give a talk on open 555 00:19:30,900 --> 00:19:35,280 Telemetry there open Telemetry gave me 556 00:19:33,660 --> 00:19:37,679 the ability to see exactly where the 557 00:19:35,280 --> 00:19:39,720 slow and fast parts were in the running 558 00:19:37,679 --> 00:19:40,559 production code 559 00:19:39,720 --> 00:19:43,500 um 560 00:19:40,559 --> 00:19:45,600 but that's not the only topic that Liz 561 00:19:43,500 --> 00:19:46,799 discusses in in her keynote because the 562 00:19:45,600 --> 00:19:50,340 other aspect of observability 563 00:19:46,799 --> 00:19:52,260 engineering is the tacit admission that 564 00:19:50,340 --> 00:19:53,820 there's no such environment action you 565 00:19:52,260 --> 00:19:55,200 simply can't simulate production 566 00:19:53,820 --> 00:19:57,480 anywhere else and so you just have to 567 00:19:55,200 --> 00:19:59,460 learn how to safely develop on 568 00:19:57,480 --> 00:20:01,559 production she's talking about things at 569 00:19:59,460 --> 00:20:03,419 the scale of Google and stuff but it 570 00:20:01,559 --> 00:20:04,740 really counted for something like what I 571 00:20:03,419 --> 00:20:07,700 was dealing with where 572 00:20:04,740 --> 00:20:07,700 the um 573 00:20:08,580 --> 00:20:13,559 the constraints that I was under were a 574 00:20:11,100 --> 00:20:14,760 bit weird they were smaller but they 575 00:20:13,559 --> 00:20:17,160 were weird because the database was 576 00:20:14,760 --> 00:20:19,559 really big on a really small box 577 00:20:17,160 --> 00:20:21,240 um to be able to develop on production I 578 00:20:19,559 --> 00:20:22,559 put Django waffle in and started doing 579 00:20:21,240 --> 00:20:23,760 feature-based development where I could 580 00:20:22,559 --> 00:20:24,960 deploy it and I could roll it out to 581 00:20:23,760 --> 00:20:26,580 just myself 582 00:20:24,960 --> 00:20:27,660 make it work for just myself make it 583 00:20:26,580 --> 00:20:29,580 work for two people make it work for 584 00:20:27,660 --> 00:20:31,799 four then one percent of the user base 585 00:20:29,580 --> 00:20:32,760 then two then four then eight 586 00:20:31,799 --> 00:20:34,980 um 587 00:20:32,760 --> 00:20:37,440 it meant that I was able to get a crap 588 00:20:34,980 --> 00:20:40,320 ton done that I simply was too scared to 589 00:20:37,440 --> 00:20:41,760 do previously so I felt like that really 590 00:20:40,320 --> 00:20:45,059 changed what I was able to do and I've 591 00:20:41,760 --> 00:20:46,679 I'm a real sort of evangelist for it now 592 00:20:45,059 --> 00:20:49,679 um 593 00:20:46,679 --> 00:20:49,679 foreign 594 00:20:49,760 --> 00:20:53,299 built this tool for myself 595 00:20:53,580 --> 00:20:56,580 but I knew other people would want to 596 00:20:55,140 --> 00:20:57,960 use it but I was only half right about 597 00:20:56,580 --> 00:20:59,340 what people wanted being able to block 598 00:20:57,960 --> 00:21:01,799 all the followers of someone attacking 599 00:20:59,340 --> 00:21:02,400 you was incredibly valuable 600 00:21:01,799 --> 00:21:04,320 um 601 00:21:02,400 --> 00:21:06,480 but I thought the killer feature was the 602 00:21:04,320 --> 00:21:10,640 temporary block I thought that was 603 00:21:06,480 --> 00:21:14,419 absolutely what made the tool useful and 604 00:21:10,640 --> 00:21:17,400 nobody gave a crap about that everybody 605 00:21:14,419 --> 00:21:18,780 the overwhelming majority of the use was 606 00:21:17,400 --> 00:21:20,039 blocking people forever they just didn't 607 00:21:18,780 --> 00:21:22,320 care about the one thing that I cared 608 00:21:20,039 --> 00:21:24,120 about so I'm sure that a product manager 609 00:21:22,320 --> 00:21:26,000 type person will 610 00:21:24,120 --> 00:21:28,140 have a good lesson for you there about 611 00:21:26,000 --> 00:21:30,600 measuring what people want I don't know 612 00:21:28,140 --> 00:21:33,860 it I was I I built what I wanted and it 613 00:21:30,600 --> 00:21:33,860 was mostly what other people wanted 614 00:21:35,340 --> 00:21:38,039 it took me a little while to come around 615 00:21:36,659 --> 00:21:39,539 to asking for money but I'm glad I did 616 00:21:38,039 --> 00:21:41,159 because it didn't stay as cheap to run 617 00:21:39,539 --> 00:21:44,580 as I would have liked 618 00:21:41,159 --> 00:21:47,520 um I never intended to monetize the site 619 00:21:44,580 --> 00:21:49,740 I never wanted to like charge people who 620 00:21:47,520 --> 00:21:51,480 needed protection who needed the tool 621 00:21:49,740 --> 00:21:53,159 but what I really hoped was that the 622 00:21:51,480 --> 00:21:55,620 people who um 623 00:21:53,159 --> 00:21:57,780 who who could afford to donate 624 00:21:55,620 --> 00:21:59,760 would do so to support the people who 625 00:21:57,780 --> 00:22:01,380 couldn't it worked out but not really 626 00:21:59,760 --> 00:22:04,200 how I wanted I'm still a little bit sad 627 00:22:01,380 --> 00:22:06,900 about this but um well sad and grateful 628 00:22:04,200 --> 00:22:08,520 the overwhelming financial support that 629 00:22:06,900 --> 00:22:10,559 came to the tool was people like 630 00:22:08,520 --> 00:22:12,600 yourselves it was friends of mine who 631 00:22:10,559 --> 00:22:15,480 knew me it wasn't necessarily the people 632 00:22:12,600 --> 00:22:17,400 who were using the site so much 633 00:22:15,480 --> 00:22:19,080 um that was an interesting lesson I 634 00:22:17,400 --> 00:22:21,720 don't really know what to take from that 635 00:22:19,080 --> 00:22:22,440 uh apart from thank you 636 00:22:21,720 --> 00:22:24,539 um 637 00:22:22,440 --> 00:22:26,400 thank you this couldn't really have it 638 00:22:24,539 --> 00:22:29,340 couldn't have helped as many people as 639 00:22:26,400 --> 00:22:31,880 it did without the support of of the 640 00:22:29,340 --> 00:22:35,900 people who did financially support it 641 00:22:31,880 --> 00:22:35,900 but it was yeah it was 642 00:22:36,299 --> 00:22:39,659 it would have been like a lot of people 643 00:22:37,860 --> 00:22:41,520 who gave me good financial support 644 00:22:39,659 --> 00:22:44,940 didn't need the tool so I'm glad they 645 00:22:41,520 --> 00:22:47,000 supported it and yet I kind of yeah 646 00:22:44,940 --> 00:22:47,000 um 647 00:22:47,039 --> 00:22:50,159 I've I discovered that some of the 648 00:22:48,419 --> 00:22:53,780 weirder some of the use cases that 649 00:22:50,159 --> 00:22:53,780 happened just weren't um 650 00:22:54,860 --> 00:23:00,659 what I think even Twitter expected 651 00:22:58,380 --> 00:23:02,580 one thing that I learned that caused me 652 00:23:00,659 --> 00:23:05,159 a lot of problems with the site is that 653 00:23:02,580 --> 00:23:07,200 um there's a gigantic subculture on 654 00:23:05,159 --> 00:23:08,700 Twitter that are constantly changing 655 00:23:07,200 --> 00:23:11,880 their usernames 656 00:23:08,700 --> 00:23:13,320 like they put on and take off usernames 657 00:23:11,880 --> 00:23:14,820 like hats in the morning they're just 658 00:23:13,320 --> 00:23:15,900 constantly changing them and I get the 659 00:23:14,820 --> 00:23:19,440 impression that Twitter didn't really 660 00:23:15,900 --> 00:23:21,419 realize that either because uh 661 00:23:19,440 --> 00:23:22,620 the oauth credentials invalidate 662 00:23:21,419 --> 00:23:24,480 themselves every time someone changes 663 00:23:22,620 --> 00:23:25,799 their screen names so that would break a 664 00:23:24,480 --> 00:23:27,480 lot of the mechanisms on the site the 665 00:23:25,799 --> 00:23:28,500 unblock on Secretary of the unblocking 666 00:23:27,480 --> 00:23:30,900 thing 667 00:23:28,500 --> 00:23:34,340 um and it just was not a use case that I 668 00:23:30,900 --> 00:23:34,340 uh ever really expected 669 00:23:35,039 --> 00:23:40,500 so 670 00:23:37,080 --> 00:23:42,659 discussion that I had in 2019 about 671 00:23:40,500 --> 00:23:44,700 how almost all 672 00:23:42,659 --> 00:23:49,020 abuse protection mechanisms can be 673 00:23:44,700 --> 00:23:50,220 weaponized for abuse and I felt that 674 00:23:49,020 --> 00:23:52,220 this couldn't because I felt that 675 00:23:50,220 --> 00:23:54,720 blocking people was one of the things 676 00:23:52,220 --> 00:23:56,700 that couldn't really be weaponized 677 00:23:54,720 --> 00:23:59,220 against someone because it really is 678 00:23:56,700 --> 00:24:00,960 curating your own online experience 679 00:23:59,220 --> 00:24:03,240 and I'm pretty sure I was right about 680 00:24:00,960 --> 00:24:06,020 that but last year there was an 681 00:24:03,240 --> 00:24:09,419 interesting thing that 682 00:24:06,020 --> 00:24:12,260 there's there's an argument that could 683 00:24:09,419 --> 00:24:14,640 be made that it 684 00:24:12,260 --> 00:24:17,520 some people who I wish didn't use 685 00:24:14,640 --> 00:24:18,960 secretary used it for a benefit so I'm 686 00:24:17,520 --> 00:24:21,720 going to tell you about how my life got 687 00:24:18,960 --> 00:24:23,400 a little bit more surreal uh about a 688 00:24:21,720 --> 00:24:25,020 year ago 689 00:24:23,400 --> 00:24:27,240 um during 690 00:24:25,020 --> 00:24:28,919 the early days of the conflict of the 691 00:24:27,240 --> 00:24:32,000 Russia Ukraine war 692 00:24:28,919 --> 00:24:32,000 um there's a gigantic 693 00:24:32,039 --> 00:24:39,059 group of people on Twitter who are 694 00:24:35,280 --> 00:24:41,880 sort of harassing people who are 695 00:24:39,059 --> 00:24:44,039 spouting a lot of Russian propaganda 696 00:24:41,880 --> 00:24:46,679 um and 697 00:24:44,039 --> 00:24:48,659 I discovered this when I discovered that 698 00:24:46,679 --> 00:24:52,080 my tool secretary got really really 699 00:24:48,659 --> 00:24:54,299 popular among Russian propagandists 700 00:24:52,080 --> 00:24:56,700 for blocking the attacks from this other 701 00:24:54,299 --> 00:24:57,780 group of people I don't know if these 702 00:24:56,700 --> 00:24:59,940 are very readable but you don't really 703 00:24:57,780 --> 00:25:01,860 need to worry about it too much things 704 00:24:59,940 --> 00:25:04,679 started to get a little bit weird 705 00:25:01,860 --> 00:25:06,900 uh a lot of people were starting to 706 00:25:04,679 --> 00:25:08,820 notice that 707 00:25:06,900 --> 00:25:10,860 this group of people were using secateur 708 00:25:08,820 --> 00:25:12,659 and going okay well what's the deal with 709 00:25:10,860 --> 00:25:13,860 that tool who's who's made that what's 710 00:25:12,659 --> 00:25:16,200 going on 711 00:25:13,860 --> 00:25:18,299 um I felt a bit weird about this because 712 00:25:16,200 --> 00:25:19,620 like on the one hand I do not at all 713 00:25:18,299 --> 00:25:22,980 support 714 00:25:19,620 --> 00:25:24,960 the Russian invasion of Ukraine and on 715 00:25:22,980 --> 00:25:27,659 the other hand I built a tool that was 716 00:25:24,960 --> 00:25:30,240 designed to stop people from harassing 717 00:25:27,659 --> 00:25:32,159 people on Twitter and this group of 718 00:25:30,240 --> 00:25:34,860 people were you know harassment on 719 00:25:32,159 --> 00:25:36,659 Twitter so it was just 720 00:25:34,860 --> 00:25:38,100 people using the tool for what it was 721 00:25:36,659 --> 00:25:38,940 used for 722 00:25:38,100 --> 00:25:41,220 um 723 00:25:38,940 --> 00:25:43,080 at a certain point they decided to start 724 00:25:41,220 --> 00:25:43,980 looking into me 725 00:25:43,080 --> 00:25:46,559 um 726 00:25:43,980 --> 00:25:50,220 so I woke up one morning about a year 727 00:25:46,559 --> 00:25:51,659 ago and suddenly had a whole lot of 728 00:25:50,220 --> 00:25:53,159 things in my inbox about how I was 729 00:25:51,659 --> 00:25:56,159 helping the Russians and like who was I 730 00:25:53,159 --> 00:25:58,980 and was I a Russian spy uh 731 00:25:56,159 --> 00:26:02,340 was I was I an agent for the Russians 732 00:25:58,980 --> 00:26:05,340 things like that it got a bit weird they 733 00:26:02,340 --> 00:26:07,500 started digging into my websites and a 734 00:26:05,340 --> 00:26:08,820 few other bits and pieces 735 00:26:07,500 --> 00:26:11,340 um 736 00:26:08,820 --> 00:26:13,380 at one point they messaged some people 737 00:26:11,340 --> 00:26:15,120 from who claimed to be from Anonymous 738 00:26:13,380 --> 00:26:16,799 and those people sort of said oh look 739 00:26:15,120 --> 00:26:20,220 like this guy's clearly not a very good 740 00:26:16,799 --> 00:26:22,559 developer the SSL cert on his Blog has 741 00:26:20,220 --> 00:26:24,000 expired uh he clearly doesn't care much 742 00:26:22,559 --> 00:26:25,500 about security 743 00:26:24,000 --> 00:26:26,760 um I was staying way out of this because 744 00:26:25,500 --> 00:26:28,260 there was nothing that I could say that 745 00:26:26,760 --> 00:26:29,580 would like alleviate it either side 746 00:26:28,260 --> 00:26:31,440 because I thought either side was kind 747 00:26:29,580 --> 00:26:32,039 of problematic 748 00:26:31,440 --> 00:26:33,779 um 749 00:26:32,039 --> 00:26:35,220 but I really wanted to reply and say 750 00:26:33,779 --> 00:26:37,500 dude 751 00:26:35,220 --> 00:26:38,700 I get I get an email every 90 days from 752 00:26:37,500 --> 00:26:40,140 my friend Lee 753 00:26:38,700 --> 00:26:41,820 to tell me that my certificate has 754 00:26:40,140 --> 00:26:43,980 expired on my blog it's the only way I 755 00:26:41,820 --> 00:26:47,120 know anyone and anyone still reads it 756 00:26:43,980 --> 00:26:47,120 and so like 757 00:26:49,620 --> 00:26:54,720 things got weirder because 758 00:26:52,200 --> 00:26:55,740 on the Russian side they started saying 759 00:26:54,720 --> 00:26:58,320 hey how come we never heard about this 760 00:26:55,740 --> 00:27:00,360 tool until a few weeks ago maybe it's a 761 00:26:58,320 --> 00:27:04,860 false flag operation by the ukrainians 762 00:27:00,360 --> 00:27:07,200 to get rip addresses so and then so they 763 00:27:04,860 --> 00:27:09,360 started um 764 00:27:07,200 --> 00:27:12,059 sort of playing mind games with each 765 00:27:09,360 --> 00:27:15,059 other about whose side I was on at this 766 00:27:12,059 --> 00:27:18,720 point though I went and talked to uh ZX 767 00:27:15,059 --> 00:27:21,240 security who uh run kiwicon and koi Khan 768 00:27:18,720 --> 00:27:22,500 um and our friends of mine from 769 00:27:21,240 --> 00:27:23,220 Wellington 770 00:27:22,500 --> 00:27:24,900 um 771 00:27:23,220 --> 00:27:27,360 just sort of go okay well look if this 772 00:27:24,900 --> 00:27:29,520 becomes a bit more targeted like how 773 00:27:27,360 --> 00:27:32,640 much of my data really is online I was 774 00:27:29,520 --> 00:27:35,220 talking to Simon and he said well Tom do 775 00:27:32,640 --> 00:27:38,159 you still live on um Acura drive and I 776 00:27:35,220 --> 00:27:39,360 was like no but [ __ ] how did you find 777 00:27:38,159 --> 00:27:42,000 that 778 00:27:39,360 --> 00:27:43,200 um I had gone through like my link I've 779 00:27:42,000 --> 00:27:44,520 gone through my LinkedIn my run keeper 780 00:27:43,200 --> 00:27:45,840 all those things like strive I turned 781 00:27:44,520 --> 00:27:47,340 off all the maps I turned off all the 782 00:27:45,840 --> 00:27:49,080 friend sharing I turned off everything 783 00:27:47,340 --> 00:27:50,520 except you know made everything private 784 00:27:49,080 --> 00:27:53,039 mode for a little while 785 00:27:50,520 --> 00:27:54,900 um but it turns out that my phone number 786 00:27:53,039 --> 00:27:56,640 and my home address 787 00:27:54,900 --> 00:27:57,659 my old home address because I hadn't 788 00:27:56,640 --> 00:27:58,860 updated in a while was still like 789 00:27:57,659 --> 00:28:00,840 registered on a bunch of my New Zealand 790 00:27:58,860 --> 00:28:04,200 domains because New Zealand only 791 00:28:00,840 --> 00:28:05,640 recently added privacy protections to 792 00:28:04,200 --> 00:28:08,279 their who is database but you have to 793 00:28:05,640 --> 00:28:09,960 opt into them uh and I neglected to do 794 00:28:08,279 --> 00:28:11,580 so for some of my some of my personal 795 00:28:09,960 --> 00:28:14,159 domains 796 00:28:11,580 --> 00:28:17,159 um finally things got wow no we're 797 00:28:14,159 --> 00:28:19,100 almost to the end of the weirdness 798 00:28:17,159 --> 00:28:22,320 um 799 00:28:19,100 --> 00:28:24,299 some of the nepho people then said to 800 00:28:22,320 --> 00:28:27,179 the Russian people that I had spoken to 801 00:28:24,299 --> 00:28:29,400 the gcsb uh 802 00:28:27,179 --> 00:28:30,539 to hand over my database now I do want 803 00:28:29,400 --> 00:28:32,460 to just point out that the database just 804 00:28:30,539 --> 00:28:34,260 had public data in it anyway right like 805 00:28:32,460 --> 00:28:36,059 just but 806 00:28:34,260 --> 00:28:37,440 they started saying oh yeah I just got 807 00:28:36,059 --> 00:28:38,760 off the phone with Tom Eastman in New 808 00:28:37,440 --> 00:28:40,320 Zealand the developer of the tool and he 809 00:28:38,760 --> 00:28:42,840 has said that he is in touch with the 810 00:28:40,320 --> 00:28:44,820 special intelligence service 811 00:28:42,840 --> 00:28:46,740 um 812 00:28:44,820 --> 00:28:49,580 again like I just didn't reply to any of 813 00:28:46,740 --> 00:28:49,580 this stuff but um 814 00:28:51,299 --> 00:28:55,380 I I I worked in Security in Wellington 815 00:28:53,940 --> 00:28:56,760 and you have friends who work in 816 00:28:55,380 --> 00:28:57,900 Security in Wellington every once in a 817 00:28:56,760 --> 00:28:59,279 while they work for they start working 818 00:28:57,900 --> 00:29:00,360 for the government and you sort of oh so 819 00:28:59,279 --> 00:29:01,919 what department of the government do you 820 00:29:00,360 --> 00:29:04,799 work for and they get really cagey about 821 00:29:01,919 --> 00:29:07,380 it they're like just the government 822 00:29:04,799 --> 00:29:10,020 just the government and and so one of my 823 00:29:07,380 --> 00:29:12,299 friends messaged me one morning he just 824 00:29:10,020 --> 00:29:15,600 pointed to that tweet 825 00:29:12,299 --> 00:29:18,659 and he said Tom what the [ __ ] is this 826 00:29:15,600 --> 00:29:21,419 and I think the I think I replied with 827 00:29:18,659 --> 00:29:23,580 like a a face palm Emoji 828 00:29:21,419 --> 00:29:27,419 um but the the biggest irony of all this 829 00:29:23,580 --> 00:29:29,760 is yes because of this weird [ __ ] uh I 830 00:29:27,419 --> 00:29:31,620 did actually have someone from a 831 00:29:29,760 --> 00:29:35,340 government Department come and talk to 832 00:29:31,620 --> 00:29:38,520 me about uh they're like like 833 00:29:35,340 --> 00:29:40,860 he yeah he was like Tom what have you 834 00:29:38,520 --> 00:29:43,020 got yourself into like dude I don't even 835 00:29:40,860 --> 00:29:45,059 I don't even know 836 00:29:43,020 --> 00:29:46,679 um I'm almost out of time and I am 837 00:29:45,059 --> 00:29:47,760 almost out of slides 838 00:29:46,679 --> 00:29:50,940 um 839 00:29:47,760 --> 00:29:53,220 I wish I had like a point to all this it 840 00:29:50,940 --> 00:29:55,020 was worth it it was really worth it I 841 00:29:53,220 --> 00:29:56,159 helped a lot of people I learned a lot 842 00:29:55,020 --> 00:29:58,919 of things 843 00:29:56,159 --> 00:29:59,940 some [ __ ] got a little weird 844 00:29:58,919 --> 00:30:02,220 um 845 00:29:59,940 --> 00:30:04,260 I'm grateful to everyone who helped me 846 00:30:02,220 --> 00:30:06,120 do it uh 847 00:30:04,260 --> 00:30:07,260 I just thought I think one of the most 848 00:30:06,120 --> 00:30:11,640 important things that I learned though 849 00:30:07,260 --> 00:30:14,220 is that there's a there's a emotional 850 00:30:11,640 --> 00:30:16,860 and physical upper limit to how much 851 00:30:14,220 --> 00:30:18,600 effort that you can put into 852 00:30:16,860 --> 00:30:20,580 um to fixing someone else's trash fire 853 00:30:18,600 --> 00:30:21,779 when they don't want it fixed you know I 854 00:30:20,580 --> 00:30:24,779 put this effort into Twitter because 855 00:30:21,779 --> 00:30:27,659 people I cared about relied on the tool 856 00:30:24,779 --> 00:30:29,640 and were using it to communicate if I 857 00:30:27,659 --> 00:30:31,140 was in the same position today I would 858 00:30:29,640 --> 00:30:34,620 say no just get off the platform like 859 00:30:31,140 --> 00:30:37,080 just just it's like the the the the the 860 00:30:34,620 --> 00:30:38,880 degradation of the platform uh in the 861 00:30:37,080 --> 00:30:39,960 last year or so has been such that I 862 00:30:38,880 --> 00:30:43,140 don't think 863 00:30:39,960 --> 00:30:45,179 there's the an ability to protect on it 864 00:30:43,140 --> 00:30:46,679 um I'm out of time pretty much the one 865 00:30:45,179 --> 00:30:51,020 other thing that I wanted to discuss but 866 00:30:46,679 --> 00:30:51,020 I'm happy to talk about outside is um 867 00:30:51,720 --> 00:30:55,020 I put I put some effort into trying to 868 00:30:53,760 --> 00:30:57,000 keep it going after the credentials were 869 00:30:55,020 --> 00:30:58,080 busted there was a couple avenues that I 870 00:30:57,000 --> 00:31:00,480 went down 871 00:30:58,080 --> 00:31:01,860 um with credentials to like old apps 872 00:31:00,480 --> 00:31:02,820 that had kind of leaked and things like 873 00:31:01,860 --> 00:31:04,860 that 874 00:31:02,820 --> 00:31:06,299 um and for reasons it just wasn't going 875 00:31:04,860 --> 00:31:09,600 to work 876 00:31:06,299 --> 00:31:12,240 um and finally this happened today which 877 00:31:09,600 --> 00:31:13,679 kind of tells you this this this gives 878 00:31:12,240 --> 00:31:15,000 you a good clear indicator of where the 879 00:31:13,679 --> 00:31:17,159 future of this platform is going anyway 880 00:31:15,000 --> 00:31:18,840 right so 881 00:31:17,159 --> 00:31:20,640 thanks everyone who helped me out with 882 00:31:18,840 --> 00:31:22,080 this thanks everyone for listening now I 883 00:31:20,640 --> 00:31:24,659 can talk about this for hours like any 884 00:31:22,080 --> 00:31:26,039 any slide that you saw here I I love 885 00:31:24,659 --> 00:31:28,559 talking about this stuff there's a lot 886 00:31:26,039 --> 00:31:33,539 more that I learned 887 00:31:28,559 --> 00:31:35,940 um but yeah thank you for uh letting me 888 00:31:33,539 --> 00:31:38,480 thank you for listening 889 00:31:35,940 --> 00:31:38,480 thank you