1 00:00:00,480 --> 00:00:03,480 foreign 2 00:00:08,340 --> 00:00:12,480 welcome back 3 00:00:10,019 --> 00:00:15,120 all right 4 00:00:12,480 --> 00:00:17,160 um now we've got beat the rush designing 5 00:00:15,120 --> 00:00:20,640 effective load tests for your web 6 00:00:17,160 --> 00:00:24,920 application by Anthony Shaw 7 00:00:20,640 --> 00:00:24,920 um take it away Anthony cool thank you 8 00:00:27,240 --> 00:00:31,140 okay hey everybody 9 00:00:29,099 --> 00:00:33,360 um I hope you're all having a good day I 10 00:00:31,140 --> 00:00:35,280 flew in this morning from Sydney 11 00:00:33,360 --> 00:00:37,860 um you might be able to tell from my 12 00:00:35,280 --> 00:00:40,440 accent that I'm not from around here 13 00:00:37,860 --> 00:00:41,820 um I'm actually English so before we get 14 00:00:40,440 --> 00:00:44,160 into load testing I'm going to talk 15 00:00:41,820 --> 00:00:45,960 about a couple of things and one is 16 00:00:44,160 --> 00:00:48,360 football the second one is T and the 17 00:00:45,960 --> 00:00:49,800 third one is the queen and how they all 18 00:00:48,360 --> 00:00:51,780 relate to 19 00:00:49,800 --> 00:00:56,280 um to load testing 20 00:00:51,780 --> 00:00:56,940 so if you want to follow along 21 00:00:56,280 --> 00:00:59,640 um 22 00:00:56,940 --> 00:01:01,320 this is my website the top link second 23 00:00:59,640 --> 00:01:04,680 one is my 24 00:01:01,320 --> 00:01:07,080 dead bird X account and then I'm also a 25 00:01:04,680 --> 00:01:09,420 mustard on if you're on Mastodon as well 26 00:01:07,080 --> 00:01:10,380 so 27 00:01:09,420 --> 00:01:13,439 um 28 00:01:10,380 --> 00:01:16,439 load testing and tea what do they have 29 00:01:13,439 --> 00:01:17,939 to do with each other so uh 30 00:01:16,439 --> 00:01:19,320 I guess over the last few weeks 31 00:01:17,939 --> 00:01:20,580 everyone's gone a bit nuts with the 32 00:01:19,320 --> 00:01:23,460 matildas 33 00:01:20,580 --> 00:01:25,140 um the last game was the most watched TV 34 00:01:23,460 --> 00:01:27,479 program in Australian history apparently 35 00:01:25,140 --> 00:01:30,479 11 million people watched it 36 00:01:27,479 --> 00:01:32,280 um I'm originally from the UK uh soccer 37 00:01:30,479 --> 00:01:34,080 is not a new thing for us we've kind of 38 00:01:32,280 --> 00:01:35,759 been into it for a while there's a 39 00:01:34,080 --> 00:01:37,140 famous problem and that's what happens 40 00:01:35,759 --> 00:01:40,860 at half time 41 00:01:37,140 --> 00:01:42,600 now at half time however many people are 42 00:01:40,860 --> 00:01:44,700 watching the game we'll go into the 43 00:01:42,600 --> 00:01:45,900 kitchen and turn the kettle on to make a 44 00:01:44,700 --> 00:01:47,280 cup of tea 45 00:01:45,900 --> 00:01:49,439 like you might think that's not really 46 00:01:47,280 --> 00:01:51,540 an issue but when you've got 20 30 47 00:01:49,439 --> 00:01:54,420 million people watching it and they all 48 00:01:51,540 --> 00:01:56,579 turn on a two kilowatt Kettle all at the 49 00:01:54,420 --> 00:01:58,439 same time it effectively melts the 50 00:01:56,579 --> 00:02:00,240 electricity grid 51 00:01:58,439 --> 00:02:02,880 um the second thing they do is to go to 52 00:02:00,240 --> 00:02:04,799 the bathroom so what they have to do 53 00:02:02,880 --> 00:02:06,659 when they know there's a big game in a 54 00:02:04,799 --> 00:02:09,420 world cup is they have to test 55 00:02:06,659 --> 00:02:11,459 effectively load test two things one is 56 00:02:09,420 --> 00:02:14,819 electricity grid and the second thing is 57 00:02:11,459 --> 00:02:17,040 a sewage system because effectively yeah 58 00:02:14,819 --> 00:02:21,840 both of them get low tested uh pretty 59 00:02:17,040 --> 00:02:23,819 hard so England versus Italy 2020 1.1 60 00:02:21,840 --> 00:02:26,459 million kettles simultaneously boarding 61 00:02:23,819 --> 00:02:27,180 was a 600 megawatt load 62 00:02:26,459 --> 00:02:29,580 um 63 00:02:27,180 --> 00:02:31,920 all in a huge Spike 64 00:02:29,580 --> 00:02:33,300 now we're going to talk about load 65 00:02:31,920 --> 00:02:35,640 testing and I want you to keep thinking 66 00:02:33,300 --> 00:02:37,920 about this analogy because not all load 67 00:02:35,640 --> 00:02:38,879 tests look like this 68 00:02:37,920 --> 00:02:41,400 um 69 00:02:38,879 --> 00:02:45,540 one that I was involved with a long time 70 00:02:41,400 --> 00:02:48,360 ago was a TV soap called Emmerdale and 71 00:02:45,540 --> 00:02:49,920 they were running a series in Emmerdale 72 00:02:48,360 --> 00:02:52,200 where you had to try and figure out who 73 00:02:49,920 --> 00:02:54,000 killed this particular character and at 74 00:02:52,200 --> 00:02:56,040 the end of the episode they'd say if you 75 00:02:54,000 --> 00:02:58,620 want to find out more try and figure out 76 00:02:56,040 --> 00:03:01,800 the clues click on this go to this 77 00:02:58,620 --> 00:03:04,140 website and effectively we had a half 78 00:03:01,800 --> 00:03:05,819 time problem with the kettles but a 79 00:03:04,140 --> 00:03:09,180 website version of that 80 00:03:05,819 --> 00:03:12,840 the other one I got involved with was 81 00:03:09,180 --> 00:03:14,519 um the Royal Palace set up a website for 82 00:03:12,840 --> 00:03:16,739 the queen 83 00:03:14,519 --> 00:03:18,620 in the unlikely situation that she 84 00:03:16,739 --> 00:03:21,900 passed away there's an obituary website 85 00:03:18,620 --> 00:03:24,540 and they asked me to load test it 86 00:03:21,900 --> 00:03:25,680 um this is a while ago so I had to load 87 00:03:24,540 --> 00:03:27,680 test that site 88 00:03:25,680 --> 00:03:30,120 um and I think when I heard the news 89 00:03:27,680 --> 00:03:32,099 quite recently of her passing I was the 90 00:03:30,120 --> 00:03:34,739 only person that went white in the face 91 00:03:32,099 --> 00:03:38,940 and immediately went on the laptop to go 92 00:03:34,739 --> 00:03:41,580 and check the latency of the website 93 00:03:38,940 --> 00:03:43,620 um it did it did hold up so that's good 94 00:03:41,580 --> 00:03:46,319 okay so 95 00:03:43,620 --> 00:03:48,060 when you're designing a load test you 96 00:03:46,319 --> 00:03:49,860 need to think about a couple of things 97 00:03:48,060 --> 00:03:52,440 now you need to think about the number 98 00:03:49,860 --> 00:03:54,000 of requests that you want to simulate 99 00:03:52,440 --> 00:03:56,700 and I think about this as being like a 100 00:03:54,000 --> 00:03:59,040 queue of people in a line a nice orderly 101 00:03:56,700 --> 00:04:00,780 cue okay the other thing you need to 102 00:03:59,040 --> 00:04:02,519 think about is concurrency 103 00:04:00,780 --> 00:04:04,680 these two things are not the same so if 104 00:04:02,519 --> 00:04:07,920 you say I want to test a thousand 105 00:04:04,680 --> 00:04:09,540 requests is that one at a time or is 106 00:04:07,920 --> 00:04:12,239 that a thousand that time 107 00:04:09,540 --> 00:04:13,500 because when you think about people and 108 00:04:12,239 --> 00:04:15,900 the strain that they can put on 109 00:04:13,500 --> 00:04:18,000 infrastructure the right hand side looks 110 00:04:15,900 --> 00:04:20,400 a bit like a stadium maybe or a gig 111 00:04:18,000 --> 00:04:21,720 that's just finished and you've got a 112 00:04:20,400 --> 00:04:23,639 lot of people all trying to leave an 113 00:04:21,720 --> 00:04:25,020 area at the same time all trying to get 114 00:04:23,639 --> 00:04:27,720 on the train at the same time so you've 115 00:04:25,020 --> 00:04:29,220 got a high concurrency issue 116 00:04:27,720 --> 00:04:31,560 so 117 00:04:29,220 --> 00:04:34,500 um what you don't want to do is to use a 118 00:04:31,560 --> 00:04:36,720 tool like Apache bench which is a great 119 00:04:34,500 --> 00:04:39,419 tool for measuring the performance of 120 00:04:36,720 --> 00:04:41,699 web servers but not applications and 121 00:04:39,419 --> 00:04:45,000 because it will effectively generate 122 00:04:41,699 --> 00:04:47,520 load on your website that looks 123 00:04:45,000 --> 00:04:50,280 absolutely nothing like real load so the 124 00:04:47,520 --> 00:04:52,199 numbers that you get back you might say 125 00:04:50,280 --> 00:04:55,080 to your engineering team well we can 126 00:04:52,199 --> 00:04:56,699 handle you know 12 requests a second and 127 00:04:55,080 --> 00:04:59,400 it's going to hold that at 800 128 00:04:56,699 --> 00:05:01,500 milliseconds which sounds great but that 129 00:04:59,400 --> 00:05:03,560 load looks nothing like your actual 130 00:05:01,500 --> 00:05:06,180 website load 131 00:05:03,560 --> 00:05:08,460 what real user patterns actually look 132 00:05:06,180 --> 00:05:10,800 like is uh this does anyone play this 133 00:05:08,460 --> 00:05:13,620 game in the room okay a few of you do 134 00:05:10,800 --> 00:05:16,919 okay so City skylines is brilliant 135 00:05:13,620 --> 00:05:18,060 um now what real user load looks like on 136 00:05:16,919 --> 00:05:20,940 a website is actually quite different 137 00:05:18,060 --> 00:05:22,919 you've got different users going to 138 00:05:20,940 --> 00:05:24,780 different parts of your apps at 139 00:05:22,919 --> 00:05:26,340 different times and at different speeds 140 00:05:24,780 --> 00:05:28,080 so 141 00:05:26,340 --> 00:05:29,940 you know your users are not robots 142 00:05:28,080 --> 00:05:31,740 they're not all clicking on things you 143 00:05:29,940 --> 00:05:33,000 know in millisecond Precision they're 144 00:05:31,740 --> 00:05:34,680 taking their time they're looking at 145 00:05:33,000 --> 00:05:35,520 things they're clicking on stuff and 146 00:05:34,680 --> 00:05:38,759 they're going in different directions 147 00:05:35,520 --> 00:05:40,919 people also have different speeds of 148 00:05:38,759 --> 00:05:42,840 their internet connections the latency 149 00:05:40,919 --> 00:05:45,000 between them and the website is also a 150 00:05:42,840 --> 00:05:46,740 bit different so 151 00:05:45,000 --> 00:05:48,660 the reason I bring this up is because 152 00:05:46,740 --> 00:05:50,820 you've got you know people going maybe 153 00:05:48,660 --> 00:05:53,160 to the home page people going to a 154 00:05:50,820 --> 00:05:55,440 different part of the app and people go 155 00:05:53,160 --> 00:05:57,240 into the search what you can end up with 156 00:05:55,440 --> 00:05:59,400 in many applications and in City 157 00:05:57,240 --> 00:06:01,139 skylines is a bottleneck where you've 158 00:05:59,400 --> 00:06:03,060 got users trying to go through a 159 00:06:01,139 --> 00:06:04,740 particular thing at the same time so we 160 00:06:03,060 --> 00:06:07,979 talked about postgres in the last talk 161 00:06:04,740 --> 00:06:10,199 sometimes the database is the bottleneck 162 00:06:07,979 --> 00:06:13,320 um sometimes it's the web server itself 163 00:06:10,199 --> 00:06:15,360 but if you're simulating user load 164 00:06:13,320 --> 00:06:18,479 instead of just trying to create network 165 00:06:15,360 --> 00:06:20,820 traffic you can find these bottlenecks a 166 00:06:18,479 --> 00:06:22,580 lot easier so not all users have the 167 00:06:20,820 --> 00:06:26,400 same speed car or the same connection 168 00:06:22,580 --> 00:06:28,860 not all users arrive at the same time 169 00:06:26,400 --> 00:06:31,500 they take different Journeys and they 170 00:06:28,860 --> 00:06:33,660 also wait between clicks so this is 171 00:06:31,500 --> 00:06:37,680 really important 172 00:06:33,660 --> 00:06:39,240 so if you want to understand your users 173 00:06:37,680 --> 00:06:41,400 in a bit more detail so if you want to 174 00:06:39,240 --> 00:06:43,199 design a load test the first thing you 175 00:06:41,400 --> 00:06:45,539 want to do is actually understand your 176 00:06:43,199 --> 00:06:47,940 users more what does the load look like 177 00:06:45,539 --> 00:06:49,620 on your current application 178 00:06:47,940 --> 00:06:52,740 if you haven't launched the application 179 00:06:49,620 --> 00:06:53,940 you've got to guess so we'll talk a bit 180 00:06:52,740 --> 00:06:56,639 more about that 181 00:06:53,940 --> 00:06:57,960 so when you're thinking about users and 182 00:06:56,639 --> 00:06:59,699 what they do with your app and what they 183 00:06:57,960 --> 00:07:02,039 do with your website they take different 184 00:06:59,699 --> 00:07:04,259 paths they'll click on different pages 185 00:07:02,039 --> 00:07:07,580 those paths you can work out 186 00:07:04,259 --> 00:07:10,680 probabilities of them you can say Okay 187 00:07:07,580 --> 00:07:12,960 60 of users will start on the home page 188 00:07:10,680 --> 00:07:15,539 twenty percent of them will then go on 189 00:07:12,960 --> 00:07:17,940 the promotions page let's say so you can 190 00:07:15,539 --> 00:07:20,039 look at your web history and see what 191 00:07:17,940 --> 00:07:22,560 Journeys do people take and then if we 192 00:07:20,039 --> 00:07:24,840 want to write a load test let's simulate 193 00:07:22,560 --> 00:07:27,960 that and most of the load testing tools 194 00:07:24,840 --> 00:07:30,599 will let you say designer a virtual user 195 00:07:27,960 --> 00:07:33,419 and actually give them a path to take 196 00:07:30,599 --> 00:07:37,680 so if you're analyzing the traffic first 197 00:07:33,419 --> 00:07:39,240 and you can group users into personas 198 00:07:37,680 --> 00:07:40,560 um so you know you've got like a casual 199 00:07:39,240 --> 00:07:43,199 Shopper maybe someone who's actually 200 00:07:40,560 --> 00:07:44,880 going to go and place an order then that 201 00:07:43,199 --> 00:07:48,060 means you can create a more realistic 202 00:07:44,880 --> 00:07:49,560 load on your application 203 00:07:48,060 --> 00:07:52,259 um there are tools that you can get as 204 00:07:49,560 --> 00:07:53,400 well that like you visualize this stuff 205 00:07:52,259 --> 00:07:55,380 um so if you're looking at your web 206 00:07:53,400 --> 00:07:56,699 traffic on your web history you can 207 00:07:55,380 --> 00:07:58,860 actually see the journeys that people 208 00:07:56,699 --> 00:08:00,780 take and then how many of them go 209 00:07:58,860 --> 00:08:02,400 through that particular page 210 00:08:00,780 --> 00:08:04,860 and once you've got that understanding 211 00:08:02,400 --> 00:08:07,680 you can simulate the load a lot more 212 00:08:04,860 --> 00:08:10,199 accurately than just throwing uh 213 00:08:07,680 --> 00:08:12,960 requests at the web server 214 00:08:10,199 --> 00:08:15,060 so another thing to take into account as 215 00:08:12,960 --> 00:08:16,800 well is timing so 216 00:08:15,060 --> 00:08:19,979 like I said before 217 00:08:16,800 --> 00:08:21,120 users real users will wait between 218 00:08:19,979 --> 00:08:23,879 clicks 219 00:08:21,120 --> 00:08:26,520 so if you want to do a proper load test 220 00:08:23,879 --> 00:08:29,940 you want to try and simulate some of 221 00:08:26,520 --> 00:08:31,560 those pauses if you've got this kind of 222 00:08:29,940 --> 00:08:33,659 analysis where you can see the user 223 00:08:31,560 --> 00:08:37,140 flows it should tell you how many 224 00:08:33,659 --> 00:08:38,880 seconds it takes between if you don't 225 00:08:37,140 --> 00:08:40,260 have that you could just ask a couple of 226 00:08:38,880 --> 00:08:41,459 colleagues to go and click through the 227 00:08:40,260 --> 00:08:43,740 site and you can time them with a 228 00:08:41,459 --> 00:08:44,880 stopwatch if you want or you can just 229 00:08:43,740 --> 00:08:48,839 guess 230 00:08:44,880 --> 00:08:50,940 um having zero weight between clicks is 231 00:08:48,839 --> 00:08:53,100 unrealistic and it creates really weird 232 00:08:50,940 --> 00:08:55,200 load and you'll actually find 233 00:08:53,100 --> 00:08:57,000 bottlenecks that won't actually come up 234 00:08:55,200 --> 00:08:59,339 in production so you'll try and 235 00:08:57,000 --> 00:09:01,560 basically prematurely optimize problems 236 00:08:59,339 --> 00:09:04,080 that wouldn't even exist when you're 237 00:09:01,560 --> 00:09:06,240 putting the app in front of users 238 00:09:04,080 --> 00:09:08,399 what users do instead is if we see this 239 00:09:06,240 --> 00:09:11,040 like a timeline they'd start off on one 240 00:09:08,399 --> 00:09:13,200 part of the application wait a bit go to 241 00:09:11,040 --> 00:09:15,240 another page click on something else 242 00:09:13,200 --> 00:09:17,220 click on something else and then maybe 243 00:09:15,240 --> 00:09:20,279 they're done and they log off 244 00:09:17,220 --> 00:09:22,560 so if you've got two users 245 00:09:20,279 --> 00:09:24,000 they're on two different timelines 246 00:09:22,560 --> 00:09:25,500 they're not synchronized with each other 247 00:09:24,000 --> 00:09:27,240 unless they're looking over each other's 248 00:09:25,500 --> 00:09:28,620 shoulder so if you've got two people 249 00:09:27,240 --> 00:09:29,820 going on the website they're going to be 250 00:09:28,620 --> 00:09:31,140 clicking on different parts of the site 251 00:09:29,820 --> 00:09:33,180 at different times 252 00:09:31,140 --> 00:09:35,279 so if you want to run a load test on 253 00:09:33,180 --> 00:09:38,600 your application you want to simulate 254 00:09:35,279 --> 00:09:38,600 some of this Randomness as well 255 00:09:38,640 --> 00:09:44,959 something else which is really commonly 256 00:09:40,560 --> 00:09:48,540 missed in load tests is if you've got a 257 00:09:44,959 --> 00:09:51,060 a an app that does a lot of Ajax it's 258 00:09:48,540 --> 00:09:53,640 got a lot of JavaScript some load 259 00:09:51,060 --> 00:09:56,640 testing tools will ignore that so if you 260 00:09:53,640 --> 00:09:58,680 just make HTTP requests to the server 261 00:09:56,640 --> 00:10:00,600 and you simulate it and you think the 262 00:09:58,680 --> 00:10:01,800 performance is brilliant then you 263 00:10:00,600 --> 00:10:04,019 actually launch it and you put it in 264 00:10:01,800 --> 00:10:06,300 front of a bunch of users and there's an 265 00:10:04,019 --> 00:10:08,640 Ajax thing that pulls every 100 266 00:10:06,300 --> 00:10:10,800 milliseconds and you've got 100 people 267 00:10:08,640 --> 00:10:12,660 leaving their browsers open you just 268 00:10:10,800 --> 00:10:13,980 ignored like a huge part of the actual 269 00:10:12,660 --> 00:10:16,260 load of your app 270 00:10:13,980 --> 00:10:20,160 so it's really important that you 271 00:10:16,260 --> 00:10:22,920 understand is there an Ajax component to 272 00:10:20,160 --> 00:10:24,600 my application and if there is what 273 00:10:22,920 --> 00:10:27,380 differences that make to the load that's 274 00:10:24,600 --> 00:10:27,380 put on the server 275 00:10:28,080 --> 00:10:33,720 okay so the tool I want to talk about is 276 00:10:30,660 --> 00:10:36,300 Locust there are other tools for load 277 00:10:33,720 --> 00:10:38,220 testing I particularly like Locust for a 278 00:10:36,300 --> 00:10:40,680 couple of reasons one is free and open 279 00:10:38,220 --> 00:10:42,720 source which was a big tick a second one 280 00:10:40,680 --> 00:10:44,519 is written in Python and also it's 281 00:10:42,720 --> 00:10:46,320 really easy to get started with and 282 00:10:44,519 --> 00:10:49,980 really easy to use 283 00:10:46,320 --> 00:10:51,360 so to use Locust you just do pip install 284 00:10:49,980 --> 00:10:54,720 Locust 285 00:10:51,360 --> 00:10:58,500 and then you run Locus f with a locust 286 00:10:54,720 --> 00:11:02,040 file a Locus file is a python file in 287 00:10:58,500 --> 00:11:03,600 which you Define virtual users and 288 00:11:02,040 --> 00:11:05,760 basically it uses each one of those 289 00:11:03,600 --> 00:11:07,800 users as a script and it will create 290 00:11:05,760 --> 00:11:09,360 load based on that script 291 00:11:07,800 --> 00:11:11,940 so we're going to give you a couple of 292 00:11:09,360 --> 00:11:13,200 examples of what a Locus script would 293 00:11:11,940 --> 00:11:16,140 look like 294 00:11:13,200 --> 00:11:19,320 so the first thing we do is we import a 295 00:11:16,140 --> 00:11:21,959 HTTP user a task and then a function 296 00:11:19,320 --> 00:11:25,320 called between and you define in a class 297 00:11:21,959 --> 00:11:27,899 so you inherit from HTTP user and you 298 00:11:25,320 --> 00:11:29,640 define a user so we you can call it what 299 00:11:27,899 --> 00:11:32,540 you like I'll call minebasic user 300 00:11:29,640 --> 00:11:34,500 because I think they're basic and 301 00:11:32,540 --> 00:11:36,480 they're going to we're going to wait 302 00:11:34,500 --> 00:11:38,880 between three and five seconds that's 303 00:11:36,480 --> 00:11:40,200 what that statement means between each 304 00:11:38,880 --> 00:11:43,260 new user 305 00:11:40,200 --> 00:11:45,240 so they've got three tasks 306 00:11:43,260 --> 00:11:47,880 um the first one is to go to a forward 307 00:11:45,240 --> 00:11:50,339 slash which is the home page the second 308 00:11:47,880 --> 00:11:52,800 task is to go to the destinations page 309 00:11:50,339 --> 00:11:54,899 and then the third task is to go to the 310 00:11:52,800 --> 00:11:57,300 specific destination 311 00:11:54,899 --> 00:12:00,120 the task decorator basically means that 312 00:11:57,300 --> 00:12:01,860 that method is a task and each user will 313 00:12:00,120 --> 00:12:04,860 do all of those tasks 314 00:12:01,860 --> 00:12:07,860 the argument three says that that task 315 00:12:04,860 --> 00:12:09,720 is three times more likely to happen so 316 00:12:07,860 --> 00:12:11,579 actually they're going to go to the home 317 00:12:09,720 --> 00:12:12,660 page three times more than they would to 318 00:12:11,579 --> 00:12:14,820 the other two 319 00:12:12,660 --> 00:12:16,200 so that's more realistic because a lot 320 00:12:14,820 --> 00:12:17,040 of people just go to the home page and 321 00:12:16,200 --> 00:12:18,779 then 322 00:12:17,040 --> 00:12:21,660 realize it's not for them and then click 323 00:12:18,779 --> 00:12:23,459 off so if this is our basic user that's 324 00:12:21,660 --> 00:12:25,320 going to create quite a bit of load to 325 00:12:23,459 --> 00:12:26,940 the home page and a bit to the 326 00:12:25,320 --> 00:12:28,920 destination and then the specific 327 00:12:26,940 --> 00:12:31,680 destination page 328 00:12:28,920 --> 00:12:34,440 so the between function means we'll wait 329 00:12:31,680 --> 00:12:38,399 between three and five seconds per task 330 00:12:34,440 --> 00:12:42,060 and then run that three times more often 331 00:12:38,399 --> 00:12:44,279 um I mentioned Ajax before so we're just 332 00:12:42,060 --> 00:12:47,339 doing a DOT get so this is a HTTP 333 00:12:44,279 --> 00:12:49,740 request on that URI 334 00:12:47,339 --> 00:12:52,440 um if you have a more complicated app if 335 00:12:49,740 --> 00:12:55,139 you can't just simulate load by going to 336 00:12:52,440 --> 00:12:57,060 URI 337 00:12:55,139 --> 00:12:58,860 um there's another option which is to 338 00:12:57,060 --> 00:13:00,959 use a playwright has anyone use 339 00:12:58,860 --> 00:13:03,660 playwright in the room okay a few of you 340 00:13:00,959 --> 00:13:07,620 uh playwright is a browser automation 341 00:13:03,660 --> 00:13:10,620 tool that has a wrapper in Python 342 00:13:07,620 --> 00:13:12,120 um what you can effectively do is if you 343 00:13:10,620 --> 00:13:15,000 install playwright 344 00:13:12,120 --> 00:13:17,639 um then you can run playwright code gen 345 00:13:15,000 --> 00:13:19,860 and then do Target equals python it will 346 00:13:17,639 --> 00:13:21,540 pop up a browser you can go to your 347 00:13:19,860 --> 00:13:23,760 website and it's recording all 348 00:13:21,540 --> 00:13:25,740 everything you do so you can click on 349 00:13:23,760 --> 00:13:27,480 the buttons you can fill in forms you 350 00:13:25,740 --> 00:13:29,940 can submit stuff what if isn't 351 00:13:27,480 --> 00:13:33,000 JavaScript and on the right hand side 352 00:13:29,940 --> 00:13:34,740 you'll see that python code window and 353 00:13:33,000 --> 00:13:37,019 it generates the code of what you've 354 00:13:34,740 --> 00:13:39,120 been clicking on so if you've got a 355 00:13:37,019 --> 00:13:41,700 complicated app and it would be quite 356 00:13:39,120 --> 00:13:43,620 time consuming and quite tedious to go 357 00:13:41,700 --> 00:13:45,480 and actually code how to click on all 358 00:13:43,620 --> 00:13:47,820 the buttons how to find that particular 359 00:13:45,480 --> 00:13:50,160 drop down then you can just use the 360 00:13:47,820 --> 00:13:52,380 playwright recorder and then there's a 361 00:13:50,160 --> 00:13:55,019 plugin between Locust and playwright so 362 00:13:52,380 --> 00:13:57,120 instead of just creating a HTTP user you 363 00:13:55,019 --> 00:13:58,800 can create a playwright user and you can 364 00:13:57,120 --> 00:14:01,320 give it that code that got generated 365 00:13:58,800 --> 00:14:03,420 here and then it will actually open up a 366 00:14:01,320 --> 00:14:04,440 browser and then drive the traffic using 367 00:14:03,420 --> 00:14:07,139 the browser 368 00:14:04,440 --> 00:14:09,720 uh one there's a link at the bottom to a 369 00:14:07,139 --> 00:14:11,459 code sample of that all working uh one 370 00:14:09,720 --> 00:14:13,860 caution I will give you if you're 371 00:14:11,459 --> 00:14:15,060 simulating a large number of users on a 372 00:14:13,860 --> 00:14:17,339 single machine 373 00:14:15,060 --> 00:14:19,800 is one thing to do that using HTTP 374 00:14:17,339 --> 00:14:21,660 traffic it's another thing to do that by 375 00:14:19,800 --> 00:14:23,700 opening a thousand instances of Chrome 376 00:14:21,660 --> 00:14:25,860 on your machine 377 00:14:23,700 --> 00:14:29,220 so don't expect the same performance 378 00:14:25,860 --> 00:14:31,980 because you won't achieve it 379 00:14:29,220 --> 00:14:34,079 um okay so going more specifically into 380 00:14:31,980 --> 00:14:36,720 into Django now 381 00:14:34,079 --> 00:14:39,899 Django has a brilliant feature 382 00:14:36,720 --> 00:14:43,199 um in that it's got built-in csrf which 383 00:14:39,899 --> 00:14:45,060 is a security layer which means that you 384 00:14:43,199 --> 00:14:47,399 can't inject stuff into forms if you're 385 00:14:45,060 --> 00:14:49,139 not actually the person who did it 386 00:14:47,399 --> 00:14:51,480 um this creates a bit of a complication 387 00:14:49,139 --> 00:14:53,880 sometimes because 388 00:14:51,480 --> 00:14:56,699 um one a Django form loads up on the 389 00:14:53,880 --> 00:14:58,320 page you need to read a hidden field and 390 00:14:56,699 --> 00:15:00,720 then you need to put that in the 391 00:14:58,320 --> 00:15:03,660 submission when you click on a form so 392 00:15:00,720 --> 00:15:07,019 if you have to do that you can do that 393 00:15:03,660 --> 00:15:09,120 using basically doing a get first and 394 00:15:07,019 --> 00:15:11,639 then to get the csrf token out of the 395 00:15:09,120 --> 00:15:15,000 cookie and then include it in the post 396 00:15:11,639 --> 00:15:18,420 request so in Locust when you're 397 00:15:15,000 --> 00:15:21,600 programming HTTP users your tasks can 398 00:15:18,420 --> 00:15:23,459 include a series of commands so not just 399 00:15:21,600 --> 00:15:27,060 to get to the URL but you can actually 400 00:15:23,459 --> 00:15:29,820 have a get to get to load the page and 401 00:15:27,060 --> 00:15:31,860 then another request which is the post 402 00:15:29,820 --> 00:15:33,240 cool so 403 00:15:31,860 --> 00:15:34,680 um that is a sample of how to do that 404 00:15:33,240 --> 00:15:36,120 and there's a link at the end of the 405 00:15:34,680 --> 00:15:38,899 talk as well if you need to copy that 406 00:15:36,120 --> 00:15:38,899 code snippet 407 00:15:39,000 --> 00:15:44,519 um now that's kind of a bit of the nuts 408 00:15:41,880 --> 00:15:46,380 and bolts of how to use Locust 409 00:15:44,519 --> 00:15:47,880 um going back to the actual part of the 410 00:15:46,380 --> 00:15:50,880 design of the load test because that's 411 00:15:47,880 --> 00:15:53,220 where there's a lot of common I guess 412 00:15:50,880 --> 00:15:55,160 pitfalls in terms of what you're testing 413 00:15:53,220 --> 00:15:57,300 and how effective that is it actually 414 00:15:55,160 --> 00:16:00,959 estimating whether your website's going 415 00:15:57,300 --> 00:16:02,639 to hold up when it hits a lot of load so 416 00:16:00,959 --> 00:16:06,300 a common mistake I see is that people 417 00:16:02,639 --> 00:16:07,920 will deploy the application and then 418 00:16:06,300 --> 00:16:10,860 they'll run a load test 419 00:16:07,920 --> 00:16:13,079 and the number of users they have on the 420 00:16:10,860 --> 00:16:15,600 application when they're spinning and up 421 00:16:13,079 --> 00:16:16,980 and testing it is like one and the 422 00:16:15,600 --> 00:16:18,420 number of products they've got in the 423 00:16:16,980 --> 00:16:20,220 database or the number of widgets 424 00:16:18,420 --> 00:16:23,220 they've got on the thing is like two or 425 00:16:20,220 --> 00:16:25,260 three and that's your test ones right 426 00:16:23,220 --> 00:16:28,620 um now when you actually look at look at 427 00:16:25,260 --> 00:16:30,420 real load if you've got 10 000 products 428 00:16:28,620 --> 00:16:32,579 in your production app and you've got 429 00:16:30,420 --> 00:16:34,680 two in your test app and you're load 430 00:16:32,579 --> 00:16:35,760 testing both of them those low patterns 431 00:16:34,680 --> 00:16:38,820 are going to look very very different 432 00:16:35,760 --> 00:16:40,680 because the number of SQL queries it's 433 00:16:38,820 --> 00:16:42,779 going to be running on one versus the 434 00:16:40,680 --> 00:16:44,880 other it's going to be different and 435 00:16:42,779 --> 00:16:46,740 also if you've got pagination on your 436 00:16:44,880 --> 00:16:49,860 page if you you know there's a lot of 437 00:16:46,740 --> 00:16:52,800 different context things so what I 438 00:16:49,860 --> 00:16:54,899 strongly recommend is that you seed test 439 00:16:52,800 --> 00:16:57,600 data into an application or a server 440 00:16:54,899 --> 00:16:59,399 before you load test it and there's a 441 00:16:57,600 --> 00:17:01,079 couple of ways that you can do that 442 00:16:59,399 --> 00:17:03,060 there's one approach that I really like 443 00:17:01,079 --> 00:17:05,699 which I'll recommend but there 444 00:17:03,060 --> 00:17:08,400 definitely are others this is to use a 445 00:17:05,699 --> 00:17:10,500 package called Nemesis there's other 446 00:17:08,400 --> 00:17:13,319 ones like Faker there's some other 447 00:17:10,500 --> 00:17:16,699 packages for generating fake data I like 448 00:17:13,319 --> 00:17:19,919 Nemesis because it's really good for 449 00:17:16,699 --> 00:17:21,360 testing like internationalization and 450 00:17:19,919 --> 00:17:24,000 you can actually test 451 00:17:21,360 --> 00:17:25,559 you can create test data in different 452 00:17:24,000 --> 00:17:27,059 languages with different contexts and 453 00:17:25,559 --> 00:17:29,940 stuff like that so it's really helpful 454 00:17:27,059 --> 00:17:33,540 so if you spun up an application and you 455 00:17:29,940 --> 00:17:34,620 wanted to create a thousand users in the 456 00:17:33,540 --> 00:17:36,720 system 457 00:17:34,620 --> 00:17:39,780 you could do that just by writing a seed 458 00:17:36,720 --> 00:17:42,299 function importing Nemesis and then 459 00:17:39,780 --> 00:17:44,640 using this person object that person 460 00:17:42,299 --> 00:17:47,100 object will generate a random name a 461 00:17:44,640 --> 00:17:48,660 random email and a random password for 462 00:17:47,100 --> 00:17:49,980 each one of those users and it's got 463 00:17:48,660 --> 00:17:52,860 loads of other properties as well you 464 00:17:49,980 --> 00:17:55,620 can put in their address their political 465 00:17:52,860 --> 00:17:58,140 affiliations their favorite type of tea 466 00:17:55,620 --> 00:18:00,120 like it's got everything in there so 467 00:17:58,140 --> 00:18:03,179 um and then once you've done that you 468 00:18:00,120 --> 00:18:06,419 want to use a bulk create call to add 469 00:18:03,179 --> 00:18:07,919 all of those users in a huge batch 470 00:18:06,419 --> 00:18:09,200 cool 471 00:18:07,919 --> 00:18:13,140 so 472 00:18:09,200 --> 00:18:15,179 here's a test question for you 473 00:18:13,140 --> 00:18:17,539 what is the fastest implementation of 474 00:18:15,179 --> 00:18:17,539 this problem 475 00:18:19,940 --> 00:18:25,380 so return 610 is definitely the that's 476 00:18:23,160 --> 00:18:28,260 the most efficient way of implementing 477 00:18:25,380 --> 00:18:31,740 that function and it's the same when it 478 00:18:28,260 --> 00:18:34,080 comes to web app so what um I've often 479 00:18:31,740 --> 00:18:36,480 found with Django and with all 480 00:18:34,080 --> 00:18:38,820 Frameworks with asp.net with other 481 00:18:36,480 --> 00:18:41,820 languages with different infrastructure 482 00:18:38,820 --> 00:18:43,140 is so often when you get a bottleneck or 483 00:18:41,820 --> 00:18:46,679 you get slow performance in an 484 00:18:43,140 --> 00:18:49,440 application the solution is to cash and 485 00:18:46,679 --> 00:18:52,080 you look at caching from the top down 486 00:18:49,440 --> 00:18:54,299 um so my lovely implementation of the 487 00:18:52,080 --> 00:18:56,700 Fibonacci function is effectively just 488 00:18:54,299 --> 00:18:58,799 caching it based on the answer and 489 00:18:56,700 --> 00:19:02,880 actually most good implementations of 490 00:18:58,799 --> 00:19:06,539 the FIB function will use an lru cache 491 00:19:02,880 --> 00:19:09,539 um this one's just cheeky uh okay so 492 00:19:06,539 --> 00:19:11,100 uh let's cache everything now in a 493 00:19:09,539 --> 00:19:11,880 Django stack 494 00:19:11,100 --> 00:19:13,320 um 495 00:19:11,880 --> 00:19:14,880 you might think okay I've used the 496 00:19:13,320 --> 00:19:16,860 caching module before 497 00:19:14,880 --> 00:19:19,440 so you're kind of familiar with that 498 00:19:16,860 --> 00:19:21,200 actually there's different parts that 499 00:19:19,440 --> 00:19:23,700 you might want to Cache depending on 500 00:19:21,200 --> 00:19:25,740 where the load is going to be and where 501 00:19:23,700 --> 00:19:27,480 you're going to see benefit 502 00:19:25,740 --> 00:19:30,480 um so at the top end you've got your 503 00:19:27,480 --> 00:19:31,080 HTTP server so this could be 504 00:19:30,480 --> 00:19:35,160 um 505 00:19:31,080 --> 00:19:37,799 Apache or could be nginx for example 506 00:19:35,160 --> 00:19:40,020 so you can cash out that layer so that 507 00:19:37,799 --> 00:19:42,299 kind of cache would be like a CDN where 508 00:19:40,020 --> 00:19:44,880 you've actually just cached the bytes 509 00:19:42,299 --> 00:19:47,580 the HTML that's been generated and that 510 00:19:44,880 --> 00:19:49,559 would look the same for everybody 511 00:19:47,580 --> 00:19:51,840 um you've got whiskey and ASCII are not 512 00:19:49,559 --> 00:19:53,820 familiar for any caching at that level 513 00:19:51,840 --> 00:19:55,620 um you kind of then got the Django 514 00:19:53,820 --> 00:19:57,480 application from there downwards and 515 00:19:55,620 --> 00:20:00,419 you've got middleware views models 516 00:19:57,480 --> 00:20:01,919 static files each one of those can kind 517 00:20:00,419 --> 00:20:03,179 of be en cached using different 518 00:20:01,919 --> 00:20:06,000 mechanisms 519 00:20:03,179 --> 00:20:07,200 so underneath that in your middleware 520 00:20:06,000 --> 00:20:08,700 you've got 521 00:20:07,200 --> 00:20:10,500 um different middleware that you want to 522 00:20:08,700 --> 00:20:12,900 plug in you've got templates you've got 523 00:20:10,500 --> 00:20:14,220 the orm and then underneath that you've 524 00:20:12,900 --> 00:20:16,620 got the database 525 00:20:14,220 --> 00:20:18,600 so when we look at caching with the 526 00:20:16,620 --> 00:20:20,400 Django app there's a whole stack of it 527 00:20:18,600 --> 00:20:23,039 there's actually different places you 528 00:20:20,400 --> 00:20:25,679 can do caching so at the top end you've 529 00:20:23,039 --> 00:20:28,020 got a CDN or like an edge cache 530 00:20:25,679 --> 00:20:30,179 so before you kind of jump in with 531 00:20:28,020 --> 00:20:33,299 Django and say okay let's cache in the 532 00:20:30,179 --> 00:20:36,840 template if the page is practically 533 00:20:33,299 --> 00:20:38,700 static or the part of the site that 534 00:20:36,840 --> 00:20:41,039 people are mostly going to is basically 535 00:20:38,700 --> 00:20:42,140 static then just cache to that whole 536 00:20:41,039 --> 00:20:45,179 page 537 00:20:42,140 --> 00:20:47,160 there's no point in introducing caching 538 00:20:45,179 --> 00:20:49,140 at a really low level when you could do 539 00:20:47,160 --> 00:20:51,299 it really efficiently at the Top If the 540 00:20:49,140 --> 00:20:53,940 generated result was always the same 541 00:20:51,299 --> 00:20:56,280 that said if the first thing people have 542 00:20:53,940 --> 00:20:58,799 to do in your Django app is log in 543 00:20:56,280 --> 00:21:00,419 then introducing a cache at the top is 544 00:20:58,799 --> 00:21:02,220 going to be pretty pointless because the 545 00:21:00,419 --> 00:21:04,260 page looks different for everybody 546 00:21:02,220 --> 00:21:06,419 so this is kind of why you have to look 547 00:21:04,260 --> 00:21:09,720 at caching in terms of the whole stack 548 00:21:06,419 --> 00:21:11,460 and which parts will be the same and 549 00:21:09,720 --> 00:21:14,100 which parts will be different and where 550 00:21:11,460 --> 00:21:16,679 does it make sense to Cache 551 00:21:14,100 --> 00:21:20,100 um the template caching in Django I 552 00:21:16,679 --> 00:21:23,340 recommend that everybody uses so 553 00:21:20,100 --> 00:21:25,559 you don't want it to have to build and 554 00:21:23,340 --> 00:21:28,260 generate the template every single time 555 00:21:25,559 --> 00:21:29,880 especially for partials if the inputs 556 00:21:28,260 --> 00:21:32,400 are the same and the output is the same 557 00:21:29,880 --> 00:21:34,320 there's no point in Computing that 558 00:21:32,400 --> 00:21:38,039 result for every single user that goes 559 00:21:34,320 --> 00:21:39,360 to the application if it can be cached 560 00:21:38,039 --> 00:21:41,820 um you've also got things on the 561 00:21:39,360 --> 00:21:43,500 database layer most of these are you 562 00:21:41,820 --> 00:21:45,720 don't have to configure but you can do 563 00:21:43,500 --> 00:21:48,919 and that's kind of how things can be 564 00:21:45,720 --> 00:21:48,919 cached in the database 565 00:21:49,320 --> 00:21:53,640 you've also got object caches you can 566 00:21:51,600 --> 00:21:55,380 just throw around all over python to 567 00:21:53,640 --> 00:21:57,000 make things a bit faster 568 00:21:55,380 --> 00:21:57,780 cool 569 00:21:57,000 --> 00:22:01,140 um 570 00:21:57,780 --> 00:22:02,640 now if you have introduced caching 571 00:22:01,140 --> 00:22:04,799 um there's another Pitfall you can fall 572 00:22:02,640 --> 00:22:06,440 into which I'll talk about which is you 573 00:22:04,799 --> 00:22:09,539 design your load test 574 00:22:06,440 --> 00:22:11,159 and in your load test you said okay 575 00:22:09,539 --> 00:22:11,940 we're going to get destination number 576 00:22:11,159 --> 00:22:14,159 one 577 00:22:11,940 --> 00:22:15,960 and you've tested it and it works 578 00:22:14,159 --> 00:22:17,880 brilliantly and you ship it to 579 00:22:15,960 --> 00:22:19,860 production and it performs horrendously 580 00:22:17,880 --> 00:22:21,600 and you're like but I tested it the 581 00:22:19,860 --> 00:22:23,400 locations page was really fast and 582 00:22:21,600 --> 00:22:24,539 really efficient and then when we 583 00:22:23,400 --> 00:22:27,120 shipped it 584 00:22:24,539 --> 00:22:29,159 it wasn't like what's the difference 585 00:22:27,120 --> 00:22:32,820 and the difference is that I hard-coded 586 00:22:29,159 --> 00:22:35,400 the id1 in my load test and it cashed 587 00:22:32,820 --> 00:22:37,500 the hell out of that page and then when 588 00:22:35,400 --> 00:22:40,140 we ship it to production we've got it 589 00:22:37,500 --> 00:22:42,960 running across 10 different servers with 590 00:22:40,140 --> 00:22:44,880 20 000 different destinations and people 591 00:22:42,960 --> 00:22:47,159 are clicking on different ones and the 592 00:22:44,880 --> 00:22:50,880 cache isn't preceded so the performance 593 00:22:47,159 --> 00:22:53,159 is terrible so if you can 594 00:22:50,880 --> 00:22:55,799 um try and introduce an element of 595 00:22:53,159 --> 00:22:58,320 Randomness to your load tests so you're 596 00:22:55,799 --> 00:22:59,840 actually testing different instances or 597 00:22:58,320 --> 00:23:03,600 different IDs 598 00:22:59,840 --> 00:23:06,480 this is across both if you're looking at 599 00:23:03,600 --> 00:23:08,340 individual records or if you're 600 00:23:06,480 --> 00:23:10,500 inputting data basically try and 601 00:23:08,340 --> 00:23:12,780 simulate as much as you can the 602 00:23:10,500 --> 00:23:13,980 randomness of a user not all of your 603 00:23:12,780 --> 00:23:15,240 users are going to click on the same 604 00:23:13,980 --> 00:23:16,679 page 605 00:23:15,240 --> 00:23:19,260 um and if that's what that looks like 606 00:23:16,679 --> 00:23:21,480 then try and test your cash 607 00:23:19,260 --> 00:23:23,580 okay so if you've got all this working 608 00:23:21,480 --> 00:23:25,860 and you've run Locust 609 00:23:23,580 --> 00:23:29,520 you're going to get a graph 610 00:23:25,860 --> 00:23:30,960 maybe that looks a bit like this 611 00:23:29,520 --> 00:23:34,440 um and I'll tell you a bit more about 612 00:23:30,960 --> 00:23:37,880 how to interpret the results so often 613 00:23:34,440 --> 00:23:40,799 what you see with most applications 614 00:23:37,880 --> 00:23:42,780 is uh so we've got two lines here one is 615 00:23:40,799 --> 00:23:45,960 the median response time 616 00:23:42,780 --> 00:23:48,600 and the other one is the 95th percentile 617 00:23:45,960 --> 00:23:52,460 um the 95th percentile means that 95 of 618 00:23:48,600 --> 00:23:54,600 responses are less than that number so 619 00:23:52,460 --> 00:23:57,600 often when you're looking at performance 620 00:23:54,600 --> 00:24:00,059 data you look at percentiles so you try 621 00:23:57,600 --> 00:24:02,159 and ignore some of the outliers and you 622 00:24:00,059 --> 00:24:03,179 say okay what performance will most 623 00:24:02,159 --> 00:24:05,580 people see 624 00:24:03,179 --> 00:24:08,159 and because the word most is objective 625 00:24:05,580 --> 00:24:10,860 you can pick a number but 95 is pretty 626 00:24:08,159 --> 00:24:13,380 typical 99 would be extreme so I'd say 627 00:24:10,860 --> 00:24:14,760 95 is normally what you should be 628 00:24:13,380 --> 00:24:16,500 looking at 629 00:24:14,760 --> 00:24:18,960 um you'll see a spike often at the 630 00:24:16,500 --> 00:24:21,600 beginning because things are warming up 631 00:24:18,960 --> 00:24:24,059 so that part is uh effectively a warm-up 632 00:24:21,600 --> 00:24:25,919 of either the cache the server the 633 00:24:24,059 --> 00:24:28,320 instances you know there might be in 634 00:24:25,919 --> 00:24:30,000 some processes which were asleep and all 635 00:24:28,320 --> 00:24:31,919 of a sudden you're throwing thousands of 636 00:24:30,000 --> 00:24:33,840 requests at them so it takes a minute 637 00:24:31,919 --> 00:24:35,940 for them to warm up 638 00:24:33,840 --> 00:24:38,760 so the line after that if you see a nice 639 00:24:35,940 --> 00:24:41,100 flat stable line for your load that you 640 00:24:38,760 --> 00:24:42,299 generated it means you're looking pretty 641 00:24:41,100 --> 00:24:44,220 good 642 00:24:42,299 --> 00:24:46,740 um and you've got a stable application 643 00:24:44,220 --> 00:24:49,080 with no obvious bottlenecks 644 00:24:46,740 --> 00:24:52,260 in Locust you'll also get detailed 645 00:24:49,080 --> 00:24:54,299 information about the specific URLs that 646 00:24:52,260 --> 00:24:55,980 you've called and then what the average 647 00:24:54,299 --> 00:24:58,200 response times were and what the 648 00:24:55,980 --> 00:24:58,740 percentiles were 649 00:24:58,200 --> 00:25:01,679 um 650 00:24:58,740 --> 00:25:03,900 generally you can ignore the the average 651 00:25:01,679 --> 00:25:06,539 or the the mean 652 00:25:03,900 --> 00:25:08,400 um because the 95th percentile is much 653 00:25:06,539 --> 00:25:10,200 more accurate representation of what 654 00:25:08,400 --> 00:25:12,720 people will see so if you're looking at 655 00:25:10,200 --> 00:25:15,240 these at this data 656 00:25:12,720 --> 00:25:17,039 um often you get really weird outliers 657 00:25:15,240 --> 00:25:18,720 because of the network or because it was 658 00:25:17,039 --> 00:25:20,400 just booting up 659 00:25:18,720 --> 00:25:21,720 um but once it's out and it's stable 660 00:25:20,400 --> 00:25:23,159 that's the number you want to be looking 661 00:25:21,720 --> 00:25:24,779 at 662 00:25:23,159 --> 00:25:28,320 okay 663 00:25:24,779 --> 00:25:30,960 so something else you can do in Locust 664 00:25:28,320 --> 00:25:33,659 is you can set a ramp so you can say 665 00:25:30,960 --> 00:25:36,240 okay we want to get to a thousand users 666 00:25:33,659 --> 00:25:37,740 virtual users but they're not all going 667 00:25:36,240 --> 00:25:39,539 to go and start at the same time so 668 00:25:37,740 --> 00:25:41,460 we're going to slowly introduce more and 669 00:25:39,539 --> 00:25:43,500 more users so one of the options when 670 00:25:41,460 --> 00:25:45,720 you start Locust is 671 00:25:43,500 --> 00:25:46,679 um how many uh what ramp time do you 672 00:25:45,720 --> 00:25:49,440 want to have 673 00:25:46,679 --> 00:25:51,299 so if you slowly introduce five ten 674 00:25:49,440 --> 00:25:53,419 users at a time over a five minute 675 00:25:51,299 --> 00:25:55,799 period or every one minute period 676 00:25:53,419 --> 00:25:58,020 that is a lot more gentle on your 677 00:25:55,799 --> 00:26:00,000 application and is a lot less likely to 678 00:25:58,020 --> 00:26:02,159 introduce bottlenecks which are 679 00:26:00,000 --> 00:26:05,460 synthetic they're not real 680 00:26:02,159 --> 00:26:07,020 um because you're unless that is you're 681 00:26:05,460 --> 00:26:08,760 trying to load test 682 00:26:07,020 --> 00:26:10,500 what happens if people will go and click 683 00:26:08,760 --> 00:26:12,260 on the website at the same time if 684 00:26:10,500 --> 00:26:14,820 you've got a um 685 00:26:12,260 --> 00:26:17,580 the census is gonna remember that 686 00:26:14,820 --> 00:26:18,659 dilemma with the census right oh who 687 00:26:17,580 --> 00:26:21,500 could have predicted that everyone would 688 00:26:18,659 --> 00:26:21,500 leave it to the last minute 689 00:26:23,880 --> 00:26:27,240 yeah unless we told everyone to do at 690 00:26:25,500 --> 00:26:29,340 the same time so if you've got that type 691 00:26:27,240 --> 00:26:31,320 of application or you're running a quiz 692 00:26:29,340 --> 00:26:33,000 for a soap opera where you go and tell 693 00:26:31,320 --> 00:26:34,860 everyone in the advert which airs at the 694 00:26:33,000 --> 00:26:36,659 same time to click on the website uh 695 00:26:34,860 --> 00:26:39,179 then don't do this but otherwise you can 696 00:26:36,659 --> 00:26:40,799 slowly ramp up your users 697 00:26:39,179 --> 00:26:42,960 um the thing that you're trying to look 698 00:26:40,799 --> 00:26:45,120 at in your graph which is the response 699 00:26:42,960 --> 00:26:46,740 times is where I've pointed the arrow so 700 00:26:45,120 --> 00:26:49,440 we saw we had a nice sort of stable 701 00:26:46,740 --> 00:26:50,940 response time in the 95th percentile and 702 00:26:49,440 --> 00:26:53,340 then all of a sudden when I got to a 703 00:26:50,940 --> 00:26:56,340 certain number of users which was 190 704 00:26:53,340 --> 00:26:58,500 then the response time start to creep up 705 00:26:56,340 --> 00:27:01,740 what that's actually showing me is that 706 00:26:58,500 --> 00:27:03,539 around 190 users there's some sort of 707 00:27:01,740 --> 00:27:05,820 bottleneck in my application so when 708 00:27:03,539 --> 00:27:07,500 you're looking at Locus results 709 00:27:05,820 --> 00:27:09,900 um if you see a stable line that's good 710 00:27:07,500 --> 00:27:11,520 if you see at some point it starts to 711 00:27:09,900 --> 00:27:13,320 climb upwards there's some sort of 712 00:27:11,520 --> 00:27:14,640 bottleneck Locust isn't going to tell 713 00:27:13,320 --> 00:27:16,440 you what the bottleneck is it's just 714 00:27:14,640 --> 00:27:18,179 going to tell you there is one 715 00:27:16,440 --> 00:27:20,640 so we can see the response times 716 00:27:18,179 --> 00:27:23,279 increasing so somewhere 717 00:27:20,640 --> 00:27:26,400 there's a queue being introduced so this 718 00:27:23,279 --> 00:27:28,380 is the city skylines we see traffic jams 719 00:27:26,400 --> 00:27:29,580 um in real applications somewhere 720 00:27:28,380 --> 00:27:32,100 there's a queue it could be a database 721 00:27:29,580 --> 00:27:34,980 could be the network server is hard to 722 00:27:32,100 --> 00:27:36,779 know but you know there is one 723 00:27:34,980 --> 00:27:38,220 um so once you've got that then you can 724 00:27:36,779 --> 00:27:40,140 start doing profiling which is a 725 00:27:38,220 --> 00:27:42,480 completely different talk and I'm not 726 00:27:40,140 --> 00:27:44,640 going to tell you how to do that 727 00:27:42,480 --> 00:27:47,640 um so yeah hopefully this gives you a 728 00:27:44,640 --> 00:27:50,520 good introduction to load testing with 729 00:27:47,640 --> 00:27:52,500 Locust and you can download Locus and 730 00:27:50,520 --> 00:27:54,900 test it out if you find it too 731 00:27:52,500 --> 00:27:56,880 cumbersome to write all the HTTP 732 00:27:54,900 --> 00:27:58,740 requests using that method then I 733 00:27:56,880 --> 00:28:01,500 recommend giving the playwright one a go 734 00:27:58,740 --> 00:28:04,140 and using the playwright user and then 735 00:28:01,500 --> 00:28:05,700 you can record tests in your browser 736 00:28:04,140 --> 00:28:09,840 um playwright's also got the benefit 737 00:28:05,700 --> 00:28:12,659 that you can export it as Pi test as a 738 00:28:09,840 --> 00:28:14,159 pi test test file so not only are you 739 00:28:12,659 --> 00:28:16,320 generating a load test but you can 740 00:28:14,159 --> 00:28:18,120 actually generate a UI test as well so 741 00:28:16,320 --> 00:28:19,980 if you go and you click through the 742 00:28:18,120 --> 00:28:21,720 website and you click on the buttons and 743 00:28:19,980 --> 00:28:24,720 you check that that button is still 744 00:28:21,720 --> 00:28:26,340 there and it's not over there you can in 745 00:28:24,720 --> 00:28:27,960 the drop down in the code generator I 746 00:28:26,340 --> 00:28:29,760 showed you Pi test is one of the options 747 00:28:27,960 --> 00:28:31,980 so you can save that and you can just 748 00:28:29,760 --> 00:28:34,140 run it in pi test introduce that into 749 00:28:31,980 --> 00:28:36,779 your CI CD and now you've got automated 750 00:28:34,140 --> 00:28:40,440 UI testing so win-win 751 00:28:36,779 --> 00:28:43,860 cool so in conclusion 752 00:28:40,440 --> 00:28:46,919 start by analyzing your users if you 753 00:28:43,860 --> 00:28:48,179 have any if you don't then pick some 754 00:28:46,919 --> 00:28:50,700 friends and get them to test the 755 00:28:48,179 --> 00:28:52,860 application and see what they do 756 00:28:50,700 --> 00:28:54,539 um try and Vary your inputs 757 00:28:52,860 --> 00:28:57,480 um so don't hard code everything to one 758 00:28:54,539 --> 00:28:59,580 vary the timing the user requests and 759 00:28:57,480 --> 00:29:01,320 the data because users are random so 760 00:28:59,580 --> 00:29:03,840 you've got to try and simulate some of 761 00:29:01,320 --> 00:29:05,700 that and remember keep remembering the 762 00:29:03,840 --> 00:29:08,460 goal is to simulate real user traffic 763 00:29:05,700 --> 00:29:11,159 and that's what you're trying to do 764 00:29:08,460 --> 00:29:14,279 and beware of false performance from 765 00:29:11,159 --> 00:29:16,340 caching which I showed you so that kind 766 00:29:14,279 --> 00:29:18,000 of comes back to the randomness element 767 00:29:16,340 --> 00:29:21,000 and 768 00:29:18,000 --> 00:29:23,100 um optimize when you finish this so once 769 00:29:21,000 --> 00:29:25,200 you've got a good Benchmark you know 770 00:29:23,100 --> 00:29:27,360 what the locus results look like once 771 00:29:25,200 --> 00:29:29,700 you've got that then you can jump into 772 00:29:27,360 --> 00:29:31,559 application start introducing caching 773 00:29:29,700 --> 00:29:33,659 start increasing the number of servers 774 00:29:31,559 --> 00:29:35,640 you have or whatever it is and then 775 00:29:33,659 --> 00:29:37,820 you've got a baseline run the same test 776 00:29:35,640 --> 00:29:40,020 again and then compare the results 777 00:29:37,820 --> 00:29:42,659 Locust actually lets you do side by side 778 00:29:40,020 --> 00:29:44,399 comparisons as well so if you need to go 779 00:29:42,659 --> 00:29:46,500 and demonstrate to whoever it is the 780 00:29:44,399 --> 00:29:48,240 engineering manager or colleague that 781 00:29:46,500 --> 00:29:50,220 you want to make this change and if we 782 00:29:48,240 --> 00:29:52,620 make that change it's going to mean you 783 00:29:50,220 --> 00:29:54,539 know the app runs faster for 100 users a 784 00:29:52,620 --> 00:29:56,580 thousand users whatever and then it's 785 00:29:54,539 --> 00:29:59,500 good to to prove that 786 00:29:56,580 --> 00:30:01,200 cool all right thank you very much 787 00:29:59,500 --> 00:30:03,230 [Applause] 788 00:30:01,200 --> 00:30:07,200 foreign 789 00:30:03,230 --> 00:30:07,200 [Applause]