foreign
welcome back
all right
um now we've got beat the rush designing
effective load tests for your web
application by Anthony Shaw
um take it away Anthony cool thank you
okay hey everybody
um I hope you're all having a good day I
flew in this morning from Sydney
um you might be able to tell from my
accent that I'm not from around here
um I'm actually English so before we get
into load testing I'm going to talk
about a couple of things and one is
football the second one is T and the
third one is the queen and how they all
relate to
um to load testing
so if you want to follow along
um
this is my website the top link second
one is my
dead bird X account and then I'm also a
mustard on if you're on Mastodon as well
so
um
load testing and tea what do they have
to do with each other so uh
I guess over the last few weeks
everyone's gone a bit nuts with the
matildas
um the last game was the most watched TV
program in Australian history apparently
11 million people watched it
um I'm originally from the UK uh soccer
is not a new thing for us we've kind of
been into it for a while there's a
famous problem and that's what happens
at half time
now at half time however many people are
watching the game we'll go into the
kitchen and turn the kettle on to make a
cup of tea
like you might think that's not really
an issue but when you've got 20 30
million people watching it and they all
turn on a two kilowatt Kettle all at the
same time it effectively melts the
electricity grid
um the second thing they do is to go to
the bathroom so what they have to do
when they know there's a big game in a
world cup is they have to test
effectively load test two things one is
electricity grid and the second thing is
a sewage system because effectively yeah
both of them get low tested uh pretty
hard so England versus Italy 2020 1.1
million kettles simultaneously boarding
was a 600 megawatt load
um
all in a huge Spike
now we're going to talk about load
testing and I want you to keep thinking
about this analogy because not all load
tests look like this
um
one that I was involved with a long time
ago was a TV soap called Emmerdale and
they were running a series in Emmerdale
where you had to try and figure out who
killed this particular character and at
the end of the episode they'd say if you
want to find out more try and figure out
the clues click on this go to this
website and effectively we had a half
time problem with the kettles but a
website version of that
the other one I got involved with was
um the Royal Palace set up a website for
the queen
in the unlikely situation that she
passed away there's an obituary website
and they asked me to load test it
um this is a while ago so I had to load
test that site
um and I think when I heard the news
quite recently of her passing I was the
only person that went white in the face
and immediately went on the laptop to go
and check the latency of the website
um it did it did hold up so that's good
okay so
when you're designing a load test you
need to think about a couple of things
now you need to think about the number
of requests that you want to simulate
and I think about this as being like a
queue of people in a line a nice orderly
cue okay the other thing you need to
think about is concurrency
these two things are not the same so if
you say I want to test a thousand
requests is that one at a time or is
that a thousand that time
because when you think about people and
the strain that they can put on
infrastructure the right hand side looks
a bit like a stadium maybe or a gig
that's just finished and you've got a
lot of people all trying to leave an
area at the same time all trying to get
on the train at the same time so you've
got a high concurrency issue
so
um what you don't want to do is to use a
tool like Apache bench which is a great
tool for measuring the performance of
web servers but not applications and
because it will effectively generate
load on your website that looks
absolutely nothing like real load so the
numbers that you get back you might say
to your engineering team well we can
handle you know 12 requests a second and
it's going to hold that at 800
milliseconds which sounds great but that
load looks nothing like your actual
website load
what real user patterns actually look
like is uh this does anyone play this
game in the room okay a few of you do
okay so City skylines is brilliant
um now what real user load looks like on
a website is actually quite different
you've got different users going to
different parts of your apps at
different times and at different speeds
so
you know your users are not robots
they're not all clicking on things you
know in millisecond Precision they're
taking their time they're looking at
things they're clicking on stuff and
they're going in different directions
people also have different speeds of
their internet connections the latency
between them and the website is also a
bit different so
the reason I bring this up is because
you've got you know people going maybe
to the home page people going to a
different part of the app and people go
into the search what you can end up with
in many applications and in City
skylines is a bottleneck where you've
got users trying to go through a
particular thing at the same time so we
talked about postgres in the last talk
sometimes the database is the bottleneck
um sometimes it's the web server itself
but if you're simulating user load
instead of just trying to create network
traffic you can find these bottlenecks a
lot easier so not all users have the
same speed car or the same connection
not all users arrive at the same time
they take different Journeys and they
also wait between clicks so this is
really important
so if you want to understand your users
in a bit more detail so if you want to
design a load test the first thing you
want to do is actually understand your
users more what does the load look like
on your current application
if you haven't launched the application
you've got to guess so we'll talk a bit
more about that
so when you're thinking about users and
what they do with your app and what they
do with your website they take different
paths they'll click on different pages
those paths you can work out
probabilities of them you can say Okay
60 of users will start on the home page
twenty percent of them will then go on
the promotions page let's say so you can
look at your web history and see what
Journeys do people take and then if we
want to write a load test let's simulate
that and most of the load testing tools
will let you say designer a virtual user
and actually give them a path to take
so if you're analyzing the traffic first
and you can group users into personas
um so you know you've got like a casual
Shopper maybe someone who's actually
going to go and place an order then that
means you can create a more realistic
load on your application
um there are tools that you can get as
well that like you visualize this stuff
um so if you're looking at your web
traffic on your web history you can
actually see the journeys that people
take and then how many of them go
through that particular page
and once you've got that understanding
you can simulate the load a lot more
accurately than just throwing uh
requests at the web server
so another thing to take into account as
well is timing so
like I said before
users real users will wait between
clicks
so if you want to do a proper load test
you want to try and simulate some of
those pauses if you've got this kind of
analysis where you can see the user
flows it should tell you how many
seconds it takes between if you don't
have that you could just ask a couple of
colleagues to go and click through the
site and you can time them with a
stopwatch if you want or you can just
guess
um having zero weight between clicks is
unrealistic and it creates really weird
load and you'll actually find
bottlenecks that won't actually come up
in production so you'll try and
basically prematurely optimize problems
that wouldn't even exist when you're
putting the app in front of users
what users do instead is if we see this
like a timeline they'd start off on one
part of the application wait a bit go to
another page click on something else
click on something else and then maybe
they're done and they log off
so if you've got two users
they're on two different timelines
they're not synchronized with each other
unless they're looking over each other's
shoulder so if you've got two people
going on the website they're going to be
clicking on different parts of the site
at different times
so if you want to run a load test on
your application you want to simulate
some of this Randomness as well
something else which is really commonly
missed in load tests is if you've got a
a an app that does a lot of Ajax it's
got a lot of JavaScript some load
testing tools will ignore that so if you
just make HTTP requests to the server
and you simulate it and you think the
performance is brilliant then you
actually launch it and you put it in
front of a bunch of users and there's an
Ajax thing that pulls every 100
milliseconds and you've got 100 people
leaving their browsers open you just
ignored like a huge part of the actual
load of your app
so it's really important that you
understand is there an Ajax component to
my application and if there is what
differences that make to the load that's
put on the server
okay so the tool I want to talk about is
Locust there are other tools for load
testing I particularly like Locust for a
couple of reasons one is free and open
source which was a big tick a second one
is written in Python and also it's
really easy to get started with and
really easy to use
so to use Locust you just do pip install
Locust
and then you run Locus f with a locust
file a Locus file is a python file in
which you Define virtual users and
basically it uses each one of those
users as a script and it will create
load based on that script
so we're going to give you a couple of
examples of what a Locus script would
look like
so the first thing we do is we import a
HTTP user a task and then a function
called between and you define in a class
so you inherit from HTTP user and you
define a user so we you can call it what
you like I'll call minebasic user
because I think they're basic and
they're going to we're going to wait
between three and five seconds that's
what that statement means between each
new user
so they've got three tasks
um the first one is to go to a forward
slash which is the home page the second
task is to go to the destinations page
and then the third task is to go to the
specific destination
the task decorator basically means that
that method is a task and each user will
do all of those tasks
the argument three says that that task
is three times more likely to happen so
actually they're going to go to the home
page three times more than they would to
the other two
so that's more realistic because a lot
of people just go to the home page and
then
realize it's not for them and then click
off so if this is our basic user that's
going to create quite a bit of load to
the home page and a bit to the
destination and then the specific
destination page
so the between function means we'll wait
between three and five seconds per task
and then run that three times more often
um I mentioned Ajax before so we're just
doing a DOT get so this is a HTTP
request on that URI
um if you have a more complicated app if
you can't just simulate load by going to
URI
um there's another option which is to
use a playwright has anyone use
playwright in the room okay a few of you
uh playwright is a browser automation
tool that has a wrapper in Python
um what you can effectively do is if you
install playwright
um then you can run playwright code gen
and then do Target equals python it will
pop up a browser you can go to your
website and it's recording all
everything you do so you can click on
the buttons you can fill in forms you
can submit stuff what if isn't
JavaScript and on the right hand side
you'll see that python code window and
it generates the code of what you've
been clicking on so if you've got a
complicated app and it would be quite
time consuming and quite tedious to go
and actually code how to click on all
the buttons how to find that particular
drop down then you can just use the
playwright recorder and then there's a
plugin between Locust and playwright so
instead of just creating a HTTP user you
can create a playwright user and you can
give it that code that got generated
here and then it will actually open up a
browser and then drive the traffic using
the browser
uh one there's a link at the bottom to a
code sample of that all working uh one
caution I will give you if you're
simulating a large number of users on a
single machine
is one thing to do that using HTTP
traffic it's another thing to do that by
opening a thousand instances of Chrome
on your machine
so don't expect the same performance
because you won't achieve it
um okay so going more specifically into
into Django now
Django has a brilliant feature
um in that it's got built-in csrf which
is a security layer which means that you
can't inject stuff into forms if you're
not actually the person who did it
um this creates a bit of a complication
sometimes because
um one a Django form loads up on the
page you need to read a hidden field and
then you need to put that in the
submission when you click on a form so
if you have to do that you can do that
using basically doing a get first and
then to get the csrf token out of the
cookie and then include it in the post
request so in Locust when you're
programming HTTP users your tasks can
include a series of commands so not just
to get to the URL but you can actually
have a get to get to load the page and
then another request which is the post
cool so
um that is a sample of how to do that
and there's a link at the end of the
talk as well if you need to copy that
code snippet
um now that's kind of a bit of the nuts
and bolts of how to use Locust
um going back to the actual part of the
design of the load test because that's
where there's a lot of common I guess
pitfalls in terms of what you're testing
and how effective that is it actually
estimating whether your website's going
to hold up when it hits a lot of load so
a common mistake I see is that people
will deploy the application and then
they'll run a load test
and the number of users they have on the
application when they're spinning and up
and testing it is like one and the
number of products they've got in the
database or the number of widgets
they've got on the thing is like two or
three and that's your test ones right
um now when you actually look at look at
real load if you've got 10 000 products
in your production app and you've got
two in your test app and you're load
testing both of them those low patterns
are going to look very very different
because the number of SQL queries it's
going to be running on one versus the
other it's going to be different and
also if you've got pagination on your
page if you you know there's a lot of
different context things so what I
strongly recommend is that you seed test
data into an application or a server
before you load test it and there's a
couple of ways that you can do that
there's one approach that I really like
which I'll recommend but there
definitely are others this is to use a
package called Nemesis there's other
ones like Faker there's some other
packages for generating fake data I like
Nemesis because it's really good for
testing like internationalization and
you can actually test
you can create test data in different
languages with different contexts and
stuff like that so it's really helpful
so if you spun up an application and you
wanted to create a thousand users in the
system
you could do that just by writing a seed
function importing Nemesis and then
using this person object that person
object will generate a random name a
random email and a random password for
each one of those users and it's got
loads of other properties as well you
can put in their address their political
affiliations their favorite type of tea
like it's got everything in there so
um and then once you've done that you
want to use a bulk create call to add
all of those users in a huge batch
cool
so
here's a test question for you
what is the fastest implementation of
this problem
so return 610 is definitely the that's
the most efficient way of implementing
that function and it's the same when it
comes to web app so what um I've often
found with Django and with all
Frameworks with asp.net with other
languages with different infrastructure
is so often when you get a bottleneck or
you get slow performance in an
application the solution is to cash and
you look at caching from the top down
um so my lovely implementation of the
Fibonacci function is effectively just
caching it based on the answer and
actually most good implementations of
the FIB function will use an lru cache
um this one's just cheeky uh okay so
uh let's cache everything now in a
Django stack
um
you might think okay I've used the
caching module before
so you're kind of familiar with that
actually there's different parts that
you might want to Cache depending on
where the load is going to be and where
you're going to see benefit
um so at the top end you've got your
HTTP server so this could be
um
Apache or could be nginx for example
so you can cash out that layer so that
kind of cache would be like a CDN where
you've actually just cached the bytes
the HTML that's been generated and that
would look the same for everybody
um you've got whiskey and ASCII are not
familiar for any caching at that level
um you kind of then got the Django
application from there downwards and
you've got middleware views models
static files each one of those can kind
of be en cached using different
mechanisms
so underneath that in your middleware
you've got
um different middleware that you want to
plug in you've got templates you've got
the orm and then underneath that you've
got the database
so when we look at caching with the
Django app there's a whole stack of it
there's actually different places you
can do caching so at the top end you've
got a CDN or like an edge cache
so before you kind of jump in with
Django and say okay let's cache in the
template if the page is practically
static or the part of the site that
people are mostly going to is basically
static then just cache to that whole
page
there's no point in introducing caching
at a really low level when you could do
it really efficiently at the Top If the
generated result was always the same
that said if the first thing people have
to do in your Django app is log in
then introducing a cache at the top is
going to be pretty pointless because the
page looks different for everybody
so this is kind of why you have to look
at caching in terms of the whole stack
and which parts will be the same and
which parts will be different and where
does it make sense to Cache
um the template caching in Django I
recommend that everybody uses so
you don't want it to have to build and
generate the template every single time
especially for partials if the inputs
are the same and the output is the same
there's no point in Computing that
result for every single user that goes
to the application if it can be cached
um you've also got things on the
database layer most of these are you
don't have to configure but you can do
and that's kind of how things can be
cached in the database
you've also got object caches you can
just throw around all over python to
make things a bit faster
cool
um
now if you have introduced caching
um there's another Pitfall you can fall
into which I'll talk about which is you
design your load test
and in your load test you said okay
we're going to get destination number
one
and you've tested it and it works
brilliantly and you ship it to
production and it performs horrendously
and you're like but I tested it the
locations page was really fast and
really efficient and then when we
shipped it
it wasn't like what's the difference
and the difference is that I hard-coded
the id1 in my load test and it cashed
the hell out of that page and then when
we ship it to production we've got it
running across 10 different servers with
20 000 different destinations and people
are clicking on different ones and the
cache isn't preceded so the performance
is terrible so if you can
um try and introduce an element of
Randomness to your load tests so you're
actually testing different instances or
different IDs
this is across both if you're looking at
individual records or if you're
inputting data basically try and
simulate as much as you can the
randomness of a user not all of your
users are going to click on the same
page
um and if that's what that looks like
then try and test your cash
okay so if you've got all this working
and you've run Locust
you're going to get a graph
maybe that looks a bit like this
um and I'll tell you a bit more about
how to interpret the results so often
what you see with most applications
is uh so we've got two lines here one is
the median response time
and the other one is the 95th percentile
um the 95th percentile means that 95 of
responses are less than that number so
often when you're looking at performance
data you look at percentiles so you try
and ignore some of the outliers and you
say okay what performance will most
people see
and because the word most is objective
you can pick a number but 95 is pretty
typical 99 would be extreme so I'd say
95 is normally what you should be
looking at
um you'll see a spike often at the
beginning because things are warming up
so that part is uh effectively a warm-up
of either the cache the server the
instances you know there might be in
some processes which were asleep and all
of a sudden you're throwing thousands of
requests at them so it takes a minute
for them to warm up
so the line after that if you see a nice
flat stable line for your load that you
generated it means you're looking pretty
good
um and you've got a stable application
with no obvious bottlenecks
in Locust you'll also get detailed
information about the specific URLs that
you've called and then what the average
response times were and what the
percentiles were
um
generally you can ignore the the average
or the the mean
um because the 95th percentile is much
more accurate representation of what
people will see so if you're looking at
these at this data
um often you get really weird outliers
because of the network or because it was
just booting up
um but once it's out and it's stable
that's the number you want to be looking
at
okay
so something else you can do in Locust
is you can set a ramp so you can say
okay we want to get to a thousand users
virtual users but they're not all going
to go and start at the same time so
we're going to slowly introduce more and
more users so one of the options when
you start Locust is
um how many uh what ramp time do you
want to have
so if you slowly introduce five ten
users at a time over a five minute
period or every one minute period
that is a lot more gentle on your
application and is a lot less likely to
introduce bottlenecks which are
synthetic they're not real
um because you're unless that is you're
trying to load test
what happens if people will go and click
on the website at the same time if
you've got a um
the census is gonna remember that
dilemma with the census right oh who
could have predicted that everyone would
leave it to the last minute
yeah unless we told everyone to do at
the same time so if you've got that type
of application or you're running a quiz
for a soap opera where you go and tell
everyone in the advert which airs at the
same time to click on the website uh
then don't do this but otherwise you can
slowly ramp up your users
um the thing that you're trying to look
at in your graph which is the response
times is where I've pointed the arrow so
we saw we had a nice sort of stable
response time in the 95th percentile and
then all of a sudden when I got to a
certain number of users which was 190
then the response time start to creep up
what that's actually showing me is that
around 190 users there's some sort of
bottleneck in my application so when
you're looking at Locus results
um if you see a stable line that's good
if you see at some point it starts to
climb upwards there's some sort of
bottleneck Locust isn't going to tell
you what the bottleneck is it's just
going to tell you there is one
so we can see the response times
increasing so somewhere
there's a queue being introduced so this
is the city skylines we see traffic jams
um in real applications somewhere
there's a queue it could be a database
could be the network server is hard to
know but you know there is one
um so once you've got that then you can
start doing profiling which is a
completely different talk and I'm not
going to tell you how to do that
um so yeah hopefully this gives you a
good introduction to load testing with
Locust and you can download Locus and
test it out if you find it too
cumbersome to write all the HTTP
requests using that method then I
recommend giving the playwright one a go
and using the playwright user and then
you can record tests in your browser
um playwright's also got the benefit
that you can export it as Pi test as a
pi test test file so not only are you
generating a load test but you can
actually generate a UI test as well so
if you go and you click through the
website and you click on the buttons and
you check that that button is still
there and it's not over there you can in
the drop down in the code generator I
showed you Pi test is one of the options
so you can save that and you can just
run it in pi test introduce that into
your CI CD and now you've got automated
UI testing so win-win
cool so in conclusion
start by analyzing your users if you
have any if you don't then pick some
friends and get them to test the
application and see what they do
um try and Vary your inputs
um so don't hard code everything to one
vary the timing the user requests and
the data because users are random so
you've got to try and simulate some of
that and remember keep remembering the
goal is to simulate real user traffic
and that's what you're trying to do
and beware of false performance from
caching which I showed you so that kind
of comes back to the randomness element
and
um optimize when you finish this so once
you've got a good Benchmark you know
what the locus results look like once
you've got that then you can jump into
application start introducing caching
start increasing the number of servers
you have or whatever it is and then
you've got a baseline run the same test
again and then compare the results
Locust actually lets you do side by side
comparisons as well so if you need to go
and demonstrate to whoever it is the
engineering manager or colleague that
you want to make this change and if we
make that change it's going to mean you
know the app runs faster for 100 users a
thousand users whatever and then it's
good to to prove that
cool all right thank you very much
[Applause]
foreign
[Applause]