and now we welcome to the science data
and analytics track of pycon australia
2021 marlene mangami
malin is a zimbabwean software engineer
and
engineer exploring speaker based on the
city of harare an advocate for using
science and technology for social good
she's a director and the vice chair for
the python software foundation and is
currently working as a software engineer
with the rapids team at nvidia
in 2017 chico founded zimbo pi a
non-profit organization that gives
zimbabwean young women access to
resources in the field of technology she
is currently pursuing a computer science
degree with a concentration in machine
learning and ai at the university of
london
her her talk is recorded so there won't
be an introduction or q a but you can
find her at home on the web
marlenemangami.com
the talk is about python and gpus maleen
will give us a brief history of python
and parallel computing including
python's infamous gill
and she'll also share how and why
pythonistas can get started with using
gpus to do more work and faster welcome
to python australia marlene mangami and
with all of you a beginner's guide to
gpu for by gpus for pythonistas
i'm so excited to be here with you today
for the virtual version of pycon line
for pycon australia um the title of my
talk today is a beginner's guide to gpus
for python easters i'm very excited
about this talk if you have any
questions after i give my talk feel free
to
find me on twitter and just send me a dm
i think i might be
around after this talk as well just to
answer any questions too
first thing something about me is that i
am
the current vice chair and a director at
the python software foundation with the
which is the non-profit organization
behind
python programming language it's a very
cool organization that i encourage you
to look into
something else about myself is that i am
a co-founder of a non-profit
organization here in zimbabwe which is
actually where i'm calling you from
i'm called zimbo-pai and this
organization just tries to get girls
excited about programming in python and
then uh
something else about myself is that
a final thing about myself actually
so that this isn't too long is that i'm
currently interning as a software
engineer with the rapids team at nvidia
and i am working on a python um gpu
based data frame library that i will
talk a little bit about um
some of the work that i'm doing in this
talk as well and the library i work on
is open source so you can also check out
the code after if you'd like
so let's go ahead and dive right in i
think a good place to start is
understanding why python easter should
be interested in gpus in the first place
typically when people think about gpus
they immediately their mind goes to
graphics cards or gaming and i know for
myself i was really surprised when i
started reading about this that
gpus were being used for anything else
outside of the gaming world and this
assumption is actually not too far off
base
in fact if we did a quick search on
wikipedia which as many of us know is
the source of all truth
we find that gpu is actually an acronym
for graphics processing unit and so the
original purpose of the gpu was to
create images for computer graphics and
gaming consoles
so that assumption that i had and i
think many people around the world have
is is actually fairly accurate since the
early 2010s we saw a sharp rise in the
use of gp gbus or general purpose
graphic processing units
and as one can imagine general purpose
gpus are gpus that can be used
for more general purposes
and typically
these sorts of gpus are used instead of
a cpu for executing tasks that can be
run in parallel so i think i have a
pretty good example uh to just
demonstrate what this means so one of
the things that i absolutely love about
conferences being remote these days is
that
i don't have to wait in line for a cup
of coffee like if i if i would like
coffee right now i just can get up and
go to my kitchen and have some coffee
but sometimes
you'll attend these conferences that are
great and and not all the time as well
but you can find yourself in these
really long lines
waiting to use a coffee machine and
sometimes it's a single coffee machine
and now you're standing in line and
you're debating whether you should go
and give your talk like a mature adult
or just stay and wait for a precious cup
of coffee goodness
but there are a number of solutions to
this problem
and
one solution that a conference
organizers if they can afford it can
consider is to add a
an extra coffee machine into the mix
and this is great because it allows
things to move much faster because the
cues
like generally become shorter and and
maybe are even cut in half or even into
a third and people are able to access
their coffee grab it and go and get on
with their day and be more productive so
we can think of the cores in our
computers in the same way we would think
about these coffee machines in many
cases utilizing more cores can make us
more productive in the work that we do
so a typical cpu contains anywhere from
between four to eight cores whereas a
gpu can contain thousands of them
so having a higher number of cores
increases the computational power that
we have and allows us to do more work
faster
so
just before i get a slew
of
emails from people who love cpus this is
not me throwing cpus under the bus i use
cpus actually there are some tasks that
are much better suited to using a cpu as
compared to a gpu so for example if you
have a task that requires a lot of
serial processing where you need to
purchase things one by one in a sequence
it would definitely be best to use a cpu
in that case um gpus are great but they
expect to run things simultaneously and
so if your computations cannot be done
simultaneously then it's abyss to stick
to a cpu and by the way you actually can
take advantage of
more cores in your cpu
by using things like multi-threading or
async io among other things to boost
your performance
locally with the cpu so from what we've
talked about so far we've established
that if you're a pythony style who needs
to
do some computations that can be run
simultaneously
gpus are a really great option for you
to consider because uh ultimately your
work will be done much faster because
your code will be running much faster
and since this is a beginner's guide
some people watching someone watching
might be uh thinking to themselves
marlene i have been using python for a
long time
and i don't understand why i would need
to speak python up it seems pretty fast
to me
and that's you know i think a good
question
uh to to to pose to to that question as
well is
um is python slow
and uh believe it or not this is
actually quite a contentious question
and when i looked online for answers to
this question i found some opinions that
were helpful
but others that were to be honest
probably borderline slander
and i personally came to the conclusion
that the slowness of python is actually
fairly relative and many developers will
likely never encounter this slowness
that people talk about um but to be fair
there is
some level of truth to that statement
and i do think that it's important that
when we are talking about python um
potentially being so that we give some
context and explain what we're comparing
python to so last year in 2020 anthony
shaw gave a really great talk at python
us and the talk was titled why is python
slow and in that talk
he
put this this really great graph that's
on the screen right now um and this is a
graph that shows the in body algorithm
and it shows how long different
languages take to run this algorithm and
uh the program models the orbits of
jupiter saturn uranus and neptune and is
pretty uh sort of intense python for
some of you some of you may know that
python is a an interpreted language that
is written and compiled in c
but if you look at the chart and you
compare
the time it takes for c to
run this algorithm c takes seven seconds
while python takes 14 minutes
i don't even know how many times
slower that is and you know someone
might be thinking okay but c is actually
a strongly typed and compiled language
while we know that python is dynamically
typed and interpreted so you know that's
not really a fair comparison but if we
look at another language like node.js
which is also
dynamically typed
uh similarly to python we still see that
python is much slower than node.js
so
if you'd if you're really interested in
this specific
example and would want to find out some
specific reasons about why this is the
case i definitely recommend you watching
anthony's talk it's a fantastic talk
but
something to note is that even though
this might be the case according to
stackoverflow's developer survey python
beats out almost every single other
language in this chart in terms of
popularity and the truth is python to me
at least i think to a lot of other
people
is a really good and fun language to
program in python is such a joy to use
and so if you really need that um if you
would want that speed up and making sure
that you have uh your code running as
fast as possible but also want to
continue enjoying coding in python there
are lots of ways to be able to do that
earlier today i talked about how
utilizing more cores through things like
multi-threading and concurrency can help
us get better performance from our
python code and in the past python
hasn't actually had a very friendly
reputation with things like
multi-threading and parallelism because
of uh something called
the guild
so
you know i wouldn't
really have talked about python and
concurrency python and speed even
if i don't at least
mention the gill exactly is going on
with python skill and if you search
online for python talks on the gill you
get this very strange mix of suggestions
um
as you can see you know some people are
talking about removing it completely
um others are talking about
understanding it more trying to help
others understand it i think
and even even more you know recently
eric snow gave a talk asking whether to
kill or not to kill
so generally it seems like it has this
really big
reputation so so what is the gill
um well the girl is python's global
interpreter lock and this is a lock in c
python that preserves its internal
shared global state and what
occasionally makes the gill
controversial is that because of it no
more than one thread can run at a time
and for people that are wanting to
utilize things like multi-threading um
you know which is running many different
threads at the same time to speed up
their work the girl can stand in their
way potentially however
in recent years the girl has become less
and less of a barrier for people who
want to speed up their python code in
this respect
and gpus are only one of a number of
different ways to get around it and i
thought
this uh this next clip that i have on
the screen is a clip from raymond
hittinger's 2017 keynote at pi bay and i
thought it was really helpful in terms
of helping understand why the gill is
important and some ways that you can
actually get around it
our friend larry hastings is working on
a project called the gillectomy to
remove the global interpreter lock from
python do you think that's a difficult
project
that is an incorrect hypothesis
uh takes about a day of work
to remove the gill
he removed the gill on the first day it
was no problem taking it out
the problem then becomes all of the
locks that you have to put in everywhere
else in order to get python uh to
function it turns out that's not
particularly hard either and a few days
later he had that done
so the gill is gone and replaced by uh
lots of other little locks on smaller
data structures problem solved any
questions
oh
are locks expensive to acquire in
release
in fact they are and so the good news is
the gill is gone it's free threaded and
it's uh dozens of times slower than
regular uh python
so you actually get a payoff for the
guild and the payoff is you don't pay
all of the performance cost of all of
these individual lock acquires and our
releases is actually a really nice thing
uh to have it gets in the way of us free
threading but we have ways of solving
that problem if you can't fully free
thread one python why don't i run eight
pythons in parallel each with their own
threads
and then it's no problem i'm taking
advantage of all of other cores or you
can combine threading and
multi-processing uh there's a number of
ways to go in fact at some point most
folks just get over the python has a
global interpreter lock go ahead and
saturate all eight cores to a hundred
percent and get full advantage of the
other machine and just simply ignore the
problem there's lots of ways to ignore
the global interpreter lock it is not
that big of deep okay so like raymond
said when we utilize all of our cores we
don't really need to worry about the
guilt and with gpus we have thousands of
cores so if we decide we wanted to even
just run one thread on each core we
would still have a really significant
speed up okay so
great now that we've looked at some of
the big performance issues that would
lead us to possibly using a gpu when
we're running python let's go ahead and
look at some practical examples of how
we can do this
so modern datasets can have as much as
millions even billions of data points
that need processing and for the average
data scientist that is working with
these large data sets just using
ordinary python for processing this
information is not very efficient and
most of the time you know they would
spend if they chose to to run python
only they could spend a lot of time just
waiting around and not being productive
but these days data scientists wanting
to perform larger computations on
datasets or wanting to filter through
information in larger quantities
there are a number of different
libraries available so pandas is
probably the most popular solution for
this
it's a data science library written in
python c and cython
that uses vectorized operations in order
to run functions at very high speeds and
although it doesn't use gpus
let's quickly take a look at how much of
a speed up we get when we try to create
a data table with pandas versus with
python
at the moment we are starting off on
google colab and this specifically for
context is
the rapids notebook rapids has a really
great um
free notebook on um if you just go to
the get started page
you can have access to this notebook and
basically it allows you to test out a
gpu and the different rapids
libraries very quickly and
we'll just walk you through some of the
basics of that
we are not yet starting off with our gpu
but just to give you some extra context
at the moment we are going to we are
importing our libraries and then we're
creating um some data using uh numpy
some random data and then
we're going to start off by creating a
pandas data frame and this is you know a
very simple way to do it and then after
we create this data frame we're going to
see how long it takes using
the time module we're going to see how
long it takes for
pandas to calculate the mean
of the data in this column
okay so
i'm just going to go ahead and do that
i'm going to create a
separate column in the pandas data frame
and i'm going to go ahead and
use the pandas rolling method um to
calculate the mean and this is a great
method that really speeds things up
um
yeah so let's go ahead and see how long
it takes for
um pandas to calculate the mean oh that
was really quick
and it takes about 0.03
seconds as you can see there which is
really fast um
now the next thing we're going to do is
we're going to figure out how long it's
going to take for
python to do the same thing and instead
of using the python rolling method we're
going to create as i'm doing now
we are going to create a for loop and
basically what this for loop is going to
do is it's uh we're going to use the
mean loop column in the data frame and
we are going to just loop through all of
the values
and calculate their means
and add them to
the
uh to the data frame as well um but this
time of course using python and not
pandas
so as you can guess this is probably
going to be longer we're going to um
just going to fill this up but we're
going to
um
yeah we're just going to we're going to
calculate this and also time it and see
how long it's going to take
for all of these loops to run let's just
comment out that code
and let's run it to see how long that's
going to take
so
the loop here
using
python looping over took a full one
second
and 16 seconds it's just
do with both that's a lot that's a big
difference
um that's a really significant
difference
so as you can see
it's possible to get a speed up of about
500 times just by switching to this
library this probably explains why most
data scientists choose to work with
pandas as compared to writing their code
in pure python
now even though you'll get a significant
speed up just by switching to this sort
of workflow
from python to pandas making use of a
gpu especially for larger datasets can
take things even further than earlier i
have been interning as a software
engineer with the rapids team at nvidia
and i've really enjoyed working uh with
the team there i've enjoyed working on
this project um qdf is an open source
python based gpu data frame library and
for
data scientists pythonistas that are
wanting to work with data
it allows you to do a lot of those
operations like loading or joining
aggregating and filtering and different
things like that
but it makes it possible for you to do
those computations on the gpu
and
how
the team has designed this
qdf is to really mirror the panda's api
so a lot of the things that if you can
do something
in pandas you should be able to do that
same thing in korea so
i think as well because the apis are so
similar in how they've been written um
it's really intuitive to make a switch
so i'm going to go ahead and show you an
example of what it looks like
right
so
here we would like to compare pandas and
qdf and see the difference in
speed for this we're going to be using
time it
to just see how
how long it takes for pandas to do some
calculations and how long it takes for
qdf to do calculations so as you can see
um
right now at the top we are creating a
pandas data frame and this data frame
has significantly more data than last
time and then i want to show just how
easy it is to convert a pandas data
frame to qdf
and uh it's it's a very similar
instruction and then
you're using the method from pandas and
just putting in um your original pandas
data frame you can also create your own
um
you can also create your own
qdf data frames from scratch if you uh
if you'd like to as well
and so we are going to start off by
you know using pandas to calculate the
mean
of the data that is in our pandas data
frame and we're going to do the same
thing as well with qdf and we're going
to time both of these
operations and see how long it takes um
to do this with pandas and then to see
how long it takes to do this with kudith
so
go ahead and run this
and see
what the time takes
so it takes um
that's great it takes about
156 milliseconds per loop for pandas and
3.56 milliseconds per loop for
qdf which is really significant
so i did want to also show you a second
calculation um which is just merging um
data together
and
i ran out of ram
to do this on google collab because it
is a much larger computation to do and
so
instead of trying to do it here we're
just going to move over to
um
to
a jupiter notebook that
is using my own local gpu
so here we are in our jeep to notebook
and uh behind the scenes we are running
a
gpu
and so we are still able to access our
cdf gpu data frame
we have a similar setup from what we had
last time we're importing our libraries
and creating the same data frame with
pandas
this time we're trying to merge the data
together at first with pandas and then
um with qdf so i'm just running that
and
yeah so i had to this was this took
quite a long time to run
particularly with pandas so i had to
speed things up
so i'm just going to jump forward a
little bit
all right so it did take quite a long
time to
get through the calculation
with pandas i actually cut out a whole
section of time
and and
here pandas to go through all of it took
about um one minute 10 seconds
plus or minus
486 milliseconds per loop
which is quite a lot of time
to go through all of those loops
it was it was a lot of data so it you
know pandas was quite slow
but next we can just let it go through
and calculate and as you can see
for um kudif it was much faster it took
about 367
milliseconds um to calculate to
calculate that for per loop
um which is a lot faster so definitely
that's a fantastic way to get speed ups
and from your python code but qdf is not
the only way that
python easters are using or can use gpus
there are lots of fields where
people using python can get speed ups in
their python code one of those fields is
mathematics you know we have in data
visualization machine learning and ai
gaming and graphics just lots of ways
that python is can utilize gpus to get
speed ups
you might be watching this and thinking
to yourself you know marlene
i can't afford a gpu or i don't know
where to get a gpu
um you can definitely google it but um
there are some great um resources online
and
right now i would say gpus are quite
accessible online
and in some cases you can actually get
them for free or use them at a really
good sort of cost rate
and
here on the screen i have some examples
of places that you could access a gpu
even right away i know for the rapids
team we actually have a
a google collab notebook on the website
i'll put a link to the website there
that you can actually take a look at and
you can
go to the colab notebook and actually
try out the
um the different libraries for free
so definitely would encourage you to do
that if you would like to
um that is actually the end of my talk i
hope you enjoyed listening to it
if you have any questions feel free to
let me know after this talk or
feel free to also reach out to me on
twitter i'm marlene underscores dmw
thank you so much for for listening
hello again uh reminder that the
previous talk was recorded so there
would be no questions
but thank you marlene mangami anyway i
mean thank you for presenting at
pyconeyou
uh there's a rest now there's a bit of a
break for 45 minutes
last block of the day will begin at
4 15 australian eastern time
so take a break maybe hydrate maybe
stretch your legs see you then
you