foreign
is an independent independent software
consultant building python-based
back-end devops and Data Systems she
also teaches python via online
month-long boot camps she's an open
source Enthusiast and has contributed to
pandas and Apache airflow please put
your hands together for povani
thank you
thank you thanks for such a nice
introduction first of all hello everyone
hello everyone
okay nice good now we are all awake I
know we are hungry but
just to put things into perspective I'm
bhavani Ravi I've been fallen in love by
with python seven years back and it's
still going on I'm an independent
software consultant I get thrown into
technical problems and I Rise Up from
there that's what I do for living
and today we are here to build our own
unit testing Library
and when I put out this talk and tell to
all my friends that the talkers accepted
the very first question was why
Center
why even your own testing Library
and
I don't have a perfect answer for that I
have an answer for that one is it's fun
remember when you all first started
programming remember that moment when
you've fallen in love with that first
program that you wrote it gave you that
spark and kept you hooked all along mine
was printing the first 10 numbers and I
changed the program to ten thousand and
then to a lakh until the system crashed
but there came a point where programs
like that got really boring and now I
can do this for fun and I hope you can
too
and there's this Rich environments quote
what you don't create you don't
understand and I pretty much love this
quote
and as a result of that I created own
flask my own version of flask
application and that was almost a year
back
it's been too long since I've had fun so
now I am after Pi test
the only problem was I've been using pi
test pretty much
extensively the last uh eight months and
the hundred percent test coverage from
the previous talk all of that was so
relatable and I love all of these
features that by test provides the test
Runner parameterized test cases expected
exceptions fixtures mocking you name
them I use them
and
show all fans how many of you have used
by test
okay everybody almost everybody if you
have not used it your logic will look
something like this and your Associated
test case which starts with test
underscore followed by you call your
logic and test whether that uh that the
logic behaves the exact same way and
that's a very simple test case this is
your parameterized test case where you
pass a bunch of data and hope that all
the data that you're passing behave the
same way as well
and you can also expect your
functionality to raise a particular
exception and a pie test will capture
that
so I thought okay let's start building
it and the first place to start is
reverse engineer it and I went and
looked into Pi test source code and this
was the reaction
I
well
it has about 130k lines of code
earliest comment was on 2007 and so far
there are 5000 PRS being merged for the
project which is a lot more than flask
flask was very rather simple what I'm
trying to say with all this is It's hard
to reverse and generate
the next option was chargpt or GitHub
co-pilot because we are all relying on
that so much but things like this llms
really work if there are previous
examples available unfortunately even if
you Google right now there are not a lot
of examples on how do you build a
listing library from scratch
so again thrown out of all the options
here
so
building my own testing Library
reflecting whatever Pi test provides as
an interface I'm gonna have all these
features and let's see how that goes
first starting with the test Runner I'm
going to be live coding this and if I
make any mistakes please be kind to me
and the test Runner this is all our
version this is not by this version so
it has an interface that looks like this
you do PI test followed by the directory
that you want to test and if we think
about it how would you write a test
Runner
first problem is finding out all the
test functions that are there in all the
test files under that for the particular
directory and then you have to run them
maintain a streak of success and failure
and if there is an error you have to
report them can we live code it
okay
I did say we are going to do things from
scratch but I kind of lied
so we're gonna
start with this so here's my test
function okay
here's my own test module which is where
we're gonna fill out things uh the test
Runner is the only thing that's going to
run everything else is a race an
accepted reception parameterize picture
which we will come back and fill in
later here is my test Runner and some of
the complex complexities I have written
away using certain functions so
I have got all the test files I've got
all the test objects here and the only
thing that we have to do right now is
run the test cases so what do we mean
when we run the test case
we have a test uh sorry we have a test
file with a bunch of functions we have
loaded all that function into our module
now we have the test function name and
the test function object and running
would running them would actually mean
calling that test function object and if
I run this
so I'm going to use the own Pi test
module and pass all the test cases all
this functionality of reading the
directory reading all the functions has
been already written so if I run this it
says success no address has been found
whether it's really working or not I
might not know so what I'm gonna do is
in my case I'm going to make it 10 and
see what happens
okay
there we have it our first assertion
error so it's not just a matter of
calling the function to run the test
case you have to also maintain a streak
of all your errors
so
let's wrap this in a very nice try and
accept
all right so we're gonna wrap it in a
nice except and there we go so we're
gonna keep a track of our exception
and under here if there are any errors
we have to print them so let's print it
for
wow life coding is hard so here we go
I'm kind of cheating right here but it's
okay so we are capturing all the errors
we are getting a trace back and we're
printing them
hmm
so okay
we have something running here and then
there is an error under test ad these
are the this is a trace bag and where
the session error is happening let's
also give some more meaningful things so
that we know something is running
running desk keys
test function name
all right so we know that it's running
certain test cases there are certain
errors let's revert the spark that's our
test Runner so now we have a test Runner
which takes all the text objects runs
them marks them success marks them error
we're good to go
the next thing is the easiest one of all
accepted exceptions so looking at this
by test.racist we are going to implement
the races what can you make out of it
the interface
races is a context manager
and context managers if you have seen it
it comes with an entry and an exit block
it is initialized with an exception type
that you're expecting
and when exiting the block you have to
ensure that the particular exception
that you have mentioned should be raised
and if not we are going to raise an
assertion error
and
oops not here yet
okay
so let's go to our PI test.races own pi
test.races and this is my template block
so right now to test this particular
aspect of it I have a very nicely
written test case right here so this is
going to address type error if I'm
passing anything that's not a integer to
my uh
ad subtract multiple divide
so let's run this now it raises a type
error
and it's not being captured it's not
being handled because we haven't
implemented this Rises yet if you've
perfectly implemented it then it would
just pass through and Mark everything as
access because the function is working
as expected so at enter we don't have to
do anything at Exit we have to check
if the first the exception type is none
or not if it's none no exception has
been raised so that is there and if the
exception type is not there then we are
raising an assertion error both the
cases Association error different logic
but how is GitHub copilot doing all that
because I have practiced this almost 100
times so far so
assertion error raised when the
exception type is none or if it's not
matched so now let's run this
okay that's weird
since we have already handled the
exception we also have to Market as true
that you don't have to do anything just
everything is nice and good
so now it has ran the test cases it has
also ran the test Rises and things are
fine and well but
is it really working well
so instead of this zero division error I
am going to mark it as typewriter as
well now our function should fail
because it's expecting something and
resulting in something
there we go we have an assertion error
my yes raised typewriter has been raised
instead of a zero division error so yeah
kind of works there
let's revert this back
and life is all good nice and successful
again
good
the next thing is parameterized test
case again this isn't this this might
look very difficult but it's also very
simple
or the drill looking at
bytest.mark.paramel trials what can you
make out of it
what python construct it is
a decorator and if you look at it it's
not just a normal decorator it's a
decorator with arguments oh scary
when it's so first you are registering
the parameters the second your uh you're
registering the function and then you're
gonna return a wrapper all of that has
to happen
and when once you have the function then
the function is being called you have to
run it n times so that all your subtest
also runs
and lock the results even if there is
failure in either one of them you have
to ensure that that is captured and
finally you will get there
so Pi tested parameterize here is my
is going to fill it the keys are look
okay first the test case the own tester
parameterize looks something like this
here I've used Python's eval don't ever
use it in production
and
yeah so to the parameters I am passing
like three four Arguments for me to test
it and your keys are going to be this
your values are going to be this first
when registering the arguments I have to
put it in a nice format so that I can
just pass it as argument so let's make a
test args
list and here we are going to
okay
different than the previous one but sure
next what we're going to do is we are
going to when the function is being
called when your test case is being
called You're creating subtests so let's
create I comma test ARG
in numerate of test arguments
then we'll print that we are running the
subdes
and also call the particular function
with the test argument
so when previously when we were calling
the test function it was just calling
that one particular test object right
now what we are calling is the wrapped
decorator so inside the lab tech Creator
instead of directly calling the function
we are calling the function n number of
times based on the parameters that is
getting registered
so
let's run this
um
yeah so you have running sub uh subtest
0 1 2 depending on the parameters if I
go and add more it's going to add more
if I change something here that also
gets handled because we have taken care
of assertion errors globally not you
have to do you don't have to do anything
particularly inside the parameterization
so that is also good
great
all good now
we have a decorator that takes arguments
and a function looped over it all good
to go
the last but the scary part is fixtures
this is a scary thing for me very easy
to use very nice feature of Pi test but
this one
suck my brain out because I have to
implement it all by myself
and fixtures are great because you can
do a ton of things from there
you can define a whole large data set
and have it available and multiple tests
can use it or you can do your setup and
tear down and there are Pi test comes
with enormous amount of features this or
to use scoping you can put it in your
conf test and the variety no wonder they
have 130k lines right they want to make
your life easier so
it's a decorator again
and you can register a function as a
fixture
and when you are running the test case
you have to ensure that you are passing
the argument which is a fixture which in
turn is a function which is decorated
appeared yeah that's how I felt too
handling test argument that is a fixture
and then you have fixtures that are just
data fixtures which is going to return
dictionary or a list or a tuple or you
have yielding fixtures which is going to
yield and then do something later so all
of these things are going on
let's try to implement this
so here is our beautiful fixture
and that's it
that's all you have to do for fixtures
because
you register a function as a fixture and
your job is done but a lot of things
that you have to do is in your test
Runner you have to ensure that you are
passing the arguments so far when we are
calling the test function we are not
passing any arguments so first thing
that we have to do is to find out what
arguments like parameters are there
in your test function so this particular
construct code variable names will give
you all the arguments of a particular
function and for args in test args we
have to see whether that particular
picture is in a fiction mapping
but we haven't defined our fiction
mapping yet so let's go to that here I
have some logic to get all the test
functions this does nothing but get all
the members of the module
finds out all the functions that starts
with test underscore this time we are
also going to check if members of one
which is the object
Dot
name is a fixture wrapper because we
have wrapped it in a fixture then we are
going to store it in the fixture app
picture mapping so great
so it's in the fixture mapping then you
call it that is your test argument
let's also create a list
of course this code is never going to be
used or being production so all the
solid yagni principles doesn't apply
here
list
all right we are not going to use it
test
s
dot append we are going to call the
fixture and get the result put it in a
test art list
and gonna pass it to
desktop list now when I run this it's
gonna pass
let's uncomment these test cases here I
have a fixture that is getting
registered and we have a
a test con which is going to execute a
particular connection which is also
going to raise an operational error
because I don't even have the database
don't
it's raising an assertion error which is
weird because
it's expecting an operational error but
it's actually getting an attribute error
what do you think is happening here
Beyond errors like that has popped up
and the problem is because of what's
happening in con fixture if we go go
here and print out what is actually
getting passed when we are running the
test case is you can see that it's a
generator this is where I said it's an
yielding function not
a direct data fixture so we have to do
something even more crazy
uh
again back to our test Runner here we
have an argument which is the fixture we
know that great
now we have to get
let's get the fixture object
and we have to check if the fixture
object has
the next argument in it so that we know
it's a yielding gen it's a generator if
it's a generator we are going to call it
and then store that value
else
we're going to call that object directly
so that's the logic
there are two types of fixtures one is
an yielding fixture which is a generator
the other one is a direct fixture object
which just passes the data and we have
handled both the cases now let's run
this
oh still an assertion error this has
also happened quite a bit of time and
still legit
it's still a generator object
now I'm confused
any help
I didn't call it yes great
oh I don't call
hmm
it's a fixture object
we're calling it and then
okay this is why I was scared of doing
this but
okay I'm gonna add more print statements
very classic programmer move
a picture object what is coming here
it's a wrapper
right
okay I'm gonna pull out another classic
programmer move is have your backup code
go through that see what it does and
just go with it so here is my perfectly
running fixture
stuff I'm gonna copy paste that
all right so
we have the fixture value here we are
going to call return on it or you're
gonna directly call next on it or just
gonna directly append it
so that's there
perfectly out of sync okay much better
still a lot of things test arcs to pass
all right
thank you so yes that is our fixture
which is a healing fixture next we call
next on it if not if it's a data fixture
that will also work
so after a very so thank you for that I
didn't I forgot to put the thumbs up
over here
so if you want to go through the same
thing you can you can go to my GitHub
repository and try it out on your own
there is a blog post associated with it
with it which needs an update since a
lot has changed in the last three months
and what can you do to have fun is you
can try implementing mocks on top of
this or try testing this testing library
or testing your own testing library or
build your own eggs and have fun as a
part of it
yeah most importantly don't forgot the
first program that you ever wrote which
was very fun and kept you hooked so far
and if you want to check out my other
works here if you want to talk about
python devops data data Ops I'm here
thank you
we have some time for questions
they're all too hungry
oh just up there
so I have a question regarding
um once you have the test completed
usually you end in a big trouble
analyzing the test because usually you
just need to look at the assertive
statements but when things grow
a bunch of things you have hundreds of
tests and analyzing that is quite
difficult so did you have any strategy
once you have everything completed just
for example instead of doing the assert
racing specific
assertions or custom exceptions do you
have something in mind
quite capture that question exactly what
you
when you are once the test is completed
and the analysis of the test is a bit
difficult so do you have any strategy
for that
all right so assertions I don't know
whether this is what you're asking but
if you do it with by test this is a pi
test version of test case that's here
um
yes so when you do it with pi test I'm
going to change things over here yeah
any number would work and I'm going to
run it with by test
I have printed it pretty badly right
because we are building it live but when
you do it with whiteest it's gonna show
you exactly where that error happens and
it also has a very nice Trace back that
you can print with more verbose things
so it doesn't matter at what level the
error occurs you will have a complete
race back of things and there are
options in pi test to do that I think I
when we do this case it's just an
assertion error happening at that
particular line but if you have it in
your logic there also it works so let's
say you are expecting five here but
instead of add I'm gonna change this to
oh
let's do something fun
10 always
then it's also going to give you that
whole Trace back
yeah so here it says you know assert on
the left you get 10 but on the 5 uh if
you're making any particular logical
error that also comes as a part of the
price back okay thank you very much
um no other questions
thank you
what do you think is the most useful
Insight you've got about the internals
of Pi test that you learn from this
re-implementation is there some part of
the internals that you know you wish
people better understood
oops
I'm gonna go back to the slide where
I was pretty overwhelmed scared uh it
was very hard to even just look at it or
understand it because it has built for
so many different use cases even
something as simple as fixture it
behaves completely differently to what
we what I have built so I have stopped
drawing inspiration from pie test way
early in the process and just built up
my own version so that's
hmm
thank you for your talk could you talk
us through the entry point of your code
how it iterates over the test files
finds the test classes finds the test
methods and then works out what to run
sure so it's pretty
but sure
all right
that's my test Runner and it gets test
files and then a bunch of test functions
so test files is nothing but reading
reading the directory if the directory
does not have test in it you skip the
directory and if it does have in it
check get all the files that starts with
tests and ends with pi and then return
them
so that's all the test files inside any
particular directory of course I have
done it for only one level
and for the test function I'm importing
the whole module like the whole file and
going through its members using whether
it's it's a function and if it if a
function starts with test underscore
that is a test function and I'm passing
them back
that's that help
thank you
okay thank you um thanks pavani for such
an engaging talk
um
it's a token of our appreciation thank
you thank you
uh one small thing a very thank you to
pycon Au they have sponsored me to come
all the way from India to present the
stock here thank you so much from the
bottom of my heart