1 00:00:11,628 --> 00:00:20,080 >> Hello, everyone and welcome to the Platypus  Hall. I am the host for today. We have Chris   2 00:00:20,080 --> 00:00:25,200 Neugebauer who is about the give our next talk.  I want to do a quick acknowledgment of country   3 00:00:25,920 --> 00:00:32,720 where I am presenting from. I am in Cambra and  wish to acknowledge the traditional custodials   4 00:00:32,720 --> 00:00:41,680 of the land. I want to acknowledge and respect  the continuing culture and contribution they   5 00:00:41,680 --> 00:00:47,840 make to this city and region. I want to  knowledge other Indigenous and island   6 00:00:47,840 --> 00:00:52,320 and first nation people from all across the  world who may be attending today's event.  7 00:00:52,320 --> 00:00:58,160 >> Thanks. I am presenting from  Petaluma in California on the   8 00:00:58,160 --> 00:01:08,320 unseated land of the Coast Miwok people. >> We have Chris Neugebauer giving a talk on   9 00:01:08,320 --> 00:01:13,920 the use and misuse of denning decorators.  This is pre-recorded so he will be   10 00:01:13,920 --> 00:01:19,200 available for questions throughout and  after the talk. If you have questions,   11 00:01:19,200 --> 00:01:25,840 put them in the chat and Q&A and he can answer  them after the talk or give commentary throughout.   12 00:01:28,080 --> 00:01:37,920 I think we are good to go. Take it away. >> Hello, there. Thanks. My name is Christopher   13 00:01:37,920 --> 00:01:46,560 Neugebauer. You can find me on the internet and  this is my talk about decorators. Now, unlike the   14 00:01:46,560 --> 00:01:53,120 talks I have given at Python conferences in 2019,  2020 which were kind of trying to investigate   15 00:01:53,120 --> 00:01:59,280 pointed questions about the future and philosophy  of Python but ended up being all about decorators,   16 00:02:00,080 --> 00:02:06,560 this year, I am going to start discussing  decorators and see where I end up. In particular,   17 00:02:06,560 --> 00:02:12,080 I am not interested in discussing ethics,  religion, or philosophy. What I will tell   18 00:02:12,080 --> 00:02:17,840 you is this talk will end up relatively basic  and probably not end up in that place. So   19 00:02:19,120 --> 00:02:23,520 we can cover a lot of ground we will skip  irrelevant details. If you think a detail   20 00:02:23,520 --> 00:02:34,800 was important that I said was irrelevant, I  apologize. These days my day job is as an engineer   21 00:02:34,800 --> 00:02:40,960 at toolchain. We are the sponsors of the Open  Source pants developer workflow system which makes   22 00:02:40,960 --> 00:02:47,040 it easier to run your favorite Python tools more  efficiently and effectively. Internally we use   23 00:02:47,040 --> 00:02:55,600 decorators a lot. I will get to that at the end of  this talk. A recap of what a decorator actually is   24 00:02:56,960 --> 00:03:04,160 first. Say you have a function which does  something really obvious like print I am a banana.   25 00:03:04,160 --> 00:03:08,800 When you call it does the thing you wrote  in the code. It prints I am a banana.   26 00:03:09,760 --> 00:03:17,440 Decorators which begin with the at symbol before  are Python's way of providing a structured way   27 00:03:17,440 --> 00:03:23,680 to do something with functions, methods and in  later versions classes. Adding this surprise   28 00:03:23,680 --> 00:03:31,840 decorator makes the function print out another  thing before it executes. Surprise. This is how   29 00:03:31,840 --> 00:03:37,920 the decorator works. It takes that function  as a parameter, creates a wrapper function,   30 00:03:37,920 --> 00:03:41,280 and then it leaves that wrapper function  in the place of the function that we wrote.   31 00:03:42,560 --> 00:03:46,240 To be more clear, when we put that  decorator statement before the function,   32 00:03:46,880 --> 00:03:51,440 once Python defines the function,  it passes the code to our decorator   33 00:03:51,440 --> 00:03:59,040 and the decorator leaves the new function in  place of the original function. So in summary,   34 00:03:59,040 --> 00:04:04,480 that means that a decorator is a function  that takes a function and leaves a function.   35 00:04:05,840 --> 00:04:11,440 Except that's not what it is at all. To show you  why, we will return to the original decorator.   36 00:04:12,880 --> 00:04:20,000 Under the hood, our modern Python decorator  function is doing the same bit as a more   37 00:04:20,000 --> 00:04:26,560 ancient Python decorator function which is the  bit at the bottom. This bit after the at symbol   38 00:04:27,472 --> 00:04:35,520 is an expression and when evaluated it is equal to  a function which is our decorator, the decorator   39 00:04:35,520 --> 00:04:40,560 is called giving the function we are trying  to decorate and what we are replacing it with.   40 00:04:41,440 --> 00:04:48,000 If you want to add a parameter you do this  and end up with a function nested three deep   41 00:04:49,040 --> 00:04:56,800 which is more difficult than the two deep. You  take a function that takes the parameters we   42 00:04:56,800 --> 00:05:02,160 define in the decorator statement and at the  bottom you return a decorate function. This   43 00:05:02,960 --> 00:05:07,280 will accept a decorator function and return  a wrapper function and the wrapper function   44 00:05:10,240 --> 00:05:12,800 calls the function we design  with the decorator statement.   45 00:05:14,160 --> 00:05:19,440 It is really confusing but it is easy enough  to remember if you just remember how that   46 00:05:19,440 --> 00:05:24,720 Python code translates to the more manual  Python code. Kind of almost makes sense.   47 00:05:26,240 --> 00:05:31,680 This means a decorator is an expression which is  a function that itself a function and returns a   48 00:05:31,680 --> 00:05:39,840 function but that's also not correct. You saw in  the last slide there a decorator just literally   49 00:05:39,840 --> 00:05:43,760 assigns the result of the decorator function  to the name of the function that's passed in.   50 00:05:43,760 --> 00:05:47,840 We don't check that value be a function  because Python doesn't do type checking.   51 00:05:47,840 --> 00:05:53,360 There is not any requirement that there be  anything returned at all. This really seems   52 00:05:53,360 --> 00:05:58,560 confusing for things that should be modifying  functions but tells us what a decorator is.   53 00:05:59,600 --> 00:06:05,840 A decorator is an expression whose value is  a function that takes a function and does   54 00:06:06,640 --> 00:06:13,520 something with it. That's why decorators are  hard to define and hard to wrap your head   55 00:06:13,520 --> 00:06:20,000 around at first. On the other hand, decorators are  flexible, unlock a lot of unexpected and useful   56 00:06:20,000 --> 00:06:25,840 design patterns in Python that aren't accessible  in other ways and all come from the observation   57 00:06:25,840 --> 00:06:31,280 that Python doesn't say what a decorator has  to do with the code you chose to decorator. It   58 00:06:31,280 --> 00:06:37,680 does whatever function can do. That's what today's  talk is about. That giant pile of question marks.   59 00:06:39,760 --> 00:06:44,080 For the rest of today's talk, we are going to look  at why Python got decorators if the first place   60 00:06:44,080 --> 00:06:49,920 and what decorator is enabled in popular  frameworks and finish with design decorators.   61 00:06:52,480 --> 00:07:01,520 Enjoy. So first up, we are going to look at how  Python ended up with a feature that is on the face   62 00:07:01,520 --> 00:07:08,640 of it more complex than it needs to be. Python  has always been an object oriented language but   63 00:07:09,920 --> 00:07:15,840 people who worked on Python came from languages  like C and C is not an object oriented language.   64 00:07:16,960 --> 00:07:21,520 What the C developers like was it  was possible for multiple developers   65 00:07:21,520 --> 00:07:25,680 working on different files to name their  variables and functions the same thing   66 00:07:26,400 --> 00:07:32,960 and not have to worry about consumers actually  using the wrong function. Name spaces enables this   67 00:07:34,720 --> 00:07:40,960 and name spaces as Tim Peters put it were one  honking great idea which Python should do more of.   68 00:07:42,400 --> 00:07:51,200 The way we did name spaces is with module and file  structure but this isn't enough. Say you have a   69 00:07:51,200 --> 00:07:57,840 class pairing a name and address, Python gives you  the ability to create precisely one constructor.   70 00:07:58,480 --> 00:08:03,280 You make the fields the parameters. In  practice you might want to create an object   71 00:08:03,280 --> 00:08:15,440 if a serialized form like a JSON dictionary.  To do this you would need a factory function   72 00:08:16,400 --> 00:08:22,640 which is a function that lives in the module next  to the class you want to instantiate which pulls   73 00:08:23,360 --> 00:08:31,520 the constructor out of the dictionary. We can't  name space things intuitively and automatically.   74 00:08:31,520 --> 00:08:36,400 It is only the name of the function making  it clear what we will get from the factory   75 00:08:36,400 --> 00:08:42,080 function and just like C we need non-clashing  names to make sure we are having uniqueness.   76 00:08:46,320 --> 00:08:50,080 We need to make sure the names of the  corresponding factory fashions don't clash.   77 00:08:51,920 --> 00:08:57,200 This is a code organization nightmare  and why name spaces with a honking idea.   78 00:08:58,320 --> 00:09:08,720 This being the late '90s and early 2000s Python  looked to the code organization which was Java.   79 00:09:10,080 --> 00:09:14,880 Java has firm ideas about how code should be  organized. One thing it allows that Python   80 00:09:14,880 --> 00:09:24,880 wouldb never allow is multiple constructors. But  one thing could be borrowed. Java has a static   81 00:09:24,880 --> 00:09:33,600 method which is a method called on the class  itself. It is a first-class language feature.   82 00:09:34,720 --> 00:09:40,080 It allows you to organize factory methods  under the class where they logically belong.   83 00:09:43,840 --> 00:09:50,720 When Python 2.2 was released it added a fancy new  feature. You could define a method inside a class   84 00:09:51,440 --> 00:09:57,520 and then after you defined it you would call it  a class method and when you want to call that   85 00:09:57,520 --> 00:10:02,720 class method you would call it on the class  itself rather than an instance of the class.   86 00:10:03,520 --> 00:10:09,120 The class method and static method were two  ways Python decided to do more name spacing.   87 00:10:10,640 --> 00:10:15,920 This way of defining something as a class method  without using a new language feature to do it   88 00:10:15,920 --> 00:10:22,240 is called the decorate pattern. People were  really happy they could name space their code.   89 00:10:23,520 --> 00:10:29,120 That pattern that we saw by the way is something  I will come back to a lot in today's talk.   90 00:10:29,120 --> 00:10:35,600 I am going to call it define, consume,  throw away. Define, consume, throw away   91 00:10:35,600 --> 00:10:41,200 is an anti-pattern in most languages but it is  something that we reluctantly accept in Python.   92 00:10:42,400 --> 00:10:47,840 Now, as I was saying, very, very quickly Python  discovered a problem with post decorate pattern.   93 00:10:48,880 --> 00:10:56,880 Say you a long method going off the end of the  screen, it is difficult to notice the method is   94 00:10:56,880 --> 00:11:03,840 actually a class method. This is a case where  it came quite difficult to read Python code   95 00:11:03,840 --> 00:11:09,440 and understand what it was doing just at a glance.  Very quickly, they decided to do something about   96 00:11:09,440 --> 00:11:16,480 that. Pep317 with the catchy name, decorators  for functions and methods, came up with some   97 00:11:16,480 --> 00:11:22,160 goals for inventing a new language feature to  make decoration easier to approach. It is an   98 00:11:22,160 --> 00:11:26,640 interesting read historically because it covers  some of the thinking around Python that time.   99 00:11:27,360 --> 00:11:31,760 There is this idea that there might be a jit  compiler and we should optimize for that.   100 00:11:33,600 --> 00:11:40,400 They pointed out decoration was hidden and  it should be moved to the top of the function   101 00:11:40,400 --> 00:11:48,800 where it is more in your face. Decorators turned  into a really interesting feature in Python.   102 00:11:49,360 --> 00:11:54,800 It took class and static method from being  something that was awkward into something   103 00:11:55,360 --> 00:12:03,600 easy to use and made more ideas in Python  more accessible. One idea of object oriented   104 00:12:03,600 --> 00:12:09,840 programming is encapsulation which is the idea a  class has ultimate responsibility for its data.   105 00:12:10,400 --> 00:12:15,600 In Java land, the way to ensure data is controlled  is to make sure the data fields are private   106 00:12:15,600 --> 00:12:21,360 and gate using a method and if you need to  change an implementation detail you change in the   107 00:12:21,360 --> 00:12:27,840 method and calls are taken care of. There is no  difference between a stored property like filling   108 00:12:27,840 --> 00:12:33,120 here where you return the value and a synthetic  property where you calculate and return that.   109 00:12:34,320 --> 00:12:40,400 Because Python never had private fields, it never  discouraged people from directly accessing field   110 00:12:40,400 --> 00:12:46,240 and objects and by convention that meant a visible  difference between the stored property, filling,   111 00:12:46,240 --> 00:12:52,560 and the synthetic property. One you call, one  you don't. It made it hard for library authors   112 00:12:52,560 --> 00:12:58,560 to change from a stored property to a synthetic  property. Python invented a decorator called   113 00:12:58,560 --> 00:13:04,160 property which came around the same time as class  and static method but was more awkward to use so   114 00:13:04,160 --> 00:13:11,680 people didn't use it. With property, you can  start a class that has stored properties and   115 00:13:11,680 --> 00:13:18,640 get methods and by replacing the get method with a  property, you can access the synthetic and stored   116 00:13:18,640 --> 00:13:26,080 property in the same way. It lets you refactor  from a stored to synthetic property without   117 00:13:26,080 --> 00:13:32,960 breaking compatibility. Now you don't need to call  awkward set and get methods nor need to know if   118 00:13:36,000 --> 00:13:41,760 it is synthetic or proper. It is clear  to the calling context what to do.   119 00:13:44,080 --> 00:13:49,760 That decorator syntax meant properties called on  immediately in Python and plenty of languages have   120 00:13:49,760 --> 00:13:55,120 stolen it sense. Properties are interesting  because unlike the decorators we started off   121 00:13:55,120 --> 00:14:00,240 showing there doesn't need to be any wrapping  going on behind the scenes. In Python's case   122 00:14:00,240 --> 00:14:06,960 property works by adding magic inside the internal  classes. We intercept an attribute look up.   123 00:14:07,600 --> 00:14:12,240 The decorator needs to do enough to tell  subsequent calls this is a property and you   124 00:14:12,240 --> 00:14:17,440 should do that property thing. This is really all  you need to do. Python's implementation is a lot   125 00:14:17,440 --> 00:14:24,160 more complex but the seeds are there. Add the  interception code and the decorator at the top.   126 00:14:28,080 --> 00:14:32,240 And then a new decorator has the same  basic effect as Python's property getters.   127 00:14:33,360 --> 00:14:38,400 Decorators and Python's model where you can  access the internal parts of the language mean   128 00:14:38,400 --> 00:14:43,840 you can alter the semantics of the language to  make libraries easier to use and more powerful.   129 00:14:44,800 --> 00:14:49,040 Decorators have been used in a lot of interesting  places throughout the Python ecosystem.   130 00:14:50,240 --> 00:14:56,720 The functools library is full of decorators  you might find interesting. I think it is worth   131 00:14:56,720 --> 00:15:05,120 calling out Django because it realizes decorators  are a useful way to apply frequently reused   132 00:15:05,120 --> 00:15:14,320 code to functions. Django uses decorators as  behavior modifiers for views. Require http method   133 00:15:14,320 --> 00:15:21,920 tells which methods are acceptable for a given  view saving you from writing boilerplate down here   134 00:15:21,920 --> 00:15:28,560 during the function. Django uses decorators all  over the place. It made the observation sometimes   135 00:15:28,560 --> 00:15:34,480 there are structured, repetitive behavior  fitting into the pre-call and post-call parts   136 00:15:34,480 --> 00:15:40,320 of a function which are better suited to using  wrapping than other object orgented behavior.   137 00:15:42,560 --> 00:15:46,400 We have seen why decorators happen, what  they brought into the world of Python,   138 00:15:46,400 --> 00:15:52,400 and techniques with code reuse, and ways  to selectively add improved semantics   139 00:15:52,400 --> 00:15:57,040 without adding new language features. These  aren't the only thing decorators let you do.   140 00:15:58,160 --> 00:16:03,440 I want to show you how decorators open up a way  to solve problems that other programs use with   141 00:16:03,440 --> 00:16:13,440 features like anonymous blocks. What I am not  convinced about is any of these are good ideas.   142 00:16:13,440 --> 00:16:18,720 Only that these are things that maybe you  should try and see if they are useful to you and   143 00:16:18,720 --> 00:16:23,600 then they will find out if they are good. The  first is this idea of a scoped block which is   144 00:16:23,600 --> 00:16:30,000 something I touched on in my 2019 talk. I didn't  have a good solution at the time. Now I do.   145 00:16:30,720 --> 00:16:34,960 One of the things that blocks helps out with  is maintain variable scope. Any given block   146 00:16:34,960 --> 00:16:40,160 of code doesn't depend on variables it defines  in a block of code. My thinking on this came   147 00:16:40,160 --> 00:16:48,400 around from my time working in con -- cotlin. If  you have a web framework that needs to produce   148 00:16:48,400 --> 00:16:54,320 a user or error response, you only need to  assign this responsible variable once. The   149 00:16:54,320 --> 00:16:59,360 user response gets assigned a user response. If  it is an error it is assigned the error response.   150 00:17:00,720 --> 00:17:04,400 This looks reasonable in Python when you  don't need to deal with error handling.   151 00:17:05,200 --> 00:17:09,600 Once you start to handle the error, you  have to define the response variable   152 00:17:09,600 --> 00:17:15,040 inside two different enter blocks and handle  it in the outer blocks. It is not nice,   153 00:17:15,040 --> 00:17:20,480 easy to make mistakes and a whole pile of  Python tools don't like this pattern of code.   154 00:17:21,840 --> 00:17:25,360 One way to refactor this is to pull  that if-statement out as a function   155 00:17:26,080 --> 00:17:29,920 which now has the same structure as the  one that doesn't have error handling   156 00:17:29,920 --> 00:17:36,800 and now you need to divert your gaze to see the  inner behavior which is bad visual locality. It   157 00:17:36,800 --> 00:17:43,200 gives you a new function to support even though  you edge -- only intended as a one off. It   158 00:17:43,760 --> 00:17:53,040 is better to put it inside a function. You don't  need to pass a parameter. What is more annoying   159 00:17:53,040 --> 00:17:58,240 is the design, consume and throw away behavior. You are writing it once so you can use it once.   160 00:18:03,120 --> 00:18:06,400 You know sometimes the code is  going to get called but not when.   161 00:18:07,520 --> 00:18:13,520 This is something a decorator can solve. This  is that decorator. It is called run. All it   162 00:18:13,520 --> 00:18:19,280 does is call the function we are decorating and  store its result. You can define a function that   163 00:18:19,280 --> 00:18:24,080 takes no parameters and returns the thing you want  stored to a variable like this string hello world   164 00:18:24,080 --> 00:18:30,160 and put the run decorator before the function and  end up with a variable called hello world instead   165 00:18:30,160 --> 00:18:36,800 of a function called hello world. This is useful  for the branching block because we can go from   166 00:18:36,800 --> 00:18:44,080 using define, consume, throw away to defining  our variable by using the run decorator here.   167 00:18:45,760 --> 00:18:50,880 This gives us the advantage of scoping out  variables. We can define them at the level   168 00:18:51,520 --> 00:18:58,640 used. We get the benefit but without the poor  visual locality of design, consume, throw away   169 00:18:59,760 --> 00:19:06,160 which is really nice. The next thing  is the idea of string transformations   170 00:19:06,160 --> 00:19:11,600 which is something Python is good at  except if one of the transformations   171 00:19:12,960 --> 00:19:18,240 needs multiple lines of code to write  effectively. Python has, for the longest time,   172 00:19:18,800 --> 00:19:24,400 avoided multi-line lambdas and this pattern is one  that let's you take advantage of this. -- pattern.   173 00:19:26,560 --> 00:19:34,080 A string transformation is along along the line of  a list or generator expression. These make a lot   174 00:19:34,080 --> 00:19:40,720 of sense when you only call a single method or a  single function on your iterable items. Or if you   175 00:19:40,720 --> 00:19:46,800 can string from one to the other with minimal  branching. If you want to do something that   176 00:19:46,800 --> 00:19:52,240 requires a bit of branching things get awkward.  You have to return to define, consume, throw   177 00:19:54,240 --> 00:20:03,280 which we know is not very good. Especially  since other languages have really begun to   178 00:20:03,280 --> 00:20:09,120 figure this out. If you have a Lambda expression  as the last argument inside parenthesis you can   179 00:20:09,120 --> 00:20:15,040 pull that Lambda outside of the parenthesis  and combined with cotlin's if-expressions   180 00:20:15,680 --> 00:20:23,200 you end up with something quite elegant here. We  can't get the list comprehension into a syntax   181 00:20:24,720 --> 00:20:31,040 but we may be able to do something with  the map or legacy and old style functions.   182 00:20:32,080 --> 00:20:38,480 Previously we would define our multi-line function  here and call map with the multi-line parameter as   183 00:20:38,480 --> 00:20:50,960 a -- multi-line function as a parameter which  is define, consume, throw. To do that we need   184 00:20:50,960 --> 00:20:57,120 a decorator called apply. Apply takes a transform  function like map or filter andane an iterable and   185 00:20:57,120 --> 00:21:05,360 accepts a multi-line function as the thing that  does transform on each element. You say apply map   186 00:21:05,360 --> 00:21:11,440 to items using the function that we are about to  define and store it to the function's name. With   187 00:21:11,440 --> 00:21:17,240 that, we can take our define, consume, throw code  and turn it into something where we say what we   188 00:21:17,240 --> 00:21:23,280 are going to do with the code before we define  it. It is exactly the same way we would do this   189 00:21:23,280 --> 00:21:30,320 with a single line Lambda expression. Effectively  unlocking multi-line lambdas here which is nice.   190 00:21:30,320 --> 00:21:35,600 That's a couple patterns where you can use  decorator to avoid define, consume, throw patterns   191 00:21:36,880 --> 00:21:41,360 with multi-line function. Now I want to talk  about the builder pattern which is established   192 00:21:41,360 --> 00:21:47,600 in the Python community but we use it in a really,  really ugly way. In Python, we like writing data   193 00:21:47,600 --> 00:21:52,160 structures in code but there is only support for  a small number of officially blessed structure.   194 00:21:59,520 --> 00:22:04,240 In its usual form there is a lot of repetition  and a side problem of defining the thing   195 00:22:04,240 --> 00:22:10,160 but not showing how it will be consumed until  later. The first things defined are often the   196 00:22:10,160 --> 00:22:16,960 last things you consume. This isn't an issue  in the reasonably flat structure but as you   197 00:22:16,960 --> 00:22:24,560 get nested the lack of clear visual locality  becomes difficult. Builders are almost always   198 00:22:24,560 --> 00:22:29,360 used like this. You create the builder, do things  with the builder and then build the builder.   199 00:22:30,720 --> 00:22:35,040 We can do that to build up a data structure  like an HTML document but with a slight twist.   200 00:22:35,920 --> 00:22:42,480 This HTML node builder class takes a tag name and  has a build method putting everything together at   201 00:22:42,480 --> 00:22:50,400 the end. What's interesting is the node method.  It is actually a decorator that creates a builder,   202 00:22:51,200 --> 00:22:55,600 passes it to the function we are decoratoring,  and then builds the builder once that function   203 00:22:55,600 --> 00:23:02,160 we are decoratoring returns. If we create one  more decorator, create the initial builder,   204 00:23:03,920 --> 00:23:07,680 you end up with a function  that defines an HTML document   205 00:23:07,680 --> 00:23:12,240 but more importantly this function sort of has  the structure of the document we are creating,   206 00:23:12,240 --> 00:23:17,360 strips out all the repetitive calls to builder,  and just to build the builder and leaves us with   207 00:23:17,360 --> 00:23:23,360 the task of doing the building stuff and adding  things to the structure. This one encourages you   208 00:23:23,360 --> 00:23:28,960 to write more functions but in doing so, you end  up with code where you interact with so much less   209 00:23:28,960 --> 00:23:34,480 boilerplate. The final pattern I want  to show is one I call the code registry.   210 00:23:35,360 --> 00:23:40,160 Unlike previous ideas which are geared toward  structuring single use functions in a nicer way,   211 00:23:40,800 --> 00:23:45,200 this one is about separating the idea of how  you define your code from how you call it.   212 00:23:45,760 --> 00:23:50,080 Example I want to show is an example of a  library for handling text processing pipelines.   213 00:23:51,200 --> 00:23:58,080 Say you want to figure out the most frequent  word you can use Linux pipelines like this   214 00:23:58,080 --> 00:24:05,120 and you will get a result, sure. To do  this task in unix, you need to know what   215 00:24:05,120 --> 00:24:10,560 all these different unix commands are and  in which order to call them to transform the   216 00:24:10,560 --> 00:24:16,080 string appropriately, tokenize the word  and normalize into lowercase and so on.   217 00:24:17,680 --> 00:24:22,560 Doing this in Python really isn't much better.  The code is more explicit but it is a lot more   218 00:24:22,560 --> 00:24:28,320 verbose with a lot more libraries. What is  cool is if I can ask Python to give me a   219 00:24:28,320 --> 00:24:34,800 result of a particular type with a particular  characteristic. Say list of instance strings,   220 00:24:35,360 --> 00:24:44,640 made up of lowercase words with no punctuation.  How do we turn that idea into reality?   221 00:24:46,080 --> 00:24:55,040 If we use a code registry which is a class with a  decorator function and an input and output scheme,   222 00:24:55,920 --> 00:25:05,200 it returns the function, we don't wrap it but put  it away inside the code registry so we can find   223 00:25:05,200 --> 00:25:11,520 it based on information we associated it with.  With the registry, we can define small functions   224 00:25:11,520 --> 00:25:16,240 that represent all the steps in transforming  the data. We can say we have a function that   225 00:25:16,240 --> 00:25:22,000 accepts an iterable of strings with attributes  and returns a counter with the same attributes   226 00:25:22,000 --> 00:25:28,480 as well as the frequency attribute. You can define  a whole bunch of rules like this which handle   227 00:25:28,480 --> 00:25:34,240 small parts of our text processing. Once you have  done that, you can make a function called find   228 00:25:34,240 --> 00:25:39,520 steps which figures out how to get from an input  type with attributes and produce an output type   229 00:25:39,520 --> 00:25:45,280 with attributes and you can write a function that  finds those steps and applies those steps and that   230 00:25:45,280 --> 00:25:50,800 will do the processing you want. If you have ever  written code in prologue this is the same thing.   231 00:25:52,080 --> 00:25:55,920 So this idea of separating the act of  calling your code from the place you   232 00:25:55,920 --> 00:26:02,800 define it is powerful and shows up in a lot of  places. In flask, for example, the app router   233 00:26:02,800 --> 00:26:08,720 is a registry you populate with decorators. The  root decorators let you specify circumstances in   234 00:26:08,720 --> 00:26:14,400 which a response giving function gets called.  Closest to home is how it is used in pants.   235 00:26:15,280 --> 00:26:20,800 Pants uses a rule registry to figure out how to  get from a set of source artifacts to shippable   236 00:26:20,800 --> 00:26:26,960 artifacts so that a developer tool workflow can  be defined as a series of small steps and then the   237 00:26:26,960 --> 00:26:32,000 engine will figure out what steps can get you from  start to finish. More importantly, third party   238 00:26:32,000 --> 00:26:37,040 developers can provide their own rules which helps  pants figure how to deal with different artifacts   239 00:26:37,920 --> 00:26:44,880 and figure how to run the rules automatically.  This is a nice, intrusive way to handle plugins.   240 00:26:47,040 --> 00:26:51,280 Even though this approach separates the  name of the function to how it gets called,   241 00:26:51,280 --> 00:26:56,400 the decorator still returns the original  function unaltered meaning you can call   242 00:26:56,400 --> 00:27:03,280 the function to directly unit test it. That's  the end of our whirlwind tour. We have seen   243 00:27:03,280 --> 00:27:09,680 decorators have allowed in Python and looked at  four design patterns that show unexplored ideas   244 00:27:09,680 --> 00:27:16,400 that are enabled by decorators. The first being  the idea of throwaway functions. They are not   245 00:27:16,400 --> 00:27:21,760 truly anonymous blocks in the way other languages  offer them but decorators are the closest thing   246 00:27:21,760 --> 00:27:34,320 we have. We also saw decorators had great for  transforming functions into the data they produce   247 00:27:34,320 --> 00:27:40,080 which is useful for building up data structures.  We saw the complicated builder pattern replaced   248 00:27:40,080 --> 00:27:46,480 with something a bit more readable and easier to  nest. We saw with code registry, they allow you   249 00:27:46,480 --> 00:27:53,040 to design with a function rather than the name  meaning you can call based on the data you have   250 00:27:53,600 --> 00:28:02,480 rather than the name. If you have questions,  you can email me at the address on the screen.   251 00:28:02,480 --> 00:28:07,760 There is a Twitter handle there. I will be in  the hallway track after this talk for questions.   252 00:28:11,040 --> 00:28:16,480 If you are after code snippets, I will share them  after this talk on my website at Chrisjm.com.   253 00:28:18,000 --> 00:28:25,680 Thank you for everyone at Pycon who made  this happen. I want to thank you for coming   254 00:28:25,680 --> 00:28:30,080 to my talk. I hope you learned something and I am  looking forward to hearing the questions you have.   255 00:28:30,800 --> 00:28:43,840 Thank you and see you next year,  hopefully, in person. Hopefully. Thank you.