onilog

posts tagged "python"

Silly django confusion

24 April 2012 3:51 PM (python | django | coding)

I'm setting up a testing environment for work, using fixtures to save and load test data and I got a little stumped by something I ran into.

I had exported one of the database tables we use to a json file that I was going to import into a fresh new database to test with. When I imported it everything seemed fine, except when looking at the actual page. So this is what I found:

SELECT * FROM app_table;
    => 3 rows of data
from app.models import Table

Table.objects.count()
    => 3

Table.objects.all()
    => []

This is weird. So I looked at the django.db.connection.queries property and I realized that it was doing a join since the model subclasses another:

from django.db import models

from app.models import SuperTable

class Table(SuperTable):...

Which, of course, means that in order to get the complete Table instance, the related SuperTable instance is also required, but in order to do a COUNT of app_table it isn't necessary. And that's where the inconsistency came from, now that I've also copied the contents of SuperTable everything is fine again.

Python Decorators

9 February 2012 0:00 AM (python | coding | decorators)

Time for me to post something about decorators, whynot.

Simple decorator

  • Functional

    From what I see, most documents start out with a decorator as a class. I actually find, for Python, the functional one much easier to start with.

    The simplest, functional, and still illustrative decorator, as found here, is:

    def entryExit(function):
        def new_function():
            print "Entering", function.__name__
            function()
            print "Exited", function.__name__
        return new_function
    
    @entryExit
    def func():
        print 'inside func'
    
    func()
    

    That's pretty easy, really. All it does is replace func with an instance of new_function. This new_function calls the original func, but before that it prints "Entering func" and afterwards it prints "Exited func". So basically this one is just a wrapper around your function.

  • OO

    I find that, for the first time looking at it, the functional decorator is easier to understand. Although it does require that you know how functional programming works, of course. This solution is somewhat clearer once you understand that the __call__ function is called when you try to use an object like a function:

    class entryExit:
        def __init__(self, function):
            self.function = function
    
        def __call__(self):
            print "Entering", self.function.__name__
            self.function()
            print "Exited", self.function.__name__
    
    @entryExit
    def func():
        print 'inside func'
    
    func()
    

    To explain it's a lot easier. At the moment @entryExit is found entryExit.__init__ is called and we save the function for later use. When func is then called, it actually calls the entryExit, which causes entryExit.__call__ to be called, which in turn calls func, here called self.function.

  • Result

    Entering func
    inside func
    Exited func
    

Known parameters

  • Functional

    Since we just saw how decorators work without any parameters, it's time we check to see how one works where we know the parameters. It's pretty much the same example:

    def entryExit(function):
        def new_function(someparam):
            print "Entering", function.__name__
            function(someparam)
            print "Exited", function.__name__
        return new_function
    
    @entryExit
    def func(param)
        print 'inside func, with param', param
    
    func('foo')
    

    Not much different there, just the idea that the entryExit function doesn't take any extra parameters, but instead only the new_function does. This is functional programming, doesn't actually have anything to do with the decorator though.

  • OO

    Once you understand the first example, this one is actually clearer than the functional one, since by now we should know to recognize that entryExit.__call__ is called, it should be clear how to handle that:

    class entryExit:
        def __init__(self, function):
            self.function = function
    
        def __call__(self, someparam):
            print "Entering", self.function.__name__
            self.function(someparam)
            print "Exited", self.function.__name__
    
    @entryExit
    def func(param):
        print 'inside func, with param', param
    
    func('foo')
    

    Since it's just entryExit.__call__ that gets called in the end, it should be clear that any param that should be used by the function should be part of the entryExit.__call__ function.

  • Result

    Entering func
    inside func, with param foo
    Exited func
    

Arbitrary arguments

  • Functional

    Having to match your decorator function to the function you want to apply it to isn't very useful in any case where there might be functions that have different parameter lengths, and writing a decorator for each isn't all that much fun. Here's how we work around that:

    def entryExit(function):
        def new_function(*args, **kwargs):
            print "Entering", function.__name__
            function(*args, **kwargs)
            print "Exited", function.__name__
        return new_function
    
    @entryExit
    def func():
        print 'inside func'
    
    @entryExit
    def func2(arg):
        print 'inside func2, with arg', arg
    
    @entryExit
    def func3(arg1, arg2):
        print 'inside func3, with args', arg1, 'and', arg2
    
    func()
    print
    func2('foo')
    print
    func3('bar', 'baz')
    

    This gets a little stranger, with the *arg and **kwargs, but that is one of the reasons Python is so dynamic.

  • OO

    From known parameters to arbitrary parameters is again a rather small step for this method:

    class entryExit:
        def __init__(self, function):
            self.function = function
    
        def __call__(self, *args, **kwargs):
            print "Entering", self.function.__name__
            self.function(*args, **kwargs)
            print "Exited", self.function.__name__
    
    @entryExit
    def func():
        print 'inside func'
    
    @entryExit
    def func2(arg):
        print 'inside func2, with arg', arg
    
    @entryExit
    def func3(arg1, arg2):
        print 'inside func3, with args', arg1, 'and', arg2
    
    func()
    print
    func2('foo')
    print
    func3('bar', 'baz')
    

    The only thing, just as with the functional example, is the *args and **kwargs parts, if you understand that then this is pretty much the same as when you know which variables are needed.

  • Result

    Entering func
    inside func
    Exited func
    
    Entering func2
    inside func2, with arg foo
    Exited func2
    
    Entering func3
    inside func3, with args bar and baz
    Exited func3
    

Mixing

  • Fixed and arbitrary arguments

    • Functional

      There's just one idea left, mixing known, or required, and arbitrary arguments. It's not difficult to extrapolate what needs to be done from the information we have so far:

      def entryExit(function):
          def new_function(arg, *args, **kwargs):
              print "Entering", function.__name__, "with arg", arg
              function(arg, *args, **kwargs)
              print "Exited", function.__name__
          return new_function
      
      @entryExit
      def func(arg):
          print 'inside func, with arg', arg
      
      @entryExit
      def func2(arg1, arg2):
          print 'inside func2, with args', arg1, 'and', arg2
      
      @entryExit
      def func3(arg1, arg2, arg3):
          print 'inside func3, with args', arg1, ',', arg2, 'and', arg3
      
      func('foo')
      print
      func2('bar', 'baz')
      print
      func3('baz', 'foo', 'bar')
      

      From what we know about both fixed and arbitrary args, this wasn't very difficult to figure out, but this does offer other possibilities as well.

    • OO

      Well this shouldn't be very hard, it's almost downright logical.

      class entryExit:
          def __init__(self, function):
              self.function = function
      
          def __call__(self, arg, *args, **kwargs):
              print "Entering", self.function.__name__, 'with arg', arg
              self.function(arg, *args, **kwargs)
              print "Exited", self.function.__name__
      
      @entryExit
      def func(arg):
          print 'inside func, with arg', arg
      
      @entryExit
      def func2(arg1, arg2):
          print 'inside func2, with args', arg1, 'and', arg2
      
      @entryExit
      def func3(arg1, arg2, arg3):
          print 'inside func3, with args', arg1, ',', arg2, 'and', arg3
      
      func('foo')
      print
      func2('bar', 'baz')
      print
      func3('baz', 'foo', 'bar')
      

      As long as entryExit.__call__ can be called with the same arguments as the decorated function, it really doens't matter how many required and how many optional arguments a decorator gets, but if it needs at least one and none are given, then yes, that will be problematic.

    • Result

      Entering func with arg foo
      inside func, with arg foo
      Exited func
      
      Entering func2 with arg bar
      inside func2, with args bar and baz
      Exited func2
      
      Entering func3 with arg baz
      inside func3, with args bar , foo and baz
      Exited func3
      
  • Hiding arguments

    • Functional

      Though it can make your code very difficult to understand or even maintain if used unwisely:

      def entryExit(function):
          def new_function(*args, **kwargs):
              print "Entering", function.__name__
              function('foo', *args, **kwargs)
              print "Exited", function.__name__
          return new_function
      
      @entryExit
      def func(arg):
          print 'inside func, with arg', arg
      
      @entryExit
      def func2(arg1, arg2):
          print 'inside func2, with args', arg1, 'and', arg2
      
      @entryExit
      def func3(arg1, arg2, arg3):
          print 'inside func3, with args', arg1, ',', arg2, 'and' arg3
      
      func()
      print
      func2('foo')
      print
      func3('bar', 'baz')
      

      This way a decorator can 'eat away' arguments from a function call. Of course this means that when you look at the code later you see the function takes 1 argument, but you don't see it being given to the function.

    • OO

      It can get very messy with all the strange ways to apply them.

      class entryExit:
          def __init__(self, function):
              self.function = function
      
          def __call__(self, *args, **kwargs):
              print "Entering", self.function.__name__
              self.function('foo', *args, **kwargs)
              print "Exited", self.function.__name__
      
      @entryExit
      def func(arg):
          print 'inside func, with arg', arg
      
      @entryExit
      def func2(arg1, arg2):
          print 'inside func2, with args', arg1, 'and', arg2
      
      @entryExit
      def func3(arg1, arg2, arg3):
          print 'inside func3, with args', arg1, ',', arg2, 'and', arg3
      
      func()
      print
      func2('foo')
      print
      func3('bar', 'baz')
      

      I believe that code could get very messy very quickly this way.

    • Result

      Entering func
      inside func with arg foo
      Exited func
      
      Entering func2
      inside func2 with args foo and foo
      Exited func
      
      Entering func3
      inside func3 with args foo , bar and baz
      Exited func
      
  • Adding arguments

    • Functional

      Another possibility would be:

      def entryExit(function):
          def new_function(arg, *args, **kwargs):
              print "Entering", function.__name__, 'with argument', arg
              function(*args, **kwargs)
              print "Exited", function.__name__
          return new_function
      
      @entryExit
      def func():
          print 'inside func'
      
      @entryExit
      def func2(arg):
          print 'inside func2 with arg', arg
      
      @entryExit
      def func3(arg1, arg2):
          print 'inside func3 with args', arg1, 'and', arg2
      
      func('foo')
      print
      func2('bar', 'baz')
      print
      func3('baz', 'foo', 'bar')
      

      This does pretty much the opposite of the previous example, this adds an argument to the function. Again you see a difference between invocation and declaration, which can get very confusing, I'm sure.

    • OO

      class entryExit:
          def __init__(self, function):
              self.function = function
      
          def __call__(self, arg, *args, **kwargs):
              print "Entering", self.function.__name__, 'with argument', arg
              self.function(*args, **kwargs)
              print "Exited", self.function.__name__
      
      @entryExit
      def func():
          print 'inside func'
      
      @entryExit
      def func2(arg):
          print 'inside func2, with arg', arg
      
      @entryExit
      def func3(arg1, arg2):
          print 'inside func3, with args', arg1, 'and', arg2
      
      func('foo')
      print
      func2('bar', 'baz')
      print
      func3('baz', 'foo', 'bar')
      
    • Result

      Entering func with argument foo
      inside func
      Exited func
      
      Entering func2 with argument bar
      inside func2, with arg baz
      Exited func2
      
      Entering func3 with argument baz
      inside func3, with args foo and bar
      Exited func3
      

Parameters to the decorator

  • Functional

    The decorator itself could also get/use some parameters, this adds another level of functional programming and gets even stranger to look at:

    def entryExit(arg):
        def entry_exit_function(function):
            def new_function(*args, **kwargs):
                print "Entering", function.__name__, 'with argument', arg
                function(*args, **kwargs)
                print "Exited", function.__name__
            return new_function
        return entry_exit_function
    
    @entryExit('foo')
    def func():
        print 'inside func'
    
    @entryExit('bar')
    def func2(arg):
        print 'inside func2, with arg', arg
    
    @entryExit('baz')
    def func3(arg1, arg2):
        print 'inside func3, with args', arg1, 'and', arg2
    
    func()
    print
    func2('baz')
    print
    func3('foo', 'bar')
    
  • OO

    class entryExit:
        def __init__(self, arg):
            self.arg = arg
    
        def __call__(self, function):
            def internal_call(*args, **kwargs):
                print "Entering", function.__name__, 'with argument', self.arg
                function(*args, **kwargs)
                print "Exited", function.__name__
            return internal_call
    
    @entryExit('foo')
    def func():
        print 'inside func'
    
    @entryExit('bar')
    def func2(arg):
        print 'inside func2, with arg', arg
    
    @entryExit('baz')
    def func3(arg1, arg2):
        print 'inside func3, with args', arg1, 'and', arg2
    
    func()
    print
    func2('baz')
    print
    func3('foo', 'bar')
    
  • Result

    Entering func with argument foo
    inside func
    Exited func
    
    Entering func2 with argument bar
    inside func2, with arg baz
    Exited func2
    
    Entering func3 with argument baz
    inside func3, with args foo and bar
    Exited func3
    

    This way the extra parameter(s) can be added per function definition, and they stay the same over each call of that function.

Multiple decorators

  • Functional

    Of course, a function can have more than one decorator:

    def entry(function):
        def new_function(*args, **kwargs):
            print "Entering"
            function(*args, **kwargs)
        return new_function
    
    def exit(function):
        def new_function(*args, **kwargs):
            function(*args, **kwargs)
            print "Exited", function.__name__
        return new_function
    
    @exit
    def func():
        print 'inside func'
    
    @entry
    def func2(arg):
        print 'inside func2, with arg', arg
    
    @entry
    @exit
    def func3(arg1, arg2):
        print 'inside func3, with args', arg1, 'and', arg2
    
    func()
    print
    func2('baz')
    print
    func3('foo', 'bar')
    
  • OO

    class entry:
        def __init__(self, function):
            self.function = function
    
        def __call__(self, *args, **kwargs):
            print "Entering"
            self.function(*args, **kwargs)
    
    class exit:
        def __init__(self, function):
            self.function = function
    
        def __call__(self, *args, **kwargs):
            self.function(*args, **kwargs)
            print "Exited", self.function.__name__
    
    @exit
    def func():
        print 'inside func'
    
    @entry
    def func2(arg):
        print 'inside func2, with arg', arg
    
    @entry
    @exit
    def func3(arg1, arg2):
        print 'inside func3, with args', arg1, 'and', arg2
    
    func()
    print
    func2('baz')
    print
    func3('foo', 'bar')
    
  • Result

    inside func
    Exited func
    
    Entering
    inside func2, with arg baz
    
    Entering
    inside func3, with args foo and bar
    Exited func3
    

    As you can see, the __name__ does change to the new function that we added.

Sources

What I've learned today and what I've written here has all been made possible thanks to the following sources: