Pages Menu
Categories Menu

Most recent articles

Python basics: Functions and arguments

Posted by on 2014-Jul-18 |

I’ve recently talked to some guys and girls about functions in Python and how to define and call them. Here’s the distillation of everything that’s interesting about the topic. [Footnote: By the way, I won’t put comments in all of my functions, because they are so very short and just for illustration. You should always put documentation in your functions, though!]

The first thing we do is define a function without any parameters. There syntax for this is:

def fun():
    print("this is fun")

This defines a function fun that can be called and that prints out the string given in its function body. What’s the “function body”, you ask? It’s all the indented lines following def fun(): (which, coincidentally, is called function head), and it’s those lines that are executed when you call the function. Let’s try it out:

>>> fun()
this is fun

As you can see, you call functions by giving them a list of arguments in parentheses — and in our case, that list is empty. If you omit the argument list (sometimes also called parameter list), something entirely different happens:

>>> fun
<function fun at 0x10796b440>

What does that mean? It means that instead of calling the function, you pointed at the function. Functions in Python behave like any other value, so you could assign them to a variable or pass them as a parameter to another function. But more about that later.

Our function doesn’t return anything, so calling it doesn’t produce a value. Well, actually, “no value” in Python means None, and we can see that, too:

>>> val = fun()
this is fun
>>> print(repr(val))

One of the most important use cases of functions, however, is to compute something, which means we have to be able to indicate what the value of a function call should be. That’s what the keyword return does. It works like this:

def fun():
    print("this is still fun")
    return "enjoyment"

And we get the value back when we call the function:

>>> val = fun()
>>> print(repr(val))

So let’s say, we want to define a function that adds two variables. To do that, we need to tell Python that our function should accept two parameters:

def add(a, b):
    return a + b

We already know what return means, but now the argument list isn’t empty any more. Instead, it has two entries, a and b, and that means you also have to call this function with two arguments:

>>> add(2, 2)

You can not call the function with any other number of parameters, or you’ll get an error:

>>> add(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: add() missing 1 required positional argument: 'b'
>>> add(2, 3, 4)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: add() takes 2 positional arguments but 3 were given

The arguments we defined here are so-called positional arguments. They are called this because their position, or order, is important. Calling a function with a different order of arguments changes the meaning, like this:

def sub(a, b):
   return a - b

>>> sub(3, 2)
>>> sub(2, 3)

Positional arguments, because they are identified by their position.

But what if we want to allow someone to only specify one argument, assuming that people always mean 0 when they don’t say anything? In that case, we can turn one of our positional arguments into keyword arguments, or default arguments [Footnote: These two words actually have distinct, specific meanings in Python 3. However, this difference is not very important in most use cases, so I’ll use them interchangeably.]. This means that we specify a value that should be used when the argument is absent in the call:

def add(a, b=0):
   return a - b

I can now leave the value for b out of the call and Python will substitute a zero for it:

>>> add(1)
>>> add(1, 2)

This argument is now a default argument, because it can be defined either by its position or by its default value. It can also be identified by is name, and that’s why it’s not a positional argument any more (as you saw above, it’s still positional, too, though). Have a look at this:

def sub(a=0, b=0):
    return a - b

Both a and b are now default arguments that I can omit, or I can identify them by their name — and then the order isn’t important any more. Here’s how you can call this new sub function:

>>> sub(3, 2)
>>> sub(3)
>>> sub()
>>> sub(a=4)
>>> sub(b=2)
>>> sub(a=4, b=2)
>>> sub(b=4, a=2)

As you saw here, we can call this function like we did before, with positional arguments. We can also skip any of the arguments and get a value of 0 for those. We can also identify the arguments by their name, regardless of the order. That’s also why they’re called keyword arguments — instead of their position I can call them by their keyword, their name.

For most parts, this can get you very far. You can now create functions with as few or as many arguments as you need, and you can make some or all of these arguments with default values, so we can skip them or call them by their name.

The hitch with keyword arguments

Of course, not all is easy in the land of computers, so there’s a hitch that we have to look out for when using keyword arguments. These arguments are only initialized when the function is defined, so they become what other languages would call static, they don’t change from call to call.

Well, what do I mean by that now?! It means that when you give a keyword argument a default value that can change, like a list, a dictionary or an object, you’ll be surprised. Let’s have a look:

def wrong_insert(a, ls=[]):
    """Insert a into the given list."""
    return ls

Now when you use insert, at first nothing surprising happens:

>>> wrong_insert(2, [])
>>> wrong_insert(3, [1, 2])
[1, 2, 3]
>>> wrong_insert(1)     # we can omit the keyword argument here!

However, now something completely strange appears:

>>> wrong_insert(1)     # again!
[1, 1]

Wait, what now?! Why is 1 twice in that list? Because you inserted twice into the same list. That default value is only created once when you define the function, so when you append something to the list, you append it to the default value of that parameter. So how do we do it right? We assign a default value that cannot be changed and test for it. There’s one value that’s perfectly suited for that task, and it’s None:

def right_insert(a, ls=None):
    if ls is None:
        ls = []
    return ls

This one works as expected:

>>> right_insert(2, [])
>>> right_insert(3, [1, 2])
[1, 2, 3]
>>> right_insert(1)
>>> right_insert(1)

Be very, very careful when you see a list, a dictionary or an object as a default value. Changing that value will happen and you probably won’t expect it.

There’s more or less only one use case for putting a mutable value (like a dictionary) into a default value, and that’s when you use it as a cache. You should certainly call it CACHE in that case, too! Here’s an example:

def fib(n, CACHE={}):
    if n in CACHE:
        return CACHE[n]
        if n < 1:
            return 1
            val = fib(n-1) + fib(n-2)
            CACHE[n] = val
            return val

That’s a neatly encapsulated cache. Since we named it in ALL-CAPS, we know that it’s sort-of global, so we’ll be on the lookout when finding such a variable.

If you see any other list, dictionary or object as a default parameter value, be suspicious!

varargs, lots of positional arguments

But what if you don’t know the number of positional arguments beforehand? Say, you want to add two or more numbers? There’s a way to do that and it’s called varargs, from “variable number of arguments”. The syntax for that in Python is:

def adder(*args):
    result = 0
    for argument in args:
        result = result + argument
    return result

And let’s try it out:

>>> adder(1)
>>> adder()
>>> adder(1, 1, 2, 3, 5, 8, 13, 21)

So what happens here is that any argument that is not positional gets stuffed into a list, and this list is called args, because we gave it that name with an asterisk in front of it in the function definition. Any other name works as well, but it’s a pretty strong convention to use args here. And as you see, we can have any number of arguments, even none at all. Of course we can still specify that we want positional arguments in front of it:

def subber(start, *args):
    result = start
    for argument in args:
        result = result - argument
    return result

You can not omit the first argument. It’s a positional argument, so you cannot omit it. But we can omit the varargs:

>>> subber(5, 2)
>>> subber(10, 1, 2, 3, 4)
>>> subber(10)
>>> subber()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: subber() missing 1 required positional argument: 'start'

Can you have more than one varargs?

def subber(*args, *moreargs):
  File "<stdin>", line 1
    def subber(*args, *moreargs):
SyntaxError: invalid syntax

No, you can’t. And what would be the meaning of that anyway? Which list should Python add the arguments to? No, no, that doesn’t mean anything, so you it’s a syntax error, as it should be.

Did you notice what simple trick Python does here for us? It takes the arguments that you pass into a function that don’t go anywhere else, stuffs them into a list and gives us that list. It’s really all that happens.

But interestingly enough, the trick works in reverse, too: When you have a list of values that you want to use as arguments to a function, you can tell python to do just that, and with the same syntax, too:

>>> arglist = [10, 5, 4]
>>> subber(*arglist)
>>> subber(10, *arglist)
>>> subber(100, 1, 2, 3, 4, *arglist)

Don’t forget that asterisk, though:

>>> subber(10, arglist)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in subber
TypeError: unsupported operand type(s) for +: 'int' and 'list'

kwargs, lots of keyword arguments and lots of other arguments

So let’s say you want to define a function that has a keyword argument, but you don’t know what keyword that’ll be, and you also don’t know how many it should be. Sounds unlikely, but it’s actually very useful when you do dynamic processing or datastructures you don’t know much about.

Of course, there’s a syntax for it as well, and it’s pretty similar to the varargs we saw before. It’s called kwargs, for keyword arguments, uses two asterisks and instead of a list of arguments you get a dictionary:

def dynamic(**kwargs):

And it works as expected:

>>> dynamic(hello='world', goodbye='lonelyness')
{'goodbye': 'lonelyness', 'hello': 'world'}

Just as we saw before, the same works in reverse: When I have a dictionary with values that I want to use to call a function, I can just tell Python to unpack them into arguments:

def do_stuff(key=0, lock=1, **kwargs):
    print("doing stuff with key {} and lock {}".format(key, lock))
    print("and more arguments: {}".format(kwargs))

>>> params = {'key': 10, 'lock': 'Locke', 'something': 'else'}
>>> do_stuff(**params)
doing stuff with key 10 and lock Locke
and more arguments: {'something': 'else'}

So now we have all parts together and we can actually stick them together, so here’s the full syntax for function definitions [at least in Python 2.7]:

def fun(a, b, *args, **kwargs): print(a, b, args, kwargs)

Try it out for yourself!

Actually, there’s one more thing: You could add in keyword arguments in the middle, between b and *args, but it wouldn’t work the way you expected it to, and that would be confusing and unintuitive. So don’t do that!

Functions are values!

So why is the args and kwargs dance so useful? Especially kwargs — what good could it be when you don’t know the parameters to your function?

To make this work for us, we have to know one more great thing about functions: they are values, just like everything else is. So you can rename functions, store them in variables, put them in lists and even pass them to other functions. Using fun from above, let’s see:

>>> fun(0, 0)
(0, 0, [], {})
>>> morefun = fun
>>> morefun(0, 0)
(0, 0, [], {})
>>> funlist = [fun, morefun]
>>> funlist[0](0, 0)
(0, 0, [], {})
>>> def funfun(f):
        print("calling f")
        f(0, 0)
>>> funfun(fun)
calling f
(0, 0, [], {})

That last bit is really interesting: we can pass functions into functions. We can also return functions from functions — because they’re simply values that behave like any other.

And finally, we can also define functions inside of functions. In a way, when you write def f(...):..., this is just shorthand for f = <some function>, and it will behave exactly like any other variable. Think of functions as any other variable, just one that you can call to get a new value.

So now let’s put all this together. See if you can figure out what this next example does:

def log_fun(f):
    def inside(*args, **kwargs):
        print("called with {} and {}".format(args, kwargs))
        result = f(*args, **kwargs)
        print("resulted in {}".format(result))
        return result
    return inside

Wow! We defined a function that takes a function, creates a function, uses a function inside the inner function, and then returns that inner function. So, what does it do? log_fun wraps the function we passed as parameter f, and returns a function that does the wrapping. The wrapping function then prints the arguments and kwarguments, calls f with them, prints the result of the call and finally returns the result of the call. All without knowing anythig about the parameters of f itself. From a parameter/return perspective, it behaves quite like f does — only it prints out the parameters and results in between. How do we use it? Let’s try:

>>> new_f = log_fun(fun)
>>> new_f(0, 0)
called with (0, 0) and {}
0 0 () {}
resulted in None

You could be even more daring and replace fun with that new logged version of fun:

>>> fun = log_fun(fun)
>>> fun(0, 0)
called with (0, 0) and {}
0 0 () {}
resulted in None

Pretty neat, eh? You simply wrapped your function in some logging, without caring what the actual parameters are. That log_fun function is pretty useful when you want to debug your function calls without changing the functions themselves.

Since wrapping, enhancing or decorating functions is such a useful concept, there’s a special syntax for it in python, with an @. These next two examples are exactly the same:

def something(x):
    print("X!", x)
something = log_fun(something)

and this one:

def something(x):
    print("X!", x)

And now you know what a Decorator is in python: It’s a function dthat takes a function f and returns a function f', and in-between does something with the original function f — effectively enhancing the things f does. Don’t be alarmed by the syntax, the two blocks above are exactly identical, semantically, so if you ever see one of the two, you can replace it with the other without fear.

There’s more to say about decorators, but there’s another document for that.


If you ever programmed in a compiled language like C, C++ or Java, you might have heard the term “overloading a function”. It’s a cool concept, and it basically means that two functions or methods can have the same name but a different parameter list. Like this (in Java):

class OverloadingDemo
    void simpleFunc()
        System.out.println('simpleFunc v1 called')

    int simpleFunc(int param)
        System.out.println('simpleFunc v2 called')
        return param * 2;

    public static void main(String[] args)

If you compiled that program and ran it, the output would be:

simpleFunc v1 called
simpleFunc v2 called

The reason this works is because when you compile this program, the compiler can match the different calls to the different function definitions. So effectively, there are two functions in this Java class, one without parameters and one with parameters.

Python works differently. When you define a function, as you remember, you simply give a value to a variable. So if you do this in Python:

def fun():
    print("fun v1 called")

def fun(a):
    print("fun v2 called")

Now there is still only one version of fun, and it’s the last one that you defined. The second definition of fun didn’t overload the function, it replaced it.

If you want your function to work the same way as the ones in the example above, you’ll have to do it yourself:

def fun(a=None):
    if a is None:
        print("fun v1 called")
        print("fun v2 called")

So, the gist is that Python does not have built-in overloading. Instead, you can assign a None value for the parameter you want to overload and check for its existence yourself.


Well, there’s a surprising amount of stuff in Python functions. The most important things you have to remember are:

  • If you want a variable number of positional arguments, use args. If you want to variable number of named arguments, use kwargs.
  • Function definitions have a lot of syntax, but they’re nothing “magic”. They just simply create a function object and assign that value to a name, the function name.
  • You can not overload a function in the Java-sense, but you can overload them in other ways, with named arguments, args, kwargs and your own separation logic.
  • Do not use a mutable value as a default parameter value.

And finally, don’t worry too much about it! =)

We’ve lost our top rating

Posted by on 2014-Jan-23 |

Yesterday, Qualys released a new SSL test with updated guidelines.

Until that moment, our page at scored straight A in their test. Now we only score A-. Check it out for yourself! Then check your own page! The reason for this is that the new test includes new guidelines on Forward Secrecy. For some combinations of browsers by a certain OS company, we don’t provide Forward Secrecy, leading to a downgrading.

Unfortunately, there’s not much I can do about this, because our web server does not support the necessary ciphers in the version we’re using. So for the moment, we’ll stay at A- and update our Forward Secrecy at some later point.


This is my current work:

SixSafe: your files, secure, forever.

Your files are encrypted securely even without TLS-FS!

Mark your live deployments with userscripts

Posted by on 2014-Jan-17 |

I recently launched a web service at its own domain. I am actively developing the service on my local machine, so I always have at least two instances running at the same time.

Thanks to that situation, I am scared whenever I change something that I’m doing the change to the wrong instance. Which would be bad.

So I built myself a little tool to help me discern between the two: a bright red tag that shows me which the live version is. It’s really simple with a Userstyle. All I did was add a new userstyle for the domain name of my service:

@-moz-document domain("") {

 body:before {

   content: "LIVE";

   font-size: 50px;
   background-color: red;
   color: #eee;

   margin: 0;
   padding: 100px 100px 5px 100px;

   position: absolute;
   top: -50px; left: -110px;

   -moz-transform: rotate(-45deg); 

You can see the result in the image: a bright red “LIVE” tag.

This solution is really great for me: It’s non-intrusive, so no-one else can see it. It’s easy to install without changing the service and it’s pretty obvious which instance I am working with. Everything I wanted!

This is my current work:

SixSafe: your files, secure, forever.

You won’t see the live tag on it unless you install the userstyle yourself!

Gamification 2.0

Posted by on 2013-Apr-23 |

A friend recently sent me a link to Jane McGonigals excellent TED talk about Gamification. Watching it, I was reminded of the Mother Of All Gamification talks, Jesse Schell’s Visions of the Gamepocalypse. I really enjoyed both of these talks, but I don’t like their conclusions.

Even Coursera now offers courses on Gamification. Which is fitting, I guess, since university courses have been "gamified" with points and regular feedback for a long time.

Even Coursera now offers courses on Gamification. Which is fitting, I guess, since university courses have been “gamified” with points and regular feedback for a long time.

What both Schell and McGonical want to do is make games that “trick” the player into doing something with himself or his surroundings at the same time. I don’t think that’s going to work. For one, because it’s dishonest, not telling the user what you’re actually trying to achieve with the game. But more importantly, because they are diluting the game design and experience with their ulterior motives. Did you play educational games when you were young? Probably Math Blaster! or Mario Teaches Typing? We had some of these when I was small and I never liked them. The reason is that you’re playing a nice little game and suddenly something interrupts your flow with a math question. That’s not a fun game mechanic. That’s not any game mechanic, actually, that’s just an annoying block between you and the game.

There is a second mainstream meaning of the word gamification, which is “turning some task into a game”. Wikipedia has a nice list of things that belong into this category, i.e. things that people have tried to do. I think that’s not the right way either, because I don’t think that achievements (as funny as they may sometimes be) are the way to use the knowledge and power we have learnt from games to make the world a better or easier place. I also think that achievements, specifically, are a pretty poor game mechanic that’s been used in recent years because they’re so easy to make (however, hard to get right), in contrast to meaningful in-game achievement without a badge, which is pretty hard to manufacture.

However, I think that gamification is something quite different: Gamification is using the knowledge of game mechanics and techniques and how they work, and apply them to different areas than computer games. Note that I am speaking only of knowledge of game mechanics, not game mechanics themselves. What I think games get right is finding and mining human traits like motivation and persistence. Games are exceedingly good at that, and good games even more so.



There’s a nice explanation on Youtube how Mega Man X gets it right. And did you know that the iconic first level of Super Mario Bros is designed exactly to show you what you can do in the game? Jumping, hitting blocks, getting coins from � blocks, enemies that are dangerous and that you can defeat them by jumping on top of them. All of that from one single screen, and in less than ten seconds. Without a single text display.

That is great gamification. They “gamified” their intro/tutorial and showed the player all these things effortlessly. Designing that level must have taken very long, but for the user it’s so incredibly smooth. So that’s one component of gamification for me: show instead of tell. Let the customer find out what’s possible on their own, without bugging them, by making it easy, almost obvious what and where things are to find. Let’s call this property “Leading”, because a properly gamified systems leads the user towards actions that are beneficial.



The second component of gamification that is most widely discussed is Motivation.

Points: 000000
Achievement locked

Can you scroll to a million points? Will you?

Let’s stay with Mario for a second: Mario has a coin counter and a points counter, and they are constantly increasing, all the time. Every time you do something, you get points or coins or both. So, in the simple gamification models of today, the designer would say “we need to motivate the player feedback to do something, so let’s add points to it!” That has been done, to great effect, and I guess it can work—for the right kind of person. But I’m not that kind of person, and to me it feels very childish to attach meaningless points to meaningful tasks, which is exactly what Jesse Schell points at in his wonderful talk. There are systems, games, out there, that try to do this sucessfully (by Miss McGonical again), so they demonstrate that it works for some situations and for some people. However, the point counter in Mario isn’t really the reason why one continues to play, is it? The point counter is merely an indicator that you’re going in the right direction and how far you’ve gotten. As such, it’s great, but it’s not the motivator. The motivator is something different: curiosity, urgency, “just one more level”, epicness, or something like that. This is what we have to take from games into the real world, not the stupid point counter. Of course, taking something so fluid and hard-to-grasp like motivation from a game and modelling a real-world task with that in mind is very hard. So it’s natural to try out simple schemes first, like the point counter. But I think that a properly gamified system should motivate the user through its properties and its whole, not through meaningless points or achievements. So, let’s take Motivation as our second component.

There are a myriad ways to model and channel this motivation, like levels, stages, medals, progress, goodies or, yes, achievements. In fact, there are so many that finding the right combination for a specific situation can be pretty hard.

Another important concept that helps form and channel user Motivation are reward schedules. And these are actually being taken into consideration when doing gamification. Some people proclaim that gamification should make tasks in the real world harder, but I don’t think that’s the case. Rather, it should move the challenge and the reward closer together, but keep the overall challenge/reward levels about the same. Which, thanks to human psychology, actually makes the task easier to do for a person. Brushing your teeth is a great example here (and the reason why Jesse Schell used it), because there’s a challenge and a reward and they are very far apart from each other. The challenge is to brush your teeth twice or thrice a day. The reward is that your teeth will not develop problems and pain for you at some point in the future. So in this case, the world we live in is badly gamified, because the effort has to be put in right now and frequently from now on, while the reward is uncertain and a long way away. Humans are just not good at that kind of schedule, so here’s something to be optimized. However, changing the reward schedule for brushing your teeth doesn’t make it any harder or easier to do, it just makes it more natural for humans to do. In a way, a good reward schedule makes it easier to mine the motivation a person has for any goal and to apply that motivation in the right direction towards achieving that goal.

There are very simple ways to do that and very complicated ways, but there are ways. I think we need to be getting better at finding them. Still, Motivation, one of the central points of Gamification.



The third point I’d like to add to the essentials is Feedback. Humans love feedback and they need it to function properly. Giving people feedback on any process allows them to get better at it, so that’s what they will do naturally. Let’s look at our two examples from above. When you brush your teeth, there is some feedback that you did well: a fresh breath, smooth teeth, and a whiter smile. If you skip once or twice, that feedback will deteriorate, so once you stopped brushing your teeth the feedback won’t work as well as before. However, there’s also very little clear indication about how good you did. There’s no way to check that you brushed everywhere, that you brushed enough and that you brushed for long enough. So the feedback loop isn’t very good, and that’s probably the reason why you don’t know anyone who’s a master a brushing their teeth. Now let’s analyze Mario. In a game of Mario, every action you take has a clear and consistent feedback value. Jumping on an enemy: happy sound, enemy dies and you get points (and sometimes even a shell). Bumping a � block: points, coins and another happy sound. Walk into an enemy: sad sound, your character falls of the screen and a live is deducted. Note that all of these actions are visible and clearly good or bad. If you do something that’s very good, like finishing a level, a small fanfare is played and you get to see a nice little cut scene. Playing Mario is one giant session of trial and feedback. Some people say that positive feedback is all that makes a game good, and I tend to agree: give a person feedback on what they’re doing and they’ll get better and better at it.

Most systems we use today are not very good at this. This is sad, because there are so many simple things one could add to most systems to make them give proper feedback, like status display, diagrams, scoreboards or suggestions for improvement. People can master any system as long as they are shown whether what they’re doing is good or bad.




Our hero begins his quest with a vow to finish this article.

There is a fourth point that is central to any gamification effort you want to undertake, and that is narrative. People crave narratives of any kind. So much in fact, that they will create their own stories for something as simple as a 32 x 16 monochrome display, showing a state machine with less than 16 states, making them bond with that state machine and get up in the middle of the night to ‘feed’ it. Note, however, that Tamagotchi is an excellent distillation of Feedback and thus masterfully gamified: people want to carry the toy around and care for it every day, no matter its simplicity, creating Motivation and Narrative out of the players themselves.
Some of the most well-known and loved toys and games of the world are simple playgrounds for creating narratives: Lego, Barbie, Playmobil, The Sims and, of course, Minecraft. All these games offer is a rich environment for creating your own stories and adventures.
At the same time, even the most simple-minded games offer some kind of background story, whether that may be furious birds or supertechnology soldiers.

The truth is that Narratives make it easier for people to understand a situation and muster up Motivation for it, even if the Narrative is quite thin. Even if you know that the princess is in another castle, here’s a good and simple reason to slog through all these complicated worlds: find the princess and rescue her. Be a hero!

Sometimes, team selections are used to create a simple but powerful Narrative: us against them. That seems to work for all recent 3d shooters, even if the teams are absolutely identical.

In a way, Narrative is also something that McGonical and Schell point at and want to use as a tool to motivate the player. It’s just that the methods they choose (point counters and badges) are quite crude and not very well-fitting, in my opinion. However, it’s also true that Narrative is probably the hardest to add to any Gamification project that is not actually a game. How would you fit a Narrative (that is not totally moronic) around brushing your teeth or doing your taxes? I don’t know! But figure it out and you’ll have the power to help people do something important. And you’ll find easy ways to collect your reward for that – even if it’s just the knowledge that you’re a hero!



So, to recap: Leading. Motivation. Feedback. Narrative. Add these four to any system you are designing, and you will have a properly gamified system that will work very well.

It’s hard to tell how to actually use these things in any real product, but I think it’s still worthwhile to start thinking about gamification on a broader scale.


This is my current work:

SixSafe: your files, secure, forever.

It’s not gamified (yet), but you should still take a look at it.

OAuth and Django — a cautionary tale

Posted by on 2013-Feb-01 |

Have you ever implemented OAuth authorization in any project of yours? Well, I have these last two weeks in a Django project I am doing, and it was not a nice experience. The following is going to be a bit of a rant against the state of authentication libraries, but take it as a cautionary tale and remember one thing:

Hours of doing can save you minutes of thinking.

So, let me tell you a little something about my experiences.

The first thing you’ll encounter on your quest is the choice between OAuth 1.0 and OAuth 2.0. There are voices for both sides, and they’re both very convincing, so what you’re going to do (if you’re like me) is first implement OAuth1, then switch to OAuth2, then scrap that and go back to OAuth1. Because the first thing you’ll try is to find and use a good OAuth1 library for Django, of which there are none. Oh, there are OAuth libraries all around, but I couldn’t get any of them to work. So you’ll say to yourself, screw this, I’m going to do OAuth2, it’s better anyway. Then you’ll spend some time reading up on bearer tokens and refresh tokens and grant types and you’ll come away more confused than enlightened. Then you’ll play around a bit with the existing implementations like Google’s wonderful OAuth playground and you’ll say to yourself: I can do this! And you’ll start and do that, reading parts of the humongous spec on the side and notice more and more that there’s loads of stuff in there that you don’t even want to think about. Then you’ll not get it done right, you won’t find any client libraries for using it either, and then finally you’ll read Eran Hammer’s rant over OAuth2 and decide that it’s really not in your best interest to implement OAuth2, and go back to searching for OAuth1 libraries you could use.

So now you’re stuck at that part again, but this time, you’re determined. You’re determined to find a stable and good implementation of OAuth1, that also offers 1.0a, because that’s important for security. You’ll skip oauthlib because you don’t trust it (a mistake, as you’ll find out later) and also because the documentation is a bit sparse. Well-written but sparse. So instead you’ll go directly for Leah Culvers six-years-old OAuth implementation. It’s a bit crummy and you can see its age, but it’s quite simple and adaptable, so you copy it verbatim into your project and start hacking away…

A mere two weeks later, you’ll have touched every part of the library and added your own crumminess to it. You’ll have an OAuth implementation that’s half-working, is neither OAuth 1.0 nor OAuth 1.0a but somewhere in between, and you’ll find new errors every other day, because you never took the time to read any of the OAuth tutorials there are — because you really couldn’t find good and simple ones, detailing the differences between OAuth versions. But, you’ll have a working authentication scheme that uses signed requests and never exchanges secrets in the clear, which is a good thing all on its own.

So let’s examine the issues you will have had by the time you’ve come this far.

  • You did some reading at the beginning about OAuth and its versions, but couldn’t find good enough material to make a decision. Go back and do that now, you’ll regret it many times if you don’t. Then pick one version and (in the case of OAuth2) one workflow and stick with that. You can extend later, if you need to. 
  • You didn’t find any good, trustworthy-looking libraries, for neither version of OAuth. That’s ok, just pick one and stick with it. Switching between libraries will not do you any good but burn your time away. 
  • Once you’ve collected some experience and read enough of the materials and the spec, write down your findings and make it easier for others and for future-you to find them and learn from it. 

As you can see, I’m in the third stage described. I’ve tried to address some of the issues I’ve found with current OAuth implementations and collect my knowledge about that stuff in code. It’s far from finished and far from done, but it will be useful once you start an OAuth1 implementation for yourself. You can find that code on GitHub along with some documentation I wrote, and I sincerely hope you can learn from my mistakes.


Posted by on 2012-Nov-08 |

A few days ago, a colleague pointed out to me that “simplicity” is not a feature. His reasoning behind this was that simplicity means different things for different persons, so simplicity can mean anything inside of a range of things.

I don’t agree with his statement at all, and I actually think that simplicity is the only feature any program can have. In the widest sense, all that computers do is provide a given task in a simpler way, usually making that task faster and more accessible to different groups of users. Some of the times, making a task simpler makes the task possible in the first place.

Even at the beginning, one of the first instances of a computer was the famous difference engine, invented by Charles Babbage. Its sole purpose? Computing tables of logarithms in a quick and reliable way. In other words: making finding a given logarithm simpler for the user. Note that the logarithm tables themselves are a way of making that task simpler in the first place.
After the difference engine came programmable looms by J. M. Jaquard, which made the production of complicated, patterned fabrics simpler.
Later, “real” programmable computers were invented by Konrad Zuse in Germany and von Neuman, Turing and others in Great Britain. Their use cases? Solving linear equations, doing the necessary computations to reverse code ciphers and computing ballistic tables. All of which make the underlying task simpler for the user. They also made these tasks a lot quicker and a lot more reliable, but these are merely side effects of making the tasks simpler for the user.

Mainframes: Making it simpler to do complex calculations of any kind for many users.

Home computers: Making it simpler to do complex calculations of any kind on your own.

The internet: Making it simpler to retrieve information that is stored in different places.

Email: Making it simpler to contact people that are not immediately available (i.e. making it simpler to send letters).

Even “modern” services and gadgets like Google (making it simpler to find some websites that store information you might be interested in), Facebook (making it simpler to have a homepage, share photos and see what others are doing), Twitter (making it simpler to tell everyone who’s interested what you’re doing) or Apple computers (making it simpler to use all the awesome stuff that nerds built for themselves) are built on that one guiding principle of simplicity.

Note that oftentimes, with simplicity come other benefits like scalability or even availability, which help adoption and development of even newer, even simpler products.

But the main thing that software, that technology does, is making stuff simple. I can’t think of a single example of a product or service that is not making something simpler.


Shameless plug: I’m doing Bit Chest at the moment, which is built around being a simple and safe file archiving solution.