This morning I was playing around and decided to try and recreate an example from Structure and Interpretation of Computer Programs (SICP). Along the way I discovered some unexpected gotchas with how Python scopes closures. The example I was trying to reproduce was the following one:

(define (make-account balance)
  (define (withdraw amount)
    (if (>= balance amount)
        (begin (set! balance (- balance amount))
               balance)
        "Insufficient funds"))
  (define (deposit amount)
    (set! balance (+ balance amount))
    balance)
  (define (dispatch m)
    (cond ((eq? m 'withdraw) withdraw)
          ((eq? m 'deposit) deposit)
          (else (error "Unknown request -- MAKE-ACCOUNT"
                       m))))
  dispatch)

Initially, thinking that Python 2.7.x closures were pretty good in my experience, I decided on the following naive implementation:

def make_account(balance):

    def deposit(amount):
        balance = balance + amount
        return balance
 
    def withdraw(amount):
        if balance >= amount:
            balance = balance - amount
            return balance
        return 'Insufficient funds'
 
    def dispatch(method_name):
        if method_name == 'deposit':
            return deposit
        elif method_name == 'withdraw':
            return withdraw
        elif method_name == 'balance':
            return balance
        else:
            raise ValueError('Unknown request -- {}'.format(method_name))
 
    return dispatch

But when you hop in the Python shell and try to run this you’ll have the following experience:

>>> account = make_account(100)
>>> account("deposit")(25)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "closure_objects.py", line 3, in deposit
    balance = balance + amount
UnboundLocalError: local variable 'balance' referenced before assignment

After digging around for a little bit I found this excellent article containing some gotcha’s regarding scoping in Python closures. To quote (with some edits for our example):

…When you say [balance = balance + amount], two things happen at different times:

1. When Python compiles the function, it sees [balance] =, and declares a new variable, scoped within inner, named [balance].

2. When Python executes the function, it needs to compute [balance + amount]. Okay, well, what’s [balance]? It’s a local variable… but, oops, it doesn’t have a value yet! Raise error.

The assignment creates a new inner variable that masks the outer variable

Luckily, Python 3 fixes the issue by introducing the nonlocal keyword, but in Python 2.X the solution is a bit hacky:

def make_account(balance):
    current_balance = [balance]
 
    def deposit(amount):
        current_balance[0] = current_balance[0] + amount
        return current_balance[0]
 
    def withdraw(amount):
        if current_balance[0] >= amount:
            current_balance[0] = current_balance[0] - amount
            return current_balance[0]
        return 'Insufficient funds'
 
    def dispatch(method_name):
        if method_name == 'deposit':
            return deposit
        elif method_name == 'withdraw':
            return withdraw
        elif method_name == 'balance':
            return current_balance[0]
        else:
            raise ValueError('Unknown request -- {}'.format(method_name))
 
    return dispatch

This works because Python leverages 3 different types of assignment, the regular = assignment, __setattr__ assignment, and __setitem__ assignment. The original = assignment attempted to created a new variable, but by triggering __setitem__ assignment instead we get the expected behavior.

Hopefully this will help stop others from getting tripped up on this.

Resources To Better Understand The Issue:

• Gotcha: Python, scoping, and closures
• Complete Gist