All About Context (and file handling)


While this post will cover context managers in Python, I’d like to first address a statement made in my introductory post:

My goal here isn’t to teach programming, there are much better resources out there for that.

Well lets just…

My goal here isn’t to teach programming, there are much better resources out there for that.

My intention behind that statement was to express that I wouldn’t be covering the fundamentals of programming as a whole, because there are much better resources for that.  Based on the evidence, I did a terrible job of expressing those intentions clearly. I doubt anyone ever intends to paint themselves into a corner, but thankfully, strikethrough and shame heals all.

In theory, leading someone to believe they won’t be taught new programming concepts, and then later teaching programming concepts isn’t terrible.  At the very least, it’s always good practice to admit to mistakes, regardless of how incredibly specific or mundane.  I also wanted to stretch this introductory section out a bit more, and this was the best I could come up with.  Anyway, let’s talk about context managers!

Context Managers

While context managers are applicable to operations outside of file handling, file open/close procedures are a common use case, and examples utilizing such operations provide a good introduction.  Although examples in this post are Python specific, the concepts covered can be related to other languages as well;  in C# for example, the using statement behaves like a context manager.  Additional information on context managers in Python, including other applications, can be found in this excellent article by Jeff Knupp.

The technical definition, along with the meaning behind ‘context manager’ can be found in Python’s official documentation in regards to the with statement:

context manager is an object that defines the runtime context to be established when executing a with statement. The context manager handles the entry into, and the exit from, the desired runtime context for the execution of the block of code.

While the technical definition explains what context managers do fundamentally, what they do in practice comes down to the following two operations (in the order they appear):

  1. Taking control of a resource
  2. Releasing control of that resource

A better explanation of context managers is covered through one of their primary uses; file handling.

File Handling

Expanding on the definition provided in the previous section, context managers can be thought of as a means of handling operations that are performed in pairs.  In Python, context managers are often used to handle file open and file close operations. While not using a context manager for file handling doesn’t mean your code won’t work, it’s something to be avoided.  The following python code will perform as expected:

file = open('insecure_password_storage.txt', 'w')

Although opening and closing a file in this manner works, problems occur if an exception is raised prior to the call to file.close().  A failure to reach this call introduces the following possibilities:

  1. Some implementations of Python will handle this close at a later point via automatic garbage collection, but allowing the loss of explicit control over that file isn’t a good practice.
  2. Python might not release control of the file at all, limiting manipulation of that file until control is released.
  3. Data may not be correctly flushed to the file itself if that file was opened in write/read-write mode

This method of file handling introduces too much uncertainty and instability to a program.  Forgetting to add a file.close(), or even a simple syntax error can introduce any of the above problems.   Errors are always a possibility, especially during testing, and more focus should be placed on solving the cause of an error rather than worrying about the external effects of that error.  Accounting for those errors can be achieved with the following:

file = open('insecure_password_storage.txt', 'w')
    file.write('12345) # closing apostrophe missing

By leaving the apostrophe off the file.write(), at runtime,  python will raise an exception.  The addition of the finally clause acts as a cleanup action; even if Python throws an exception, anything under the finally block executes; in this case, the file.close() operation.

This method of file handling acts as a context manager, and was the primary method of opening/closing a file prior to the addition of the with statement in Python 2.5. Although the try-finally approach handles exceptions raised prior to file closure, missing calls to  file.close()  aren’t accounted for.

It’s easy to forget to add a piece of code (unless I’m the only one who does it, then just play along).  I personally don’t like having to remember to close a file using the method above considering the consequences of not doing it (that sounds better than ‘I like being lazy’ at least).  To achieve the same effect as a try-finally approach, but with the added benefit of writing less code, use the with statement:

# File write operation
with open('insecure_password_storage.txt', 'w') as file:

The with statement in the above code replaces the functionality of the try-finally approach.  Apart from the syntax difference, the biggest change is the exclusion of  file.close().  The with statement acts as a context manager, so any file opened under the with block closes at the end of the block.  Just like the try-finally method, even if Python raises an exception prior to the file being closed, the with statement ensures the resource is released.  Read operations utilizing a with statement operate in the same manner:

# File read operation
with open('insecure_password_storage.txt', 'r') as file:
    my_passwords = file.readlines()

The with block always handles any indented code under it, so the contents of a file can be stored inside of a variable and used outside the block. Although examples have covered basic file read/write calls, any operation that need to be performed on an open file can be performed using a with statement.  A file.close() method never needs to be included in the with statement, and it shouldn’t as the with statement was intended to handle operations like this.

Final Thoughts

Use the with statement method for file open/close operations; doing so will improve your code’s stability, and it will avoid introducing file system problems on top of code errors.   At the very least, you are cutting out an entire line of code from every file operation; that’s a 33% decrease!

Those looking for information on the underlying methods behind the with statement can find more information here.

Ohh, and don’t store your passwords in text files.