Debugging in Jupyter notebooks
A summary of the different ways of debugging in a jupyter notebook
Intro
I have used Jupyter notebooks pretty extensively for personal projects and research experiments. But sometimes after I take an extended break from coding I find myself forgetting some of the details about how debuggers work within Jupyter notebooks. This post is mostly a reminder for myself of how to use debuggers in a notebook but hopefully it can also help others who need the same reminder.
Throughout this notebook, we will be using ipdb instead of python's built-in pdb. ipdb builds upon pdb and offers better syntax highlighting among other features.
First, we will make sure that ipdb will be used by default instead of pdb by setting the environment variable (if it isn't set already).
%set_env PYTHONBREAKPOINT=IPython.core.debugger.set_trace
Setting a breakpoint within the notebook
Until python 3.7, there was only one way of setting a breakpoint in a python script. Fortunately, python 3.7 introduced the breakpoint() function with PEP 553 that makes it more convenient to set a breakpoint.
Prior to 3.7, you had to import the debugger and call set_trace():
def foo():
print("before breakpoint")
from IPython.core.debugger import set_trace; set_trace()
print("after breakpoint")
return
foo()
Fortunately, starting with python 3.7 you can do the same thing with breakpoint(). Note that python will look at the PYTHONBREAKPOINT environment variable to determine which debugger to drop into. This is why we set this variable early on.
def foo():
print("before breakpoint")
breakpoint()
print("after breakpoint")
return
foo()
Debugging exceptions post-mortem
Jupyter notebooks allow you to do post-mortem debugging by dropping the debugger into the code leading up an unhandled exception. This is very useful when developing code because it allows you to examine the logic around the exception without having to manually set a breakpoint yourself. All you need to do is the line magic %debug
def bar():
print("This function is about to fail with an error")
raise ValueError
bar()
Now we use the next cell to drop into the code right before the exception to examine the cause post-mortem.
%debug
You can also use the cell magic %%debug instead of breakpoint() at the top of a cell. However, I have never used %%debug because it seems to debug the execution of the cell by the notebook which causes some information to be hidden.
%%debug
print("You won't see the code for this line in the stack trace")
bar()