Running some code whenever your Jupyter notebook starts is handy and easy.

You put a some code in $(ipython locate)/profile_default/startup/00-mycode.py and improve your workflow instantly. You can avoid writing import numpy as np in every notebook, check that you’re not running a notebook with production credentials etc.

There’s a big catch, though — they fail silently. If there’s an error in running the code from startup scripts, the notebook will run as nothing has happened.

Having them fail silently is not a big deal if you’re using them to import numpy — you’ll figure it out first time you reference the variable. But if you’re using them to set up some security guardrails and they fail, you’re in trouble.

How to make startup scripts fail hard?

Unfortunately, there is no way to just flip some config flag and make them fail, but there is a workaround — you can write your startup script as a ipython extension. I’ll explain how to do it, and then explain the anatomy of the solution.

I’ve create a tiny repository with everything needed. There are three steps

  1. Clone the repo.
  2. Copy your startup code to profile_changes/startup_extensions/extension_example.py. The load_ipython_extension function is not necessary unless you need a handle to ipython.
  3. Copy both files to the default ipython profile with following command
cp -r profile_changes/* $(ipython locate)/profile_default

These steps will make sure that your code is loaded every time a kernel starts, and that it fails if your script fails.

How does it work?

The solution consists of two components.

1. Very simple ipython extension

You’ve seen the content of that one while you were creating the script. Instead of startup scripts, we use a slim ipython extension because we can configure ipython to fail if extension loading fails.

The code is located in .ipython/profile_default/startup_extensions/extension_example.py, which we later on instruct ipython to load on startup.

2. IPython config

We change the config (e.g. ~/.ipython/profile_default/ipython_config.py) to load your extension. The change looks something like:

import sys
import os
sys.path.append(os.path.join(os.path.dirname(__file__), "startup_extensions"))

c.InteractiveShellApp.reraise_ipython_extension_failures = True
c.InteractiveShellApp.extensions.append(
  'extension_example'
)

It does 3 things

  • Makes ipython fails if loading any extensions fail. Without that, extensions are no better than your startup scripts (line 5)
  • Makes sure that ipython can find your extensions (lines 1-3)
  • Loads your extension_example.py (line 6).

“name ‘np’ is not defined” or How to make sure imported packages are visible in notebooks?

Variables created/imported in the extension won’t be visible in notebooks by default. That’s because extensions are loaded as python packages, and don’t share the namespace with your notebooks.

Your extensions can make these variables visible, but you have to do it explicitly by pushing it to ipython.user_ns. Here’s an example how:

import numpy as np

def load_ipython_extension(ipython):
  # This will make np visible in user namespace (notebook)
  ipython.user_ns['np'] = np