Seeing “ImportError: No module named tensorflow” but you know you installed it? Sometimes you can import packages from the console, but not from the Jupyter notebook? !pip install tensorflow sometimes just does not work?

It’s not about you. It’s not about python being flaky. It’s actually about your machine running multiple python installations (environments). Let’s build a basic understanding of what’s happening there and what causes your packages missing even after being installed. Once you understand that, you’ll have fewer problems installing and importing packages you need.

You are running multiple environments of python. That is fine!

Not fine like this! It is actually fine and not your fault. Because not all python2 tools are ported to python3, it is possible that your operating system runs both versions without you touching anything.

For example, I have both python 2 and python 3 installed. When I call python in my console – python 2.7.10 gets invoked; when I call python3, python 3.7.1 is invoked. It gets even better. I have multiple environments of python 3.6.3 (yes, all the same version) and they load dynamically depending on the project I am currently working on (thanks to conda). I create and destroy python environments daily.

So how do I know what environment is currently running? Easy! By running import sys; sys.prefix in interactive console. That reveals what environment of python you are currently in.

Different python environment means different sets of packages

Unless you’ve modified $PYTHONPATH variable (it’s ok if you haven’t heard of it), each of these environments will use a completely separate set of packages. That is the reason you can’t import the package and you know you installed it. You installed it in the wrong environment!

But you’re running Jupyter and not the console and how does it all fit together? We’re getting there just now.

Jupyter kernel you use is not using the same python environment as your console.

Jupyter does not run python the same way your console does. It has a concept of a kernel (if you are not familiar with that concept, think about it as python environment registered with Jupyter). The kernel running your notebook likely uses a different python environment and definitely does not have all the environment variables set as your console does. That is fine, too. We know how to figure out which environment is running our code so we can do exactly the same in Jupyter notebook.

Aha!!! You know which environment Jupyter uses. Now you just have to:

  • make sure your console (temporarily) uses the same python environment as your Jupyter notebook.
  • install the package with conda install or pip install (if you don’t know what is the difference, quickly go to this guide).

Cool, cool, cool. But you don’t know how to make your console use the same environment? All good, we’ll do it from your Jupyter notebook, but not like the last time you did it.

You are installing packages straight from Jupyter notebook and you are doing it wrong.

Browsing through StackOverflow about similar issues made me realise people are suggesting the thing that won’t work most of the time. If you are installing packages by running

!conda install tensorflow
# or if you want to use pip
!pip install tensorflow

you are using very fragile commands (if run in notebook) and that’s the reason packages you installed can’t be imported. It is not fine this time. But we will fix it 🙏. If you are interested in low-level details about why it does not work, read this great blog post from Jake Vanderplas.

Instead, use these commands:

import sys
!conda install --yes --prefix {sys.prefix} tensorflow
# or if you want to use pip
!{sys.executable} -m pip install tensorflow

and you will not have problems with damn ImportError again.

Understanding these concepts helped me and my teammates to understand what’s happening and why we were getting these errors. If you think I made a mistake, I missed something or you just want to say thanks, please do that in comments 🗣.