Guides for tooling
-
How to make startup scripts for Jupyter kernels reliable?
Running some code whenever your Jupyter notebook starts is handy and easy.
-
Github Actions: using python version from .python-version file (pyenv)
Github Actions: how to use pyenv's python version - one from .python-version file. -
Why requirements.txt isn't enough
If you're using only requirements.txt to manage your dependencies, you're in trouble. -
The minimal conda cheatsheet
The smallest conda cheatsheet you'll find around. -
Overview of python dependency management tools
I briefly describe each tool (pip, venv, pip-tools, pyenv, conda, pipenv, poetry and Docker), why it’s created and the problems it’s tackling. You can find a table summarizing all the information and the usual setups people use. -
SageMaker: install Jupyter extensions in restart-proof way
“Every time my notebook shuts down and restarts, I lose notebook extensions and have to reinstall them from the terminal”, my teammate said. Eventually, he gave up reinstalling them.
-
SageMaker: SSH to notebook instances
If you’re using SageMaker as a development machine, you’ll need SSH access to notebook instances sooner or later.
-
SageMaker: save your conda environments after the machine restarts
Are you creating conda environments and installing packages from scratch every time you start a SageMaker machine?
-
SageMaker: automatically stop your instances when idle
If your company is running on AWS, it’s likely that AWS Sagemaker is a central piece of the infrastructure you use daily. It’s fantastic how easy it is to start an instance and get a lot of CPU and GPU resources for your experimentation.
-
Importing packages in Jupyter notebooks
Seeing “ImportError: No module named tensorflow” but you know you installed it? Sometimes you can import packages from the console, but not from the Jupyter notebook? !pip install tensorflow sometimes just does not work?
-
The difference between conda and pip and how not to break your environment again?
One of the first things I’ve noticed while coming into Machine Learning Engineer role was package management mess. People often ask me “Should I use conda or pip to install packages?”. “Is conda just python with preinstalled packages?”. “I am getting some compiler errors. I thought we’re using just python?!”.