As mentioned in a previous post, I recently finished writing the second edition of Learn Ansible. This had me using Python a lot, and my one key takeaway is that managing Python has become a massive chore. Now, with the book complete, I thought I would strip back on my machine, start from scratch - or as close to clean as I could get - and develop a cleaner, more streamlined way of managing Python on my local machine.
I have tried a few methods in the past—see these posts from December 2019, January 2021 and October 2021. While these worked, they quickly became a chore to maintain due to all the different things that were trying to use Python. For example, there is the version of Python built into macOS, Homebrew tries to maintain its own Python installations for packages it downloads, and then there is my stuff on top.
Introducing Conda
I did a little research and chose Conda - which can be found here . While we will cover the Conda command in this post - I am using Miniconda .
Miniconda is described as the following:
Miniconda is a free minimal installer for conda. It is a small bootstrap version of Anaconda that includes only conda, Python, the packages they both depend on, and a small number of other useful packages (like pip, zlib, and a few others).
So, as I wanted to keep things as simple as possible, I used Miniconda rather than the more full-fat Anaconda.
Installing and configuring Miniconda on macOS
The installation guide for Miniconda is simple if you are using Homebrew , you just need to run the following command:
brew install miniconda
Once installed, we must ensure it is loaded when we open our terminal. To add the right lines to ~/.zshrc run the following command:
conda init zsh
This adds the following code to the end of ~/.zshrc:
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/opt/homebrew/Caskroom/miniconda/base/bin/conda' 'shell.zsh' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/opt/homebrew/Caskroom/miniconda/base/etc/profile.d/conda.sh" ]; then
. "/opt/homebrew/Caskroom/miniconda/base/etc/profile.d/conda.sh"
else
export PATH="/opt/homebrew/Caskroom/miniconda/base/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize
I have already updated my dotfiles to include the above and also to use the Powerline10k theme so it shows the Conda environment that is currently selected in my terminal:
As you can see from the prompt above, where it shows base on the right-hand side - with base being the default environment.
Adding an Ansible environment
In this example, I will create an environment for running Ansible; let’s start by creating the environment itself.
Creating the environment
To add an environment for running just Ansible, I ran the following command:
conda create -n ansible python=3.12
The command gave me the following output:
Channels:
- defaults
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /opt/homebrew/Caskroom/miniconda/base/envs/ansible
added / updated specs:
- python=3.12
The following packages will be downloaded:
package | build
---------------------------|-----------------
bzip2-1.0.8 | h80987f9_5 129 KB
ca-certificates-2024.3.11 | hca03da5_0 128 KB
python-3.12.2 | h99e199e_0 14.0 MB
tzdata-2024a | h04d1e81_0 116 KB
xz-5.4.6 | h80987f9_0 372 KB
------------------------------------------------------------
Total: 14.8 MB
The following NEW packages will be INSTALLED:
bzip2 pkgs/main/osx-arm64::bzip2-1.0.8-h80987f9_5
ca-certificates pkgs/main/osx-arm64::ca-certificates-2024.3.11-hca03da5_0
expat pkgs/main/osx-arm64::expat-2.5.0-h313beb8_0
libcxx pkgs/main/osx-arm64::libcxx-14.0.6-h848a8c0_0
libffi pkgs/main/osx-arm64::libffi-3.4.4-hca03da5_0
ncurses pkgs/main/osx-arm64::ncurses-6.4-h313beb8_0
openssl pkgs/main/osx-arm64::openssl-3.0.13-h1a28f6b_0
pip pkgs/main/osx-arm64::pip-23.3.1-py312hca03da5_0
python pkgs/main/osx-arm64::python-3.12.2-h99e199e_0
readline pkgs/main/osx-arm64::readline-8.2-h1a28f6b_0
setuptools pkgs/main/osx-arm64::setuptools-68.2.2-py312hca03da5_0
sqlite pkgs/main/osx-arm64::sqlite-3.41.2-h80987f9_0
tk pkgs/main/osx-arm64::tk-8.6.12-hb8d0fd4_0
tzdata pkgs/main/noarch::tzdata-2024a-h04d1e81_0
wheel pkgs/main/osx-arm64::wheel-0.41.2-py312hca03da5_0
xz pkgs/main/osx-arm64::xz-5.4.6-h80987f9_0
zlib pkgs/main/osx-arm64::zlib-1.2.13-h5a0b063_0
Proceed ([y]/n)? y
Downloading and Extracting Packages:
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate ansible
#
# To deactivate an active environment, use
#
# $ conda deactivate
Activating the environment
With the dedicated Ansible environment created, we can activate using the following command:
conda activate ansible
We can now check the path of the python binary by running which python:
which python
/opt/homebrew/Caskroom/miniconda/base/envs/ansible/bin/python
As you can see, it is using the python binary within our environment, which means that Ansible can now be installed along with some collections:
python -m pip install ansible
ansible-galaxy collection install azure.azcollection
python -m pip install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt
The screen below shows an example of what the terminal looks like when switched to the Ansible environment:
So far so good !!!
Digging Deeper
Here are some notes on package management and virtual environment management.
Pip vs Conda
As well as being a Python environment manager, Conda also provides it own package management, so rather that running pip using
python -m pip install ansible
I could have run the following command:
conda install conda-forge::ansible
This would install Ansible from the main Anaconda site, which hosts its own files at conda-forge/ansible for Ansible, if I had taken this route the installation would have looked like the following:
Channels:
- defaults
- conda-forge
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /opt/homebrew/Caskroom/miniconda/base/envs/ansible
added / updated specs:
- conda-forge::ansible
The following packages will be downloaded:
package | build
---------------------------|-----------------
ansible-9.4.0 | pyh707e725_0 20.7 MB conda-forge
ansible-core-2.16.5 | pyh707e725_0 1.3 MB conda-forge
cryptography-42.0.5 | py312hd4332d6_0 1.4 MB
jinja2-3.1.3 | py312hca03da5_0 327 KB
markupsafe-2.1.3 | py312h80987f9_0 24 KB
packaging-23.2 | py312hca03da5_0 169 KB
pyyaml-6.0.1 | py312h80987f9_0 172 KB
resolvelib-0.8.1 | py312hca03da5_0 30 KB
yaml-0.2.5 | h1a28f6b_0 71 KB
------------------------------------------------------------
Total: 24.1 MB
The following NEW packages will be INSTALLED:
ansible conda-forge/noarch::ansible-9.4.0-pyh707e725_0
ansible-core conda-forge/noarch::ansible-core-2.16.5-pyh707e725_0
cffi pkgs/main/osx-arm64::cffi-1.16.0-py312h80987f9_0
cryptography pkgs/main/osx-arm64::cryptography-42.0.5-py312hd4332d6_0
jinja2 pkgs/main/osx-arm64::jinja2-3.1.3-py312hca03da5_0
markupsafe pkgs/main/osx-arm64::markupsafe-2.1.3-py312h80987f9_0
packaging pkgs/main/osx-arm64::packaging-23.2-py312hca03da5_0
pycparser pkgs/main/noarch::pycparser-2.21-pyhd3eb1b0_0
pyyaml pkgs/main/osx-arm64::pyyaml-6.0.1-py312h80987f9_0
resolvelib pkgs/main/osx-arm64::resolvelib-0.8.1-py312hca03da5_0
yaml pkgs/main/osx-arm64::yaml-0.2.5-h1a28f6b_0
Proceed ([y]/n)? n
CondaSystemExit: Exiting.
However, in my case, as I wanted to keep things simple - I am sticking to what I know and will use pip.
I did, however, add the following alias for pip in my dotfile to call python -m pip each time I run the pip command:
alias pip='python -m pip'
This means that if I run pip install ansible , the command python -m pip install ansible would be run, so I can be sure that I am not calling the pip executable from some other random place.
Installing a different Python version
Now, not every piece of code supports the latest and greatest version of Python. Luckily, as you might have already guessed, we can define what version of Python to install when you create your virtual environment using Conda.
conda create -n test python=3.10
As you can see, we are requesting that Python 3.10 be installed instead of Python 3.12, which is what our other virtual environments use.
Channels:
- defaults
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /opt/homebrew/Caskroom/miniconda/base/envs/test
added / updated specs:
- python=3.10
The following packages will be downloaded:
package | build
---------------------------|-----------------
pip-23.3.1 | py310hca03da5_0 2.7 MB
python-3.10.14 | hb885b13_0 13.0 MB
setuptools-68.2.2 | py310hca03da5_0 942 KB
wheel-0.41.2 | py310hca03da5_0 107 KB
------------------------------------------------------------
Total: 16.7 MB
The following NEW packages will be INSTALLED:
bzip2 pkgs/main/osx-arm64::bzip2-1.0.8-h80987f9_5
ca-certificates pkgs/main/osx-arm64::ca-certificates-2024.3.11-hca03da5_0
libffi pkgs/main/osx-arm64::libffi-3.4.4-hca03da5_0
ncurses pkgs/main/osx-arm64::ncurses-6.4-h313beb8_0
openssl pkgs/main/osx-arm64::openssl-3.0.13-h1a28f6b_0
pip pkgs/main/osx-arm64::pip-23.3.1-py310hca03da5_0
python pkgs/main/osx-arm64::python-3.10.14-hb885b13_0
readline pkgs/main/osx-arm64::readline-8.2-h1a28f6b_0
setuptools pkgs/main/osx-arm64::setuptools-68.2.2-py310hca03da5_0
sqlite pkgs/main/osx-arm64::sqlite-3.41.2-h80987f9_0
tk pkgs/main/osx-arm64::tk-8.6.12-hb8d0fd4_0
tzdata pkgs/main/noarch::tzdata-2024a-h04d1e81_0
wheel pkgs/main/osx-arm64::wheel-0.41.2-py310hca03da5_0
xz pkgs/main/osx-arm64::xz-5.4.6-h80987f9_0
zlib pkgs/main/osx-arm64::zlib-1.2.13-h5a0b063_0
Proceed ([y]/n)? y
Downloading and Extracting Packages:
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate test
#
# To deactivate an active environment, use
#
# $ conda deactivate
As you can see from the screen below, switching to the new virtual environment loads the older version:
Managing Virtual Environments
You can list all of your virtual environments using the command below:
conda env list
This will return something like the following:
# conda environments:
#
base * /opt/homebrew/Caskroom/miniconda/base
ansible /opt/homebrew/Caskroom/miniconda/base/envs/ansible
discogs /opt/homebrew/Caskroom/miniconda/base/envs/discogs
test /opt/homebrew/Caskroom/miniconda/base/envs/test
Now let’s remove the test virtual environment:
conda remove -n test --all
This will nuke everything to do with the test virtual environment:
Remove all packages in environment /opt/homebrew/Caskroom/miniconda/base/envs/test:
## Package Plan ##
environment location: /opt/homebrew/Caskroom/miniconda/base/envs/test
The following packages will be REMOVED:
bzip2-1.0.8-h80987f9_5
ca-certificates-2024.3.11-hca03da5_0
libffi-3.4.4-hca03da5_0
ncurses-6.4-h313beb8_0
openssl-3.0.13-h1a28f6b_0
pip-23.3.1-py310hca03da5_0
python-3.10.14-hb885b13_0
readline-8.2-h1a28f6b_0
setuptools-68.2.2-py310hca03da5_0
sqlite-3.41.2-h80987f9_0
tk-8.6.12-hb8d0fd4_0
tzdata-2024a-h04d1e81_0
wheel-0.41.2-py310hca03da5_0
xz-5.4.6-h80987f9_0
zlib-1.2.13-h5a0b063_0
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Everything found within the environment (/opt/homebrew/Caskroom/miniconda/base/envs/test), including any conda environment configurations and any non-conda files, will be deleted. Do you wish to continue?
(y/[n])? y
Updating packages
Even if, like me, you use pip there are still some packages managed by Conda in your Virtual environment, you can update them by switching to the virtual environment you want to update and then run:
conda update --all
Conclusion
Managing Python environments can be a daunting task, especially when dealing with multiple projects and dependencies. However, by leveraging the power of Conda and Miniconda, you can streamline your Python environment management process and ensure a clean, efficient setup on your local machine.
With Conda, you can easily create isolated Python environments for different projects, specifying the exact Python version and package requirements. This allows you to maintain separate environments for each project, avoiding conflicts and ensuring compatibility.
By following the steps outlined in this post, you can install and configure Miniconda on your macOS system, create dedicated environments for tools like Ansible, and manage packages using either Conda’s package management or the familiar pip command.
Additionally, Conda provides flexibility in installing different Python versions within each environment, enabling you to work with projects that require specific Python versions seamlessly.
With the ability to list, remove, and update virtual environments and packages, Conda offers a comprehensive solution for Python environment management. By incorporating Conda into your workflow, you can enhance your productivity, maintain a clean development setup, and efficiently manage your Python projects.
So, whether you’re a beginner or an experienced Python developer, embracing Conda for Python environment management can greatly simplify your development process and help you focus on writing awesome code!