pyenv lets you manage multiple versions of Python on your computer.

This blog post focuses on how pyenv uses the shim design pattern to provide a wonderful user experience (it doesn’t focus on installing and using the software like other posts).

Once pyenv is installed you can easily switch between one project that uses Python 3.6, another project that uses Python 3.7, and a system-wide default Python, without any additional thought.

Managing Python versions and packages can be a huge pain. It’s important to understand how pyenv works to debug issues. Complexities are layered on top of different Python versions… good luck understanding virtual environments if you don’t understand Python versioning.

pyenv is a fork of rbenv, a project to manage Ruby versions. The “shim design philosophy” used by pyenv and rbenv has proven to be a winner for maintaining multiple versions of a programming language on a given machine. Learning the shim design philosophy will make you a better programmer and teach you a powerful design pattern you can use with your own programs.

Homebrew’s package index. Homebrew Formulae. Package for scientific computing with Python: Analytics: Installs (30 days) boost-python3: 1,870. I use this installed python 3.6.6rc1 but brew still points python3 to 3.7.0. Since I'm using Heroku and Pipfile use 3.6, I use the other answer to get 3.6.5 installed (3.6.6 isn't available in GitHub, as far as I can tell). – John Pang Jul 2 '18 at 9:43. Python 3.6 has been officially released and is being used on many machines. In fact, if you were to install Python 3 via Homebrew (at the time of this writing), Python 3.6 would be installed. This is important because you need to check your file paths. Aug 15, 2018 15 Aug 2018 How to downgrade Homebrew Python to 3.6 A new Py­thon ver­sion has been re­leased, but Tensor­Flow does not yet sup­port it on ac­count of a break­ing change with async and await be­ing made re­served keywords. Python 3.6.8 was the last bugfix release for 3.6.x. Following the release of 3.6.8, we plan to provide security fixes for Python 3.6 as needed through 2021, five years following its initial release. Security fix releases are source-only releases; binary installers are not provided.

pyenv installation

You can install pyenv with Homebrew using the brew install pyenv command.

Homebrew runs this formula. We’ll cover how Homebrew works in another post.

Type which pyenv to see that there is a pyenv executable in the /usr/local/bin directory.

Enter open /usr/local/bin to view the pyenv executable in the directory.

The pyenv executable is run whenever we run commands like pyenv versions or pyenv install --list. Simply run pyenv without any arguments to see a listing of all the pyenv commands that can be run.

Let’s dig into where pyenv installs Python code in the filesystem.

Python installations

Run pyenv versions to see the Python versions you currently have installed on your machine.

Run pyenv install --list to see the Python versions that can be installed.

One of the Python versions that can be installed is 3.6.10 – let’s install it with pyenv install 3.6.10. Here’s the console output:

Python 3.6.10 was downloaded to the /Users/matthewpowers/.pyenv/versions/3.6.10 directory.

Python versions

Type pyenv versions to see all the Python versions available on your machine.

My machine has the system Python version, 3.6.10, and 3.7.5.

Python 3.7.5 is the “current selected” version of Python that pyenv will use (as indicated by the * next to 3.7.5 when pyenv versions is run).

If you run python when 3.7.5 is the selected Python version, then a Python 3.7.5 shell will be started. Understanding how pyenv knows to launch a 3.7.5 shell when the python command is run is the central focus of this post.

The /Users/matthewpowers/.pyenv/version file only contains a single line of data.

pyenv global

Let’s view the current “global” python version with pyenv global:

The global Python version is set in the /Users/matthewpowers/.pyenv/version file.

Let’s change the global version to 3.6.10 with pyenv global 3.6.10.

We can run pyenv versions and see that pyenv is using 3.6.10 by default now.

Let’s inspect the contents of the /Users/matthewpowers/.pyenv/version file to decipher what the pyenv global 3.6.10 command did under the hood.

Looks like pyenv global 3.6.10 just clobbered the /Users/matthewpowers/.pyenv/version file with a new Python version number.

Let’s make a bad decision and clobber /Users/matthewpowers/.pyenv/version with echo and > to see if we can change the global Python version ourselves (don’t do this – we’re just hacking to learn how this all works).

The pyenv global version is set in the /Users/matthewpowers/.pyenv/version file. This approach simple and intuitive. Let’s see how we can modify Python versions for different projects.

pyenv local

Let’s create a ~/Documents/project1/ folder and cd into the directory.

We can run pyenv version to see this project is using Python 3.7.5 (it’s using the global Python version by default):

Let’s change the Python version to 3.6.10 for this project.

pyenv added a .python-version file to the project1 directory.

Let’s inspect the contents of /Users/matthewpowers/Documents/project1/.python-version:

Homebrew Install Python 3.6

The /Users/matthewpowers/.pyenv/version file that sets the global Python version is unchanged.

pyenv is somehow performing an analysis like “yes, I can see you have a global Python version set, but the python version command is being run from the project1 directory, and that has a .python-version file that takes precedence over the global Python version”.

Let’s create a ~/Documents/project2/ folder and set the local Python version to “system”.

Let’s verify that project2 is using the system Python version.

Let’s run some commands to recap and demonstrate that the home directory is using a Python version of 3.7.5, project1 is using 3.6.10, and project2 is using the system Python.

Let’s keep digging and see how pyenv is switching these Python versions magically.

PATH basics

PATH is an environment variable that specifies an ordered list of folders where executables are saved.

Run echo $PATH on your machine to see the path on your machine.

The PATH is an ordered list of directories delimited by a colon. It’s easier to visualize a bulleted list:

• /usr/local/bin
• /usr/bin
• /bin
• /usr/sbin
• /sbin

When you enter a command in Terminal, your computer will look for an executable in /usr/local/bin first, then /usr/bin, then /bin, then /bin/usr/sbin and finally /sbin.

When you run echo hi, your Terminal will start by looking for an executable named echo in the /usr/local/bin directory. There isn’t a echo executable in /usr/local/bin/ (run open /usr/local/bin/ on your machine and visually inspect to verify). echo isn’t in /usr/bin either. An echo executable is stored in /bin. The Terminal will use whatever executable it finds first.

You can also find where executables are located with the whereis command:

pyenv changes PATH

pyenv adds this code to the ~/.bash_profile which changes the PATH on your machine:

Run echo $PATH to see the PATH is different now: /Users/matthewpowers/.pyenv/shims:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin.

The /Users/matthewpowers/.pyenv/shims directory has been added before all the other directories in the PATH.

All Terminal commands will go through /Users/matthewpowers/.pyenv/shims first now. This allows pyenv to “intercept” any relevant Python commands.

• The $ python command will go to /Users/matthewpowers/.pyenv/shims/python.
• The $ pip command will go to /Users/matthewpowers/.pyenv/shims/pip.

/Users/matthewpowers/.pyenv/shims doesn’t contain an executable named echo, so the echo hi there command will still use the echo executable that’s in /bin.

Rehashing design pattern

Once you have pyenv installed and run a simple command like pip install pandas, what exactly happens? How does pyenv execute this code?

The pyenv README has a great high level description on shims and the rehashing pattern. From the README:

Shims are lightweight executables that simply pass your command along to pyenv. So with pyenv installed, when you run, say, pip, your operating system will do the following:

• Search your PATH for an executable file named pip
• Find the pyenv shim named pip at the beginning of your PATH
• Run the shim named pip, which in turn passes the command along to pyenv

Let’s describe these steps in more detail:

• When you run pip install pandas, your Terminal will search your PATH for a pip executable. It’ll find that executable in the /Users/matthewpowers/.pyenv/shims directory that is at the front of your path. pyenv is intercepting this command and handling it before it can be picked up by another executable lower in the PATH hierarchy. The python command will use the executable in /usr/bin, the dreaded system Python version, it it’s not intercepted.
• /Users/matthewpowers/.pyenv/shims/pip is an executable file that’s referred to as a “shim”. A shim intercepts a call and redirects the operation elsewhere.
• The shim redirects the command to pyenv. Let’s take a look at the code with cat /Users/matthewpowers/.pyenv/shims/pip:

We can see that exec '/usr/local/Cellar/pyenv/1.2.18/libexec/pyenv' is “passing the command along to pyenv”.

Intercepting the python command

The python command will be intercepted by /Users/matthewpowers/.pyenv/shims/python, as you can see by running which python: Oraciones cientificas pdf.

The code in /Users/matthewpowers/.pyenv/shims/python is exactly the same as the code in /Users/matthewpowers/.pyenv/shims/pip.

This is what cat /Users/matthewpowers/.pyenv/shims/python returns:

The shim passes the command to pyenv without doing any processing.

pyenv executable

Let’s look at the pyenv executable that’s being passed commands from the shims.

Homebrew Python 3.6 Software

Run cat /usr/local/Cellar/pyenv/1.2.18/libexec/pyenv to inspect the contents of the pyenv executable. Prepare yourself for 150 lines of Bash code.

Here is where this script is defined in the pyenv repo.

pip install for different Python versions

Suppose you’re using Python 3.7.5 and run pip install pandas.

The Pandas code will be stored in .pyenv/versions/3.7.5/lib/python3.7/site-packages.

If you switch to Python 3.6.10 with pyenv shell 3.6.10, you’ll need to reinstall pandas with another pip install pandas. The Python 3.6.10 pandas will be stored in /Users/matthewpowers/.pyenv/versions/3.6.10/lib/python3.6/site-packages.

pyenv doesn’t share library versions across different Python versions.

Homebrew Python 3.6 Crack

See pyenv-virtualenv for more details about managing virtual environments with pyenv.

pyenv is not bootstrapped by Python

You might wonder why the pyenv codebase is almost entirely Shell and Roff code.

pyenv intentionally avoids Python. If pyenv was written in Python, then the system would need Python installed to run pyenv commands. pyenv can install Python on machines that don’t have any version of Python installed (most machines come with an old Python version pre-installed).

You don’t want a program that installs Python to depend on Python.

The system version of Python that comes pre-installed in machines causes a huge headache for programmers that don’t understand the PATH, executable, etc. The shell scripting patterns you’ve learned in this post will save you from a lot of Python development hell.

Conclusion

We learned a lot about pyenv and have a good understanding of how it works.

Homebrew Python 3.6 Online

The shims are injected at the beginning of your PATH so pyenv can route commands to the right Python version. pyenv is using versions specified in text files to see what version of Python should be used to run the commands.

pyenv offers an elegant user interface for managing different Python versions for end users. The codebase is amazingly clean.

Programs like these make you sit back with awe – you’re starstruck that other programmers can architect such beauty.

Study their design patterns and try to copy them! Copy what you like.