When working with conda packages, you might encounter packages with names like pytorch-mutex, _libgcc-mutex or the blas package that seem to serve a special coordination role. These are "mutex" packages—a solution for managing mutually exclusive variants in the conda ecosystem.
Mutex packages are useful to guide the package solver to select a "subtree" from all dependencies.
What Are Mutex Packages?
A mutex package is a single package name with different build strings that enforces mutually exclusive choices between software variants. The key insight is that we allow only
one package with the same name
to be installed in an environment at any time.
For example:
pytorch-mutex, version=1.0, build=cuda
pytorch-mutex, version=1.0, build=cpu
Since both packages share the name pytorch-mutex, the conda package solver ensures that only one can be installed, effectively creating a mutex (mutual exclusion) mechanism.
How Mutex Packages Work
Multiple related packages can depend on the same mutex package, with specific build string requirements:
When you install pytorch-gpu, conda automatically selects the pytorch-mutex=1.0=cuda in the environment. This prevents any pytorch-family CPU-variant packages from being installed, since they would require the conflicting pytorch-mutex=1.0=cpu.
BLAS Library Selection
One of the most common uses is coordinating BLAS implementations. There are multiple BLAS implementations that have different names but expose the same shared library APIs. In conda-forge, we have mkl, blis, and openblas.
To coordinate these three, we need to have an empty mutex metapackage that serves as a selector between the implementations. This package is simply called blas and comes in three variants (one for each BLAS implementation).
Packages like NumPy, SciPy, and other scientific libraries can depend on the blas metapackage. The mutex package ensures that only one of the three actual BLAS implementations is going to be installed (OpenBLAS, BLIS or MKL).
Users can easily toggle the BLAS variant by also explicitly installing blas=*=mkl or their desired variant in their environment.
GPU vs CPU Variants
Machine learning frameworks often use mutex packages to coordinate GPU/CPU variants:
This ensures that all PyTorch-related packages (torchvision, torchaudio, etc.) align on the same compute backend.
Implementation Pattern
Creating a mutex package involves:
Define the mutex package with different build strings for each variant
Make related packages depend on the appropriate mutex variant
Let conda's single-name constraint enforce mutual exclusion
Since mutex packages are empty, it's straightforward to create one. The package will only contain some metadata.
First, create a recipe.yaml
And then place a variants.yaml file next to it:
This would create two packages, one my-mutex-1.0.0-gpu and my-mutex-1.0.0-cpu that you can use as part of your dependency tree.
Conclusion
Mutex packages represent an elegant solution to variant management in Conda. By leveraging conda's fundamental constraint that only one package with a given name can be installed, they provide a neat trick to coordinate variants across complex software stacks.
More documentation about Mutex packages can be found here:
If you want to chat more about tricks like this, join us on Discord!
# pytorch-gpu package depends on CUDA mutexrequirements: run: - pytorch-mutex * cuda# pytorch-cpu package depends on CPU mutex requirements: run: - pytorch-mutex * cpu
:focal(smart))
