How to Test a Python Distribution

Introduction

Your code is good. Really good.

You enforced consistent style with black, isort, and pydocstyle.

You checked for a wide range of performance, correctness, and readability issues with flake8, mypy, pylint, and ruff.

You ran extensive tests with pytest and hypothesis.

You even scanned for legal and security issues with tools like pip-audit, pip-licenses, and safety.

So finally, FINALLY, it’s time to package it up into a tarball and upload it to PyPI.

python -m build .

... right?

Hopefully! But let’s check.

  • Are those distributions valid zip or tar files?

  • Will their READMEs look pretty when rendered on the PyPI homepage?

  • Do they have correctly-formatted platform tags?

  • Are they as small as possible, to be kind to package repositories and users with weak internet connections?

  • Are they free from filepaths and file names that some operating systems will struggle with?

You checked those things, right? And in continuous integration, with open-source tools, not with manual steps and random tar incantations copied from Stack Overflow?

If yes, great! I’m proud of you.

If you’re feeling like you could use some help with that… read on.

Quickstart

After building at least one wheel and sdist…

python -m build -o dist .

Run the following on those distributions to catch a wide range of packaging issues.

# are all the sdists valid gzipped tar files?
gunzip -tv dist/*.tar.gz

# are all the wheels valid zip files?
zip -T dist/*.whl

# is the package metadata well-formatted?
pyroma --min=10 dist/*.tar.gz
twine check --strict dist/*

# is the distribution properly structured and portable?
check-wheel-contents dist/*.whl
pydistcheck --inspect dist/*

List of Tools

The following open-source tools can be used to detect (and in some cases repair) a wide range of Python packaging issues.

  • abi3audit (link) = detect ABI incompatibilities in wheels with CPython extensions

  • auditwheel (link) = detect and repair issues in Linux wheels that link to external shared libraries

  • auditwheel-emscripten (link) = like auditwheel, but focused on Python-in-a-web-browser applications (e.g. pyodide auditwheel)

  • auditwheel-symbols (link) = detect which symbols in a Linux wheel’s shared library are causing auditwheel to sugggest a more recent manylinux tag

  • check-manifest (link) = check that sdists contain all the files you expect them to, based on what you’ve checked into version control

  • check-wheel-contents (link) = detect unnecessary files, import issues, portability problems in wheels

  • delocate (link) = detect and repair issues in macOS wheels that link to external shared libraries

  • delvewheel (link) = detect and repair issues in Windows wheels that link to external shared libraries

  • pydistcheck (link) = detect portability problems in wheels and sdists

  • pyroma (link) = detect incomplete or malformed metadata in sdists

  • twine (link) = detect issues in package metadata (via twine check)

  • wheel-inspect (link) = dump summary information about wheels into machine-readable formats