Publishing your first Python package to PyPI

Posted on March 14, 2017 in Development • 3 min read

In this post, I describe how to publish a Python package to the PyPI index so that it becomes available from the pip package manager. The procedure has been tested on a few personal projects.

Introductions: PyPI and pip

  • The Python Package Index, abbreviated as PyPI, is the official third-party software repository for Python. It is analogous to CPAN, the repository for Perl.

  • pip, a recursive acronym that can stand for either "Pip Installs Packages" or "Pip Installs Python", is a package management system used to install and manage software packages written in Python. pip supports installing from PyPI, version control, local projects, and directly from distribution files.

Sign up on PyPI servers

Create two accounts, you can use the same credentials (% and space characters might create problems):

  • To publish packages on PyPI, you must create an account: Manual user registration.

  • test PyPI is an additional Python Package Index where packages are maintained temporarily and is used solely for testing. To publish packages on test PyPI, you must create an account: Manual user registration.

Create the ~/.pypirc configuration file:

[distutils]
index-servers =
  pypi
  pypitest

[pypi]
repository=https://pypi.python.org/pypi
username=your_username
password=your_password

[pypitest]
repository=https://testpypi.python.org/pypi
username=your_username
password=your_password

Secure the configuration file with chmod 600 ~/.pypirc.

Prepare your package

The official documentation on Packaging and Distributing Projects describes in detail the required files, and provides a PyPA sample project.

These are the files that must be present in your project's top directory:

  • setup.py: Information on dependencies, entry points, author, description, version, keywords, requirements, files to be included in the package
  • setup.cfg: Option defaults for setup.py commands
  • README.rst: README file in reStructuredText format, that will be published on the PyPI's project page
  • MANIFEST.in: The best method to include additional files in your package
  • <your package>: A top-level package containing the project's Python modules and additional files

The official documentation lacks important details on how to specify which files to include in the package in a portable way.

Register and publish your package

Packages can be distributed as Source Distributions, Wheels, Universal Wheels and Platform Wheels [doc]. The following steps describe how to publish a Source Distribution.

Let's register your package on the PyPI test server:

python setup.py register -r pypitest

You register the package only once.

Build and upload the package to the PyPI test server:

python setup.py sdist upload -r pypitest

Install (or upgrade) the package xyz from the PyPI test server:

pip install --extra-index-url https://testpypi.python.org/pypi xyz --upgrade

You can see the PyPI's website of your package xyz at https://testpypi.python.org/pypi/xyz.

If everything works smoothly, you can switch to the PyPI production server:

python setup.py register -r pypi
python setup.py sdist upload -r pypi
pip install xyz --upgrade

Congratulations! You've published your first Python package on PyPI.

Python packages published on PyPI are not verified, checked or audited, therefore install only packages from authors you do trust. Read more here.

Publish your package as a wheel

Pure Python packages can be published as Pure Python Wheels. A wheel is a built package that can be installed without needing to go through the “build” process. Installing wheels is substantially faster for the end user than installing from a source distribution, even with pure Python packages.

Install the wheel package:

pip install wheel

To build and upload the source and wheel distributions:

python setup.py sdist bdist_wheel --universal upload

If you get the error "error: invalid command 'bdist_wheel'" , try to specify the Python version and update setuptools: pip install setuptools --upgrade.

Add additional files to your package

There are thee methods: MANIFEST.in, data_files and package_data. MANIFEST.in seems the simplest, most portable and reliable method. The other two did not work for me, and their use is discouraged by the community even if the official documentation lists them as the preferred choice.

To add recursively the contents of a directory named data, follow these steps:

  1. The data directory must be placed inside your top-level package <project directory>/<your package>/. E.g., xyz/xyz/. It will NOT work if you keep it on the project's top directory.

  2. Set include_package_data to True in setup.py:

setup( ...,
  include_package_data = True,
  ...)
  1. In MANIFEST.in, add this line to add the contents of directory data inside package xyz:
recursive-include xyz/data *

Documentation on MANIFEST.in format and features: The MANIFEST.in template.

Install the package from directory

To install the package from its local development directory:

python setup.py install

Useful resources

That's all folks!