How to publish your package to PyPI, 2018

1. Overview
2. Audience
3. Relevance
4. Motivation
5. Demonstration
6. Features
7. Limitations
8. Installation
9. Configuration
   1. Files
   2. __init__.py
   3. License
   4. README
      1. README.md
      2. README.rst
   5. setup.cfg
      1. long_description
      2. classifiers
      3. zip_safe
      4. packages
      5. console_scripts
      6. bdist_wheel
   6. Dependencies
      1. Difference between requirements.txt and install_requires
         1. install_requires
         2. requirements.txt
      2. setup.py
      3. requirements.txt
      4. MANIFEST.in
   7. pyroma
10. TestPyPI
    1. What is TestPyPI
    2. Registration
    3. Twine settings
    4. Build package
    5. Publish to TestPyPI
    6. Visit TestPyPI
    7. virtualenv
       1. Why virtual environment?
       2. Using virtualenv and virtualenvwrapper
       3. Checking
11. PyPI
12. Updating
13. Automatic updating, release and changelog
    1. Demonstration
    2. Features
    3. Limitations
    4. Installation and setting-up
       1. .release-it.json
    5. Usage
14. Testing environment

1. Overview ¶

Tutorial, how you can add your Python package to PyPI — Python Package Index.

If you want, that users install your Python package via command:

pip install <your_package>

Read this article.

2. Audience ¶

Python developers, that:

1. publish package to PyPI first time,
2. already published packages to PyPI, but want to do it more qualitatively.

3. Relevance ¶

This article is relevant for February 2018. In the future, the data in this article may be obsolete.

4. Motivation ¶

One more article? Why?

I read some articles about PyPI publishing → I think, that articles, which I read, have disadvantages:

1. outdated;
2. not show real examples and demonstrations;
3. not all-in-one; I needed force read other resources;
4. They do not describe all the difficulties the developer has encountered, when try to publish package on PyPI.

I don’t want, that another beginner Python developers take a lot of time for PyPI publishing as me. And I try to write an article, that:

1. non-outdated at February 2018;
2. show examples and demos;
3. all-in-one; I try to add in my article all information, that I need for first PyPI publishing;
4. described all problems, to solve that personally I spent my time.

5. Demonstration ¶

For examples in this article, I select my Erichek package.

pypi.python.org/pypi/erichek page at February 2018:

pypi.org/project/erichek Warehouse page at February 2018:

You can install Erichek, use command:

pip install erichek

If you want to have similar behavior, read on.

If I have explained something incomprehensibly, see files of Erichek GitHub repository.

6. Features ¶

1. This article — cross platform solution.

7. Limitations ¶

1. You must be able to write working Python packages.
2. Erichek and another my packages solely for Python 3. I don’t want support Python 2. If you want support Python 2, possibly, you will need some other action.
3. If you can have a configuration different, that Erichek, possibly, you will need in another actions.

8. Installation ¶

Please, install via pip (pip install <package>):

1. twine,
2. wheel,
3. virtualenv,
4. virtualenvwrapper for Linux/macOS, virtualenvwrapper-win for Windows,
5. pyroma,
6. update your setuptools, to be sure, that pyroma works correct — pip install -U setuptools.

9. Configuration ¶

9.1. Files ¶

Simplify configuration. Real project configuration see in Erichek GitHub repository.

│   .release-it.json
│   CHANGELOG.md
│   LICENSE.md
│   MANIFEST.in
│   README.rst
│   package.json
│   requirements.txt
│   setup.cfg
│   setup.py
│
├───erichek
│       __init__.py
│       __main__.py

In Software Engineering site recommends use uppercase for README and some another text files.

9.2. __init__.py ¶

See answer, why __init__.py need. This file may be blank or contains content.

9.3. License ¶

Add text of license for your package to LICENSE.md file.

If your editor is Sublime Text 3, you can use License Snippets package.

9.4. README ¶

9.4.1. README.md ¶

If you preferred write README in Markdown, you can have problems.

At February 2018 PyPI doesn’t support Readme.md without dependencies. Possibly, it seems like there is no easy way to use a markdown README for PyPI. Solutions involve requiring pandoc locally, which is a heavy dependency.

9.4.2. README.rst ¶

I recommend write README in reStructuredText — you need use file README.rst. I add to my README.rst next information:

1. Short package description,
2. Badges,
3. Link to long package description on my site.

I prefer to add long description to personal site, not to README, because I have much more options in the design of the description.

9.5. setup.cfg ¶

See example setup.cfg. Copy content of this file to your setup.cfg and change example values of parameters to your real values.

In the subsections below I describe the places that caused me difficulties.

9.5.1. long_description ¶

my value:

long_description = file: Readme.rst

Set your README file as value. Letters must be in same register as in file. For example, if your file is Readme.rst, you need to set file: README.rst, not file: Readme.rst or file: README.RST.

9.5.2. classifiers ¶

You can use solely classifiers from this list. If no, you can’t publish your package to PyPI.

Possibly, UNIX users can select classifiers via pypi-classifiers GUI, but I can’t set this program for Windows 10.

9.5.3. zip_safe ¶

my value:

zip_safe = False

If you build your package use Wheels, you don’t need this parameter, but pyroma will show lower value. And so I recommend add zip_safe = False or zip_safe = True (unimportant) to your setup.cfg file.

9.5.4. packages ¶

my value:

packages = find:

If no packages = find:, users doesn’t download folder with your Python module.

9.5.5. console_scripts ¶

What is it. If Erichek user want run Erichek, he/she wrote in console:

python "path/to/__main__.py"

But since console_scripts exist in setup.cfg, Erichek user merely can use command, that get same behavior:

erichek

my value:

console_scripts =
	erichek = erichek.__main__:main

Parameter and values:

• erichek — name of command, that users of your package will run.
• erichek.__main__ — relative path to your main module. For me it __main__.py file in erichek folder.
• :main — function, that run, when you run your module. For my __main__.py it main() function.

9.5.6. bdist_wheel ¶

my value:

[bdist_wheel]
python-tag = py3

See what is Wheels and why .whl preferred than .egg.

Erichek — Python 3 package, so I have python-tag = py3. If your package support Python 2 and 3 both, you need use next code:

[bdist_wheel]
universal = 1

9.6. Dependencies ¶

In this section I tell, what you need, that dependencies of your package from file requirements.txt automatically install for user. You don’t need to write same dependencies in requirements.txt and setup.cfg/setup.py.

9.6.1. Difference between requirements.txt and install_requires ¶

Simplified definition.

9.6.1.1. install_requires ¶

install_requires parameter in setup.cfg or setup.py show, which packages will install, if user install your package via pip. For example, user install clize package:

pip install clize

Part of clize setup.py:

install_requires=[
	'six',
	'sigtools >= 2.0',
	'attrs >= 17.4.0',
	'od',
	'docutils',
],

In clize installation process six, sigtools, docutils, od and attrs packages automatically install for user, if this packages no already installed.

D:\>pip install clize
Collecting clize
	Using cached clize-4.0.3-py2.py3-none-any.whl
Requirement already satisfied: six in c:\python36\lib\site-packages (from clize)
Requirement already satisfied: sigtools>=2.0 in c:\python36\lib\site-packages (from clize)
Requirement already satisfied: docutils in c:\python36\lib\site-packages (from clize)
Collecting od (from clize)
	Using cached od-1.0-py3-none-any.whl
Requirement already satisfied: attrs>=17.4.0 in c:\python36\lib\site-packages (from clize)
Installing collected packages: od, clize
Successfully installed clize-4.0.3 od-1.0

9.6.1.2. requirements.txt ¶

For example, I am Python package developer and have local dependencies for my package.

1. If I want to update local dependencies, I update versions in requirements.txt → I run in console:

   pip install -r requirements.txt -t . --upgrade
   

   it update my local dependencies.

2. Services as Dependabot or Pyup check dependencies from requirements.txt and update numbers of outdated versions.

9.6.2. setup.py ¶

In this and 2 next sections I wrote, how you can get install_requires behavior, use solely requirements.txt file. You don’t need add your dependencies to install_requires.

Add this code to your setup.py:

from setuptools import setup
from pip.req import parse_requirements
# parse_requirements() returns generator of pip.req.InstallRequirement objects
install_reqs = parse_requirements('requirements.txt', session='hack')
# reqs is a list of requirements
# e.g. ['django==1.5.1', 'mezzanine==1.4.6']
reqs = [str(ir.req) for ir in install_reqs]
setup(
	install_requires = reqs
)

from setuptools import setup

9.6.3. requirements.txt ¶

Add list of your dependencies to requirements.txt.

9.6.4. MANIFEST.in ¶

Add a line to MANIFEST.in file:

include requirements.txt

If no, users can’t install your package.

9.7. pyroma ¶

pyroma — PyPI configuration validator.

If you set all your configuration files, run command in root directory of your Python module:

pyroma .

You need get 10/10 final rating, example:

$ pyroma .
------------------------------
Checking .
Registered VCS backend: git
Registered VCS backend: hg
Registered VCS backend: svn
Registered VCS backend: bzr
Found erichek
------------------------------
Final rating: 10/10
Your cheese is so fresh most people think it's a cream: Mascarpone
------------------------------

If no 10/10, please, see pyroma messages and try fix your package.

10. TestPyPI ¶

10.1. What is TestPyPI ¶

If you publish your package first time or not sure that everything is doing right, I recommend at first publish package to TestPyPI. TestPyPI — is a service, where you can test uploading, downloading and display your package on PyPI site before you share package to PyPI.

10.2. Registration ¶

Please, register on TestPyPI and PyPI sites.

It would be nice, if you will use same username and password for both sites, that Twine works correct.

10.3. Twine settings ¶

Add environment variables TWINE_USERNAME and TWINE_PASSWORD with values — your PyPI and TestPyPI username and password.

See, how you can add environment variables for:

• UNIX,
• Windows.

10.4. Build package ¶

Print in your terminal:

python setup.py bdist_wheel

10.5. Publish to TestPyPI ¶

Enter in your terminal:

twine upload --repository-url https://test.pypi.org/legacy/ dist/*

If no errors in your configuration, package must successful upload to TestPyPI.

10.6. Visit TestPyPI ¶

After uploading visit 2 pages:

• https://testpypi.python.org/pypi/<your_package>,
• https://test.pypi.org/project/<your_package>.

Examples for Erichek at February 2018:

• test.pypi.org/project/erichek:

• testpypi.python.org/pypi/erichek:

If the result suits you, in next step install package, that you upload, use virtual environment.

10.7. virtualenv ¶

I recommend install your TestPyPI package, use virtual environment.

10.7.1. Why virtual environment? ¶

Citation from Pro Python Best Practices book:

Virtualenv is like building a moat around the house. It prevents a fire from spreading — in both directions. Likewise, a virtual environment prevents that Python projects interfere with each other.

Reasons of use virtual environment for TestPyPI installation:

1. You can have bugs in your package. Bugs may have a negative impact of your environment. If you will use virtual environment, these bugs does not affect your global Python environment.
2. You can have some globally dependencies in your environment, but users of your package may haven’t them. If you use virtual environment, you see, how your package will install and works without, possibly, pre-installed global dependencies on your machine.

10.7.2. Using virtualenv and virtualenvwrapper ¶

Open terminal in any directory. Run these commands:

C:\Users\SashaChernykh>mkvirtualenv erichekenv
Using base prefix 'c:\\python36'
New python executable in C:\Users\SashaChernykh\Envs\erichekenv\Scripts\python.exe
Installing setuptools, pip, wheel...done.
(erichekenv) C:\Users\SashaChernykh>toggleglobalsitepackages
	Disabled global site-packages
(erichekenv) C:\Users\SashaChernykh>pip install --extra-index-url https://test.pypi.org/simple/ erichek

Where:

• erichekenv — name of your virtual environment; you can use any name instead of erichek, if virtual environment with this name not already exist;
• erichek — name of your package; use real name of your package instead of erichek.

Could not find a version that satisfies the requirement mypackage==4.14.7 (from versions: 4.14.6)
No matching distribution found for mypackage==4.14.7

10.7.3. Checking ¶

If you can’t have bugs in installation process, check, that you package works correct. For example, Erichek check, contains errors in .txt files in a folder or no. I run erichek console command.

If error in one of text file:

C:\ErichekExamples>erichek
[2018-02-08 08:49:43.058353] NOTICE: eric_encoding logbook: All files in Windows-1251 encoding
[2018-02-08 08:49:43.060355] NOTICE: eric_body logbook: All files contains <body>
[2018-02-08 08:49:43.062358] NOTICE: eric_asterisks logbook: All needest lines contains asterisks
[2018-02-08 08:49:43.063357] ERROR: eric_head logbook: Air_crashes.txt not contains «Постоянный адрес пакета:»
[2018-02-08 08:49:43.064359] ERROR: eric_head logbook: One or more packages not contains one or more head data. Please, add correct head data to your package.
[2018-02-08 08:49:43.064359] ERROR: summary logbook: You have errors in your packages. Please, fix them.

If all .txt files correct:

C:\ErichekExamples>erichek
[2018-02-08 08:51:23.840987] NOTICE: eric_encoding logbook: All files in Windows-1251 encoding
[2018-02-08 08:51:23.841997] NOTICE: eric_body logbook: All files contains <body>
[2018-02-08 08:51:23.843991] NOTICE: eric_asterisks logbook: All needest lines contains asterisks
[2018-02-08 08:51:23.844992] NOTICE: eric_head logbook: All files contains correct head data
[2018-02-08 08:51:23.844992] NOTICE: summary logbook: Congratulations! You haven’t errors in your packages!

Erichek works as expected.

After succsessful checking you can remove your virtual environment (erichekenv in example):

(erichekenv) C:\Users\SashaChernykh>rmvirtualenv erichekenv
	Deleted C:\Users\SashaChernykh\Envs\erichekenv
C:\Users\SashaChernykh>

11. PyPI ¶

If all actions from TestPyPI section success for you, you can publish your package to PyPI and then install it.

1. change version in your setup.cfg file (it must be semver-compatible);
2. in root folder of your package open terminal and print:

   python setup.py bdist_wheel && twine upload dist/*
   

3. visit https://pypi.python.org/pypi/<your_package> and https://pypi.org/project/<your_package>, as you visit TestPyPI and check, is everything okay;

4. install your package, use pip:

   pip install <your_package>
   

5. check, that your package works correct as in previous section.

If all okay, congratulations! You successful publish your package to PyPI!

12. Updating ¶

That update your package in PyPI:

1. Make new changes,
2. Make same actions as in section above.

13. Automatic updating, release and changelog ¶

13.1. Demonstration ¶

I enter command to the terminal:

release-it -n -V

I get behavior:

1. All commits description add to CHANGELOG.md:
2. New release publish to GitHub:
3. Version updates in setup.cfg:
4. New version of Erichek publish to PyPI.

If you want to have same behavior, read on.

13.2. Features ¶

See these features.

13.3. Limitations ¶

See these limitations.

13.4. Installation and setting-up ¶

1. you need to install all from this section except tee, cat, mv and js-beautify;
2. please, follow these, these and these instructions;
3. create a file package.json in root folder of your repository.

13.4.1. .release-it.json ¶

Create a file .release-it.json in root folder of your repository with content:

{
	"buildCommand": "changelog -u https:\/\/${repo.host}\/${repo.repository} -f CHANGELOG.md && sed -i 's\/^version = .*$\/version = ${version}\/g' setup.cfg && python setup.py bdist_wheel && twine upload dist\/*",
	"changelogCommand": "changelog -f -",
	"github": {
		"release": true,
		"tokenRef": "TOKEN_GITHUB_FOR_RELEASE_IT"
	},
	"npm": {
		"publish": false
	},
	"safeBump": false
}

Where:

• sed -i 's\/^version = .*$\/version = ${version}\/g' setup.cfg — command for replacing version in your setup.cfg to the newest, see find and replace via sed:
• python setup.py bdist_wheel && twine upload dist\/* — build and publish your package.

&& sed -i 's\/^VERSION = \".*\"$\/VERSION = \"${version}\"\/g' \"erichek\/eric_config.py\"

See here, if you want to know details about another parts of .release-it.json.

13.5. Usage ¶

See usage instructions.

14. Testing environment ¶

• Windows 10 Enterprise LTSB 64-bit EN,
• Python 3.6.4,
• setuptools 38.4.1,
• pip 9.0.1,
• Twine 1.9.1,
• Wheel 0.30.0,
• virtualenv 15.1.0,
• virtualenvwrapper-win 1.2.5,
• pyroma 2.3,
• Node.js 9.4.0,
• git 2.16.0.windows.2,
• release-it 7.0.0,
• changelog 1.7.0,
• sed (GNU sed) 4.2.2.