Specifications

Naming Specification

Versioning Specification

How to version your Python projects

The version of a package is defined in the setup.py file. A setup.py looks something like this:

from distutils.core import setup

setup(name='MyProject', version='1.0', author='Tarek',
      author_email='tarek@ziade.org',
      url='http://example.com',)

And this gives you the power to register and upload your project to The Python Package Index (PyPI), as simply as:

$ python setup.py register sdist upload

Your project is then added in PyPI, among over 9000 other projects and people can start using it, by downloading the archive you have created with the sdist command.

With pip they can even install it with a simple command, ala apt-get:

$ pip install MyProject

Why you need version numbers?

As soon as you start publishing your project to the world, you need to version it. For example, the first version of your software can be 1.0.

Everytime you are releasing a new version with new features and bugfixes, raising the version number will let your end users know that it is a newer version. This is usally done by incrementing the version, so your next version could be 1.1.

Once this new version is made available to the world on PyPI, people will be able to install it, for example using pip:

$ pip install --upgrade MyProject

The upgrade option here means that Pip will look for all published versions of MyProject on PyPI and see which one is the latest, then upgrade your system to that version if you are not up-to-date.

Now imagine that you have introduced a small bug in 1.1 that makes your project unusable on Windows. You are working on it, but you know it will take you some time to resolve it.

The best strategy here is to tell you Windows user to stick with 1.0 until you have fixed the issue. They can downgrade to 1.0 because it is still available on PyPI:

$ pip install MyProject==1.0

This is possible because pip is able to sort the various versions of your project, as long as it follows a standard version scheme.

Standard versioning schemes

The two most commone schemes used to version a software are date-based schemes and sequence-based schemes.

Date scheme

Some projects use dates for the version numbers. That was the case for Wine before it started using a sequence-based scheme. A date scheme is usually using a YYYY-MM-DD form so versions can be sorted alphanumerically:

  • 2009-08-10
  • 2005-02-03
  • etc.

This versioning scheme has a few limitations:

  • you need to add hours to the scheme if you do more than one release per day.
  • if you have maintenance branches over older releases, the overall order will not work anymore: a maintenance release of an older version will appear to be newer than a recent release.

Although, date schemes are commonly used as extra markers in versions schemes.

Sequence-based scheme

The most common scheme is the sequence-based scheme, where each version is a sequence of numerical values usually separated by dots (.):

  • 1.0
  • 1.1
  • 1.0.1
  • 1.3.4

This scheme removes the limitations we have seen with date-based schemes since you can release maintenance versions and keep the proper order: versions are ordered by comparing alphanumerically each segment of the version.

The most frequent sequence-based scheme is:

MAJOR.MINOR[.MICRO]

where MAJOR designates a major revision number for the software, like 2 or 3 for Python. Usually, raising a major revision number means that you are adding a lot of features, breaking backward-compatibility or drastically changing the APIs or ABIs.

MINOR usually groups moderate changes to the software like bug fixes or minor improvements. Most of the time, end users can upgrade with no risks their software to a new minor release. In case an API changes, the end users will be notified with deprecation warnings. In other words, API and ABI stability is usually a promise between two minor releases.

Some softwares use a third level: MICRO. This level is used when the release cycle of minor release is quite long. In that case, micro releases are dedicated to bug fixes.

Choosing between a major-based scheme and a micro-based one is really a matter of taste. The most important thing is to document how your version scheme works and what your end users should expect when you release a new version. So be sure to define your release cycle properly before you start releasing to the wild !

Development releases

When working for the next version of your application, you might need to release a development version in order to share it with other developers. Many projects provide nightly builds, which are daily snapshots of the code repository people can install to try out a cutting edge version.

Like regular versions, development versions have to be numbered so they can be sorted and compared with any other version. The simplest way to perform this numbering is to use the current repository version number when creating the release. The next version is suffixed by a dev number. Of course this implies that your code lives in a repository, like Subversion or Mercurial.

Examples:

  • 1.2.dev1234 : a development version of the future 1.2 release. 1234 is the current repository version.
  • 1.3a2.dev12 : a development version of the future 1.3 alpha 2 release. 12 is the current repository version.

Package managers that will install those versions will need to sort them correctly: 1.2.dev1234 < 1.2 < 1.3a2.dev12 < 1.3a2 < 1.3

Here’s an example of a setup.py file in a project that lives in a Mercurial repository, that will automatically generate the right development version when creating a source distribution.:

from distutils.command.sdist import sdist
import os

class sdist_hg(sdist):

    user_options = sdist.user_options + [
            ('dev', None, "Add a dev marker")
            ]

    def initialize_options(self):
        sdist.initialize_options(self)
        self.dev = 0

    def run(self):
        if self.dev:
            suffix = '.dev%d' % self.get_tip_revision()
            self.distribution.metadata.version += suffix
        sdist.run(self)

    def get_tip_revision(self, path=os.getcwd()):
        from mercurial.hg import repository
        from mercurial.ui import ui
        from mercurial import node
        repo = repository(ui(), path)
        tip = repo.changelog.tip()
        return repo.changelog.rev(tip)

setup(name='MyProject',
      version='1.0',
      packages=['package'],
      cmdclass={'sdist': sdist_hg})

This development marker will be added when the dev option is used.

Pre-releases

Major versions of a software are often preceded by pre-releases. These previews are comparable to development versions and are released for testing purposes. Publishing them will let your end-users try them out and send you valuable feedback before the final version is published. It also helps avoiding releases of brown-bag versions. A brown-bag version is a version that is badly broken and that cannot be used by some of all of the end users.

Usually, pre-releases are organized in three phases:

  • alpha releases: early pre-releases. A lot of changes can occur between alphas and the final release, like feature additions or refactorings. But they are minor changes and the software should stay pretty unchanged by the time the first beta is reached.
  • beta releases: at this stage, no new features are added and developers are tracking remaning bugs.
  • release candidate releases: a release candidate is an ultimate release before the final release. Unless something bad happens, nothing is changed.

Of course, the number of pre-releases varies a lot depending on the size of your project and the numbers of end-users. Python itself has several alpha, beta and release candidate releases when a minor version is being released, because it impacts a lot of people and organizations.

Small projects often don’t have any pre-releases because they can publish a new final version anytime. Some small projects do have pre-releases even if they could skip it because it is benefic for increasing feedback, and has a positive psychological effect on end-users. For instance, having a series of pre-releases for your “1.0” version will increase the chances to publish a rock-solid version. And the “1.0” milestone can mean a lot to your end users: round numbers like this make them feel that your software has reached an important step in its maturity.

Post-releases

Some projects use post-releases when they need to publish a version that simply doesn’t fit in the next series or can’t be a new release in the current series. Let’s take an example: “1.9” is the last release of your “1.x” series, and you have started a backward incompatible “2.x” series. You want all your users to drop any “1.x” release as soon as possible. You have released debug versions in the “1.x” series for quite a time and stated that “1.9” was the last one. Although, a user finds a really bad bug in “1.9” and asks you to fix it because he can’t switch to the new series. In that case it is useful to publish a post release.

This use case is quite rare, and it’s likely that you will never have to do post releases.

Defining your release cycle

A sofware usually follows this cycle:

  • internal development versions might be released, for testing purposes. These releases can be private to a circle of developers, but they still need to be versioned. It’s a good practice to do them because, unlike grabbing the current development version out of the code repository, it will help testing your release process: if there’s a bug in your setup.py file, or in any part of your release/deploy/install process, they can be caught in early stage that way.
  • one or several pre-release are made for getting end-users feedback and avoiding releasing a brown bag release, as we saw in the previous section. Having just release candidates, or also apha and betas is up to you. There’s no rule about it, but it is considered overkill to have alphas and betas for small projects (small here would be less that 5k SLOC. or less than 50 end-users)
  • a final version is released to the world, usually starting at 1.0 or 0.1. Micro versions are considered overkill, unless the release cycle of your minor version lasts for months, and you want to keep the same major number for years.
  • bugfixes and new features are added and new versions are released
  • when a new major version is started, like “2.0”, bug fixes releases may continue in the “1.x” series, depending on the cost of moving to “2.0” for your end-users.

Using a PEP 386 compatible scheme

PEP 386 defines a standard version scheme that should be used by Python applications that are publishing releases at PyPI. It ensures interoperability among all major package managers and installers. In other words, it will make sure that all versions of a project are proprely recognized and sorted as long as a PEP 386-compatible scheme is used.

XXX put a link, more info

XXX say here that ppl should look at how big projects are doing (python, zope, twisted)