Tag “Setuptools”

Since I started developing in Python, I use Setuptools in each of my projects. So I was sure that this approach is obvious and nobody needs an explanation what benefits it brings. I think, it happened because the first thing I’ve learned was Pylons web framework. There was no other way developing project rather than using Setuptools. However I was wondered to know how much people develop applications without packaging and get troubles, which already solved in Setuptools.

Let’s consider a typical application. It usually consists of single package that includes a number of modules and subpackages:

MyProject/                  # a project folder
    myproject/              # a root package of the project

If you are going to use Setuptools, you have to add at least a single file at the project folder—setup.py with the following contents:

from setuptools import setup

    long_description='Forty two',
    author='John Doe',

This script adds metadata to the project and tells Python how to install it. For example the following command installs project into Python site-packages:

$ python setup.py install

...and this one installs it in development mode, i.e. creates a link to the code instead of copy it:

$ python setup.py develop

If you ever developed a library and published it on PyPI the code above should be familiar to you. So I’m not going to discuss why do you need Setuptools in library development process. What I’m going to consider is why do you need Setuptools in application development. For example you develop a web site. It should work on your local development environment and production one. You are not going to distribute it via PyPI. So why do you need to add extra steps of deployment—packaging and installation? What issues Setuptools can solve?

Mess with import path

Each application has at least one main module. This module is usually executable script or contains a special object, which will be used by third-party applications. For example, uWSGI application server requires a module with a callable object application which will be served by uWSGI. Obviously, this module should import another ones from the project. Because of this, it usually contains dirty hacks around sys.path. For example, if module1.py from the example above is executable, it might contain the following patch:


import os
import sys

# Makes ``myproject`` package discoverable by Python
# adding its parent directory to import path
root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path = [root] + sys.path

from myproject import module2

And the relative import just doesn’t work:

from . import module2
# You will get
# ValueError: Attempted relative import in non-package

When your application is installed into Python site-packages using Setuptools, you will never have the problems with importing modules. Any import relative or absolute just works. And completely no hacks.

Executable scripts

There is one more trouble with executable scripts. You will have to specify a path when you call it:

$ /path/to/myproject/dosomething.py

...or create a symlink to /usr/bin

$ sudo ln -s /path/to/myproject/dosomething.py /usr/bin/dosomething
$ dosomething

Setuptools automate this routine. You just need to add a special entry point into setup() function:

    # ...
    dosomething = myproject.module1:dosomething

It creates a console script dosomething that will call dosomething() function from module myproject.module1 each time the script is executed. And this feature even works in the virtual environment. As soon as you activate virtual environment, each executable script will be available at console.

Entry points

Entry points are not limited by creating console scripts. It is a powerful feature with a lot of use cases. In a nutshell, it helps packages to communicate each other. For example, an application can scan packages for a special entry point and use them as plugins.

Entry points are usually described using ini-file syntax. Where section name is entry point group name, key is entry point name, and value is Python path to target object, i.e.:

entry_point_name = package.module:object

For instance, application can discover entry points from group myproject.plugins to load plugins defined in separate packages:

import pkg_resources

plugins = {}
for entry_point in pkg_resources.iter_entry_points('myproject.plugin'):
    plugins[entry_point.name] = entry_point.load()

Another use case is to make your application pluggable. For example, the common way to deploy Pyramid applications is using PasteDeploy-compatible entry points, which return WSGI application factory:

main = myproject.wsgi:get_application

Requirements and wheels

You can also specify application requirements in the setup() function:

    # ...
    install_requires=['Pyramid', 'lxml', 'requests'],

The third-party packages will be downloaded from PyPI each time you install the application. Additionally you can use wheels. It helps to speedup installation process dramatically and also freeze versions of third-party packages. Make sure you are using latest version of Setuptools, Pip and Wheel:

$ pip intall -U pip setuptools wheel

Then pack your application with its dependencies into wheelhouse directory using the following script:



python setup.py bdist_wheel

mkdir -p "${WHEELHOUSE}"
pip wheel \
    --use-wheel \
    --wheel-dir "${WHEELHOUSE}" \
    --find-links "${WHEELHOUSE}" \
    --requirement "${REQUIREMENTS}"

cp dist/*.whl "${WHEELHOUSE}/"

Now, you can copy wheelhouse directory to any machine and install your application even without an Internet connection:

$ pip install --use-wheel --no-index --find-links=wheelhouse MyProject

Want more?

The features described above are not only available ones. A lot of other cool things you can find in the official documentation. I hope I've awoken your interest.