I’m a Python n00b, but an experienced software engineer and Python packaging is a rats nest.

requirements.txt or setup.py? Or one of the myriad packaging tools? It’s hard to see straight off.

The requirements.txt or setup.py really baked my noodle for a while. Finally I ended up at:

• Use setup.py for most purposes (I’m mainly interested in writing small utilities and command line applications).

• Use the install_requires in setup.py to specify dependencies.

• Use pip install --user -e . in preference to python setup.py develop as the former handles more cases, even when specified in setup.py (see below).

• Use pip freeze --user > requirements.txt (not entirely sure on this one yet)

• Use python setup.py build sdist bdist_wheel to produce wheel for deployment.

• Replace test command in setup.py to run the full pytest suite.

On a side note; adding tox to setup to run tests on oldest supported Python version and current version.

Issues with python setup.py develop

I ran into an issue including the following in my setup.py:

1
install_requires = ["PyYAML>=3.11"]
 

When I ran this using python setup.py develop it threw all sorts of errors.

However, running it using pip install --user -e . worked perfectly.

This seems to stem from setup.py develop using eggs whereas pip install -e uses wheel.

Environment

I decided to use concourse CI for all my pipeline needs, so it is natural to set up build/test environments as docker containers. Since I’m setting these up I may as well leverage them into a development environment. To this end I take the build/test image and layer on top of it a ‘developer’ image. Currently I’m using the following:

Dockerfile.dev

1
2
3
4
5
6
7
8
9
10
11
12
13
FROM wanmgr:latest
MAINTAINER Mark Bools <saltyvagrant@gmail.com>

RUN apt-get update -y \
  && apt-get upgrade -y \
  && apt-get -y install git tmux vim

COPY . /app
WORKDIR /app

RUN pip install --user -e .

CMD ["tmux"]
 

This is very much a ‘work in progress’, but so far it is very satisfying.

The Project in Question

This is my first attempt at a substantial Python project. It’s actually a rewrite (and development off) a small service script to monitor multiple ‘weak’ WAN connections (slow unreliable ISPs) to load balance traffic from the LAN to the internet, ensuring when an ISP fails it is pulled from the gateway pool.

https://gitlab.com/SaltyVagrant/wanmanager

This is very much a work in progress, but here is where this post ends.