By Vincent Driessen
on Wednesday, September 24, 2014

I used to find writing command line tools tedious. Not so much the writing of the core of the tool itself, but all the peripheral stuff you had to do to actually finish one.

The Language?

The first issue is to pick the language to implement it in: do I use Python, which I'm intimitely familiar with, or a Unix shell script? With shell scripts, the syntax is pretty terrible, but the tool typically fits in a single file and there's hardly any overhead running them. On the other hand, making sure the tool works under all circumstances can be tricky. Shell scripts are notorious for breaking when you feed them arguments with spaces. The burden of making sure you properly quote all the variable interpolations in the script is on the programmer. It's possible to do, just unnecessarily hard.

On the other hand, Python is so much more expressive. There are a ton of libraries out there ready to use, and Python itself includes a lot of batteries already in its standard library, of course.

Distribution?

Python comes with its own set of problems, though. Python runtime environments are typically a mess, and I don't want to further pollute people's already cluttered global Python environments. With Python, installing a package is typically just a pip install <pkg> away, but it requires another tedious step: writing a setup.py.

If it comes to distributing the script, a shell script may be much easier. With shell scripts it's either a single file that needs to be copied somewhere. Manually, or via a make install command, which involves adding a Makefile and dealing with subtle differences for each Unix platform, not to even mention trying to run it on Windows machines.

Argument Parsing?

Each script will at some stage require some options or arguments. How should we do the argument parsing? Do I use getopt or getopts? Does it even matter? Can it take --long-form-options? Or do I resign myself to poor man's arg parsing again? The latter has too often become the default choice.

Standing on the Shoulders of Giants

Lately, a few fantastic projects have taken away most of the tedious work surrounding the building of command line tools, and almost make it trivial now.

Click

Click is a Python library written by Armin Ronacher that deals with all the handling of command line option and argument parsing and comes with fantastic defaults. This project is a great step towards more consistent and standard CLI interfaces. Besides solving the options and argument parsing, it also has a ton of useful features packaged, like smart colorized terminal output, file abstractions, subcommands, and rendering progress bars.

It solves the argument parsing problem.

pipsi

Using pipsi (also by Armin!), users can install any Python command line script into an isolated Python runtime environment, so it solves the global cluttered Python environment problem entirely.

Cookiecutter

Cookiecutter (by the awesome Audrey Roy Greenfield) is a project generator, based on a predefined project template. It will read the template, ask the user a few questions to fill in the blanks, and generates a new project for you.

cookiecutter-python-cli

cookiecutter-python-cli is one such Cookiecutter template I wrote that uses all of the above: it sports a predefined setup.py, a package structure that's extensible, and test cases and a test runner to get you started.

Putting it Together

Let's build a new high quality CLI in Python in under 60 seconds now.

First, install pipsi and follow its instructions:

$ curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python

Next, using pipsi, install Cookiecutter in its own isolated runtime environment:

$ pipsi install cookiecutter

Now use Cookiecutter to create your brand new project, based on my CLI template:

$ cd ~/Desktop
$ cookiecutter https://github.com/nvie/cookiecutter-python-cli.git
Cloning into 'cookiecutter-python-cli'...
remote: Counting objects: 64, done.
remote: Total 64 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (64/64), done.
Checking connectivity... done.
full_name (default is "Vincent Driessen")?
email (default is "vincent@3rdcloud.com")?
github_username (default is "nvie")?
project_name (default is "My Tool")?
repo_name (default is "python-mytool")?
pypi_name (default is "mytool")?
script_name (default is "my-tool")?
package_name (default is "my_tool")?
project_short_description (default is "My Tool does one thing, and one thing well.")?
release_date (default is "2014-09-04")?
year (default is "2014")?
version (default is "0.1.0")?

When you're done, you'll have a project where you can run tox to run your test suite on all important Python versions. If you don't need the test cases, simply remove the tests/ directory.

$ cd python-mytool/
$ tox
...
  py26: commands succeeded
  py27: commands succeeded
  py33: commands succeeded
  py34: commands succeeded
  pypy: commands succeeded
  flake8: commands succeeded
  congratulations :)

Let's install and run it without further modifications:

$ pipsi install --editable .
...
$ my-tool
Hello, world.
$ my-tool --as-cowboy
Howdy, world.
$ my-tool --as-cowboy Vincent
Howdy, Vincent.

You can edit the setup.py to your liking. The default provided version should already work out of the box. When you're done implementing your tool, you can either upload it to PyPI or just keep it to yourself locally:

$ pipsi install <pkgname>  # install from PyPI
$ pipsi install --editable ../path/to/project/dir   # install locally

If you have any improvements for this template, please submit a pull request. Thanks!

If you want to get in touch, I'm @nvie on Twitter.