Complete Me!

In this post I talk about the new ZSH completion script generation in clap. If you're only here for the snacks, jump on down to the good bits!

Some Back Story

My personal schedule has been absolutely insane over the past few months. I felt like I was falling behind on some of my hobby projects. Not only do I enjoy working on these projects, as they're a form of mental stress relief (...mostly), but with projects like clap where there are actual users, I also feel a need to provide support for something that many people are counting on.

Over the past weekend, I've finally had some excellent, albeit short lived free time. Even though there's a long list of things I'd love to accomplish with clap, one of the most impactful (after catching up on some bug reports/fixes of course) is completion script generation.

Completion Scripts

As you may, or may not, know shell completion scripts are perhaps the best thing since sliced bread. Even though I adore command line applications, I can very rarely remember which switches to turn on, what subcommands exist, etc. Seriously, I feel like a 95 year old man at times...

The wonderful thing about completion scripts is I don't have to remember. And with ZSH at least, I get a helpful hint at what I'm actually doing. You'll see what I mean shortly! Suffice it to say that when you're running a command, or trying to remember what to type to invoke the command, you simply double-tap your <tab> key, and see a list of valid possibilities. In most shells this also a context sensitive list (more on that in a bit). You may see something like:

$ rustup <tab><tab>
-h           man          set          toolchain    --verbose    component
help         override     show         update       --version    default  
--help       run          target       -v           which        doc
install      self         telemetry    -V  

If you're in the middle typing a particular command, hitting the <tab> key once will finish it for you, or print any possibilities if there are ambiguities

$ rustup ov<tab>
$ rustup override

$ rustup --ve<tab>
--verbose   --version

The problem with these completion scripts is that you have to write them. And they can be tedious, redundant, and arcane. All the worst things, rolled into one heated mess. For some relativity, the BASH completion script for rustup is 950 lines with no comments. (⊙_☉)

On top of that, if you ever change your CLI, say add an option, change the name of an old command, etc. you'll have to re-write these scripts. It can be a nightmare.

This is why I originally added completion script generation to clap. It simply looks at your valid flags, options, commands, etc. and generates a script for you. Yay!

Completion Script Generation in clap

clap has supported generating completion scripts (at least Bash and Fish) for a little while now. But as an avid user of robbyrussell/oh-my-zsh with ZSH as my primary shell, I felt I was missing out on some completing goodness.

That's why I'm proud to (finally) say that as of v2.16 ZSH support has been added.

Two Ways

So now that we know what completion scripts are, we can start to use them and bask in their all knowing light. Completion scripts can be generated in one of two ways in clap either at compile time, in which case they'll be output alongside your binary, or at runtime. Actually you can do both...or none. So really four ways.

Let's create a new project real quick to demo it.

$ cargo new fake --bin
$ cd fake

Then add clap as a dependency of the project.

# Cargo.toml
[package]
## ... snip

[dependencies]
clap = "~2.16.1"  

Next we mock up a very simple CLI. One of the requirements of generating completion scripts is that our CLI code be abstracted out into some kind of function because we'll need to call it from a few places.

// src/cli.rs
use clap::{App, Arg, SubCommand};  
pub fn build_cli() -> App<'static, 'static> {  
    App::new("fake")
        .arg(Arg::with_name("file")
            .help("Some input file"))
        .arg(Arg::with_name("config")
            .short("c")
            .long("config")
            .takes_value(true)
            .value_name("FILE")
            .help("Optionally use some config file"))
        .arg(Arg::with_name("verbose")
            .short("v")
            .long("verbose")
            .help("Output verbose information"))
        .subcommand(SubCommand::with_name("test")
            .about("Tests things for us...")
            .arg(Arg::with_name("totest")
                .long("thing")
                .help("The thing to test")
                .possible_values(&["thing1", "other-thing", "stuff"])))
}
// src/main.rs
extern crate clap;

mod cli;

fn main() {  
    let matches = cli::build_cli().get_matches();
}

Our little CLI has a positional argument, a flag, an option, a subcommand which also has it's own option. It might be time we start considering a completion script...

Compile Time

Using the compile time method is nice, if you already have a nice way to distribute these scripts. It's a little bit more work to set up, but not much. The upside is that it doesn't require any additional code at runtime.

We'll do this with a Rust build.rs "Build Script" which allows us to run arbitrary Rust code at compile time without the need for a nightly compiler or procedural macros.

First, we tell our project that we are in fact using a build script:

# Cargo.toml
[package]
# snip...
build = "build.rs"  

Then we create our build script. The idea is that we'll build our CLI, then generate the script and save it to a file.

// build.rs
extern crate clap;

use clap::Shell;

include!("src/cli.rs");

fn main() {  
    let mut app = build_cli();
    app.gen_completions("fake",          // We specify the bin name manually
                        Shell::Bash,      // Which shell to build completions for
                        env!("OUT_DIR")); // Where write the completions to
}

That's it! Now when you run cargo build you'll have a new fake.bash-completion file sitting alongside your fake binary.

Now you might be asking why we have to specify the binary name manually? It's because we aren't actually running the application, so clap doesn't have a chance to figure out what the end result binary name actually ended up being.

If you want to generate for more than one shell, it's as simple as changing to:

// build.rs

fn main() {  
    let mut app = build_cli();
    app.gen_completions("fake", Shell::Bash, env!("OUT_DIR")); 
    app.gen_completions("fake", Shell::Fish, env!("OUT_DIR")); 
    app.gen_completions("fake", Shell::Zsh,  env!("OUT_DIR")); 
}

Runtime

Compile time is all well and good, but that whole build.rs thing is a bit messy. Plus you have to distribute all those scripts to your end users, or make them available, and you don't know why which shell they're using, or where they install their completion scripts too!

Alas, there is another way. You can add an option, or subcommand to your CLI which generates the scripts and outputs them to stdout. This allows the user to pick which shell, and then simply redirect stdout to desired file/location of their choosing.

Let's do that.

Delete the build.rs and remove the build = "build.rs" from your Cargo.toml.

Now all we have to do is add this option our CLI: Since we already have a subcommand, let's add another called completions. Although it'd be just as simple to create a --completions <SHELL> option, or something similar.

// src/cli.rs

// snip..

    App::new("fake")
        // Snip...
        .subcommand(SubCommand::with_name("completions")
            .about("Generates completion scripts for your shell")
            .arg(Arg::with_name("SHELL")
                .required(true)
                .possible_values(&["bash", "fish", "zsh"])
                .help("The shell to generate the script for")))

Then we add the logic to our src/main.rs

// src/main.rs

use std::io;  
use clap::Shell;

fn main() {  
    let matches = cli::build_cli().get_matches();
    match matches.subcommand() {
        ("completions", Some(sub_matches)) => {
            let shell = sub_matches.value_of("SHELL").unwrap();
            cli::build_cli().gen_completions_to("fake", shell.parse::<Shell>().unwrap(), &mut io::stdout());
        },
        (_, _) => unimplemented!(), // for brevity
    }
}

And we're done! It may look slightly different from the compile time code, but the gist is that we're using the gen_completions_to which allows us to specify an io::Write object (such as stdout), and we're just turning the &str value of <SHELL> into a clap::Shell enum variant since clap::Shell implements FromStr.

Also note, the unrwap() call here is actually safe, because clap ensures that only valid &str values are used, and that one such value has been used.

Now you could run something like:

$ fake completions bash > /etc/bash_completion.d/fake.bash-completion

Also, if you don't like that completions subcommand messing up your CLI help message, you can always add the line setting(AppSettings::Hidden) which will make it a secret (⌐■_■)

More Demos!

I've recently submitted a PR to rustup which includes the ability to generate these scripts. But for a demo of both the Bash and Zsh shell completion I give you..

ZSH

or GIFV

Bash

or GIFV

Closing Comments

Some technical notes not covered in the demos, but the ZSH implementation supports things like:

  • Automatic conflict resolution (i.e. if --a conflicts with --b, and you've already typed --a then your completion list will not display --b)
  • Automatic possible value lists (i.e. in the above example, things like only being able to type zsh, bash, or fish)
  • Switch stacking (i.e. allowing you do things like -a -b -c becoming -abc)

Happy completing!