clap v2.0 is released!

I'm [happy|relieved|excited] to announce the release of clap v2.0.0! I spent a good bit of time debating a major version bump. Now that it's released I'd be hard pressed to tell you exactly what I was afraid of. This release, in my opinion, is a dramatic improvement, and feels like a valiant contribution to the Rust community...or at least the portion that uses clap.

If you're only here for the free snacks and new features, jump on down to "v2 - Phew!". Otherwise, walk with me, lets converse about this journey.

Prologue

If you haven't heard of clap before - and I mean Command Line Argument Parser (Not anything else which unfortunately shares the same name. But hey, it's easy to remember and naming things is like 95% of the battle, right?) - it's exactly what the name implies an argument parser. But it started as a learning platform for me to try and wrap my head around this new hotness I'd heard about, called Rust.

The project has since grown from purely a learning platform into a very usable library for getting Real Work® done. In fact, I'm now only one of 27 contributors as of this writing (if you count @homu and @yo-bot... /r/botsrights). It's been amazing to watch PRs come in from around the globe. This is something I truly love about the Rust community.

I'm not a programmer by day, but I do use programming to make my paid job easier, or at least automated (note, the two are not always synonymous). It also helps me feel less guilty about taking extended coffee breaks.

I do use programming to make my paid job easier, or at least automated (note, the two are not always synonymous).

Either way, command line utilities in general are of a particular use to me, and I find them fascinating. Granted, the fact remains that I can almost never remember the proper incantations to invoke their magic without a hefty amount of Googleage. Perhaps this is why I'm more comfortable writing my own than memorizing others. Add to this, there is a decent amount of horribly designed interfaces...yes, I'm looking at you git.

I kid, git ♥

I've learned so much by working on clap; about Rust, about myself, about OSS in general, and a good bit about parsing arguments.

v0 - Be Gentle

The initial offerings of clap were embarrassing to say the least. I was am still figuring out how to properly speak crab. This coupled with coming from a non-systems programming background meant it was a slightly uphill battle.

I struggled with something I believe many do in the OSS community...is my code good enough?

...is my code good enough?

Putting your best effort out in the open, in black and white, for all the world to see and judge can be an intimidating process. At times it's difficult to separate criticisms of code, from criticisms of ones self - because coding can be such a personal thing that the two are often intermixed.

I can say, without a doubt; the Rust community is one of the friendliest and most professional communities I've ever been a part of.

the Rust community is one of the friendliest and most professional communities I've ever been a part of.

If there is anyone out there still holding on to a private code base for fear of critique, you needn't worry; the water really is warm. I say thanks to all those who work so hard to ensure this is the case!

Even though clap started as a learning project, enough people gave it a shot or offered friendly advice and PRs that I began trying to make it into a releasable product. To all those who jumped on board from the beginning, I tip my top hat to you fine ladies and gentlemen.

In my opinion, for a library to be of real use, it needs to form a contract with it's consumers regarding breakage. For clap this meant stabilizing the API and releasing a 1.0 with strong backwards compatibility guarantees. This is very important to me.

v1 - Share All the Things!

Another game changing aspect of Rust is the package management scenario. In fact, without something as amazing as cargo and crates.io, projects like clap wouldn't get anywhere. To use clap in your own applications it's literally adding a single line to your Cargo.toml (or better yet, installing the excellent cargo-edit command, and running a single $ cargo add clap).

install the excellent cargo-edit command, and run a single $ cargo add clap

Because this process is so simple, clap got a fair amount of use. Like any project, the more I and others used it, the more we began to see where certain decisions in v1 needed to be tweaked, namely:

  • The inability to handle invalid UTF-8
  • Too many lifetimes!
  • Vecs everywhere!
  • The inability to support external subcommands (i.e. similar to cargo)
  • The public API began to grow unwieldy with settings
  • The from_usage parser had some unintuitive design flaws

This meant a v2 had to be cut. My motto, and the way I intend to keep running clap is to iterate as much as required to build the best library I can, all while maintaining a breaking changes contract with my consumers. If that means clap reaches v6 next year (very unlikely), so be it.

v2 - Phew!

This new version of clap marks a huge step forward in the project. Let me point out some of the highlights by addressing the pain points above

Invalid UTF-8

Invalid UTF-8 is perfectly legal on POSIX compliant OSs where many of these little CLI utilities see use. clap's official stance is argument names, flags, and help must be valid UTF-8 encodable Rust strings. The values provided at runtime to those arguments (such as file names, etc.) do not need to be valid UTF-8 encoded strings.

External Subcommands

If one wanted to create an application similar to cargo where it's possible to take a portion of the runtime arguments and pass them to a child process process, it's now possible and just as easy as dealing with internal subcommands thanks to Rust's pattern matching. See the AppSettings::AllowExternalSubcommands setting in the documentation for details.

Lifetimes

While you didn't always see the lifetimes in clap unless browsing the documentation or you needed to do some funky return type handling, when you DID see the lifetimes your eyes would bleed. The signature of the App struct was something like App<'a, 'b, 'c, 'd, 'e, 'f> Which has now been paired down to App<'a, 'b> without losing any functionality. The same applies to other structs as well, like Arg

Iterators, Yay!

This one is a two'fer!

When building a CLI with clap it's often common to want to specify physically grouped arguments all passed to your application at once, which preivously required allocating an Vec<Arg> to pass in...no longer says I.

You can pass inline slices

myapp.args(&[  
    Arg::with_name("a"),
    Arg::with_name("b"),
    Arg::with_name("c")
])

The second part is when getting multiple values out of an argument. Before this meant creating a Vec<&str> but has since changed to returning a Values struct which implements Iterator and lets you decide which end container to use. This prevents things like,

// v1
let vals: HashSet<&str> = matches.values_of("arg")  
                                 .unwrap()        // Extra Vec<&str> created
                                 .iter()          // manually iter
                                 .collect();

// v2
let vals: HashSet<&str> = matches.values_of("arg")  
                                 .unwrap()
                                 .collect();

So Many Settings

The public API began to grow unwieldy with "settings" options. I.e. one has the ability to tweak the behavior of clap with various true/false toggles. Each of these toggles was originally implemented as separate method of App, due to their relatively small number. This quickly changed as clap grew and new features were requested. The number of "settings" quickly began to pollute the public API. A solution was derived to use a AppSettings enum, which also made adding new settings a breeze.

Unfortunately, clap had already built up quite a few "settings" methods which couldn't be removed due to backwards compatibility. v2 makes cleaning that API possible!

From Usage

The from_usage parser, which allows various argument settings by simply parsing a usage string, had several unintuitive design flaws. Such as prioritizing naming of arguments using the value name instead of the flag. I.e to access an argument that was created using -c, --config=<FILE> you had to access, "FILE"...but what if you had more than one argument which accepted a file?! Plus, you couldn't set value names in a usage string, which is a common thing you'd want to do.

To contrast, here is the difference between v1 and v2:

// v1
-c, --config <CFILE>  // accessed with "CFILE"
-o <OFILE>            // accessed with "OFILE"
--input <IN_FILE>     // accessed with "IN_FILE"

// v2
-c, --config <FILE> // accessed with "config"
-o <FILE>           // accessed with "o"
--input <FILE>      // accessed with "input"

The usage parser now does the intuitive thing and uses the short or long to derive the name, using the value name as exactly that, a value name!
Non Breaking Changes

There have been a huge number of non-breaking changes as well. The ones I'm most proud of aren't even code.

  • The documentation has been vastly improved
  • The regression testing is far better
  • Errors and error messages are much improved
  • Performance is getting even better!
  • A number of smaller yet key features have been implemented.

Most notably these "small yet key features" is the universal short/long forms for option arguments, and the ability to parse delimited values.

By universal short/long I mean all of these are valid forms for parsing options (assuming the short and long has been defined):

// valid shorts
$ program -o val
$ program -o=val
$ program -oval

// valid longs
$ program --option val
$ program --option=val

Which goes very nicely with delimited values. Spaces as a delimiter has always been around and will continue to be. But now there is an "additional delimiter" which defaults to comma (,). For example, the above examples with three values each using delimited values looks like:

// shorts with 3 values (comma delimited)
$ program -o val,val2,val3
$ program -o=val,val2,val3
$ program -oval,val2,val3

// longs with 3 values (comma delimited)
$ program --option val,val2,val3
$ program --option=val,val2,val3

And you even have the ability to change the delimiter! So as a silly example, a very quick implementation of something that accepts only valid IPs (assuming each octet could be 0-255):

extern crate clap;  
use clap::{App, Arg};  
fn main() {  
    let m = App::new("ipval")
                 .arg(Arg::from_usage("-i, --ip-addr <X.X.X.X> 'An IP Address'")
                     .number_of_values(4)
                     .validator(valid_octet)
                     .value_delimiter("."))
                 .get_matches();
    let ip = m.values_of("ip-addr").unwrap().collect::<Vec<_>>().join(".");

    println!("{} is a valid IP", ip);
}

fn valid_octet(o: String) -> Result<(), String> {  
    if let Err(..) = o.parse::<u8>() {
        return Err(format!("'{}' must be a number between 0 and 255", o));
    }
    Ok(())
}

Here's the output from the above code:

$ ipval -i 10.20.30.40
10.20.30.40 is a valid IP

$ ipval -i 10.2000.30.40
error: '2000' must be a number between 0 and 255

$ ipval -i 10.20.30
error: The argument '--ip-addr <X.X.X.X>' requires 4 values, but 3 were provided

USAGE:  
    ipval [FLAGS] --ip-addr <X.X.X.X>

For more information try --help

$ ipval -i 10.20.30.40.50
error: The argument '--ip-addr <X.X.X.X>' requires 4 values, but 5 were provided

USAGE:  
    ipval [FLAGS] --ip-addr <X.X.X.X>

For more information try --help  

And keep in mind, these are all valid permutations with no code changes!

$ ipval -i=10.20.30.40
$ ipval -i10.20.30.40
$ ipval -i 10.20.30.40
$ ipval --ip-addr 10.20.30.40
$ ipval --ip-addr=10.20.30.40

Wrap IT Up!

Bottom line, I'm excited to see where clap goes next! Check out the repository, or the documentation for all the details and examples.

If you're looking for a project to test out some Rust, jump on in, I'd be more than happy to assist with anyone getting started!

I can't say it enough, but thank you to all 27 contributors, as well as those who have contributed in so many other ways!

clap Contributors (In alphabetical order)

  • Alexander Kuvaev (Core Collaborator)
  • Alex Gulyás
  • Benjamin Sago
  • Brad Urani
  • Dabo Ross
  • Georg Brandl
  • Paul Blouët
  • grossws
  • Ivan Dmitrievsky
  • Homu
  • Huon Wilson
  • J/A
  • Jacob Helwig
  • James McGlashan (Core Collaborator)
  • Jimmy Cuadra
  • Kevin K. (Me)
  • Markus Unterwaditzer
  • messense
  • Nelson Chen
  • Ross Nelson
  • Sebastian Thiel
  • Severen Redwood
  • SungRim Huh (Core Collaborator)
  • Tshepang Lekhonkhobe
  • Vincent Prouillet
  • Vlad
  • Yo-Bot

PS.

Sorry, I lied about the snacks...unless you count the cookies being served.