Integration testing Cargo binaries with assert_cmd
Cargo allows us to execute a few different kinds of tests,
including integration and unit. For now we'll talk about
integration tests for our CLI. Integration tests
live in /tests and only have access to the public
interface of a crate.
In this case, we'll run the garden binary using
assert_cmd and assert various things about the
execution and output. For example, that the binary
exits successfully and that stderr has no output.
cargo test allows us to run all tests.
To restrict the number of tests that run to only the two
that work, we can use cargo test help to restrict
cargo to only running tests that match the keyword "help",
or use #[ignore] on a test we want to ignore.
The third test will be ignored because we have yet to build out our functionality.
Chris Biscardi: [0:00] Cargo allows us to execute many different kinds of tests, including integration tests, unit tests, benchmark tests and documentation tests. For now, we're just going to talk about integration tests for our CLI. Integration tests live in /tests in your project directory and only have access to the public interface of a crate.
[0:22] To spawn new commands that interact with them, such as the guardian binary defined in our Cargo terminal, we'll use the cert command. We'll add it to our devDependencies because we don't intend to ship this with our project.
[0:34] Now, even though we have no tests currently, we can still run Cargo tests. Note that we've run zero tests, and the test result is OK. We'll create a new file at testsintegration.rs. This is where integration tests live.
[0:52] Although the file name does not matter, the directory name does. To spawn the garden subcommand, we'll use command from the assert_cmd crate. Tests are just like regular functions, except they have this attribute macro on top, letting Cargo know that they are tests.
[1:11] This allows us later to abstract functionality into other functions that don't count as tests. In this case, we'll use a utility from the assert_cmd crate to pull out the garden binary from the Cargo.toml and build it. Now we can unwrap this, but we can also return a result here.
[1:30] Note that after bringing result to this scope and call it err, we can use that result as our return type. We can then continue to use the question mark instead of the unwrap, but at the end of our test we'll have to return one of the result variants, in this case OK, with the value unit, which is what's specified in our type signature.
[1:48] Since we have a mutable command here, we can continue to add arguments like help, and then we can call assert on that command to give us an assert value that then we can use to test against a successful execution of the binary itself and do things like check to make sure that standard error is equal to an empty string. If we cargo test now, everything works.
[2:10] There's another trick we can use if we cargo install cargo-watch. I already have it installed, so you'll see a couple of dependencies installing here. If we cargo watch -x test, Cargo will run our tests and then wait for anything to change.
[2:28] If we add another test, in much the same way that we did the first one, but this time call out to the right subcommand, and pass help to the right subcommand. We can see that two tests have passed.
[2:40] Finally, if we want to test the right subcommand itself, we can take --help off and actually execute the right subcommand. Note that this panics, because we haven't finished the functionality of the right subcommand and that it's still panicking on "not yet implemented."
[2:58] Also note that we don't have to return a result here. If we ignore this test, we can see that test right is ignored. We have two passing tests. We've also seen that we can choose to unwrap, which will panic or return the error, depending on if we want a result type returned or a unit type returned from our function.
Hey Tommy! :)
I totally understand that the annotations can be confusing. The extension that is adding them is called rust-analyzer, which is a major part of the Rust project and represents the future for Rust IDE tooling. While I try to remove anything that might be extraneous information in my courses I made the decision to keep rust-analyzer in this series because it shows information that is otherwise hard to come by for beginners, such as inferred types of local variables and chained expressions and the names of function arguments. The inferred types are shown in the same place you would write them in the source code in this lesson, so you can choose to also type them out or not.
This does mean that there is a difference between what is shown on screen during the video and what is in the source files. I believe that this difference is beneficial since rust-analyzer is the suggested editor integration for Rust and the information it provides is hard to glean without more familiarity with Rust and both sources are made available. This is also the reason I take the time to create and link to individual commits for each lesson, so that if there is any confusion it can be resolved by checking the source code.
Thanks, Chris, for both the quick reply and the clear explanation.
I went into this thinking it was for beginners and only realized after my comment that it was probably assumed that I knew more about Rust.
I figured out pretty quickly that those were annotations, but where I got stuck was realizing that in:
let mut cmd: Command = Comand::...
That it was ": Command" that I needed to ignore rather than "Command =".
And that just comes down to me not knowing enough about Rust.
So I have bounced over to running through the rustlings exercises for now and will come back once I have finished those and have a better handle on more of the basics of the language.
Looks like the API has changed:
use assert_cmd::assert::Assert;
use assert_cmd::Command;
use color_eyre::eyre::Result;
#[test]
fn test_help() -> Result<()> {
let mut cmd: Command = Command::cargo_bin("garden")?;
let assert: Assert = cmd.arg("help").assert();
assert.success().stderr("");
Ok(())
}
The parameter annotations in gray in your editor are confusing if you don't know Rust. I had to go find the code on Github to figure out how to get my test of help to work.