| CARGO-BENCH(1) |
| |
| NAME |
| cargo-bench — Execute benchmarks of a package |
| |
| SYNOPSIS |
| cargo bench [options] [benchname] [-- bench-options] |
| |
| DESCRIPTION |
| Compile and execute benchmarks. |
| |
| The benchmark filtering argument benchname and all the arguments |
| following the two dashes (--) are passed to the benchmark binaries and |
| thus to libtest (rustc’s built in unit-test and micro-benchmarking |
| framework). If you are passing arguments to both Cargo and the binary, |
| the ones after -- go to the binary, the ones before go to Cargo. For |
| details about libtest’s arguments see the output of cargo bench -- |
| --help and check out the rustc book’s chapter on how tests work at |
| <https://doc.rust-lang.org/rustc/tests/index.html>. |
| |
| As an example, this will run only the benchmark named foo (and skip |
| other similarly named benchmarks like foobar): |
| |
| cargo bench -- foo --exact |
| |
| Benchmarks are built with the --test option to rustc which creates a |
| special executable by linking your code with libtest. The executable |
| automatically runs all functions annotated with the #[bench] attribute. |
| Cargo passes the --bench flag to the test harness to tell it to run only |
| benchmarks, regardless of whether the harness is libtest or a custom |
| harness. |
| |
| The libtest harness may be disabled by setting harness = false in the |
| target manifest settings, in which case your code will need to provide |
| its own main function to handle running benchmarks. |
| |
| Note: The #[bench] attribute |
| <https://doc.rust-lang.org/nightly/unstable-book/library-features/test.html> |
| is currently unstable and only available on the nightly channel |
| <https://doc.rust-lang.org/book/appendix-07-nightly-rust.html>. There |
| are some packages available on crates.io |
| <https://crates.io/keywords/benchmark> that may help with running |
| benchmarks on the stable channel, such as Criterion |
| <https://crates.io/crates/criterion>. |
| |
| By default, cargo bench uses the bench profile |
| <https://doc.rust-lang.org/cargo/reference/profiles.html#bench>, which |
| enables optimizations and disables debugging information. If you need to |
| debug a benchmark, you can use the --profile=dev command-line option to |
| switch to the dev profile. You can then run the debug-enabled benchmark |
| within a debugger. |
| |
| Working directory of benchmarks |
| The working directory of every benchmark is set to the root directory of |
| the package the benchmark belongs to. Setting the working directory of |
| benchmarks to the package’s root directory makes it possible for |
| benchmarks to reliably access the package’s files using relative |
| paths, regardless from where cargo bench was executed from. |
| |
| OPTIONS |
| Benchmark Options |
| --no-run |
| Compile, but don’t run benchmarks. |
| |
| --no-fail-fast |
| Run all benchmarks regardless of failure. Without this flag, Cargo |
| will exit after the first executable fails. The Rust test harness |
| will run all benchmarks within the executable to completion, this |
| flag only applies to the executable as a whole. |
| |
| Package Selection |
| By default, when no package selection options are given, the packages |
| selected depend on the selected manifest file (based on the current |
| working directory if --manifest-path is not given). If the manifest is |
| the root of a workspace then the workspaces default members are |
| selected, otherwise only the package defined by the manifest will be |
| selected. |
| |
| The default members of a workspace can be set explicitly with the |
| workspace.default-members key in the root manifest. If this is not set, |
| a virtual workspace will include all workspace members (equivalent to |
| passing --workspace), and a non-virtual workspace will include only the |
| root crate itself. |
| |
| -p spec…, --package spec… |
| Benchmark only the specified packages. See cargo-pkgid(1) for the |
| SPEC format. This flag may be specified multiple times and supports |
| common Unix glob patterns like *, ? and []. However, to avoid your |
| shell accidentally expanding glob patterns before Cargo handles |
| them, you must use single quotes or double quotes around each |
| pattern. |
| |
| --workspace |
| Benchmark all members in the workspace. |
| |
| --all |
| Deprecated alias for --workspace. |
| |
| --exclude SPEC… |
| Exclude the specified packages. Must be used in conjunction with the |
| --workspace flag. This flag may be specified multiple times and |
| supports common Unix glob patterns like *, ? and []. However, to |
| avoid your shell accidentally expanding glob patterns before Cargo |
| handles them, you must use single quotes or double quotes around |
| each pattern. |
| |
| Target Selection |
| When no target selection options are given, cargo bench will build the |
| following targets of the selected packages: |
| |
| o lib — used to link with binaries and benchmarks |
| |
| o bins (only if benchmark targets are built and required features are |
| available) |
| |
| o lib as a benchmark |
| |
| o bins as benchmarks |
| |
| o benchmark targets |
| |
| The default behavior can be changed by setting the bench flag for the |
| target in the manifest settings. Setting examples to bench = true will |
| build and run the example as a benchmark, replacing the example’s main |
| function with the libtest harness. |
| |
| Setting targets to bench = false will stop them from being bencharmked |
| by default. Target selection options that take a target by name (such as |
| --example foo) ignore the bench flag and will always benchmark the given |
| target. |
| |
| See Configuring a target |
| <https://doc.rust-lang.org/cargo/reference/cargo-targets.html#configuring-a-target> |
| for more information on per-target settings. |
| |
| Binary targets are automatically built if there is an integration test |
| or benchmark being selected to benchmark. This allows an integration |
| test to execute the binary to exercise and test its behavior. The |
| CARGO_BIN_EXE_<name> environment variable |
| <https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-crates> |
| is set when the integration test is built so that it can use the env |
| macro <https://doc.rust-lang.org/std/macro.env.html> to locate the |
| executable. |
| |
| Passing target selection flags will benchmark only the specified |
| targets. |
| |
| Note that --bin, --example, --test and --bench flags also support common |
| Unix glob patterns like *, ? and []. However, to avoid your shell |
| accidentally expanding glob patterns before Cargo handles them, you must |
| use single quotes or double quotes around each glob pattern. |
| |
| --lib |
| Benchmark the package’s library. |
| |
| --bin name… |
| Benchmark the specified binary. This flag may be specified multiple |
| times and supports common Unix glob patterns. |
| |
| --bins |
| Benchmark all binary targets. |
| |
| --example name… |
| Benchmark the specified example. This flag may be specified multiple |
| times and supports common Unix glob patterns. |
| |
| --examples |
| Benchmark all example targets. |
| |
| --test name… |
| Benchmark the specified integration test. This flag may be specified |
| multiple times and supports common Unix glob patterns. |
| |
| --tests |
| Benchmark all targets in test mode that have the test = true |
| manifest flag set. By default this includes the library and binaries |
| built as unittests, and integration tests. Be aware that this will |
| also build any required dependencies, so the lib target may be built |
| twice (once as a unittest, and once as a dependency for binaries, |
| integration tests, etc.). Targets may be enabled or disabled by |
| setting the test flag in the manifest settings for the target. |
| |
| --bench name… |
| Benchmark the specified benchmark. This flag may be specified |
| multiple times and supports common Unix glob patterns. |
| |
| --benches |
| Benchmark all targets in benchmark mode that have the bench = true |
| manifest flag set. By default this includes the library and binaries |
| built as benchmarks, and bench targets. Be aware that this will also |
| build any required dependencies, so the lib target may be built |
| twice (once as a benchmark, and once as a dependency for binaries, |
| benchmarks, etc.). Targets may be enabled or disabled by setting the |
| bench flag in the manifest settings for the target. |
| |
| --all-targets |
| Benchmark all targets. This is equivalent to specifying --lib --bins |
| --tests --benches --examples. |
| |
| Feature Selection |
| The feature flags allow you to control which features are enabled. When |
| no feature options are given, the default feature is activated for every |
| selected package. |
| |
| See the features documentation |
| <https://doc.rust-lang.org/cargo/reference/features.html#command-line-feature-options> |
| for more details. |
| |
| -F features, --features features |
| Space or comma separated list of features to activate. Features of |
| workspace members may be enabled with package-name/feature-name |
| syntax. This flag may be specified multiple times, which enables all |
| specified features. |
| |
| --all-features |
| Activate all available features of all selected packages. |
| |
| --no-default-features |
| Do not activate the default feature of the selected packages. |
| |
| Compilation Options |
| --target triple |
| Benchmark for the given architecture. The default is the host |
| architecture. The general format of the triple is |
| <arch><sub>-<vendor>-<sys>-<abi>. Run rustc --print target-list for |
| a list of supported targets. This flag may be specified multiple |
| times. |
| |
| This may also be specified with the build.target config value |
| <https://doc.rust-lang.org/cargo/reference/config.html>. |
| |
| Note that specifying this flag makes Cargo run in a different mode |
| where the target artifacts are placed in a separate directory. See |
| the build cache |
| <https://doc.rust-lang.org/cargo/guide/build-cache.html> |
| documentation for more details. |
| |
| --profile name |
| Benchmark with the given profile. See the reference |
| <https://doc.rust-lang.org/cargo/reference/profiles.html> for more |
| details on profiles. |
| |
| --ignore-rust-version |
| Benchmark the target even if the selected Rust compiler is older |
| than the required Rust version as configured in the project’s |
| rust-version field. |
| |
| --timings=fmts |
| Output information how long each compilation takes, and track |
| concurrency information over time. Accepts an optional |
| comma-separated list of output formats; --timings without an |
| argument will default to --timings=html. Specifying an output format |
| (rather than the default) is unstable and requires |
| -Zunstable-options. Valid output formats: |
| |
| o html (unstable, requires -Zunstable-options): Write a |
| human-readable file cargo-timing.html to the target/cargo-timings |
| directory with a report of the compilation. Also write a report |
| to the same directory with a timestamp in the filename if you |
| want to look at older runs. HTML output is suitable for human |
| consumption only, and does not provide machine-readable timing |
| data. |
| |
| o json (unstable, requires -Zunstable-options): Emit |
| machine-readable JSON information about timing information. |
| |
| Output Options |
| --target-dir directory |
| Directory for all generated artifacts and intermediate files. May |
| also be specified with the CARGO_TARGET_DIR environment variable, or |
| the build.target-dir config value |
| <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to |
| target in the root of the workspace. |
| |
| Display Options |
| By default the Rust test harness hides output from benchmark execution |
| to keep results readable. Benchmark output can be recovered (e.g., for |
| debugging) by passing --nocapture to the benchmark binaries: |
| |
| cargo bench -- --nocapture |
| |
| -v, --verbose |
| Use verbose output. May be specified twice for “very verbose” |
| output which includes extra output such as dependency warnings and |
| build script output. May also be specified with the term.verbose |
| config value |
| <https://doc.rust-lang.org/cargo/reference/config.html>. |
| |
| -q, --quiet |
| Do not print cargo log messages. May also be specified with the |
| term.quiet config value |
| <https://doc.rust-lang.org/cargo/reference/config.html>. |
| |
| --color when |
| Control when colored output is used. Valid values: |
| |
| o auto (default): Automatically detect if color support is |
| available on the terminal. |
| |
| o always: Always display colors. |
| |
| o never: Never display colors. |
| |
| May also be specified with the term.color config value |
| <https://doc.rust-lang.org/cargo/reference/config.html>. |
| |
| --message-format fmt |
| The output format for diagnostic messages. Can be specified multiple |
| times and consists of comma-separated values. Valid values: |
| |
| o human (default): Display in a human-readable text format. |
| Conflicts with short and json. |
| |
| o short: Emit shorter, human-readable text messages. Conflicts with |
| human and json. |
| |
| o json: Emit JSON messages to stdout. See the reference |
| <https://doc.rust-lang.org/cargo/reference/external-tools.html#json-messages> |
| for more details. Conflicts with human and short. |
| |
| o json-diagnostic-short: Ensure the rendered field of JSON messages |
| contains the “short” rendering from rustc. Cannot be used |
| with human or short. |
| |
| o json-diagnostic-rendered-ansi: Ensure the rendered field of JSON |
| messages contains embedded ANSI color codes for respecting |
| rustc’s default color scheme. Cannot be used with human or |
| short. |
| |
| o json-render-diagnostics: Instruct Cargo to not include rustc |
| diagnostics in JSON messages printed, but instead Cargo itself |
| should render the JSON diagnostics coming from rustc. Cargo’s |
| own JSON diagnostics and others coming from rustc are still |
| emitted. Cannot be used with human or short. |
| |
| Manifest Options |
| --manifest-path path |
| Path to the Cargo.toml file. By default, Cargo searches for the |
| Cargo.toml file in the current directory or any parent directory. |
| |
| --frozen, --locked |
| Either of these flags requires that the Cargo.lock file is |
| up-to-date. If the lock file is missing, or it needs to be updated, |
| Cargo will exit with an error. The --frozen flag also prevents Cargo |
| from attempting to access the network to determine if it is |
| out-of-date. |
| |
| These may be used in environments where you want to assert that the |
| Cargo.lock file is up-to-date (such as a CI build) or want to avoid |
| network access. |
| |
| --offline |
| Prevents Cargo from accessing the network for any reason. Without |
| this flag, Cargo will stop with an error if it needs to access the |
| network and the network is not available. With this flag, Cargo will |
| attempt to proceed without the network if possible. |
| |
| Beware that this may result in different dependency resolution than |
| online mode. Cargo will restrict itself to crates that are |
| downloaded locally, even if there might be a newer version as |
| indicated in the local copy of the index. See the cargo-fetch(1) |
| command to download dependencies before going offline. |
| |
| May also be specified with the net.offline config value |
| <https://doc.rust-lang.org/cargo/reference/config.html>. |
| |
| Common Options |
| +toolchain |
| If Cargo has been installed with rustup, and the first argument to |
| cargo begins with +, it will be interpreted as a rustup toolchain |
| name (such as +stable or +nightly). See the rustup documentation |
| <https://rust-lang.github.io/rustup/overrides.html> for more |
| information about how toolchain overrides work. |
| |
| --config KEY=VALUE or PATH |
| Overrides a Cargo configuration value. The argument should be in |
| TOML syntax of KEY=VALUE, or provided as a path to an extra |
| configuration file. This flag may be specified multiple times. See |
| the command-line overrides section |
| <https://doc.rust-lang.org/cargo/reference/config.html#command-line-overrides> |
| for more information. |
| |
| -C PATH |
| Changes the current working directory before executing any specified |
| operations. This affects things like where cargo looks by default |
| for the project manifest (Cargo.toml), as well as the directories |
| searched for discovering .cargo/config.toml, for example. This |
| option must appear before the command name, for example cargo -C |
| path/to/my-project build. |
| |
| This option is only available on the nightly channel |
| <https://doc.rust-lang.org/book/appendix-07-nightly-rust.html> and |
| requires the -Z unstable-options flag to enable (see #10098 |
| <https://github.com/rust-lang/cargo/issues/10098>). |
| |
| -h, --help |
| Prints help information. |
| |
| -Z flag |
| Unstable (nightly-only) flags to Cargo. Run cargo -Z help for |
| details. |
| |
| Miscellaneous Options |
| The --jobs argument affects the building of the benchmark executable but |
| does not affect how many threads are used when running the benchmarks. |
| The Rust test harness runs benchmarks serially in a single thread. |
| |
| -j N, --jobs N |
| Number of parallel jobs to run. May also be specified with the |
| build.jobs config value |
| <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to |
| the number of logical CPUs. If negative, it sets the maximum number |
| of parallel jobs to the number of logical CPUs plus provided value. |
| If a string default is provided, it sets the value back to defaults. |
| Should not be 0. |
| |
| While cargo bench involves compilation, it does not provide a |
| --keep-going flag. Use --no-fail-fast to run as many benchmarks as |
| possible without stopping at the first failure. To “compile” as many |
| benchmarks as possible, use --benches to build benchmark binaries |
| separately. For example: |
| |
| cargo build --benches --release --keep-going |
| cargo bench --no-fail-fast |
| |
| ENVIRONMENT |
| See the reference |
| <https://doc.rust-lang.org/cargo/reference/environment-variables.html> |
| for details on environment variables that Cargo reads. |
| |
| EXIT STATUS |
| o 0: Cargo succeeded. |
| |
| o 101: Cargo failed to complete. |
| |
| EXAMPLES |
| 1. Build and execute all the benchmarks of the current package: |
| |
| cargo bench |
| |
| 2. Run only a specific benchmark within a specific benchmark target: |
| |
| cargo bench --bench bench_name -- modname::some_benchmark |
| |
| SEE ALSO |
| cargo(1), cargo-test(1) |
| |