295 lines
7.2 KiB
Groff
295 lines
7.2 KiB
Groff
.\"
|
|
.\" This file and its contents are supplied under the terms of the
|
|
.\" Common Development and Distribution License ("CDDL"), version 1.0.
|
|
.\" You may only use this file in accordance with the terms of version
|
|
.\" 1.0 of the CDDL.
|
|
.\"
|
|
.\" A full copy of the text of the CDDL should have accompanied this
|
|
.\" source. A copy of the CDDL is also available via the Internet at
|
|
.\" http://www.illumos.org/license/CDDL.
|
|
.\"
|
|
.\" Copyright (c) 2012 by Delphix. All rights reserved.
|
|
.\"
|
|
.Dd May 26, 2021
|
|
.Dt RUN 1
|
|
.Os
|
|
.
|
|
.Sh NAME
|
|
.Nm run
|
|
.Nd find, execute, and log the results of tests
|
|
.Sh SYNOPSIS
|
|
.Nm
|
|
.Op Fl dgq
|
|
.Op Fl o Ar outputdir
|
|
.Op Fl pP Ar script
|
|
.Op Fl t seconds
|
|
.Op Fl uxX Ar username
|
|
.Ar pathname Ns No …
|
|
.Pp
|
|
.Nm
|
|
.Fl w Ar runfile
|
|
.Op Fl gq
|
|
.Op Fl o Ar outputdir
|
|
.Op Fl pP Ar script
|
|
.Op Fl t seconds
|
|
.Op Fl uxX Ar username
|
|
.Ar pathname Ns No …
|
|
.Pp
|
|
.Nm
|
|
.Fl c Ar runfile
|
|
.Op Fl dq
|
|
.Pp
|
|
.Nm
|
|
.Op Fl h
|
|
.
|
|
.Sh DESCRIPTION
|
|
.Nm
|
|
command has three basic modes of operation.
|
|
With neither
|
|
.Fl c
|
|
nor
|
|
.Fl w ,
|
|
.Nm
|
|
processes the arguments provided on
|
|
the command line, adding them to the list for this run.
|
|
If a specified
|
|
.Ar pathname
|
|
is an executable file, it is added as a test.
|
|
If a specified
|
|
.Ar pathname
|
|
is a directory, the behavior depends upon the presence of
|
|
.Fl g .
|
|
If
|
|
.Fl g
|
|
is specified, the directory is treated as a test group.
|
|
See the section on
|
|
.Sy Test Groups
|
|
below.
|
|
Without
|
|
.Fl g ,
|
|
.Nm
|
|
simply descends into the directory looking for executable files.
|
|
The tests are then executed, and the results are logged.
|
|
.Pp
|
|
With
|
|
.Fl w ,
|
|
.Nm
|
|
finds tests in the manner described above.
|
|
Rather than executing the tests and logging the results, the test configuration
|
|
is stored in a
|
|
.Ar runfile ,
|
|
which can be used in future invocations, or edited
|
|
to modify which tests are executed and which options are applied.
|
|
Options included on the command line with
|
|
.Fl w
|
|
become defaults in the
|
|
.Ar runfile .
|
|
.Pp
|
|
With
|
|
.Fl c ,
|
|
.Nm
|
|
parses a
|
|
.Ar runfile ,
|
|
which can specify a series of tests and test groups to be executed.
|
|
The tests are then executed, and the results are logged.
|
|
.
|
|
.Ss Test Groups
|
|
A test group is comprised of a set of executable files, all of which exist in
|
|
one directory.
|
|
The options specified on the command line or in a
|
|
.Ar runfile
|
|
apply to individual tests in the group.
|
|
The exception is options pertaining to pre and post scripts, which act on all tests as a group.
|
|
Rather than running before and after each test,
|
|
these scripts are run only once each at the start and end of the test group.
|
|
.Ss Test Execution
|
|
The specified tests run serially, and are typically assigned results according
|
|
to exit values.
|
|
Tests that exit zero and non-zero are marked
|
|
.Sy PASS
|
|
and
|
|
.Sy FAIL ,
|
|
respectively.
|
|
When a pre script fails for a test group, only the post script is executed,
|
|
and the remaining tests are marked
|
|
.Sy SKIPPED .
|
|
Any test that exceeds
|
|
its
|
|
.Ar timeout
|
|
is terminated, and marked
|
|
.Sy KILLED .
|
|
.Pp
|
|
By default, tests are executed with the credentials of the
|
|
.Nm
|
|
script.
|
|
Executing tests with other credentials is done via
|
|
.Xr sudo 1m ,
|
|
which must
|
|
be configured to allow execution without prompting for a password.
|
|
Environment variables from the calling shell are available to individual tests.
|
|
During test execution, the working directory is changed to
|
|
.Ar outputdir .
|
|
.
|
|
.Ss Output Logging
|
|
By default,
|
|
.Nm
|
|
will print one line on standard output at the conclusion
|
|
of each test indicating the test name, result and elapsed time.
|
|
Additionally, for each invocation of
|
|
.Nm ,
|
|
a directory is created using the ISO 8601 date format.
|
|
Within this directory is a file named
|
|
.Sy log
|
|
containing all the
|
|
test output with timestamps, and a directory for each test.
|
|
Within the test directories, there is one file each for standard output,
|
|
standard error and merged output.
|
|
The default location for the
|
|
.Ar outputdir
|
|
is
|
|
.Pa /var/tmp/test_results .
|
|
.Ss "Runfiles"
|
|
The
|
|
.Ar runfile
|
|
is an INI-style configuration file that describes a test run.
|
|
The file has one section named
|
|
.Sy DEFAULT ,
|
|
which contains configuration option
|
|
names and their values in
|
|
.Sy name No = Ar value
|
|
format.
|
|
The values in this section apply to all the subsequent sections,
|
|
unless they are also specified there, in which case the default is overridden.
|
|
The remaining section names are the absolute pathnames of files and directories,
|
|
describing tests and test groups respectively.
|
|
The legal option names are:
|
|
.Bl -tag -width "tests = ['filename', …]"
|
|
.It Sy outputdir No = Ar pathname
|
|
The name of the directory that holds test logs.
|
|
.It Sy pre No = Ar script
|
|
Run
|
|
.Ar script
|
|
prior to the test or test group.
|
|
.It Sy pre_user No = Ar username
|
|
Execute the pre script as
|
|
.Ar username .
|
|
.It Sy post No = Ar script
|
|
Run
|
|
.Ar script
|
|
after the test or test group.
|
|
.It Sy post_user No = Ar username
|
|
Execute the post script as
|
|
.Ar username .
|
|
.It Sy quiet No = Sy True Ns | Ns Sy False
|
|
If
|
|
.Sy True ,
|
|
only the results summary is printed to standard out.
|
|
.It Sy tests No = [ Ns Ar 'filename' , No … ]
|
|
Specify a list of
|
|
.Ar filenames
|
|
for this test group.
|
|
Only the basename of the absolute path is required.
|
|
This option is only valid for test groups, and each
|
|
.Ar filename
|
|
must be single quoted.
|
|
.It Sy timeout No = Ar n
|
|
A timeout value of
|
|
.Ar n
|
|
seconds.
|
|
.It Sy user No = Ar username
|
|
Execute the test or test group as
|
|
.Ar username .
|
|
.El
|
|
.
|
|
.Sh OPTIONS
|
|
.Bl -tag -width "-o outputdir"
|
|
.It Fl c Ar runfile
|
|
Specify a
|
|
.Ar runfile
|
|
to be consumed by the run command.
|
|
.It Fl d
|
|
Dry run mode.
|
|
Execute no tests, but print a description of each test that would have been run.
|
|
.It Fl g
|
|
Create test groups from any directories found while searching for tests.
|
|
.It Fl o Ar outputdir
|
|
Specify the directory in which to write test results.
|
|
.It Fl p Ar script
|
|
Run
|
|
.Ar script
|
|
prior to any test or test group.
|
|
.It Fl P Ar script
|
|
Run
|
|
.Ar script
|
|
after any test or test group.
|
|
.It Fl q
|
|
Print only the results summary to the standard output.
|
|
.It Fl s Ar script
|
|
Run
|
|
.Ar script
|
|
as a failsafe after any test is killed.
|
|
.It Fl S Ar username
|
|
Execute the failsafe script as
|
|
.Ar username .
|
|
.It Fl t Ar n
|
|
Specify a timeout value of
|
|
.Ar n
|
|
seconds per test.
|
|
.It Fl u Ar username
|
|
Execute tests or test groups as
|
|
.Ar username .
|
|
.It Fl w Ar runfile
|
|
Specify the name of the
|
|
.Ar runfile
|
|
to create.
|
|
.It Fl x Ar username
|
|
Execute the pre script as
|
|
.Ar username .
|
|
.It Fl X Ar username
|
|
Execute the post script as
|
|
.Ar username .
|
|
.El
|
|
.
|
|
.Sh EXAMPLES
|
|
.Bl -tag -width "-h"
|
|
.It Sy Example 1 : No Running ad-hoc tests.
|
|
This example demonstrates the simplest invocation of
|
|
.Nm .
|
|
.Bd -literal
|
|
.No % Nm run Ar my-tests
|
|
Test: /home/jkennedy/my-tests/test-01 [00:02] [PASS]
|
|
Test: /home/jkennedy/my-tests/test-02 [00:04] [PASS]
|
|
Test: /home/jkennedy/my-tests/test-03 [00:01] [PASS]
|
|
|
|
Results Summary
|
|
PASS 3
|
|
|
|
Running Time: 00:00:07
|
|
Percent passed: 100.0%
|
|
Log directory: /var/tmp/test_results/20120923T180654
|
|
.Ed
|
|
.It Sy Example 2 : No Creating a Ar runfile No for future use.
|
|
This example demonstrates creating a
|
|
.Ar runfile
|
|
with non-default options.
|
|
.Bd -literal
|
|
.No % Nm run Fl p Ar setup Fl x Ar root Fl g Fl w Ar new-tests.run Ar new-tests
|
|
.No % Nm cat Pa new-tests.run
|
|
[DEFAULT]
|
|
pre = setup
|
|
post_user =
|
|
quiet = False
|
|
user =
|
|
timeout = 60
|
|
post =
|
|
pre_user = root
|
|
outputdir = /var/tmp/test_results
|
|
|
|
[/home/jkennedy/new-tests]
|
|
tests = ['test-01', 'test-02', 'test-03']
|
|
.Ed
|
|
.El
|
|
.
|
|
.Sh SEE ALSO
|
|
.Xr sudo 1m
|