test-runner.1: modernise
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Closes #12125
This commit is contained in:
наб
Brian Behlendorf
parent
d44449025c
commit
4910903c07
|
|
@ -11,313 +11,241 @@
|
|||
.\"
|
||||
.\" Copyright (c) 2012 by Delphix. All rights reserved.
|
||||
.\"
|
||||
.TH run 1 "23 Sep 2012"
|
||||
.SH NAME
|
||||
run \- find, execute, and log the results of tests
|
||||
.SH SYNOPSIS
|
||||
.LP
|
||||
.nf
|
||||
\fBrun\fR [\fB-dgq] [\fB-o\fR \fIoutputdir\fR] [\fB-pP\fR \fIscript\fR] [\fB-t\fR \fIseconds\fR] [\fB-uxX\fR \fIusername\fR]
|
||||
\fIpathname\fR ...
|
||||
.fi
|
||||
|
||||
.LP
|
||||
.nf
|
||||
\fBrun\fR \fB-w\fR \fIrunfile\fR [\fB-gq\fR] [\fB-o\fR \fIoutputdir\fR] [\fB-pP\fR \fIscript\fR] [\fB-t\fR \fIseconds\fR]
|
||||
[\fB-uxX\fR \fIusername\fR] \fIpathname\fR ...
|
||||
.fi
|
||||
|
||||
.LP
|
||||
.nf
|
||||
\fBrun\fR \fB-c\fR \fIrunfile\fR [\fB-dq\fR]
|
||||
.fi
|
||||
|
||||
.LP
|
||||
.nf
|
||||
\fBrun\fR [\fB-h\fR]
|
||||
.fi
|
||||
|
||||
.SH DESCRIPTION
|
||||
.sp
|
||||
.LP
|
||||
The \fBrun\fR command has three basic modes of operation. With neither the
|
||||
\fB-c\fR nor the \fB-w\fR option, \fBrun\fR processes the arguments provided on
|
||||
.Dd March 10, 2020
|
||||
.Dt RUN 1
|
||||
.Os
|
||||
.
|
||||
.Sh NAME
|
||||
.Nm run
|
||||
.Nd find, execute, and log the results of tests
|
||||
.Sh SYNOPSIS
|
||||
.Nm
|
||||
.Op Fl dgq
|
||||
.Op Fl o Ar outputdir
|
||||
.Op Fl pP Ar script
|
||||
.Op Fl t seconds
|
||||
.Op Fl uxX Ar username
|
||||
.Ar pathname Ns No …
|
||||
.Pp
|
||||
.Nm
|
||||
.Fl w Ar runfile
|
||||
.Op Fl gq
|
||||
.Op Fl o Ar outputdir
|
||||
.Op Fl pP Ar script
|
||||
.Op Fl t seconds
|
||||
.Op Fl uxX Ar username
|
||||
.Ar pathname Ns No …
|
||||
.Pp
|
||||
.Nm
|
||||
.Fl c Ar runfile
|
||||
.Op Fl dq
|
||||
.Pp
|
||||
.Nm
|
||||
.Op Fl h
|
||||
.
|
||||
.Sh DESCRIPTION
|
||||
.Nm
|
||||
command has three basic modes of operation. With neither
|
||||
.Fl c
|
||||
nor
|
||||
.Fl w ,
|
||||
.Nm
|
||||
processes the arguments provided on
|
||||
the command line, adding them to the list for this run. If a specified
|
||||
\fIpathname\fR is an executable file, it is added as a test. If a specified
|
||||
\fIpathname\fR is a directory, the behavior depends upon the \fB-g\fR option.
|
||||
If \fB-g\fR is specified, the directory is treated as a test group. See the
|
||||
section on "Test Groups" below. Without the \fB-g\fR option, \fBrun\fR simply
|
||||
descends into the directory looking for executable files. The tests are then
|
||||
executed, and the results are logged.
|
||||
|
||||
With the \fB-w\fR option, \fBrun\fR finds tests in the manner described above.
|
||||
.Ar pathname
|
||||
is an executable file, it is added as a test. If a specified
|
||||
.Ar pathname
|
||||
is a directory, the behavior depends upon the presence of
|
||||
.Fl g .
|
||||
If
|
||||
.Fl g
|
||||
is specified, the directory is treated as a test group. See the
|
||||
section on
|
||||
.Sy Test Groups
|
||||
below. Without
|
||||
.Fl g ,
|
||||
.Nm
|
||||
simply descends into the directory looking for executable files.
|
||||
The tests are then executed, and the results are logged.
|
||||
.Pp
|
||||
With
|
||||
.Fl w ,
|
||||
.Nm
|
||||
finds tests in the manner described above.
|
||||
Rather than executing the tests and logging the results, the test configuration
|
||||
is stored in a \fIrunfile\fR which can be used in future invocations, or edited
|
||||
is stored in a
|
||||
.Ar runfile ,
|
||||
which can be used in future invocations, or edited
|
||||
to modify which tests are executed and which options are applied. Options
|
||||
included on the command line with \fB-w\fR become defaults in the
|
||||
\fIrunfile\fR.
|
||||
|
||||
With the \fB-c\fR option, \fBrun\fR parses a \fIrunfile\fR, which can specify a
|
||||
series of tests and test groups to be executed. The tests are then executed,
|
||||
and the results are logged.
|
||||
.sp
|
||||
.SS "Test Groups"
|
||||
.sp
|
||||
.LP
|
||||
included on the command line with
|
||||
.Fl w
|
||||
become defaults in the
|
||||
.Ar runfile .
|
||||
.Pp
|
||||
With
|
||||
.Fl c ,
|
||||
.Nm
|
||||
parses a
|
||||
.Ar runfile ,
|
||||
which can specify a series of tests and test groups to be executed.
|
||||
The tests are then executed, and the results are logged.
|
||||
.
|
||||
.Ss Test Groups
|
||||
A test group is comprised of a set of executable files, all of which exist in
|
||||
one directory. The options specified on the command line or in a \fIrunfile\fR
|
||||
one directory. The options specified on the command line or in a
|
||||
.Ar runfile
|
||||
apply to individual tests in the group. The exception is options pertaining to
|
||||
pre and post scripts, which act on all tests as a group. Rather than running
|
||||
before and after each test, these scripts are run only once each at the start
|
||||
and end of the test group.
|
||||
.SS "Test Execution"
|
||||
.sp
|
||||
.LP
|
||||
.Ss Test Execution
|
||||
The specified tests run serially, and are typically assigned results according
|
||||
to exit values. Tests that exit zero and non-zero are marked "PASS" and "FAIL"
|
||||
to exit values. Tests that exit zero and non-zero are marked
|
||||
.Sy PASS
|
||||
and
|
||||
.Sy FAIL ,
|
||||
respectively. When a pre script fails for a test group, only the post script is
|
||||
executed, and the remaining tests are marked "SKIPPED." Any test that exceeds
|
||||
its \fItimeout\fR is terminated, and marked "KILLED."
|
||||
|
||||
By default, tests are executed with the credentials of the \fBrun\fR script.
|
||||
Executing tests with other credentials is done via \fBsudo\fR(1m), which must
|
||||
executed, and the remaining tests are marked
|
||||
.Sy SKIPPED .
|
||||
Any test that exceeds
|
||||
its
|
||||
.Ar timeout
|
||||
is terminated, and marked
|
||||
.Sy KILLED .
|
||||
.Pp
|
||||
By default, tests are executed with the credentials of the
|
||||
.Sy
|
||||
script.
|
||||
Executing tests with other credentials is done via
|
||||
.Xr sudo 1m ,
|
||||
which must
|
||||
be configured to allow execution without prompting for a password. Environment
|
||||
variables from the calling shell are available to individual tests. During test
|
||||
execution, the working directory is changed to \fIoutputdir\fR.
|
||||
.SS "Output Logging"
|
||||
.sp
|
||||
.LP
|
||||
By default, \fBrun\fR will print one line on standard output at the conclusion
|
||||
execution, the working directory is changed to
|
||||
.Ar outputdir .
|
||||
.
|
||||
.Ss Output Logging
|
||||
By default,
|
||||
.Nm
|
||||
will print one line on standard output at the conclusion
|
||||
of each test indicating the test name, result and elapsed time. Additionally,
|
||||
for each invocation of \fBrun\fR, a directory is created using the ISO 8601
|
||||
date format. Within this directory is a file named \fIlog\fR containing all the
|
||||
for each invocation of
|
||||
.Nm ,
|
||||
a directory is created using the ISO 8601
|
||||
date format. Within this directory is a file named
|
||||
.Sy log
|
||||
containing all the
|
||||
test output with timestamps, and a directory for each test. Within the test
|
||||
directories, there is one file each for standard output, standard error and
|
||||
merged output. The default location for the \fIoutputdir\fR is
|
||||
\fI/var/tmp/test_results\fR.
|
||||
.SS "Runfiles"
|
||||
.sp
|
||||
.LP
|
||||
The \fIrunfile\fR is an ini style configuration file that describes a test run.
|
||||
The file has one section named "DEFAULT," which contains configuration option
|
||||
names and their values in "name = value" format. The values in this section
|
||||
merged output. The default location for the
|
||||
.Ar outputdir
|
||||
is
|
||||
.Pa /var/tmp/test_results .
|
||||
.Ss "Runfiles"
|
||||
The
|
||||
.Ar runfile
|
||||
is an INI-style configuration file that describes a test run.
|
||||
The file has one section named
|
||||
.Sy DEFAULT ,
|
||||
which contains configuration option
|
||||
names and their values in
|
||||
.Sy name No = Ar value
|
||||
format. The values in this section
|
||||
apply to all the subsequent sections, unless they are also specified there, in
|
||||
which case the default is overridden. The remaining section names are the
|
||||
absolute pathnames of files and directories, describing tests and test groups
|
||||
respectively. The legal option names are:
|
||||
.sp
|
||||
.ne 2
|
||||
.na
|
||||
\fBoutputdir\fR = \fIpathname\fR
|
||||
.ad
|
||||
.sp .6
|
||||
.RS 4n
|
||||
.Bl -tag -width "tests = ['filename', …]"
|
||||
.It Sy outputdir No = Ar pathname
|
||||
The name of the directory that holds test logs.
|
||||
.RE
|
||||
.sp
|
||||
.ne 2
|
||||
.na
|
||||
\fBpre\fR = \fIscript\fR
|
||||
.ad
|
||||
.sp .6
|
||||
.RS 4n
|
||||
Run \fIscript\fR prior to the test or test group.
|
||||
.RE
|
||||
.sp
|
||||
.ne 2
|
||||
.na
|
||||
\fBpre_user\fR = \fIusername\fR
|
||||
.ad
|
||||
.sp .6
|
||||
.RS 4n
|
||||
Execute the pre script as \fIusername\fR.
|
||||
.RE
|
||||
.sp
|
||||
.ne 2
|
||||
.na
|
||||
\fBpost\fR = \fIscript\fR
|
||||
.ad
|
||||
.sp .6
|
||||
.RS 4n
|
||||
Run \fIscript\fR after the test or test group.
|
||||
.RE
|
||||
.sp
|
||||
.ne 2
|
||||
.na
|
||||
\fBpost_user\fR = \fIusername\fR
|
||||
.ad
|
||||
.sp .6
|
||||
.RS 4n
|
||||
Execute the post script as \fIusername\fR.
|
||||
.RE
|
||||
.sp
|
||||
.ne 2
|
||||
.na
|
||||
\fBquiet\fR = [\fITrue\fR|\fIFalse\fR]
|
||||
.ad
|
||||
.sp .6
|
||||
.RS 4n
|
||||
If set to True, only the results summary is printed to standard out.
|
||||
.RE
|
||||
.sp
|
||||
.ne 2
|
||||
.na
|
||||
\fBtests\fR = [\fI'filename'\fR [,...]]
|
||||
.ad
|
||||
.sp .6
|
||||
.RS 4n
|
||||
Specify a list of \fIfilenames\fR for this test group. Only the basename of the
|
||||
.It Sy pre No = Ar script
|
||||
Run
|
||||
.Ar script
|
||||
prior to the test or test group.
|
||||
.It Sy pre_user No = Ar username
|
||||
Execute the pre script as
|
||||
.Ar username .
|
||||
.It Sy post No = Ar script
|
||||
Run
|
||||
.Ar script
|
||||
after the test or test group.
|
||||
.It Sy post_user No = Ar username
|
||||
Execute the post script as
|
||||
.Ar username .
|
||||
.It Sy quiet No = Sy True Ns | Ns Sy False
|
||||
If
|
||||
.Sy True ,
|
||||
only the results summary is printed to standard out.
|
||||
.It Sy tests No = [ Ns Ar 'filename' Ns , …]
|
||||
Specify a list of
|
||||
.Ar filenames
|
||||
for this test group. Only the basename of the
|
||||
absolute path is required. This option is only valid for test groups, and each
|
||||
\fIfilename\fR must be single quoted.
|
||||
.RE
|
||||
.sp
|
||||
.ne 2
|
||||
.na
|
||||
\fBtimeout\fR = \fIn\fR
|
||||
.ad
|
||||
.sp .6
|
||||
.RS 4n
|
||||
A timeout value of \fIn\fR seconds.
|
||||
.RE
|
||||
.sp
|
||||
.ne 2
|
||||
.na
|
||||
\fBuser\fR = \fIusername\fR
|
||||
.ad
|
||||
.sp .6
|
||||
.RS 4n
|
||||
Execute the test or test group as \fIusername\fR.
|
||||
.RE
|
||||
|
||||
.SH OPTIONS
|
||||
.sp
|
||||
.LP
|
||||
The following options are available for the \fBrun\fR command.
|
||||
.sp
|
||||
.ne 2
|
||||
.na
|
||||
\fB-c\fR \fIrunfile\fR
|
||||
.ad
|
||||
.RS 6n
|
||||
Specify a \fIrunfile\fR to be consumed by the run command.
|
||||
.RE
|
||||
|
||||
.ne 2
|
||||
.na
|
||||
\fB-d\fR
|
||||
.ad
|
||||
.RS 6n
|
||||
.Ar filename
|
||||
must be single quoted.
|
||||
.It Sy timeout No = Ar n
|
||||
A timeout value of
|
||||
.Ar n
|
||||
seconds.
|
||||
.It Sy user No = Ar username
|
||||
Execute the test or test group as
|
||||
.Ar username .
|
||||
.El
|
||||
.
|
||||
.Sh OPTIONS
|
||||
.Bl -tag -width "-o outputdir"
|
||||
.It Fl c Ar runfile
|
||||
Specify a
|
||||
.Ar runfile
|
||||
to be consumed by the run command.
|
||||
.It Fl d
|
||||
Dry run mode. Execute no tests, but print a description of each test that would
|
||||
have been run.
|
||||
.RE
|
||||
|
||||
.ne 2
|
||||
.na
|
||||
\fB-g\fR
|
||||
.ad
|
||||
.RS 6n
|
||||
.It Fl g
|
||||
Create test groups from any directories found while searching for tests.
|
||||
.RE
|
||||
|
||||
.ne 2
|
||||
.na
|
||||
\fB-o\fR \fIoutputdir\fR
|
||||
.ad
|
||||
.RS 6n
|
||||
.It Fl o Ar outputdir
|
||||
Specify the directory in which to write test results.
|
||||
.RE
|
||||
|
||||
.ne 2
|
||||
.na
|
||||
\fB-p\fR \fIscript\fR
|
||||
.ad
|
||||
.RS 6n
|
||||
Run \fIscript\fR prior to any test or test group.
|
||||
.RE
|
||||
|
||||
.ne 2
|
||||
.na
|
||||
\fB-P\fR \fIscript\fR
|
||||
.ad
|
||||
.RS 6n
|
||||
Run \fIscript\fR after any test or test group.
|
||||
.RE
|
||||
|
||||
.ne 2
|
||||
.na
|
||||
\fB-q\fR
|
||||
.ad
|
||||
.RS 6n
|
||||
.It Fl p Ar script
|
||||
Run
|
||||
.Ar script
|
||||
prior to any test or test group.
|
||||
.It Fl P Ar script
|
||||
Run
|
||||
.Ar script
|
||||
after any test or test group.
|
||||
.It Fl q
|
||||
Print only the results summary to the standard output.
|
||||
.RE
|
||||
|
||||
.ne 2
|
||||
.na
|
||||
\fB-s\fR \fIscript\fR
|
||||
.ad
|
||||
.RS 6n
|
||||
Run \fIscript\fR as a failsafe after any test is killed.
|
||||
.RE
|
||||
|
||||
.ne 2
|
||||
.na
|
||||
\fB-S\fR \fIusername\fR
|
||||
.ad
|
||||
.RS 6n
|
||||
Execute the failsafe script as \fIusername\fR.
|
||||
.RE
|
||||
|
||||
.ne 2
|
||||
.na
|
||||
\fB-t\fR \fIn\fR
|
||||
.ad
|
||||
.RS 6n
|
||||
Specify a timeout value of \fIn\fR seconds per test.
|
||||
.RE
|
||||
|
||||
.ne 2
|
||||
.na
|
||||
\fB-u\fR \fIusername\fR
|
||||
.ad
|
||||
.RS 6n
|
||||
Execute tests or test groups as \fIusername\fR.
|
||||
.RE
|
||||
|
||||
.ne 2
|
||||
.na
|
||||
\fB-w\fR \fIrunfile\fR
|
||||
.ad
|
||||
.RS 6n
|
||||
Specify the name of the \fIrunfile\fR to create.
|
||||
.RE
|
||||
|
||||
.ne 2
|
||||
.na
|
||||
\fB-x\fR \fIusername\fR
|
||||
.ad
|
||||
.RS 6n
|
||||
Execute the pre script as \fIusername\fR.
|
||||
.RE
|
||||
|
||||
.ne 2
|
||||
.na
|
||||
\fB-X\fR \fIusername\fR
|
||||
.ad
|
||||
.RS 6n
|
||||
Execute the post script as \fIusername\fR.
|
||||
.RE
|
||||
|
||||
.SH EXAMPLES
|
||||
.LP
|
||||
\fBExample 1\fR Running ad-hoc tests.
|
||||
.sp
|
||||
.LP
|
||||
This example demonstrates the simplest invocation of \fBrun\fR.
|
||||
|
||||
.sp
|
||||
.in +2
|
||||
.nf
|
||||
% \fBrun my-tests\fR
|
||||
.It Fl s Ar script
|
||||
Run
|
||||
.Ar script as a failsafe after any test is killed.
|
||||
.It Fl S Ar username
|
||||
Execute the failsafe script as
|
||||
.Ar username .
|
||||
.It Fl t Ar n
|
||||
Specify a timeout value of
|
||||
.Ar n seconds per test.
|
||||
.It Fl u Ar username
|
||||
Execute tests or test groups as
|
||||
.Ar username .
|
||||
.It Fl w Ar runfile
|
||||
Specify the name of the
|
||||
.Ar runfile
|
||||
to create.
|
||||
.It Fl x Ar username
|
||||
Execute the pre script as
|
||||
.Ar username .
|
||||
.It Fl X Ar username
|
||||
Execute the post script as
|
||||
.Ar username .
|
||||
.El
|
||||
.
|
||||
.Sh EXAMPLES
|
||||
.Bl -tag -width "-h"
|
||||
.It Sy Example 1 Ns : Running ad-hoc tests.
|
||||
This example demonstrates the simplest invocation of
|
||||
.Nm .
|
||||
.Bd -literal
|
||||
.No % Nm run Ar my-tests
|
||||
Test: /home/jkennedy/my-tests/test-01 [00:02] [PASS]
|
||||
Test: /home/jkennedy/my-tests/test-02 [00:04] [PASS]
|
||||
Test: /home/jkennedy/my-tests/test-03 [00:01] [PASS]
|
||||
|
|
@ -328,20 +256,14 @@ PASS 3
|
|||
Running Time: 00:00:07
|
||||
Percent passed: 100.0%
|
||||
Log directory: /var/tmp/test_results/20120923T180654
|
||||
.fi
|
||||
.in -2
|
||||
|
||||
.LP
|
||||
\fBExample 2\fR Creating a \fIrunfile\fR for future use.
|
||||
.sp
|
||||
.LP
|
||||
This example demonstrates creating a \fIrunfile\fR with non default options.
|
||||
|
||||
.sp
|
||||
.in +2
|
||||
.nf
|
||||
% \fBrun -p setup -x root -g -w new-tests.run new-tests\fR
|
||||
% \fBcat new-tests.run\fR
|
||||
.Ed
|
||||
.It Sy Example 2 Ns : Creating a Ar runfile No for future use.
|
||||
This example demonstrates creating a
|
||||
.Ar runfile
|
||||
with non-default options.
|
||||
.Bd -literal
|
||||
.No % Nm run Fl p Ar setup Fl x Ar root Fl g Fl w Ar new-tests.run Ar new-tests
|
||||
.No % Nm cat Pa new-tests.run
|
||||
[DEFAULT]
|
||||
pre = setup
|
||||
post_user =
|
||||
|
|
@ -354,33 +276,8 @@ outputdir = /var/tmp/test_results
|
|||
|
||||
[/home/jkennedy/new-tests]
|
||||
tests = ['test-01', 'test-02', 'test-03']
|
||||
.fi
|
||||
.in -2
|
||||
|
||||
.SH EXIT STATUS
|
||||
.sp
|
||||
.LP
|
||||
The following exit values are returned:
|
||||
.sp
|
||||
.ne 2
|
||||
.na
|
||||
\fB\fB0\fR\fR
|
||||
.ad
|
||||
.sp .6
|
||||
.RS 4n
|
||||
Successful completion.
|
||||
.RE
|
||||
.sp
|
||||
.ne 2
|
||||
.na
|
||||
\fB\fB1\fR\fR
|
||||
.ad
|
||||
.sp .6
|
||||
.RS 4n
|
||||
An error occurred.
|
||||
.RE
|
||||
|
||||
.SH SEE ALSO
|
||||
.sp
|
||||
.LP
|
||||
\fBsudo\fR(1m)
|
||||
.Ed
|
||||
.El
|
||||
.
|
||||
.Sh SEE ALSO
|
||||
.Xr sudo 1m
|
||||
|
|
|
|||
Loading…
Reference in New Issue