From 4910903c076527252057dce2b00cd20d8264a2f2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=BD=D0=B0=D0=B1?= Date: Wed, 26 May 2021 20:15:21 +0200 Subject: [PATCH] test-runner.1: modernise MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #12125 --- tests/test-runner/man/test-runner.1 | 547 +++++++++++----------------- 1 file changed, 222 insertions(+), 325 deletions(-) diff --git a/tests/test-runner/man/test-runner.1 b/tests/test-runner/man/test-runner.1 index 19c636c8cb..039a440df5 100644 --- a/tests/test-runner/man/test-runner.1 +++ b/tests/test-runner/man/test-runner.1 @@ -11,313 +11,241 @@ .\" .\" Copyright (c) 2012 by Delphix. All rights reserved. .\" -.TH run 1 "23 Sep 2012" -.SH NAME -run \- find, execute, and log the results of tests -.SH SYNOPSIS -.LP -.nf -\fBrun\fR [\fB-dgq] [\fB-o\fR \fIoutputdir\fR] [\fB-pP\fR \fIscript\fR] [\fB-t\fR \fIseconds\fR] [\fB-uxX\fR \fIusername\fR] - \fIpathname\fR ... -.fi - -.LP -.nf -\fBrun\fR \fB-w\fR \fIrunfile\fR [\fB-gq\fR] [\fB-o\fR \fIoutputdir\fR] [\fB-pP\fR \fIscript\fR] [\fB-t\fR \fIseconds\fR] - [\fB-uxX\fR \fIusername\fR] \fIpathname\fR ... -.fi - -.LP -.nf -\fBrun\fR \fB-c\fR \fIrunfile\fR [\fB-dq\fR] -.fi - -.LP -.nf -\fBrun\fR [\fB-h\fR] -.fi - -.SH DESCRIPTION -.sp -.LP -The \fBrun\fR command has three basic modes of operation. With neither the -\fB-c\fR nor the \fB-w\fR option, \fBrun\fR processes the arguments provided on +.Dd March 10, 2020 +.Dt RUN 1 +.Os +. +.Sh NAME +.Nm run +.Nd find, execute, and log the results of tests +.Sh SYNOPSIS +.Nm +.Op Fl dgq +.Op Fl o Ar outputdir +.Op Fl pP Ar script +.Op Fl t seconds +.Op Fl uxX Ar username +.Ar pathname Ns No … +.Pp +.Nm +.Fl w Ar runfile +.Op Fl gq +.Op Fl o Ar outputdir +.Op Fl pP Ar script +.Op Fl t seconds +.Op Fl uxX Ar username +.Ar pathname Ns No … +.Pp +.Nm +.Fl c Ar runfile +.Op Fl dq +.Pp +.Nm +.Op Fl h +. +.Sh DESCRIPTION +.Nm +command has three basic modes of operation. With neither +.Fl c +nor +.Fl w , +.Nm +processes the arguments provided on the command line, adding them to the list for this run. If a specified -\fIpathname\fR is an executable file, it is added as a test. If a specified -\fIpathname\fR is a directory, the behavior depends upon the \fB-g\fR option. -If \fB-g\fR is specified, the directory is treated as a test group. See the -section on "Test Groups" below. Without the \fB-g\fR option, \fBrun\fR simply -descends into the directory looking for executable files. The tests are then -executed, and the results are logged. - -With the \fB-w\fR option, \fBrun\fR finds tests in the manner described above. +.Ar pathname +is an executable file, it is added as a test. If a specified +.Ar pathname +is a directory, the behavior depends upon the presence of +.Fl g . +If +.Fl g +is specified, the directory is treated as a test group. See the +section on +.Sy Test Groups +below. Without +.Fl g , +.Nm +simply descends into the directory looking for executable files. +The tests are then executed, and the results are logged. +.Pp +With +.Fl w , +.Nm +finds tests in the manner described above. Rather than executing the tests and logging the results, the test configuration -is stored in a \fIrunfile\fR which can be used in future invocations, or edited +is stored in a +.Ar runfile , +which can be used in future invocations, or edited to modify which tests are executed and which options are applied. Options -included on the command line with \fB-w\fR become defaults in the -\fIrunfile\fR. - -With the \fB-c\fR option, \fBrun\fR parses a \fIrunfile\fR, which can specify a -series of tests and test groups to be executed. The tests are then executed, -and the results are logged. -.sp -.SS "Test Groups" -.sp -.LP +included on the command line with +.Fl w +become defaults in the +.Ar runfile . +.Pp +With +.Fl c , +.Nm +parses a +.Ar runfile , +which can specify a series of tests and test groups to be executed. +The tests are then executed, and the results are logged. +. +.Ss Test Groups A test group is comprised of a set of executable files, all of which exist in -one directory. The options specified on the command line or in a \fIrunfile\fR +one directory. The options specified on the command line or in a +.Ar runfile apply to individual tests in the group. The exception is options pertaining to pre and post scripts, which act on all tests as a group. Rather than running before and after each test, these scripts are run only once each at the start and end of the test group. -.SS "Test Execution" -.sp -.LP +.Ss Test Execution The specified tests run serially, and are typically assigned results according -to exit values. Tests that exit zero and non-zero are marked "PASS" and "FAIL" +to exit values. Tests that exit zero and non-zero are marked +.Sy PASS +and +.Sy FAIL , respectively. When a pre script fails for a test group, only the post script is -executed, and the remaining tests are marked "SKIPPED." Any test that exceeds -its \fItimeout\fR is terminated, and marked "KILLED." - -By default, tests are executed with the credentials of the \fBrun\fR script. -Executing tests with other credentials is done via \fBsudo\fR(1m), which must +executed, and the remaining tests are marked +.Sy SKIPPED . +Any test that exceeds +its +.Ar timeout +is terminated, and marked +.Sy KILLED . +.Pp +By default, tests are executed with the credentials of the +.Sy +script. +Executing tests with other credentials is done via +.Xr sudo 1m , +which must be configured to allow execution without prompting for a password. Environment variables from the calling shell are available to individual tests. During test -execution, the working directory is changed to \fIoutputdir\fR. -.SS "Output Logging" -.sp -.LP -By default, \fBrun\fR will print one line on standard output at the conclusion +execution, the working directory is changed to +.Ar outputdir . +. +.Ss Output Logging +By default, +.Nm +will print one line on standard output at the conclusion of each test indicating the test name, result and elapsed time. Additionally, -for each invocation of \fBrun\fR, a directory is created using the ISO 8601 -date format. Within this directory is a file named \fIlog\fR containing all the +for each invocation of +.Nm , +a directory is created using the ISO 8601 +date format. Within this directory is a file named +.Sy log +containing all the test output with timestamps, and a directory for each test. Within the test directories, there is one file each for standard output, standard error and -merged output. The default location for the \fIoutputdir\fR is -\fI/var/tmp/test_results\fR. -.SS "Runfiles" -.sp -.LP -The \fIrunfile\fR is an ini style configuration file that describes a test run. -The file has one section named "DEFAULT," which contains configuration option -names and their values in "name = value" format. The values in this section +merged output. The default location for the +.Ar outputdir +is +.Pa /var/tmp/test_results . +.Ss "Runfiles" +The +.Ar runfile +is an INI-style configuration file that describes a test run. +The file has one section named +.Sy DEFAULT , +which contains configuration option +names and their values in +.Sy name No = Ar value +format. The values in this section apply to all the subsequent sections, unless they are also specified there, in which case the default is overridden. The remaining section names are the absolute pathnames of files and directories, describing tests and test groups respectively. The legal option names are: -.sp -.ne 2 -.na -\fBoutputdir\fR = \fIpathname\fR -.ad -.sp .6 -.RS 4n +.Bl -tag -width "tests = ['filename', …]" +.It Sy outputdir No = Ar pathname The name of the directory that holds test logs. -.RE -.sp -.ne 2 -.na -\fBpre\fR = \fIscript\fR -.ad -.sp .6 -.RS 4n -Run \fIscript\fR prior to the test or test group. -.RE -.sp -.ne 2 -.na -\fBpre_user\fR = \fIusername\fR -.ad -.sp .6 -.RS 4n -Execute the pre script as \fIusername\fR. -.RE -.sp -.ne 2 -.na -\fBpost\fR = \fIscript\fR -.ad -.sp .6 -.RS 4n -Run \fIscript\fR after the test or test group. -.RE -.sp -.ne 2 -.na -\fBpost_user\fR = \fIusername\fR -.ad -.sp .6 -.RS 4n -Execute the post script as \fIusername\fR. -.RE -.sp -.ne 2 -.na -\fBquiet\fR = [\fITrue\fR|\fIFalse\fR] -.ad -.sp .6 -.RS 4n -If set to True, only the results summary is printed to standard out. -.RE -.sp -.ne 2 -.na -\fBtests\fR = [\fI'filename'\fR [,...]] -.ad -.sp .6 -.RS 4n -Specify a list of \fIfilenames\fR for this test group. Only the basename of the +.It Sy pre No = Ar script +Run +.Ar script +prior to the test or test group. +.It Sy pre_user No = Ar username +Execute the pre script as +.Ar username . +.It Sy post No = Ar script +Run +.Ar script +after the test or test group. +.It Sy post_user No = Ar username +Execute the post script as +.Ar username . +.It Sy quiet No = Sy True Ns | Ns Sy False +If +.Sy True , +only the results summary is printed to standard out. +.It Sy tests No = [ Ns Ar 'filename' Ns , …] +Specify a list of +.Ar filenames +for this test group. Only the basename of the absolute path is required. This option is only valid for test groups, and each -\fIfilename\fR must be single quoted. -.RE -.sp -.ne 2 -.na -\fBtimeout\fR = \fIn\fR -.ad -.sp .6 -.RS 4n -A timeout value of \fIn\fR seconds. -.RE -.sp -.ne 2 -.na -\fBuser\fR = \fIusername\fR -.ad -.sp .6 -.RS 4n -Execute the test or test group as \fIusername\fR. -.RE - -.SH OPTIONS -.sp -.LP -The following options are available for the \fBrun\fR command. -.sp -.ne 2 -.na -\fB-c\fR \fIrunfile\fR -.ad -.RS 6n -Specify a \fIrunfile\fR to be consumed by the run command. -.RE - -.ne 2 -.na -\fB-d\fR -.ad -.RS 6n +.Ar filename +must be single quoted. +.It Sy timeout No = Ar n +A timeout value of +.Ar n +seconds. +.It Sy user No = Ar username +Execute the test or test group as +.Ar username . +.El +. +.Sh OPTIONS +.Bl -tag -width "-o outputdir" +.It Fl c Ar runfile +Specify a +.Ar runfile +to be consumed by the run command. +.It Fl d Dry run mode. Execute no tests, but print a description of each test that would have been run. -.RE - -.ne 2 -.na -\fB-g\fR -.ad -.RS 6n +.It Fl g Create test groups from any directories found while searching for tests. -.RE - -.ne 2 -.na -\fB-o\fR \fIoutputdir\fR -.ad -.RS 6n +.It Fl o Ar outputdir Specify the directory in which to write test results. -.RE - -.ne 2 -.na -\fB-p\fR \fIscript\fR -.ad -.RS 6n -Run \fIscript\fR prior to any test or test group. -.RE - -.ne 2 -.na -\fB-P\fR \fIscript\fR -.ad -.RS 6n -Run \fIscript\fR after any test or test group. -.RE - -.ne 2 -.na -\fB-q\fR -.ad -.RS 6n +.It Fl p Ar script +Run +.Ar script +prior to any test or test group. +.It Fl P Ar script +Run +.Ar script +after any test or test group. +.It Fl q Print only the results summary to the standard output. -.RE - -.ne 2 -.na -\fB-s\fR \fIscript\fR -.ad -.RS 6n -Run \fIscript\fR as a failsafe after any test is killed. -.RE - -.ne 2 -.na -\fB-S\fR \fIusername\fR -.ad -.RS 6n -Execute the failsafe script as \fIusername\fR. -.RE - -.ne 2 -.na -\fB-t\fR \fIn\fR -.ad -.RS 6n -Specify a timeout value of \fIn\fR seconds per test. -.RE - -.ne 2 -.na -\fB-u\fR \fIusername\fR -.ad -.RS 6n -Execute tests or test groups as \fIusername\fR. -.RE - -.ne 2 -.na -\fB-w\fR \fIrunfile\fR -.ad -.RS 6n -Specify the name of the \fIrunfile\fR to create. -.RE - -.ne 2 -.na -\fB-x\fR \fIusername\fR -.ad -.RS 6n -Execute the pre script as \fIusername\fR. -.RE - -.ne 2 -.na -\fB-X\fR \fIusername\fR -.ad -.RS 6n -Execute the post script as \fIusername\fR. -.RE - -.SH EXAMPLES -.LP -\fBExample 1\fR Running ad-hoc tests. -.sp -.LP -This example demonstrates the simplest invocation of \fBrun\fR. - -.sp -.in +2 -.nf -% \fBrun my-tests\fR +.It Fl s Ar script +Run +.Ar script as a failsafe after any test is killed. +.It Fl S Ar username +Execute the failsafe script as +.Ar username . +.It Fl t Ar n +Specify a timeout value of +.Ar n seconds per test. +.It Fl u Ar username +Execute tests or test groups as +.Ar username . +.It Fl w Ar runfile +Specify the name of the +.Ar runfile +to create. +.It Fl x Ar username +Execute the pre script as +.Ar username . +.It Fl X Ar username +Execute the post script as +.Ar username . +.El +. +.Sh EXAMPLES +.Bl -tag -width "-h" +.It Sy Example 1 Ns : Running ad-hoc tests. +This example demonstrates the simplest invocation of +.Nm . +.Bd -literal +.No % Nm run Ar my-tests Test: /home/jkennedy/my-tests/test-01 [00:02] [PASS] Test: /home/jkennedy/my-tests/test-02 [00:04] [PASS] Test: /home/jkennedy/my-tests/test-03 [00:01] [PASS] @@ -328,20 +256,14 @@ PASS 3 Running Time: 00:00:07 Percent passed: 100.0% Log directory: /var/tmp/test_results/20120923T180654 -.fi -.in -2 - -.LP -\fBExample 2\fR Creating a \fIrunfile\fR for future use. -.sp -.LP -This example demonstrates creating a \fIrunfile\fR with non default options. - -.sp -.in +2 -.nf -% \fBrun -p setup -x root -g -w new-tests.run new-tests\fR -% \fBcat new-tests.run\fR +.Ed +.It Sy Example 2 Ns : Creating a Ar runfile No for future use. +This example demonstrates creating a +.Ar runfile +with non-default options. +.Bd -literal +.No % Nm run Fl p Ar setup Fl x Ar root Fl g Fl w Ar new-tests.run Ar new-tests +.No % Nm cat Pa new-tests.run [DEFAULT] pre = setup post_user = @@ -354,33 +276,8 @@ outputdir = /var/tmp/test_results [/home/jkennedy/new-tests] tests = ['test-01', 'test-02', 'test-03'] -.fi -.in -2 - -.SH EXIT STATUS -.sp -.LP -The following exit values are returned: -.sp -.ne 2 -.na -\fB\fB0\fR\fR -.ad -.sp .6 -.RS 4n -Successful completion. -.RE -.sp -.ne 2 -.na -\fB\fB1\fR\fR -.ad -.sp .6 -.RS 4n -An error occurred. -.RE - -.SH SEE ALSO -.sp -.LP -\fBsudo\fR(1m) +.Ed +.El +. +.Sh SEE ALSO +.Xr sudo 1m