While "diff -r" is the most straightforward way of comparing directory
trees for differences, it has two major issues:
* File metadata is not compared, which means that subtle bugs may be
missed even if a test is written that exercises the buggy behaviour.
* diff(1) doesn't know how to compare special files -- it assumes they
are always different, which means that a test using diff(1) on
special files will always fail (resulting in such tests not being
added).
rsync can be used in a very similar manner to diff (with the -ni flags),
but has the additional benefit of being able to detect and resolve many
more differences between directory trees. In addition, rsync has a
standard set of features and flags while diffs feature set depends on
whether you're using GNU or BSD binutils.
Note that for several of the test cases we expect that file timestamps
will not match. For example, the ctime for a file creation or modify
event is stored in the intent log but not the mtime. Thus when replaying
the log the correct ctime is set but the current mtime is used. This is
the expected behavior, so to prevent these tests from failing, there's a
replay_directory_diff function which ignores those kinds of changes.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
Closes#12588
Majority of the software installed by default in GitHub runners is
irrelevant to OpenZFS. Reclaimed space could be used for more/bigger
vdev files. File deletion happens in the background, leveraging
`systemd-run` - the workflow is not significantly slowed down.
Before
```
Filesystem Size Used Avail Use% Mounted on
/dev/root 84G 53G 31G 63% /
```
After
```
Filesystem Size Used Avail Use% Mounted on
/dev/root 84G 15G 70G 18% /
```
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Signed-off-by: szubersk <szuberskidamian@gmail.com>
Closes#13066
- Move build dependencies moved to
`.github/workflows/build-dependencies.txt` shared among workflows.
- Change `ubuntu-latest` -> `ubuntu-20.04` to avoid unexpected
runner environment updates in `zloop` workflow.
- Change `ubuntu-20.04` -> `ubuntu-latest` to track changes in
runner environment in `checkstyle` workflow.
- Kernel buffer is flushed before ZTS invocation to avoid storing
the same data after each test case run.
- `make` is invoked with consistent set of options to reduce
clutter in logs.
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: szubersk <szuberskidamian@gmail.com>
Closes#13037
`configure` now accepts `--enable-asan` and `--enable-ubsan` switches
which results in passing `-fsanitize=address`
and `-fsanitize=undefined`, respectively, to the compiler. Those
flags are enabled in GitHub workflows for ZTS and zloop. Errors
reported by both instrumentations are corrected, except for:
- Memory leak reporting is (temporarily) suppressed. The cost of
fixing them is relatively high compared to the gains.
- Checksum computing functions in `module/zcommon/zfs_fletcher*`
have UBSan errors suppressed. It is completely impractical
to enforce 64-byte payload alignment there due to performance
impact.
- There's no ASan heap poisoning in `module/zstd/lib/zstd.c`. A custom
memory allocator is used there rendering that measure
unfeasible.
- Memory leaks detection has to be suppressed for `cmd/zvol_id`.
`zvol_id` is run by udev with the help of `ptrace(2)`. Tracing is
incompatible with memory leaks detection.
Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: szubersk <szuberskidamian@gmail.com>
Closes#12928
If we die from timeout of the whole GH action run, we don't run the
collect step afterward, which can make it hard to investigate the
timeout.
If we timeout first in the test action, though, it qualifies as
failure, and collects appropriately.
(330 minutes seems like an acceptable tradeoff between the 6h
timeout by default on the action and the 4h and change "functional"
usually takes.)
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes#12999
Deprecation of Python versions below 3.6 gives opportunity to unify the
build and install requirements for OpenZFS packages. The minimal
supported Python version is 3.6 as this is the most recent Python
package CentOS/RHEL 7 users can get.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: szubersk <szuberskidamian@gmail.com>
Closes#12925
The upload artifact functionality in github can't handle colons in
filenames. The current code handles this for files under the most
recent set of results. With the ability to rerun failed tests, now
there can be multiple sets of results, and they all need to be
processed in the same way.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: John Kennedy <john.kennedy@delphix.com>
Closes#12815
This was a project proposed as part of the Quality theme for the
hackthon for the 2021 OpenZFS Developer Summit. The idea is to improve
the usability of the automated tests that get run when a PR is created
by having failing tests automatically rerun in order to make flaky
tests less impactful.
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes#12740
cloud-init added a hook which triggers on every device add/rm
event, which results in holding open devices for a while after
they're created/destroyed.
So let's shove an exclusion rule for that into the GH workflows
until it gets fixed.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes#12644Closes#12669
We use docker image instead.
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes#12529
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes#12529
It is useful to have control over the number of iterations of zloop so
we can easily produce "x core dumps found *in y iterations*" metrics.
Using random values for run/pass times doesn't improve coverage in a
meaningful way.
Randomizing run time could be seen as a compromise between running a
greater variety of shorter tests versus a smaller variety of longer
tests within a fixed time span. However, it is not desirable when
running a fixed number of iterations.
Pass time already incorporates randomness within ztest.
Either parameter can be passed to ztest explicitly if the defaults are
not satisfactory.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes#12411
So commit author can just download them as
artifacts and commit.
Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes#12379
- Remove the "SPL Version" line, the repositories have been merged
since the 0.8 release and we no longer need to ask about this.
- Simply ask for the kernel version / patch level and add a hint
about how to get this information on Linux and FreeBSD.
- Remove "Status: Triage Needed" from the template, in practice
we really haven't been using this label so let's step setting it.
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes: #12340
Libera have made a webchat client available. This change builds on #12127.
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Jonathon Fernyhough <jonathon@m2x.dev>
Closes#12251
It turns out the ax_python_devel.m4 version check assumes that
("3.X+1.0" >= "3.X.0") is True in Python, which is not when X+1
is 10 or above and X is not. (Also presumably X+1=100 and ...)
So let's remake the check to behave consistently, using the
"packaging" or (if absent) the "distlib" modules.
(Also, update the Github workflows to use the new packages.)
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: John Kennedy <john.kennedy@delphix.com>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes: #12073
While Libera doesn't yet have a webchat client, we should at least
direct them to the right network. Once a webchat client is available,
we can direct them to it.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes#12127
Add support for http and https to the keylocation properly to
allow encryption keys to be fetched from the specified URL.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Issue #9543Closes#9947Closes#11956
Recently we've been running out of free space in the ubuntu 20.04
environment resulting in test failures. This appears to be caused
by a change in the default available free space and not because of
any change in OpenZFS. Try and avoid this failure by applying a
suggested workaround which removes some unnecessary files.
https://github.com/actions/virtual-environments/issues/2840
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes#11826
Our checkstyle doesn't work well on Ubuntu 20.04,
temporary pin it to 18.04.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes#11713
Instead of creating issues with type "question"
Forward to the GitHub Discussion system.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl>
Closes#11383
Run zfs-tests with sanity.run for brief results. Timeouts
are rare, so minimize false positives by increasing the
default from 60 to 180 seconds.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes#11304
Extend the CI checkstyle workflow to perform the library ABI
checks in the master branch. The intent is not to prevent any
ABI changes but to detect them immediately so when they're
made it's done intentionally.
When the changing the ABI the `make storeabi` target can be
used to generate a new .abi file which can be included with
the commit. This depends on the libabigail utility which is
available from the majority of distribution package managers.
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes#11287
This change documents the currently used branching structure.
It has been cut down to not include any controversial changes.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl>
Closes#10976
As mentioned in the #OpenZFS IRC channel (thanks "Toomas Soome"):
The OpenZFS PR Template still mentions "ZFS on Linux".
This changes that reference and updates the URLs.
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Garrett Fields <ghfields@gmail.com>
Closes#10868
This commit closes issues labeled with:
"Status: Feedback requested" after 1 month, if the
label is not removed or the author has not responded
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Richard Laager <rlaager@wiktel.com>
Signed-off-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl>
Closes#10807Closes#10808
By accident previous PR broke the bug report form.
This commit fixes it
(and is actually tested completely to work)
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl>
Closes#10821
Run checkbashisms on checkstyle too.
Reviewed-by: Kjeld Schouten <kjeld@schouten-lebbing.nl>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes#10811
- Add Status: Triage Needed to bug reports
Currently "Type: Defect" is auto added.
Adding a triage tag, makes sure all issues are reviewed by a maintainer
It also opens up some options to priorities defects in the near future.
- Prevent future StaleBot Spam
StaleBot will limit itself to 6 actions per hour
This should prevent future floods of StaleBot activity
(aka Spam)
- StaleBot: Ignore issues that are being worked on
Ignore the following Issues:
- tagged: "Status: Work in Progress"
- Having a maintainer assigned
- Being part of a project
- Having a milestone tag
- Rename Ignore "Type: Understood" to "Bot: Not Stale"
This Commits changes the general ignore tag for StaleBot from:
"Type: Understood"
to
"Bot: Not Stale"
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl>
Closes#10813
This file configures the following stale-bot:
https://github.com/apps/stale
It is set to mark issues as "Stale" after 365 days
It is also set to auto-close the issue 90 days after.
Please be aware that this issue also requires-
The listed stale-bot being added to the repo.
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl>
Closes#10778
A big portion of issues are of "Type: Question".
This PR adds a separate issue template for those.
It also automatically adds the "Type: Question" tag.
in addition it adds "Type: Defect" to all bug reports by default
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl>
Closes#10779
Github has started using a new issue templating structure.
This commit moves the current template and adds one additional one.
- Moves issue template to new issue-template folder
- Adds feature request template
- removes the following warning when viewing issue template
Reviewed-by: Richard Laager <rlaager@wiktel.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl>
Closes#10759
Use github workflow to run checkstyle
- use free (for OS projects) resources
- starts for every commit and branch
- work on forks, contributors may use it
before creating PRs
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes#10705
Update stale references to "ZFS on Linux" to "OpenZFS" in
CONTRIBUTING.md.
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes#10602
Move this file out of .github and add it to distribution.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu>
Closes#10506
Use direct links to new documentation resource.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes#10395
Originally, I wanted to point to directly to
https://zfsonlinux.topicbox.com/groups/zfs-discuss
as the text refers to that specific mailing list, but George Melikov
requested to change it to the general to give users the overview.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Marcel Schilling <marcel.schilling@uni-luebeck.de>
Closes#10367Closes#10369
Add the FreeBSD platform code to the OpenZFS repository. As of this
commit the source can be compiled and tested on FreeBSD 11 and 12.
Subsequent commits are now required to compile on FreeBSD and Linux.
Additionally, they must pass the ZFS Test Suite on FreeBSD which is
being run by the CI. As of this commit 1230 tests pass on FreeBSD
and there are no unexpected failures.
Reviewed-by: Sean Eric Fagan <sef@ixsystems.com>
Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Reviewed-by: Richard Laager <rlaager@wiktel.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Co-authored-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Matt Macy <mmacy@FreeBSD.org>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes#898Closes#8987
Most of the projects I work on don't use tabs, and while authoring my
first patch I had to wrestle with my editor to not introduce
whitespace editors.
The `.editorconfig` file is supported by a large number of editors
out of the box, and many more with plugins.
As a first-time contributor, I can't say for certain these settings
are totally correct, but thus far git and my editor are satisfied
enough.
I considered adding `git config --local format.signOff true` but
wanted to respect the warning:
format.signOff
A boolean value which lets you enable the -s/--signoff
option of format-patch by default. Note: Adding the
Signed-off-by: line to a patch should be a conscious act and
means that you certify you have the rights to submit this
work under the same open source license. Please see the
SubmittingPatches document for further discussion.
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Graham Christensen <graham@grahamc.com>
Closes#9892
- Splits PR advice into two sections.
- Add "co-authored-by" instructions.
- Add description of draft PR and when using it is appropriate.
- Reword ZFS Test Suite checklist question.
- Link to zfs-tests.sh and zloop.sh.
Reviewed-By: Marcel Schilling <marcel.schilling@mdc-berlin.de>
Reviewed-By: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-By: Richard Laager <rlaager@wiktel.com>
Co-Authored-By: Marcel Schilling <marcel.schilling@mdc-berlin.de>
Co-Authored-By: Brian Behlendorf <behlendorf1@llnl.gov>
Co-Authored-By: Richard Laager <rlaager@wiktel.com>
Signed-off-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl>
Closes#9753
Rely on ax_code_coverage to exclude test directories.
- Removes broken codecov ignore
- Places ignore section in ax_code_coverage
- Forwards users from codecov to LCOV for ignores
Reviewed-by: Prakash Surya <prakash.surya@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Richard Laager <rlaager@wiktel.com>
Signed-off-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl>
Closes#9726
Aleksa Sarai
Rich Ercolani
Damian Szuberski
John Wren Kennedy
Paul Dagnelie
Damian Szuberski
George Melikov
Ryan Moeller
Brian Behlendorf
Jonathon
наб
Kjeld Schouten-Lebbing
Garrett Fields
Evan Harris
Arvind Sankar
Marcel Schilling
Matthew Macy
Graham Christensen