Commit Graph

2623 Commits

Author SHA1 Message Date
georglauterbach 7900e3444e
correct small mistake during conflict resolution 2024-01-24 17:18:23 +01:00
georglauterbach f077f38f11
Merge branch 'master' into tests/new_filters 2024-01-24 17:15:35 +01:00
Georg Lauterbach 00018e7e2b
general: update base image to Debian 12 ("Bookworm") (#3403)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2024-01-24 17:05:55 +01:00
Brennan Kinney 0c7e49e654
release: v13.3.1 (#3817)
* chore: Bump `VERSION`

* chore: Update `CHANGELOG.md`

---------

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2024-01-24 14:25:13 +01:00
Brennan Kinney 2cf5856961
chore: Raise awareness of v13 breaking change better (#3818)
Several issues have been raised where this was not an obvious breaking change to the reader. Additional context on impact relevance has been included.
2024-01-25 00:46:56 +13:00
Brennan Kinney a01aa66a90
Apply suggestions from code review 2024-01-24 15:14:46 +13:00
Brennan Kinney 9a53fb0463
docs: Minor revisions to Dovecot Sieve page (#3811)
- The old Dovecot wiki link needed to be updated to the new location.
- The new docs are not entirely compatible AFAIK, thus making the existing examples/docs a bit outdated / incompatible. A warning admonition has been added early on to raise awareness to the reader.
- Minor formatting revisions to the content.
2024-01-24 11:47:32 +13:00
georglauterbach fa1f3b005e
added explicit `_regexp` filters for logs 2024-01-23 19:30:16 +01:00
georglauterbach b64eb7517f
Merge branch 'master' into tests/new_filters 2024-01-23 19:13:38 +01:00
Brennan Kinney 611a66bf98
fix: Correctly support multiple Dovecot PassDBs (#3812)
* fix: Dovecot PassDB should restrict allowed auth mechanisms

This prevents PassDBs incompatible with certain auth mechanisms from logging failures which accidentally triggers Fail2Ban.

Instead only allow the PassDB to be authenticated against when it's compatible with the auth mechanism used.

* tests: Use `curl` for OAuth2 login test-cases instead of netcat

`curl` provides this capability for both IMAP and SMTP authentication with a bearer token. It supports both `XOAUTH2` and `OAUTHBEARER` mechanisms, as these updated test-cases demonstrate.

* chore: Add entry to `CHANGELOG.md`
2024-01-23 19:11:05 +01:00
georglauterbach be828dd06a
Merge branch 'master' into tests/new_filters 2024-01-23 19:09:32 +01:00
Georg Lauterbach f9549829c5
Apply suggestions from code review
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-23 18:56:27 +01:00
Brennan Kinney d40a17f7e0
fix: Ensure correct ownership for the Rspamd DKIM directory (#3813)
The UID / GID shifted during a new release. Until DKIM handling is refactored in a new major release, this fix ensures the content maintains the expected `_rspamd` ownership.
2024-01-23 11:51:10 +01:00
dependabot[bot] 315f33c9fe
chore(deps): Bump anchore/scan-action from 3.5.0 to 3.6.0 (#3808)
Bumps [anchore/scan-action](https://github.com/anchore/scan-action) from 3.5.0 to 3.6.0.
- [Release notes](https://github.com/anchore/scan-action/releases)
- [Changelog](https://github.com/anchore/scan-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anchore/scan-action/compare/v3.5.0...v3.6.0)

---
updated-dependencies:
- dependency-name: anchore/scan-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2024-01-23 15:36:53 +13:00
dependabot[bot] 37f4c853b2
chore(deps): Bump actions/cache from 3 to 4 (#3807) 2024-01-22 21:02:49 +01:00
github-actions[bot] 3cbcdb2d65
docs: update `CONTRIBUTORS.md` (#3798)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2024-01-21 12:31:10 +01:00
Jam Balaya 41b471fb14
fix(typo): comment on mailserver.env (#3799)
Thanks!
2024-01-21 12:21:29 +01:00
Brennan Kinney 5da61f97ef
chore: Update OAuth2 to use new log helper 2024-01-21 13:51:17 +13:00
Brennan Kinney 1b98de4862
Apply suggestions from code review 2024-01-21 13:33:12 +13:00
georglauterbach 709b56b5f5
add CHANGELOG entry 2024-01-20 12:45:26 +01:00
Georg Lauterbach 9ae8336336
Merge branch 'master' into tests/new_filters 2024-01-20 12:43:21 +01:00
georglauterbach ca0127e44c
follow-up of previous commit 2024-01-20 12:42:44 +01:00
Georg Lauterbach b78978caed
release: v13.3.0 (#3781)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-20 12:33:05 +01:00
georglauterbach b3a6a7b103
adjust helpers to new helper filter 2024-01-20 12:25:23 +01:00
georglauterbach aaeea8af81
update `_filter_service_log` to not use `-i -E`
Moreover, I added a function to print the whole mail log. Appropriate
comments were added to this function to indicate that one should only
use this function when necessary.
2024-01-20 12:11:26 +01:00
georglauterbach 143ca0c27d
use more distinct names for MSG_ID headers 2024-01-20 12:01:46 +01:00
georglauterbach 5622761c41
fix tests 2024-01-20 12:01:33 +01:00
Brennan Kinney f3a7f08f96
tests: Revise OAuth2 tests (#3795)
* tests: OAuth2 - Replace Python `/userinfo` endpoint with Caddy

Better documented, easier flow and separation of concerns via Caddy.

The python code had additional noise related to setting up a basic API which is abstracted away via `Caddyfile` config that's dedicated to this task.

* tests: OAuth2 - Minimize noise + Improve test assertion

Caddyfile can use an Access Token instead of a JWT. Much smaller and correct for this OAuth2 configuration. This new value has been documented inline.

Likewise the `sub` field returned is not important to this test. `email_verified` is kept as it may be helpful for further coverage testing.

The actual test-case has better assertions for success and failure by checking for Dovecot logs we expect instead of netcat response.

`oauth2` to `auth` for the Caddy container hostname is not necessary, just a more generic subdomain choice.

* tests: OAuth2 - Caddyfile `imap/xoauth2` route dynamic via query string

This way is more flexible and doesn't require modifying the `Caddyfile` directly, while still easy to use.

Additionally simplifies understanding the Caddyfile to maintainers by removing the `route` directive that was required to ensure a deterministic order of vars.

* tests: OAuth2 - `/imap/xoauth2` respond with IMAP commands for netcat

Since this is the only intended usage, might as well have it respond with the full file content.

* tests: OAuth2 - Implement coverage for `OAUTHBEARER`

Caddyfile route for `/imap/` now accepts any subpath to support handling both `xoauth2` and `oauthbearer` subpaths.

Both SASL mechanisms represent the same information, with `XOAUTH2` being a common mechanism to encounter defined by Google, whilst `OAUTHBEARER` is the newer variant standardized by RFC 7628 but not yet as widely adopted.

The request to `/userinfo` endpoint will be the same, only the `credentials` value to be encoded differs.

Instead of repeating the block for a similar route, this difference is handled via the Caddyfile `map` directive.

We match the path context (_`/xoauth2` or `/oauthbearer`, the `/imap` prefix was stripped by `handle_path` earlier_), when there is a valid match, `sasl_mechanism` and `credentials` map vars are created and assigned to be referenced by the later `respond` directive.

---

Repeat the same test-case logic, DRY with log asserts extracted to a common function call. This should be fine as the auth method will be sufficient to match against or a common failure caught.

* tests: OAuth2 - Minor revisions

Separate test cases and additional comment on creating the same base64 encoded credentials via CLI as an alternative to running Caddy.

Added a simple `compose.yaml` for troubleshooting or running the container for the `/imap/xoauth2` / `/imap/oauthbearer` endpoints.

* tests: OAuth2 - Route endpoints in Caddyfile with snippets instead

`reverse_proxy` was a bit more convenient, but the additional internal ports weren't really relevant. It also added noise to logging when troubleshooting.

The `import` directive with Snippet blocks instead is a bit cleaner, but when used in a single file snippets must be defined prior to referencing them with the `import` directive.

---

`compose.yaml` inlines the examples, with slight modification to `localhost:80`, since the Caddyfile examples `auth.example.test` is more relevant to the tests which can use it, and not applicable to troubleshooting locally outside of tests.

* chore: Add entry to `CHANGELOG.md`

* chore: Additional context on access token
2024-01-20 10:49:09 +01:00
Brennan Kinney 7f825835f4
Apply suggestions from code review 2024-01-20 22:19:03 +13:00
Brennan Kinney 7c39c154db
Apply suggestions from code review
- `_mid` / `MID` => `_msgid` / `MSG_ID`
- Revised documentation / tooltip comments
2024-01-20 19:59:28 +13:00
Brennan Kinney a5d536201b
docs: Add maintenance comment for `reject_unknown_sender_domain` (#3793)
I figured this was a useful comment to reference related to the setting if it's ever being changed or needs to be better understood (linked issue is a common failure that can be encountered related to this restriction).
2024-01-20 17:51:32 +13:00
Roy Sindre Norangshol deb0d2d09a
docs: Guidance for binding outbound SMTP with multiple interfaces available (#3465)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-19 14:58:20 +13:00
Andreas Perhab 9cdbef2b36
setup/dkim: chown created dkim directories and keys to config user (#3783)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-18 10:41:55 +01:00
Brennan Kinney 437114c5dd
tests: Revise `process_check_restart.bats` (#3780) 2024-01-17 09:46:22 +00:00
Brennan Kinney 2d59aac5a1
chore: Add maintenance comment for `sed` usage (#3789)
This is a more explicit reminder for any future contributors that get thrown off by the usage of `sed` here and may be inclined to change it.

Add a link to reference a comment where it's already been explored what the alternative `sed` invocations available are.
2024-01-17 20:54:27 +13:00
georglauterbach 6503f24b61
use new filters in tests 2024-01-16 21:02:16 +01:00
georglauterbach 51a3b3f5f3
add new helper for asserting non-existence of log message 2024-01-16 20:47:51 +01:00
georglauterbach 745e59dbc2
use new filters (and sending) functions 2024-01-16 19:22:08 +01:00
georglauterbach 74d63af683
new filters for searching logs with MID 2024-01-16 19:19:19 +01:00
georglauterbach 9c621f1bd9
use new function for tracking logs
The new function, called `_send_email_with_mid`, aligns with suggestions
from @polarethene and is heavily simplified compared to its predecessor
`_send_email_and_get_id`. New helpers will be introduced to filter logs
according to the MID constructed in this function.
2024-01-16 19:18:04 +01:00
georglauterbach 3a64a7a97c
add ShellCheck global directives 2024-01-16 19:13:38 +01:00
georglauterbach 87b92f6f88
move log/filter functions into own file 2024-01-16 11:58:55 +01:00
Georg Lauterbach 068ceb1d1a
docs: misc improvements (#3773)
* correct misc typos

We also seem to be favoring `behavior` over `behaviour`.

* bump MkDocs version

* resolve errors shown when buildg docs

* improve the Rspamd page

* behaviour -> behavior

Streamline the usage of this word. The majority used behavior, so I
opted to go with this way of spelling it.

* Apply suggestions from code review

---------

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-16 08:38:08 +00:00
dependabot[bot] 2bf5234250
chore(deps): Bump anchore/scan-action from 3.4.0 to 3.5.0 (#3782)
Bumps [anchore/scan-action](https://github.com/anchore/scan-action) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/anchore/scan-action/releases)
- [Changelog](https://github.com/anchore/scan-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anchore/scan-action/compare/v3.4.0...v3.5.0)

---
updated-dependencies:
- dependency-name: anchore/scan-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-16 10:18:13 +13:00
Brennan Kinney 265440b2bb
fix: Ensure `.svbin` files are newer than `.sieve` source files (#3779) 2024-01-15 10:34:15 +01:00
Den ce6ebcc021
docs: Rspamd DKIM config simplify via `path` setting (#3702)
docs: Rspamd DKIM config (`dkim_signing.conf`) example has been simplified via `path` + `selector` settings.

---------

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-15 13:10:03 +13:00
Brennan Kinney 1449629479
fix: Revert quoting `SA_SPAM_SUBJECT` in `mailserver.env` (#3767)
In Docker Compose `.env` files are parsed properly when values are wrapped with quotes. Trailing white-space is also discarded, like it would be with shell variables.

This is not the case with `docker run` or other CRI like `podman` (_including it's compose equivalent support_). Those will parse the quotes to be included in a literal string value. Trailing white-space is also retained.

Hence a default with a trailing space is not compatible across CRI. This change documents the default with additional context on how to include a trailing white-space with a custom value for the users CRI choice. It additionally clearly communicates the opt-out value for this feature.
2024-01-15 11:23:23 +13:00
Georg Lauterbach 3a142f9726
tests: small adjustments (#3772) 2024-01-14 23:14:03 +01:00
github-actions[bot] f794f65caa
docs: updated `CONTRIBUTORS.md` (#3777)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-01-14 14:08:27 +01:00
Joerg Sonnenberger e3331b0f44
feat: Add MTA-STS support for outbound mail (#3592)
* feat: add support for MTA-STS for outgoing mails

* Hook-up mta-sts-daemon into basic process handling test

* fix: Call python script directly

The python3 shebang will run it, which will now meet the expectations of the process testing via pgrep. fail2ban has the same approach.

---------

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-13 21:37:20 +13:00