Commit Graph

2604 Commits

Author SHA1 Message Date
georglauterbach 709b56b5f5
add CHANGELOG entry 2024-01-20 12:45:26 +01:00
Georg Lauterbach 9ae8336336
Merge branch 'master' into tests/new_filters 2024-01-20 12:43:21 +01:00
georglauterbach ca0127e44c
follow-up of previous commit 2024-01-20 12:42:44 +01:00
Georg Lauterbach b78978caed
release: v13.3.0 (#3781)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-20 12:33:05 +01:00
georglauterbach b3a6a7b103
adjust helpers to new helper filter 2024-01-20 12:25:23 +01:00
georglauterbach aaeea8af81
update `_filter_service_log` to not use `-i -E`
Moreover, I added a function to print the whole mail log. Appropriate
comments were added to this function to indicate that one should only
use this function when necessary.
2024-01-20 12:11:26 +01:00
georglauterbach 143ca0c27d
use more distinct names for MSG_ID headers 2024-01-20 12:01:46 +01:00
georglauterbach 5622761c41
fix tests 2024-01-20 12:01:33 +01:00
Brennan Kinney f3a7f08f96
tests: Revise OAuth2 tests (#3795)
* tests: OAuth2 - Replace Python `/userinfo` endpoint with Caddy

Better documented, easier flow and separation of concerns via Caddy.

The python code had additional noise related to setting up a basic API which is abstracted away via `Caddyfile` config that's dedicated to this task.

* tests: OAuth2 - Minimize noise + Improve test assertion

Caddyfile can use an Access Token instead of a JWT. Much smaller and correct for this OAuth2 configuration. This new value has been documented inline.

Likewise the `sub` field returned is not important to this test. `email_verified` is kept as it may be helpful for further coverage testing.

The actual test-case has better assertions for success and failure by checking for Dovecot logs we expect instead of netcat response.

`oauth2` to `auth` for the Caddy container hostname is not necessary, just a more generic subdomain choice.

* tests: OAuth2 - Caddyfile `imap/xoauth2` route dynamic via query string

This way is more flexible and doesn't require modifying the `Caddyfile` directly, while still easy to use.

Additionally simplifies understanding the Caddyfile to maintainers by removing the `route` directive that was required to ensure a deterministic order of vars.

* tests: OAuth2 - `/imap/xoauth2` respond with IMAP commands for netcat

Since this is the only intended usage, might as well have it respond with the full file content.

* tests: OAuth2 - Implement coverage for `OAUTHBEARER`

Caddyfile route for `/imap/` now accepts any subpath to support handling both `xoauth2` and `oauthbearer` subpaths.

Both SASL mechanisms represent the same information, with `XOAUTH2` being a common mechanism to encounter defined by Google, whilst `OAUTHBEARER` is the newer variant standardized by RFC 7628 but not yet as widely adopted.

The request to `/userinfo` endpoint will be the same, only the `credentials` value to be encoded differs.

Instead of repeating the block for a similar route, this difference is handled via the Caddyfile `map` directive.

We match the path context (_`/xoauth2` or `/oauthbearer`, the `/imap` prefix was stripped by `handle_path` earlier_), when there is a valid match, `sasl_mechanism` and `credentials` map vars are created and assigned to be referenced by the later `respond` directive.

---

Repeat the same test-case logic, DRY with log asserts extracted to a common function call. This should be fine as the auth method will be sufficient to match against or a common failure caught.

* tests: OAuth2 - Minor revisions

Separate test cases and additional comment on creating the same base64 encoded credentials via CLI as an alternative to running Caddy.

Added a simple `compose.yaml` for troubleshooting or running the container for the `/imap/xoauth2` / `/imap/oauthbearer` endpoints.

* tests: OAuth2 - Route endpoints in Caddyfile with snippets instead

`reverse_proxy` was a bit more convenient, but the additional internal ports weren't really relevant. It also added noise to logging when troubleshooting.

The `import` directive with Snippet blocks instead is a bit cleaner, but when used in a single file snippets must be defined prior to referencing them with the `import` directive.

---

`compose.yaml` inlines the examples, with slight modification to `localhost:80`, since the Caddyfile examples `auth.example.test` is more relevant to the tests which can use it, and not applicable to troubleshooting locally outside of tests.

* chore: Add entry to `CHANGELOG.md`

* chore: Additional context on access token
2024-01-20 10:49:09 +01:00
Brennan Kinney 7f825835f4
Apply suggestions from code review 2024-01-20 22:19:03 +13:00
Brennan Kinney 7c39c154db
Apply suggestions from code review
- `_mid` / `MID` => `_msgid` / `MSG_ID`
- Revised documentation / tooltip comments
2024-01-20 19:59:28 +13:00
Brennan Kinney a5d536201b
docs: Add maintenance comment for `reject_unknown_sender_domain` (#3793)
I figured this was a useful comment to reference related to the setting if it's ever being changed or needs to be better understood (linked issue is a common failure that can be encountered related to this restriction).
2024-01-20 17:51:32 +13:00
Roy Sindre Norangshol deb0d2d09a
docs: Guidance for binding outbound SMTP with multiple interfaces available (#3465)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-19 14:58:20 +13:00
Andreas Perhab 9cdbef2b36
setup/dkim: chown created dkim directories and keys to config user (#3783)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-18 10:41:55 +01:00
Brennan Kinney 437114c5dd
tests: Revise `process_check_restart.bats` (#3780) 2024-01-17 09:46:22 +00:00
Brennan Kinney 2d59aac5a1
chore: Add maintenance comment for `sed` usage (#3789)
This is a more explicit reminder for any future contributors that get thrown off by the usage of `sed` here and may be inclined to change it.

Add a link to reference a comment where it's already been explored what the alternative `sed` invocations available are.
2024-01-17 20:54:27 +13:00
georglauterbach 6503f24b61
use new filters in tests 2024-01-16 21:02:16 +01:00
georglauterbach 51a3b3f5f3
add new helper for asserting non-existence of log message 2024-01-16 20:47:51 +01:00
georglauterbach 745e59dbc2
use new filters (and sending) functions 2024-01-16 19:22:08 +01:00
georglauterbach 74d63af683
new filters for searching logs with MID 2024-01-16 19:19:19 +01:00
georglauterbach 9c621f1bd9
use new function for tracking logs
The new function, called `_send_email_with_mid`, aligns with suggestions
from @polarethene and is heavily simplified compared to its predecessor
`_send_email_and_get_id`. New helpers will be introduced to filter logs
according to the MID constructed in this function.
2024-01-16 19:18:04 +01:00
georglauterbach 3a64a7a97c
add ShellCheck global directives 2024-01-16 19:13:38 +01:00
georglauterbach 87b92f6f88
move log/filter functions into own file 2024-01-16 11:58:55 +01:00
Georg Lauterbach 068ceb1d1a
docs: misc improvements (#3773)
* correct misc typos

We also seem to be favoring `behavior` over `behaviour`.

* bump MkDocs version

* resolve errors shown when buildg docs

* improve the Rspamd page

* behaviour -> behavior

Streamline the usage of this word. The majority used behavior, so I
opted to go with this way of spelling it.

* Apply suggestions from code review

---------

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-16 08:38:08 +00:00
dependabot[bot] 2bf5234250
chore(deps): Bump anchore/scan-action from 3.4.0 to 3.5.0 (#3782)
Bumps [anchore/scan-action](https://github.com/anchore/scan-action) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/anchore/scan-action/releases)
- [Changelog](https://github.com/anchore/scan-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anchore/scan-action/compare/v3.4.0...v3.5.0)

---
updated-dependencies:
- dependency-name: anchore/scan-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-16 10:18:13 +13:00
Brennan Kinney 265440b2bb
fix: Ensure `.svbin` files are newer than `.sieve` source files (#3779) 2024-01-15 10:34:15 +01:00
Den ce6ebcc021
docs: Rspamd DKIM config simplify via `path` setting (#3702)
docs: Rspamd DKIM config (`dkim_signing.conf`) example has been simplified via `path` + `selector` settings.

---------

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-15 13:10:03 +13:00
Brennan Kinney 1449629479
fix: Revert quoting `SA_SPAM_SUBJECT` in `mailserver.env` (#3767)
In Docker Compose `.env` files are parsed properly when values are wrapped with quotes. Trailing white-space is also discarded, like it would be with shell variables.

This is not the case with `docker run` or other CRI like `podman` (_including it's compose equivalent support_). Those will parse the quotes to be included in a literal string value. Trailing white-space is also retained.

Hence a default with a trailing space is not compatible across CRI. This change documents the default with additional context on how to include a trailing white-space with a custom value for the users CRI choice. It additionally clearly communicates the opt-out value for this feature.
2024-01-15 11:23:23 +13:00
Georg Lauterbach 3a142f9726
tests: small adjustments (#3772) 2024-01-14 23:14:03 +01:00
github-actions[bot] f794f65caa
docs: updated `CONTRIBUTORS.md` (#3777)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-01-14 14:08:27 +01:00
Joerg Sonnenberger e3331b0f44
feat: Add MTA-STS support for outbound mail (#3592)
* feat: add support for MTA-STS for outgoing mails

* Hook-up mta-sts-daemon into basic process handling test

* fix: Call python script directly

The python3 shebang will run it, which will now meet the expectations of the process testing via pgrep. fail2ban has the same approach.

---------

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-13 21:37:20 +13:00
Casper 71e1102749
Tiny #3480 follow up: Add missing ENABLE_OAUTH2 var (#3775) 2024-01-12 23:48:14 +01:00
Keval Kapdee 52c4582f7b
feat: Auth - OAuth2 (Dovecot PassDB) (#3480)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-13 09:45:14 +13:00
Georg Lauterbach 06fab3f129
tests: streamline tests and helpers further (#3747)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-11 10:34:08 +01:00
Casper aba218e6d7
Fix jaq: Download platform specific binary (#3766)
* choose architecture dynamically
2024-01-10 12:31:30 +13:00
dependabot[bot] 8d8936dfac
chore(deps): Bump anchore/scan-action from 3.3.8 to 3.4.0 (#3761)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-08 17:40:50 +00:00
dependabot[bot] ac25fb495b
chore(deps): Bump docker/metadata-action from 5.4.0 to 5.5.0 (#3762)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-08 18:39:28 +01:00
Brennan Kinney 5e28c17cf4
docs: SpamAssassin ENV docs refactor (#3756)
* chore: Log `SPAMASSASSIN_SPAM_TO_INBOX=1` ENV correctly

ENV name logged was incomplete.

* docs: Update SA related ENV docs

* fix: Log level `warning` should be `warn`

* docs: FAQ - Revise outdated SA entry

* chore: Antispam / Antivirus => Anti-spam / Anti-virus

* docs: ENV - Additional revisions to SA ENV

* docs: ENV - Move `ENABLE_SPAMASSASSIN_KAM`
2024-01-08 03:07:38 +01:00
github-actions[bot] 8e20173200
docs: updated `CONTRIBUTORS.md` (#3757)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-01-08 09:40:01 +13:00
Brennan Kinney 6d666512c1
ci: `.gitattributes` - Ensure `eol=lf` for shell scripts (#3755)
* ci: `.gitattributes` - Ensure `eol=lf` for shell scripts

- These files should always use LF for line endings during a checkout.
- `Dockerfile` does not like building with HereDoc `RUN` scripts that expect LF.
2024-01-08 09:34:24 +13:00
Casper 0eb4ac7714
tests: Replace `wc -l` with `grep -c` (#3752) 2024-01-06 17:07:00 +01:00
Brennan Kinney 6082d5f8d0
chore: Disable `smtputf8` support in config directly (#3750)
* chore: Disable `smtputf8` support in config

This was always configured disabled at runtime, better to just set explicitly in `main.cf` unless config diverges when Dovecot is enabled to opt-out of this feature.
2024-01-05 23:18:30 +01:00
Georg Lauterbach 04f4ae4569
Rspamd: add custom symbol scores for SPF, DKIM & DMARC (#3726) 2024-01-05 09:07:31 +01:00
Georg Lauterbach 8f391e4d5a
release: v13.2.0 (#3746)
* bump `VERSION` & adjust `CHANGELOG.md`

* chore: Bump the unreleased base compare tag in `CHANGELOG.md`

---------

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-04 22:14:51 +01:00
Georg Lauterbach bf69ef248e
Postfix: add `smtpd_data_restrictions = reject_unauth_pipelining` (#3744)
* add `smtpd_data_restrictions = reject_unauth_pipelining`

* fix: Skip restriction if trusted

* add changelog entry

* revert change to `postfix-amavis.cf`

* Update CHANGELOG.md

---------

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-04 22:13:13 +01:00
Georg Lauterbach 25c7024cc4
security(Postfix): Protect against "SMTP Smuggling" attack (#3727)
View `CHANGELOG.md` entry and PR for details.

---------

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-03 14:02:59 +13:00
Georg Lauterbach 9e81517fe3
tests: Use `swaks` instead of `nc` for sending mail (#3732)
See associated `CHANGELOG.md` entry for details.

---------

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2024-01-03 13:17:54 +13:00
Brennan Kinney 0889b0ff06
fix: `supervisor-app.conf` - Correct the log location for `postgrey` (#3724)
* fix: `supervisor-app.conf` - Correct `postgrey` log location

Looks like this should have been like every other service and reference a log file(s) based on program name in the supervisor log directory.

* tests: Adjust log location for `postgrey_enabled.bats`
2023-12-30 09:59:09 +13:00
Georg Lauterbach 8392e3c1a8
release: v13.1.0 (#3720)
Co-authored-by: Casper <casperklein@users.noreply.github.com>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2023-12-29 13:58:54 +01:00
Brennan Kinney 72517d3f82
docs: Debugging - Delivery failure from service downtime (#3718)
* docs: Debugging - Delivery failure from service downtime

Services may be temporarily down, such as when restarted when certificates are updated due to the `check-for-changes.sh` service. This is another known source of intermittent delivery failures.
2023-12-26 09:53:32 +13:00