Merge branch 'master' into simplify-getmail

.github/workflows/generic_build.yml

@@ -79,7 +79,7 @@ jobs:
           platforms: arm64
 
       - name: 'Set up Docker Buildx'
-        uses: docker/setup-buildx-action@v3.0.0
+        uses: docker/setup-buildx-action@v3.1.0
 
       # NOTE: AMD64 can build within 2 minutes
       - name: 'Build images'

.github/workflows/generic_publish.yml

@@ -40,7 +40,7 @@ jobs:
           platforms: arm64
 
       - name: 'Set up Docker Buildx'
-        uses: docker/setup-buildx-action@v3.0.0
+        uses: docker/setup-buildx-action@v3.1.0
 
       # Try get the cached build layers from a prior `generic_build.yml` job.
       # NOTE: Until adopting `type=gha` scoped cache exporter (in `docker/build-push-action`),

.github/workflows/generic_test.yml

@@ -38,7 +38,7 @@ jobs:
       # Ensures consistent BuildKit version (not coupled to Docker Engine),
       # and increased compatibility of the build cache vs mixing buildx drivers.
       - name: 'Set up Docker Buildx'
-        uses: docker/setup-buildx-action@v3.0.0
+        uses: docker/setup-buildx-action@v3.1.0
 
       # Importing from the cache should create the image within approx 30 seconds:
       # NOTE: `qemu` step is not needed as we only test for AMD64.

.github/workflows/generic_vulnerability-scan.yml

@@ -37,7 +37,7 @@ jobs:
       # Ensures consistent BuildKit version (not coupled to Docker Engine),
       # and increased compatibility of the build cache vs mixing buildx drivers.
       - name: 'Set up Docker Buildx'
-        uses: docker/setup-buildx-action@v3.0.0
+        uses: docker/setup-buildx-action@v3.1.0
 
       # Importing from the cache should create the image within approx 30 seconds:
       # NOTE: `qemu` step is not needed as we only test for AMD64.

CHANGELOG.md

@@ -36,6 +36,10 @@ The most noteworthy change of this release is the update of the container's base
       - DMS `main.cf` has renamed `postscreen_dnsbl_whitelist_threshold` to `postscreen_dnsbl_allowlist_threshold` as part of this change.
     - `smtpd_relay_restrictions` (relay policy) is now evaluated after `smtpd_recipient_restrictions` (spam policy). Previously it was evaluated before `smtpd_recipient_restrictions`. Mail to be relayed via DMS must now pass through the spam policy first.
     - The TLS fingerprint policy has changed the default from MD5 to SHA256 (_DMS does not modify this Postfix parameter, but may affect any user customizations that do_).
+- **Dovecot**
+  - The "Junk" mailbox (folder) is now referenced by it's [special-use flag `\Junk`](https://docker-mailserver.github.io/docker-mailserver/v13.3/examples/use-cases/imap-folders/) instead of an explicit mailbox. ([#3925](https://github.com/docker-mailserver/docker-mailserver/pull/3925))
+    - This provides compatibility for the Junk mailbox when it's folder name differs (_eg: Renamed to "Spam"_).
+    - Potential breakage if your deployment modifies our `spam_to_junk.sieve` sieve script (_which is created during container startup when ENV `MOVE_SPAM_TO_JUNK=1`_) that handles storing spam mail into a users "Junk" mailbox folder.
 - **rsyslog:**
   - Debian 12 adjusted the `rsyslog` configuration for the default file template from `RSYSLOG_TraditionalFileFormat` to `RSYSLOG_FileFormat` (_upstream default since 2012_). This change may affect you if you have any monitoring / analysis of log output (_eg: `mail.log` / `docker logs`_).
     - The two formats are roughly equivalent to [RFC 3164](https://www.rfc-editor.org/rfc/rfc3164)) and [RFC 5424](https://datatracker.ietf.org/doc/html/rfc5424#section-1) respectively.
@@ -52,7 +56,7 @@ The most noteworthy change of this release is the update of the container's base
       - `smtp_sasl_security_options = noanonymous` (_credentials are mandatory for outbound mail delivery_)
       - `smtp_tls_security_level = encrypt` (_the outbound MTA connection must always be secure due to credentials sent_)
 - **Environment Variables**:
-  - `SA_SPAM_SUBJECT` has been renamed into `SPAM_SUBJECT` to become anti-spam service agnostic. ([3820](https://github.com/docker-mailserver/docker-mailserver/pull/3820))
+  - `SA_SPAM_SUBJECT` has been renamed into `SPAM_SUBJECT` to become anti-spam service agnostic. ([#3820](https://github.com/docker-mailserver/docker-mailserver/pull/3820))
     - As this functionality is now handled in Dovecot via a Sieve script instead of the respective anti-spam service during Postfix processing, this feature will only apply to mail stored in Dovecot. If you have relied on this feature in a different context, it will no longer be available.
     - Rspamd previously handled this functionality via the `rewrite_subject` action which as now been disabled by default in favor of the new approach with `SPAM_SUBJECT`.
     - `SA_SPAM_SUBJECT` is now deprecated and will log a warning if used. The value is copied as a fallback to `SPAM_SUBJECT`.
@@ -61,6 +65,8 @@ The most noteworthy change of this release is the update of the container's base
     - The feature to include [`_SCORE_` tag](https://spamassassin.apache.org/full/4.0.x/doc/Mail_SpamAssassin_Conf.html#rewrite_header-subject-from-to-STRING) in your value to be replaced by the associated spam score is no longer available.
 - **Supervisord**:
   - `supervisor-app.conf` renamed to `dms-services.conf`
+- **Rspamd**:
+  - the Redis history key has been changed in order to not incorporate the hostname of the container (which is desirable in Kubernetes environments) ([#3927](https://github.com/docker-mailserver/docker-mailserver/pull/3927))
 
 ### Added
 
@@ -80,7 +86,7 @@ The most noteworthy change of this release is the update of the container's base
 - **Rspamd**:
   - The `rewrite_subject` action, is now disabled by default. It has been replaced with the new `SPAM_SUBJECT` environment variable, which implements the functionality via a Sieve script instead which is anti-spam service agnostic ([#3820](https://github.com/docker-mailserver/docker-mailserver/pull/3820))
   - `RSPAMD_NEURAL` was added and is disabled by default. If switched on it will enable the experimental Rspamd "Neural network" module to add a layer of analysis to spam detection ([#3833](https://github.com/docker-mailserver/docker-mailserver/pull/3833))
-  - The symbol weights of SPF, DKIM and DMARC have been adjusted again. Fixes a bug and includes more appropriate combinations of symbols ([#3913](https://github.com/docker-mailserver/docker-mailserver/pull/3913))
+  - The symbol weights of SPF, DKIM and DMARC have been adjusted again. Fixes a bug and includes more appropriate combinations of symbols ([#3913](https://github.com/docker-mailserver/docker-mailserver/pull/3913), [#3923](https://github.com/docker-mailserver/docker-mailserver/pull/3923))
 
 ### Fixes
 

Dockerfile

@@ -104,7 +104,6 @@ EOF
 # -----------------------------------------------
 
 COPY target/rspamd/local.d/ /etc/rspamd/local.d/
-COPY target/rspamd/scores.d/* /etc/rspamd/scores.d/
 
 # -----------------------------------------------
 # --- OAUTH2 ------------------------------------

docs/content/config/environment.md

@@ -328,10 +328,12 @@ Note: More information at <https://dovecot.org/doc/dovecot-example.conf>
 
 ##### MOVE_SPAM_TO_JUNK
 
-- 0 => Spam messages will be delivered in the mailbox.
-- **1** => Spam messages will be delivered in the `Junk` folder.
+- 0 => Spam messages will be delivered to the inbox.
+- **1** => Spam messages will be delivered to the Junk mailbox.
 
-Routes mail identified as spam into the recipient(s) Junk folder (_via a Dovecot Sieve script_).
+Routes mail identified as spam into the recipient(s) Junk mailbox (_a specialized folder for junk associated to the [special-use flag `\Junk`][docs::dovecot::special-use-flag], handled via a Dovecot sieve script internally_).
+
+[docs::dovecot::special-use-flag]: ../examples/use-cases/imap-folders.md
 
 !!! info
 

docs/content/config/security/ssl.md

@@ -134,6 +134,8 @@ Certbot provisions certificates to `/etc/letsencrypt`. Add a volume to store the
     ```
 
     This process can also be [automated via _cron_ or _systemd timers_][certbot::automated-renewal].
+    
+    - [Example with a systemd timer][certbot::automated-renewal::example-systemd-timer]
 
 !!! note "Using a different ACME CA"
 
@@ -903,6 +905,7 @@ Despite this, if you must use non-standard DH parameters or you would like to sw
 [certbot::standalone]: https://certbot.eff.org/docs/using.html#standalone
 [certbot::renew]: https://certbot.eff.org/docs/using.html#renewing-certificates
 [certbot::automated-renewal]: https://certbot.eff.org/docs/using.html#automated-renewals
+[certbot::automated-renewal::example-systemd-timer]: https://github.com/orgs/docker-mailserver/discussions/3917#discussioncomment-8661690
 [certbot::custom-ca]: https://certbot.eff.org/docs/using.htmlchanging-the-acme-server
 [certbot::webroot]: https://certbot.eff.org/docs/using.html#webroot
 

target/dovecot/90-sieve.conf

@@ -51,7 +51,7 @@ plugin {
   # deprecated imapflags extension in addition to all extensions were already
   # enabled by default.
   #sieve_extensions = +notify +imapflags
-  sieve_extensions = +notify +imapflags +vnd.dovecot.pipe +vnd.dovecot.filter
+  sieve_extensions = +notify +imapflags +special-use +vnd.dovecot.pipe +vnd.dovecot.filter
 
   # Which Sieve language extensions are ONLY available in global scripts. This
   # can be used to restrict the use of certain Sieve extensions to administrator

target/rspamd/local.d/composites.conf

@@ -0,0 +1,18 @@
+# In addition to `policies_group.conf`, this file contains
+# symbols that are applied when certain other symbols are
+# applied (or not applied).
+#
+# We are especially interested in the `policy` field, because
+# there are cases in which `remove_weight` is undesirable.
+
+# When neither SPF, DKIM, nor DMARC are available, we want
+# to increase the base score so we apply at least greylisting.
+AUTH_NA {
+  score = 2.5;
+  policy = "leave";
+}
+
+AUTH_NA_OR_FAIL {
+  score = 1;
+  policy = "leave";
+}

target/scripts/startup/setup.d/security/misc.sh

@@ -304,11 +304,11 @@ function _setup_spam_to_junk() {
     _log 'debug' 'Spam emails will be moved to the Junk folder'
     mkdir -p /usr/lib/dovecot/sieve-global/after/
     cat >/usr/lib/dovecot/sieve-global/after/spam_to_junk.sieve << EOF
-require ["fileinto","mailbox"];
+require ["fileinto","special-use"];
 
 if anyof (header :contains "X-Spam-Flag" "YES",
           header :contains "X-Spam" "Yes") {
-    fileinto "Junk";
+    fileinto :specialuse "\\\\Junk" "Junk";
 }
 EOF
     sievec /usr/lib/dovecot/sieve-global/after/spam_to_junk.sieve

target/scripts/startup/setup.d/security/rspamd.sh

@@ -126,6 +126,15 @@ function __rspamd__setup_redis() {
 servers = "127.0.0.1:6379";
 expand_keys = true;
 
+EOF
+
+    # We do not use `{{HOSTNAME}}` but only `{{COMPRES}}` to better support
+    # Kubernetes, see https://github.com/orgs/docker-mailserver/discussions/3922
+    cat >"${RSPAMD_LOCAL_D}/history_redis.conf" << "EOF"
+# documentation: https://rspamd.com/doc/modules/history_redis.html
+
+key_prefix = "rs_history{{COMPRESS}}";
+
 EOF
 
     # Here we adjust the Redis default configuration that we supply to Redis when starting it.

test/config/junk-mailbox/user-patches.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+##
+# This user script will be executed between configuration and starting daemons
+# To enable it you must save it in your config directory as "user-patches.sh"
+##
+
+echo "[user-patches.sh] Adjusting 'Junk' mailbox name to verify delivery to Junk mailbox based on special-use flag instead of mailbox's name"
+
+sed -i -e 's/mailbox Junk/mailbox Spam/' /etc/dovecot/conf.d/15-mailboxes.conf
+
+### Before / After ###
+
+# mailbox Junk {
+#   auto = subscribe
+#   special_use = \Junk
+# }
+
+# mailbox Spam {
+#   auto = subscribe
+#   special_use = \Junk
+# }

test/tests/parallel/set1/spam_virus/rspamd_full.bats

@@ -86,6 +86,18 @@ function teardown_file() { _default_teardown ; }
   refute_line --regexp 'rewrite_subject = [0-9]+;'
 }
 
+@test 'Rspamd Redis configuration is correct' {
+  _run_in_container rspamadm configdump redis
+  assert_success
+  assert_line 'expand_keys = true;'
+  assert_line 'servers = "127.0.0.1:6379";'
+
+  _run_in_container rspamadm configdump history_redis
+  assert_success
+  assert_line 'compress = true;'
+  assert_line 'key_prefix = "rs_history{{COMPRESS}}";'
+}
+
 @test "contents of '/etc/rspamd/override.d/' are copied" {
   local OVERRIDE_D='/etc/rspamd/override.d'
   _file_exists_in_container "${OVERRIDE_D}/testmodule_complicated.conf"
@@ -232,7 +244,7 @@ function teardown_file() { _default_teardown ; }
   _service_log_should_contain_string 'rspamd' 'add header "Gtube pattern"'
 
   _print_mail_log_for_msgid 'rspamd-test-email-header'
-  assert_output --partial "fileinto action: stored mail into mailbox 'Junk'"
+  assert_output --partial "fileinto action: stored mail into mailbox [SPECIAL-USE \\Junk]"
 
   _count_files_in_directory_in_container /var/mail/localhost.localdomain/user1/new/ 2
   _count_files_in_directory_in_container /var/mail/localhost.localdomain/user1/.Junk/new/ 1

test/tests/parallel/set1/spam_virus/spam_junk_folder.bats

@@ -49,9 +49,11 @@ function teardown() { _default_teardown ; }
   _should_receive_spam_at '/var/mail/localhost.localdomain/user1/new/'
 }
 
-@test "(enabled + MOVE_SPAM_TO_JUNK=1) should deliver spam message into Junk folder" {
+@test "(enabled + MOVE_SPAM_TO_JUNK=1) should deliver spam message into Junk mailbox" {
   export CONTAINER_NAME=${CONTAINER3_NAME}
 
+  _init_with_defaults
+
   local CUSTOM_SETUP_ARGUMENTS=(
     --env ENABLE_AMAVIS=1
     --env ENABLE_SPAMASSASSIN=1
@@ -60,14 +62,17 @@ function teardown() { _default_teardown ; }
     --env MOVE_SPAM_TO_JUNK=1
     --env PERMIT_DOCKER=container
   )
-  _init_with_defaults
+
+  # Adjust 'Junk' mailbox name to verify delivery to Junk mailbox based on special-use flag instead of mailbox's name
+  mv "${TEST_TMP_CONFIG}/junk-mailbox/user-patches.sh" "${TEST_TMP_CONFIG}/"
+
   _common_container_setup 'CUSTOM_SETUP_ARGUMENTS'
 
   _should_send_spam_message
   _should_be_received_by_amavis 'Passed SPAM {RelayedTaggedInbound,Quarantined}'
 
-  # Should move delivered spam to Junk folder
-  _should_receive_spam_at '/var/mail/localhost.localdomain/user1/.Junk/new/'
+  # Should move delivered spam to the Junk mailbox (adjusted to be located at '.Spam/')
+  _should_receive_spam_at '/var/mail/localhost.localdomain/user1/.Spam/new/'
 }
 
 # NOTE: Same as test for `CONTAINER3_NAME`, only differing by ENV `MARK_SPAM_AS_READ=1` + `_should_receive_spam_at` location