* docs: Relocate account / auth pages into a common section
* docs: Update references to relocated pages
* docs: Add account management overview page
Updates remaining links to account sections on this page instead (_for `accounts`, `aliases`, `quotas`_).
This page will cover the features and defer to separate pages for more specific content where relevant.
* docs: Correct relocated pages titles and links
* docs: Accounts (Dovecot Master) - Minor revisions
* docs: Fix highlighting roundcube PHP snippet in OAuth2 page
* docs: Accounts (File) - Refactor
- Manual method not necessary to document.
- Condense `setup` example guidance.
- Quotas / Aliases content migrated to Overview when not specific about file provisioner.
Some of the content is this commit is not a complete revision.
* chore: Temporary commit
* docs(refactor): Sub-addressing section
Much better docs on the sub-addressing feature supported by Postfix and Dovecot, along with the guidance with usage in Sieve.
* docs: Revise accounts section
Add some context regarding DMS accounts and their distinction/overlap from the email address functionality, and it's relevant context for receiving/sending.
File provisioner, minor revisions to referencing associated config files and account management.
* docs: Minor adjustments
* docs: Refactor the quota section
Better documented with links and coverage over the workaround details we've implemented.
* docs: Revise the quota section
Minor revisions with phrasing, admonitions for structure and better explanation of the feature functionality/purpose.
* docs: Alias section refactor
Extensively covers known issues and technical details that have been discussed often enough.
The improvements should benefit both users and maintainers.
* docs: Refactor master accounts page
This rewrite should more clearly document the feature, along with a better example and additional links for reference.
* docs: OAuth2 revision
Minor update to this page:
- Links extracted to bottom of page as per convention.
- ENV file example converted to preferred `compose.yaml` ENV settings.
* docs: Sieve minor revisions
- Correct link to subaddressing section
- Make the config file example snippets intended filename less ambiguous.
- Minor rephrasng.
* docs: Revise accounts overview section
Revised the account section and added additional clarity for common confusion with relation to sender address and multi-domain support.
Top of the page now clarifies it's a technical reference and directs users to the related pages for configuration / caveats.
Technical Overview links to Dovecot docs were missing.
* docs: Another revision pass
File based provisioner docs:
- Sections indent with info admonitions.
- Accounts section expanded with config format and example.
- Quotas section expanded and shifted to bottom (alphabetical sort).
- Split into `setup` CLI and config reference groups.
Overview page:
- Sections indent with info admonitions.
- Revised content.
* docs(chore): Shift sub-addressing section
This is related to accounts and aliases, but not provisioners, thus extract out of the accounts parent section.
* docs: Document `postfix-accounts.cf` third column
This lacked documentation but was community contributed feature to allow further customization of a Dovecot Account.
It has caveats as DMS does not take these into consideration anywhere in scripts. Documenting officially for better awareness.
* docs: Revise and expand supplementary pages
Better outline the OAuth2 login process, the two supported login mechanisms and their docs/rfcs, along with documenting caveat with mail client compatibility.
Add a verification tip for the OAuth2 support, showing how `curl` can be used, along with caveat presently affecting the `curl` in DMS v14.
Additionally note the feature still isn't documented fully, providing the user with additional references for more information.
`ACCOUNT_PROVISIONER` ENV docs minimized. No `OIDC` provisioner plans, the OAuth2 docs page now mentions SCIM 2.0 API as the next step towards resolving that concern. The tip admonition was removed as it no longer provides value, instead we link to the Account Management overview page.
Dovecot Master Accounts docs page now lightly document the `setup` CLI and config format for the feature.
* docs: Fix broken anchor links
Some anchor links to different parts of our docs have gone stale. This branch also broke a few itself that I missed.
The build now only reports issues with anchor links to Content Tabs, which it must not be aware of during the build (_MKDocs Material specific feature?_)
* docs(lint): Fix indentation level
* chore: Add entry to `CHANGELOG.md` + corrections
* docs: change some absolute links to relative links
* docs: change most hard-coded links to `edge` to point to `latest`
* Apply suggestions from code review
* docs: revert 404 page to edge and change canonical link to `latest
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
* add functionality for filtering mail log by ID
This was not planned, but as @polarthene mentioned in
https://github.com/docker-mailserver/docker-mailserver/pull/3033#issuecomment-1407169569
, filtering the mail log by email ID would be (the only) correct
approach for the Rspamd test (to eliminate race conditions).
I asserted the currect state, and came to the conclusion that this might
(or actually is) something we want in more than one place. So I went
ahead and implemented a solution.
The solution for acquiring the ID is a bit slower because it ensures the
mail queue is empty _before_ and _after_ the mail is sent. This is the
tradeoff one has to make if they want to send multiple emails in one
test file and get their IDs.
I hope you like this approach. I will provide another PR that adjusts
our current tests to use these new functions.
* added note about our helper functions in the docs
I think our work for our custom test framework should be noted in the
docs for newcomers to better understand what they should do.
* adjust Rspamd test to use new helpers for sending
* improve filter helpers further
* add sanity check when acquiring mail ID
* re-add `refute_output` to test which should now work well
`tls.bash` helper was adapted to the new helper scripts location. The `setup.bash` helper saw a bugfix (expanding the array properly) and updates the container default config to configure for IPv4 explicitly.
The IPv4 default was added after recent Docker pushes and I saw weird IPv6 related errors in the logs.. now we're sure IPv4 is the default during tests.
Added functionality to check if a process is running:
- This change adds a helper function to check whether a program is running inside a container or not.
- This added the need for a function like `_run_in_container` but allowing for providing an explicit container name.
- Future PRs can use this helper function now to check whether a process is running or not. This was done for the tests of Fail2Ban, but can be used for other tests in the future as well.
---
chore: Restructured BATS flags in `Makefile`
The `Makefile` has seen a bit of a restructuring when it comes to flags:
1. The `MAKEFLAGS` variables is used by `make`, and allows for adding additional flags that can be used within in recursive calls (via `$(MAKE)`) too, thus DRY approach.
2. The flags for calling BATS were adjusted. `--no-parallelize-within-files` has been added as well to ensure tests _inside_ a single file are run sequentially.
`dms-test` prefix matching changed to expect a `_` suffix as a delimiter.
---
docs: Add a note regarding output from running tests in parallel
These updates support running tests that have been relocated into `serial` and `parallel/set*` directories.
- `make tests` now calls the two make targets beneath it. The only difference is that `serial` continues the "1 test at a time" approach used prior to this PR, while the `parallel` target increases the `--jobs` arg to run multiple tests concurrently (_configured by `PARALLEL_JOBS`_).
- The `test/%` target leverages Bash syntax magic to ease running single tests without providing the exact path.
- This syntax also supports providing multiple test names (eg: `make test/clamav,template`) to run.
- `**` (globstar) allows for future improvements that can group multiple test files into sub-directories by their scope (eg: anti-spam, ssl, etc).
---
chore: Add `shopt -s globstar` to other targets
I realized that other targets should have this as well in case it is not set.
It is better to be more explicit here than to have weird errors due to `**` not expanding properly.
---
fix(Makefile): Add back `.PHONY` targets
I encountered `make` telling me the target was already up-to-date, which of course is nonsense.
I therefore added back the `.PHONY` targets to ensure tests are always run.
---
docs: Added instructions for running a single test
See https://github.com/docker-mailserver/docker-mailserver/pull/2857/files#r1008582760
* outsourcing env variable setup
This commit contains major parts of the work of refactoring the setup
and usage of environment variables. It outsources the setup into its own
script and provides dedicated functions to be executed at a later point in time.
A **new** env variable was added: `USER_PROVISIONG` which provides a
better way of defining which method / protocol to use when it comes to
setting up users. This way, the `ENABLE_LDAP` variable is deprecated,
but all of this is backwards compatible due to a "compatibility layer", a function provided by the new variables script.
This is not a breaking change. It mostly refators internal scripts. The
only change facing the user-side is the deprecation of `ENABLE_LDAP`. We
can prolong the period of deprecation for this variable as long as we
want, because the new function that ensures backwards compatibility
provides a clean interface for the future.
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
* added new `_log` function
With `_log`, the `_notify` method wa rendered obsolete. `_notify` was
not completely removed due to test failures in `check-for-changes.sh`.
The new `_log` function properly uses log levels such as `trace`,
`debug`, `info`, `warn` and `error`. It provides a cleaner solution
and renders `DMS_DEBUG` obsolete too (as only `_notify` depends on it).
* converted all helper script to new `_log` function
* converted all startup stacks to new `log` function
* `start-mailserver.sh` now uses new `_log` function
* final test and misc small script adjustments
* updated documentation
"Brief" summary/overview of changes. See the PR discussion or individual commits from the PR for more details.
---
Only applies to the `docs/content/**` content (_and `setup` command_). `target/` and `test/` can be normalized at a later date.
* Normalize to `example.com`
- Domains normalized to `example.com`: `mywebserver.com`, `myserver.tld`, `domain.com`, `domain.tld`, `mydomain.net`, `my-domain.tld`, `my-domain.com`, `example.org`, `whoami.com`.
- Alternative domains normalized to `not-example.com`: `otherdomain.com`, `otherdomain.tld`, `domain2.tld`, `mybackupmx.com`, `whoareyou.org`.
- Email addresses normalized to `admin@example.com` (in `ssl.md`): `foo@bar.com`, `yourcurrentemail@gmail.com`, `email@email.com`, `admin@domain.tld`.
- Email addresses normalized to `external-account@gmail.com`: `bill@gates321boom.com`, `external@gmail.com`, `myemail@gmail.com`, `real-email-address@external-domain.com`.
- **`faq.md`:** A FAQ entry title with `sample.domain.com` changed to `subdomain.example.com`.
- **`mail-fetchmail.md`:** Config examples with FQDNs for `imap`/`pop3` used `example.com` domain for a third-party, changed to `gmail.com` as more familiar third-party/external MTA.
* Normalize config volume path
- Normalizing local config path references to `./docker-data/dms/config/`: `./config/`, `config/`, \``config`\`, `/etc/` (_volume mount src path prefix_).
- Normalize DMS volume paths to `docker-data/dms/mail-{data,state,log}`: `./mail`, `./mail-state` `./data/mail`, `./data/state`, `./data/logs`, `./data/maildata`, `./data/mailstate`, `./data/maillogs`, (_dropped/converted data volumes: `maildata`, `mailstate`_).
- Other docker images also adopt the `docker-data/{service name}/` prefix.
* `ssl.md` - Use `dms/custom-certs` where appropriate.
* Apply normalizations to README and example `docker-compose.yml`
---
Common terms, sometimes interchangeably used or now invalid depending on context: `mail`, `mail container`, `mail server`, `mail-server`, `mailserver`,`docker-mailserver`, `Docker Mailserver`.
Rough transformations applied to most matches (_conditionally, depending on context_):
- 'Docker Mailserver' => '`docker-mailserver`'
- 'mail container' => '`docker-mailserver`' (_optionally retaining ' container'_)
- 'mail server' => 'mail-server' / '`docker-mailserver`'
- 'mail-server' => '`docker-mailserver`'
- 'mailserver' => 'mail-server' / '`docker-mailserver`'
Additionally I checked `docker run` (_plus `exec`, `logs`, etc, sub-commands_) and `docker-compose` commands. Often finding usage of `mail` instead of the expected `mailserver`
Additionally changes `mailserver` hostname in k8s to `mail` to align with other non-k8s examples.
---
* drive-by revisions
Mostly minor revisions or improvements to docs that aren't related to normalization effort.
Brennan Kinney
Andreas Perhab
Georg Lauterbach
Casper
Jack Pearson
NorseGaud
William Desportes
Georg Lauterbach
Nathan Pierce
Semir Patel
wernerfred