Pawel Foremski pjf@gna.org Christian Löschenkohl christian.loeschenkohl@linea7.com 2004 Pawel Foremski and Christian Löschenkohl This file is free documentation. Describes design, configuration and every-day usage of Helios qmail. 23-10-2004 Helios qmail is an advanced, top integrated and feature rich distribution of qmail based on qmail-sql project. It is mostly written in PHP to make it`s adaption easy. Helios qmail has most of the features that modern MTA should have. This manual describes Helios qmail in detail - it`s design, configuration, users managment and extending. Note that you will not find here installation instructions - you can find them in document called Helios qmail installation guide, which is bundled with Helios qmail package and available for download in several formats from http://helios-qmail.sourceforge.net/#DOCS. This document is also distributed in the same way. Helios qmail is an advanced, top integrated and feature rich distribution of qmail based on qmail-sql project. It has been created because stock qmail is quite useless these days - this project collects many popular patches, tools and extensions and links them using easy-to-adapt scripts and APIs. Helios qmail can be treated as an MTA which combines security, stability and reliability of qmail with flexibility, modern functionality and ease of use. As of 0.9.0 release, Helios qmail supports: MySQL database SMTP service: SSL/TLS SMTP authorization SMTP-after-POP/IMAP ESMTP SIZE support antispam protections like: SPF (Sender Policy Framework) antispoof RBL checks wildcards support in badhelo, badmailfrom and badrcptto files envelope sender and HELO/EHLO domains DNS checks tarpitting advanced queue filtering - supports several content scanners, such as: SpamAssassin Bogofilter Clam AntiVirus Mks_Vir extensive logging SPP (SMTP plugin patch) support POP3 service: SSL/TLS PLAIN and APOP authorization IMAP service - BINC IMAP SSL/TLS integrated and centralized configuration in /var/qmail/control PHP5 libraries providing nice API for shell scripts and/or web applications single-uid delivery new mail notification support (through Jabber) many PHP scripts to help managing the system (add/delete/modify/show users/domains/aliases/etc settings) Maildir++ support and more... If you are wondering why script language has been chosen to extend qmail, the answer is simple - to ease adaption of the system and to make it flexible. PHP has been chosen for such function because it is one of the most advanced and top-level script languages and to make Helios qmail easy to integrate with web applications. Helios qmail has been designed also to make it`s extending easy. Want an example? Here is a short PHP code (a spp module) which implements checking for maximal number of recipients: #!/usr/local/bin/php <!php if( $_ENV['SMTPRCPTCOUNT'] >= $_ENV['MAXRECIPIENTS'] ) { echo("550 sorry, too many recipients (#5.7.1)\r\n"); exit(1); } exit(0); ?> You do not know PHP? No problem, here is a bash implementation: #!/bin/bash if [ $SMTPRCPTCOUNT -ge $MAXRECIPIENTS ]; then echo "550 sorry, too many recipients (#5.7.1)" exit 1 fi exit 0 Such module can be written also in any other language. Helios qmail supports extending many of it`s elements in such (easy) way. Helios qmail is intended to be used by users who have already successfuly installed qmail and have been using it for some time. In other words, it is not recommended for qmail beginners, because of one reason - they probably will not benefit of most of it`s features. If you are a beginner, try stock qmail first and read some more documentation about it, like Life with qmail. If you are an advanced user you will gain from it`s flexibility and many integrated features. Probably the most important advantage of Helios qmail is that it is database driven - it causes speed improvement and easier managment of users and domains. If you are a power user, almost a "qmail hacker", you will probably love it`s extensibility - API, spp plugins, advanced content scanning, etc... It`s just made for buliding big mail systems, integrating it with other applications and making any adaptions. You will not find installation instructions here - you can find them in document called Helios qmail installation guide, which is bundled with Helios qmail package and available for download in several formats from http://helios-qmail.sourceforge.net/#DOCS. The main configuration file is located in /var/qmail/control/qmailconfig (it is a symbolic link to sqlserver file for compatibility with qmail-sql). It is divided into three parts: "System configuration", "Database configuration" and "Advanced options". Be sure to restart daemontools after changing this file. This part contains several global options. mail_root directory Main directory for mail (without trailing slash); "domain/" and "domain/login" directories will be created there. default_user username Contains real username to be assigned to new Helios qmail user. default_group group Contains real group to be assigned to new Helios qmail user. rcpthosts_from_db { yes, no } Should Helios qmail read rcpthosts from the database if the rcpthosts file is missing. If it is turned off, and rcpthosts is missing, your mailserver will act as an open-relay. allowed_recipients_from_db { yes, no } Verify or not whether local envelope recipients exists during SMTP session. localpop domain A domain to add to login during authorization if it is missing. domain_separators chars A list of chars that can be used during authorization by clients to separate login from domain name. check_host { yes, no } Set it to no if you do not want SQL queries to contain virtual host name (not recommended). globaldomain domain If set, system searches for login@globaldomain user if it do not find it in requested domain. host_postmaster email Default postmaster`s email for new domains. autohomedir path Path to program creating user`s home directory on demand. Be sure to correct this options after first installation. connectionstring txt Configures connection to the database. These options are in the format required by PostgreSQL (PQconnectdb). For compatibility reasons this field is parsed to get the parameters for connecting with a MySQL server. An example: connectionstring user=userconsult password=readonly dbname=userdb host=localhost Other MySQL options (eg. port number) are not recognized. popconnectionstring txt The same as connectionstring, but this field contains data used for writing to the database (for example by POP3). connection_timeout secs How long (in seconds) should Helios qmail try to connect to the database before returning an error. "0" means no limit. query_timeout secs How long (in seconds) should Helios qmail wait for an answer from the database before returning an error. "0" means no limit. There are several next directives after these options letting you to change the database structure (such as table and field names). This section contains information regarding configuration of SMTP service. Some qmail options (not Helios qmail specific) are not mentioned here - for more information see qmail documentation or Life with qmail. SMTP service can be easily configured using /var/qmail/smtpconfig file. It is a shell script, so you can also add here some start instructions if you want - this file is reloaded each time daemontools is started. Be sure to restart daemontools after changing this file. This file is well commented, you should not have problems modifying it. However, below are some notes regarding it. If you want Helios qmail to automatically authorize users basing on POP3 or IMAP authorization which took place some time before, enable SMTP-after-POP3/IMAP. Simply set AUTHEXPIRE environment variable to number of minutes that POP3/IMAP authorization should be valid for SMTP too and export it. You can also do it using SMTP TCP rules if you want host-specific configuration. It is highly recommended to set memory limit on qmail SMTP service to prevent it from DoS attacks. Required amount of memory can vary on several systems, so if you see errors in your logs saying that it cannot allocate memory, increase it. You should also do it if you are going to use a content scanner which needs a lot of memory - remember that it is executed within SMTP session. If you have chosen to enable SSL/TLS support in Helios qmail, you can also configure some of stunnel options in /var/qmail/smtpconfig-stunnel file. You can set rules for tcpserver using /var/qmail/control/smtp-tcprules file. See http://cr.yp.to/ucspi-tcp/tcprules.html for more information on TCP rules and it`s usage. Always use /var/qmail/helios/qmailctl.sh cdb to reload TCP rules after changing it. Configuration of POP3 service is similar to SMTP configuration - you can do this by editing /var/qmail/control/pop3config and (if you have enabled SSL/TLS support) /var/qmail/control/pop3config-stunnel files. Be sure to restart daemontools after changing these files. Helios qmail supports IMAP using BINC IMAP - it is an independent project, so it`s configuration is beyond this document. For such information consult document called Life With BINC IMAP. However, you can configure some elements of BINC IMAP related to Helios qmail in similar to SMTP and POP3 configuration way, using /var/qmail/control/imapconfig, /var/qmail/control/imapconfig-stunnel and /var/qmail/control/imapconfig-binc, which is the main configuration file of BINC IMAP. Be sure to restart daemontools after changing these files. Helios qmail has an advanced content scanner (a qmail-qfilter filter) which scans mail message before it is accepted for delivery. It delays SMTP session, but queue scanner has many advantages - eg. it allows you to deny mail without bouncing it (for example if it is a virus message). Helios qmail once has delivery time scanners, but it was a waste of system resources to eg. scan mail to x recipients x times. You can add several queue filters (executed by qmail-qfilter) to /var/qmail/queue file. Consult qmail-qfilter documentation for more information on writing such programs. Helios qmail contains one big qmail-qfilter filter called helios-scanner.php. It allows you to create your own filtering rules (eg. matching a regexps, passing to an external scanner) and to define custom actions (like adding a header, denying mail with special error codes, eg. virus warning). Helios scanner can be configured using /var/qmail/control/scanner file. It`s syntax is quite intuitive and it is well commented - you should not have problems editing it. Each scanner section starts with it`s name put between square brackets ('[' and ']') - eg. [section], than you can define some options (currently only max_size which contains maximal size of mail message to run checks included in this section), checks (scanners) to execute and actions to take if scanners detect something. If you are defining some texts, remember that you only need to escape (using '\') characters that you are using as opening and closing char (like " and '). Be sure to enable Helios scanner in /var/qmail/control/queue before trying to use it Scanners are defined between <scanners> and </scanners> tags. Each section should have defined scanners, if not, it is assumed that actions should be taken without checking for message content. However, if <scanners> are empty (but there are opening and closing tags), section is skipped. Below is the list of commands that can be included in scanners definitions. match_txt txt Searches for txt in whole message. match_txti txt Case insensitive version of match_txt. match_regexp txt Searches for a regular expression (POSIX Extended) defined in txt in whole message. match_pcre txt Searches for a regular expression (Perl-Compatible) defined in txt in whole message. Consult PHP documentation for documentation of txt`s syntax. match_header_* The same as match_*, but it searches only in message headers. match_body_* The same as match_*, but it searches only in message body, without it`s headers. scan_spamc "/path/to/spamc" "spamc-options" and scan_spamassassin "/path/to/spamassassin" "spamassassin-options" Pass the message to SpamAssassin (using spamc or spamassassin programs) and check if it is a spam. If is, execute actions. It is recommended to use SpamAssassin in daemon mode, because otherwise you will have to increase SMTP memory limit even to about 50 megabytes (!). scan_clamdscan "/path/to/clamdscan" "clamdscan-options" and scan_clamscan "/path/to/clamscan" "clamscan-options" Pass the message to Clam AntiVirus (using clamdscan or clamscan programs) and check it for viruses. If the message is a virus, execute actions. It is recommended to use Clam AntiVirus in daemon mode. scan_bogofilter "/path/to/bogofilter" Pass the message to Bogofilter and check if it is a spam. Actions are defined in the same way as scanners are - between <actions> and </actions> tags. Here is a list of supported actions. del_header header-name Deletes header which name is header-name. del_header_pcre header-name Version of del_header which supports Perl-compatible regular expressions in header-name. add_body_top txt and add_body_end txt Adds txt to the beginning or end of message body. add_header name contents Adds header "name: contents". add_header_safe name contents [ suffix = 'Old' ] Like above, but if name header already exists, it is moved to name-suffix first. mod_header name content [ new-name = name ] Changes header content and name (if new-name is specified). add_subject txt Add txt at the beginning of 'Subject' header. add_subject_end txt Add txt at the end of 'Subject' header. exec_filter filter Pass mail through filter program. drop Silently drops the message. exit_virus Exits with error code 121, which means that mail will be denied and following text will be displayed: "554 YOUR MESSAGE CONTAINS A VIRUS! (#5.6.0)". exit_spam Exits with error code 122, which means that mail will be denied and following text will be displayed: "554 we do not accept SPAM (#5.6.0)". exit_error Exits with error code 31, which means that mail will be denied and following text will be displayed: "554 mail server permanently rejected message (#5.3.0)". exit_temp_error Exits with error code 71, which means that mail will be temporarily denied and following text will be displayed: "451 mail server temporarily rejected message (#4.3.0)". exit error-code Exits with error-code. exit_ok Immediately accepts mail for delivery (skips remaining scanners). Helios qmail has support for most of modern spam prevention techniques - eg. RBL, SPF, content scanning, tarpitting. This section describes how to enable and use some of them. Also the next section (regarding spp plugins) and the previous section (regarding content scanning) contain some information about Helios qmail features that can be used to prevent spam. RBL (Realtime Blackhole List) is a system which lists IP addresses of hosts whose owners refuse to stop the proliferation of spam. Each time some host connects to your SMTP service, you can check whether it is listed in specified RBL list and block connection if it is. To enable RBL checks edit SMTPRBLS variable in /var/qmail/control/smtpconfig file and restart daemontools. SPF (Sender Policy Framework) is a system which protects the envelope return-path. It means that you can check if remote host has rights to send mail from specified address. See http://spf.pobox.com/ for details. SPF is implemented as a spp plugin, so you can enable it by editing /var/qmail/control/smtpplugins file (you do not need to restart anything) if you have enabled it during installation. Apart of that you also need to set up an environmental variable called SPF (either using TCP rules or SMTP configure file) to one of the following values: 0 - never do SPF lookups 1 - reject mails when SPF resolves to fail (deny) - recommended setting 2 - reject mails when SPF resolves to softfail 3 - reject mails when SPF resolves to neutral 4 - reject mails when SPF resolves to none 5 - reject mails when SPF resolves to error 6 - reject mails when SPF resolves to unknown These files allows you to deny several strings in SMTP commands. /var/qmail/control/badmailfrom You can put here mail addresses that will not be accepted as argument of SMTP 'MAIL FROM:' command. /var/qmail/control/badrcptto You can put here mail addresses that will not be accepted as argument of SMTP 'RCPT TO:' command. /var/qmail/control/badhelo You can put here host names that will not be accepted as argument of SMTP 'HELO' and 'EHLO' commands. Note that you need to enable spp plugin called badhelo.php first. Tarpitting allows you to delay SMTP session if the message that remote host is currently sending to you has too many recipients. To turn it on, enable tarpit.php plugin in /var/qmail/control/smtpplugin file, set TARPITCOUNT variable to number of recipients after which tarpitting should be started and TARPITDELAY to number of seconds to sleep after 'RCPT TO'. You can also set TARPITDELAY to one the following values: NORMAL - sleep (number of recipients - TARPITCOUNT + 1) x 2 seconds MEDIUM - sleep (number of recipients - TARPITCOUNT + 1) x 5 seconds HARD - sleep (number of recipients - TARPITCOUNT + 1)2 seconds RAND - sleep for random number of seconds (between 1 and 50) qmail-spp (see http://qmail-spp.sourceforge.net/ for more iformation) is a small patch for qmail which allows you to enhance it`s functionality using external modules (written independently, in any programming language) without necessity of recompiling it or even restarting. Helios qmail comes with it and lets you for easy enabling and disabling several spp plugins by editing /var/qmail/control/smtpplugins file. Here is a list of modules bundled with Helios qmail with short description. If you want to write your own module, see section called "spp plugins" in chapter 4. antispoof.php - enables address spoofing prevention. See "Antispoof" section in next chapter for details. Needs ANTISPOOF variable to be set to run. badhelo.php - enables support for denying bad host names in HELO/EHLO SMTP commands. See section 4.3 for details. Needs BADHELO variable to be set to run. badmailfrom.php - badrcptto.php - helodnscheck.php - maxbouncercpts.php - allows for setting maximal number of envelope recipients for a bounce message in MAXBOUNCERECIPIENTS variable ("" and "0" means no recipients). Recommended setting: "1". maxrcpts.php - allows for setting maximal number of envelope recipients in MAXRECIPIENTS variable ("" and "0" means no recipients). mfdnscheck.php - checks if envelope sender domain exists. Needs MFDNSCHECK variable to be set to run check. rcptexists.php - spf - enables support for SPF. See section 4.2 for details. tarpit.php - implements tarpitting. See section 4.4 for details. There are also more things that can be configured. Here is a short list of /var/qmail/control/ files that can be used to change these settings. For more information consult documentation of qmail. bouncemaxbytes - sets maximal size of a bounce message (in bytes) that can be sent back to the sender. If the message is over that limit, it is truncated. bouncemessage - custom text of a bounce message bouncesubject - custom subject of a bounce message doublebouncemessage - custom text of a double bounce message doublebouncesubject - custom subject of a double bounce message defaultdelivery - sets default .qmail file This chapter describes users managment using scripts bundled with Helios qmail package which are located in /var/qmail/helios/ directory (if you have not chosen to change default home dir of qmail). However, it is also possible to do that using only SQL queries (for advanced control) or a web interface, which is very easy to write using Helios qmail API. If you want to write your own administering interface, see section 1 in the next chapter. Be sure to run these scripts as root user or any other user which has permissions to read /var/qmail/control/qmailconfig file. Helios qmail comes with simple scripts for adding and deleting domains, users and aliases. They are located in /var/qmail/helios/ directory and called helios-(add|del)(domain|user|alias).php, eg. helios-adddomain.php, helios-deluser.php, helios-delalias.php and so on. Their usage is very intuitive, so they will not be described extensively here - simply execute such script with --help option to print short information on using it. Always run /var/qmail/helios/qmailctl.sh hup after adding or deleting a domain to let qmail know about it. There is only one important thing - when you are adding a domain (using helios-adddomain.php script) be sure that you will not create a loop postmaster alias - such situation can be caused by adding your main domain (usually this from /var/qmail/control/me file) without -p option. This will cause that a postmaster@YOUR-MAIN-DOMAIN alias pointing to host postmaster, which is set to postmaster@YOUR-MAIN-DOMAIN by default, will be created, which will lead to mail loops. Always use -p option with helios-adddomain.php script when adding the domain of host postmaster`s email address (see information on host_postmaster option in main configuration file). You can change user password using /var/qmail/helios/helios-passwd.php script. Execute it with --help option for usage information. For example type /var/qmail/helios/helios-passwd.php login domain foo to set login@domain`s password to "foo". You can also type only /var/qmail/helios/helios-passwd.php login domain to read user`s password from the keyboard. To set user`s .qmail file use /var/qmail/helios/helios-setdotqmail.php script. Simply pass it`s contents on standard input (eg. using redirection of echo command output) and give login, domain and optional extension as script arguments. For example type echo "./Maildir/" | /var/qmail/helios/helios-passwd.php johny domain.org ext to set johny-ext@domain.org`s .qmail to "./Maildir/". If you want to delete user`s .qmail file use /var/qmail/helios/helios-deldotqmail.php script. ".qmail file" is not a file - in fact it is saved in the database. One of the most unique features of Helios qmail is it`s support for prevention of e-mail addresses spoofing. It is similar to SPF, but it deal only with local domains, which allows for more precise controlling of rights to specified local e-mail address. It is basing on SMTP authorization and allows you to specify which SMTP user is allowed to mail from given e-mail address. Antispoof in conjunction with SPF creates a complete system for stopping envelope sender forgeries. Helios qmail comes with antispoof disabled by default, because it is important to know how to use it before enabling it. You can turn it on by editing /var/qmail/control/smtpplugins file (see section 5 in previous chapter for details). You can edit each user`s list of permitted addresses by using /var/qmail/helios/helios-antispoofadmin.php script (execute it with --help option for usage information). You can also edit following virtual wildcard users for matching a wider group of users: [:all:]@[:all:] - matches any authorized user [:all:]@domain - matches any user in specified domain user@[:all:] - matches specified user in any domain [:none:]@[:none:] - matches not authorized user Such list should contain e-mail addresses (one per line) which user can use as envelope sender address. But there are also some useful strings which you can use as login and/or domain name. Below are the lists of such special strings. List of strings which can be used as e-mail address: [:any:] - allow user to mail from any e-mail address [:none:] - do not allow user to send mails from any e-mail address [:aliases:] - allow user to mail from any of his aliases [:valid user:] - allow user to mail from e-mail address of any valid user (which exists in the system) List of strings which can be used as login: [:any:] - matches any login name [:none:] - disallows mailing from any login in specified domain, eg. [:none:]@foo.com disallows user to mail from *@foo.com [:login:] - matches login of authorized user, eg. [:login:]@foo.com in joe@domain.org`s list allows him to mail from joe@foo.com. Useful with wildcards. [:my users:] - matches any valid login name (which exists in the system) List of strings which can be used as domain name: [:any:] - matches any domain name [:none:] - disallows mailing from specified login at any domain, eg. admin@[:none:] disallows user to mail from admin@* [:domain:] - matches domain name of authorized user, eg. webmaster@[:domain:] in tom@domain.org`s list allows him to mail from webmaster@domain.org. Useful with wildcards. [:my domains:] - matches any valid domain name (which exists in the system) You can also use exclamation mark as first character to negate matching, eg. "! [:any:]@[:my domains:]" allows mailing from any foreign address. Helios qmail comes with following antispoof settings by default: User Permissions list Explanation [:none:]@[:none:] ! [:any:]@[:my domains:] This setting is required to allow remote hosts (which do not authorize on your server) to send mail to you (from foreign addresses). [:any:]@[:any:] [:login:]@[:domain:] Allows any authorized user to use his mail address. [:aliases:] Allows any authorized user to use his aliases as envelope sender address. ! [:any:]@[:my domains:] Allows any authorized user to mail from any foreign address. If the user is not allowed to mail from given e-mail address (but he tries to do this), and he is not authorized, he gets "530 authentication required (#4.7.1)" error, but if he has already authorized, he gets "550 permission denied (#5.7.0)" error. All such tries are logged. Helios qmail supports notifying about new mails through Jabber protocol. This allows you to send notifications to your mobile phone, other instant messaging communicators, etc. using Jabber transports, so it can be really useful (for example when you are waiting for an important e-mail). New mail notification works as follows: when new mail arrives, it is passed to /var/qmail/helios/helios-notify.php script (see default .qmail file in /var/qmail/control/defaultdelivery). Then, it reads the message and creates a notify message containing some information about mail, like sender, subject and beginning of text of the message. Next, it logs on a Jabber server using user`s JID and password (there is no system-wide account for sending Jabber notifications) and sends notify message to the given Jabber address (called "send-to"). To set up a new mail notification for specified user (or a group of users - wildcard string "[:all:]" can be used as login and/or domain name) use /var/qmail/helios/helios-notifyadmin.php script (execute it with --help option for usage information). There are two scripts for getting information about users in your Helios qmail system. It`s usage is trivial, so they will not be described much here. For more information, execute them with --help option. /var/qmail/helios/helios-finger.php - shows complete information about given user(s) /var/qmail/helios/helios-listdomain.php - shows all users in given domain This chapter contains only some notes on extending Helios qmail, please do not treat it as a "HOW-TO". If you want to use Helios qmail API you should be an experienced PHP programmer. There is no extensive documentation of it, because it is no problem for such person to look at the source code and just use it. However, such documentation may be written some day, but now there are only few notes about it. First of all, main Helios qmail API consist of four libraries: low-level - QmailSql.php and QmailSqlUser.php and high-level - QmailHelios.php and QmailHeliosUser.php. It can also be divided in another way: system-oriented - QmailSql.php and QmailHelios.php and user-oriented - QmailSqlUser.php and QmailHeliosUser.php. All libraries are located in /var/qmail/helios/lib directory. QmailSql.php and QmailSqlUser.php are 100% compatible with stock qmail-sql, so you can use them for writing tools for it. If you will have problems with using the API look at the /var/qmail/helios/helios-*.php scripts and user-oriented classes - they are all like a one big example. Please take a look at qmail-spp documentation for more information.