Difference between revisions of "Mailserver"

Revision as of 09:10, 17 December 2013

Note: If you install the Debian/Ubuntu packages of Group-Office all of this will be configured automatically and you can easily upgrade that. This page is just a reference for setting it up on other systems then Debian or Ubuntu. Read about installing the Debian packages here: Installing_on_Debian_or_Ubuntu#Complete_mailserver_solution

Group-Office can work well with any existing mail environment that supports IMAP. But to use some very interesting integrated features like:

1. Automatic mailbox creation when you create a Group-Office user. This will also immediately setup the account in Group-Office. The password of Group-Office and the mailbox will stay in sync.
2. Sieve filtering rules including vacation auto response
3. Complete postfix user and domain management module in Group-Office
4. With fetchmail you can have your remote mail delivered to your own server mailboxes. It's described at the page Use POP3 on a remote catch-all mailbox for local delivery

You can setup a mailserver for Group-Office that is built from widely used and stable software:

1. Group-Office module called postfixadmin
2. Postfix
3. Dovecot
4. MySQL
5. Spamassassin Anti spam
6. ClamAV Antivirus
7. Amavis

I've written a tutorial about how to install this on Debian 5. But it works just as well on other Linux distributions. This was written at the time Debian 5 (Lenny) was stable. Some of this may have changed.

Install all the necessary software on the server

On Debian you can do:

apt-get install php5 php5-mysql php5-imap php5-gd php5-cli php5-curl apache2 postfix postfix-mysql mysql-server mysql-client phpmyadmin subversion dovecot-imapd libemail-valid-perl libmail-sendmail-perl spamassassin spamc zip tnef libwbxml2-utils sudo openssl pyzor razor rsync mailutils libmime-charset-perl libmime-encwords-perl

You can install a Debian 5 base system and run this command. This will install all packages that are required for Group-Office and the mailserver.

Install Group-Office

Follow the installation instructions that come with Group-Office

Install postfixadmin module

Login to Group-Office and go to Admin menu -> modules. From there click "Install" to install the Postfix admin module.

Configure postfix

Postfix must be configured to use MySQL:

The following variables in main.cf. Start with filling some basic info:

# postconf -e 'myhostname = mx-1.example.com'
# postconf -e 'smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)'
# postconf -e 'biff = no'
# postconf -e 'append_dot_mydomain = no'
# postconf -e 'myorigin = example.com'
# postconf -e 'mydestination = $myhostname, localhost, localhost.localdomain'
# postconf -e 'mynetworks = 127.0.0.0/8'
# postconf -e 'recipient_delimiter = +'
# postconf -e 'inet_interfaces = all'
# postconf -e 'proxy_read_maps = $local_recipient_maps $mydestination \
    $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps \
    $virtual_mailbox_domains $relay_recipient_maps $relay_domains \
    $canonical_maps $sender_canonical_maps $recipient_canonical_maps \
    $relocated_maps $transport_maps $mynetworks $virtual_mailbox_limit_maps' 
    
# postconf -e 'smtpd_recipient_restrictions = \
	permit_mynetworks, \
	reject_unauth_destination, \
	reject_rbl_client zen.spamhaus.org, \
	permit'
# postconf -e 'message_size_limit=52428800'
# postconf -e 'mailbox_size_limit=0'

Now configure it for Virtual users in MySQL:

# postconf -e 'virtual_alias_domains ='
# postconf -e 'virtual_alias_maps = proxy:mysql:$config_directory/mysql_virtual_alias_maps.cf'
# postconf -e 'virtual_mailbox_domains = proxy:mysql:$config_directory/mysql_virtual_domains_maps.cf'
# postconf -e 'virtual_mailbox_maps = proxy:mysql:$config_directory/mysql_virtual_mailbox_maps.cf'
# postconf -e 'virtual_mailbox_base = /home/vmail'
# postconf -e 'virtual_minimum_uid = 150'
# postconf -e 'virtual_uid_maps = static:150'
# postconf -e 'virtual_gid_maps = static:8'
# postconf -e 'virtual_create_maildirsize = yes'
# postconf -e 'virtual_mailbox_extended = yes'
# postconf -e 'virtual_mailbox_limit_maps = proxy:mysql:/etc/postfix/mysql_virtual_mailbox_limit_maps.cf'
# postconf -e 'virtual_mailbox_limit_override = yes'
# postconf -e 'virtual_maildir_limit_message = "The user you are trying to reach has exceeded their quota."'
# postconf -e 'virtual_overquota_bounce = yes'
# postconf -e 'transport_maps = proxy:mysql:/etc/postfix/mysql_virtual_transports.cf'
# postconf -e 'relay_domains = proxy:mysql:/etc/postfix/mysql_relay_domains_maps.cf'

We will use Dovecot delivery:

# postconf -e 'virtual_transport=dovecot'
# postconf -e 'dovecot_destination_recipient_limit=1'
# postconf -e 'smtpd_data_restrictions = reject_unauth_pipelining, reject_multi_recipient_bounce, permit'

Postfix was installed with MySQL support, but that doesn't mean it already knows how to use our database. It needs to be provided with various SQL-query information for each type of table in our database. This information is stored in the MySQL files defined in the main.cf file:

/etc/postfix/mysql_virtual_alias_maps.cf

user = groupoffice_database_user
password = groupoffice_database_password
hosts = localhost
dbname = groupoffice
table = pa_aliases
select_field = goto
where_field = address
additional_conditions = and active = '1'

/etc/postfix/mysql_virtual_domains_maps.cf

user = groupoffice_database_user
password = groupoffice_database_password
hosts = localhost
dbname = groupoffice
table = pa_domains
select_field = domain
where_field = domain
additional_conditions = and backupmx = '0' and active = '1'

/etc/postfix/mysql_virtual_mailbox_limit_maps.cf

user = groupoffice_database_user
password = groupoffice_database_password
hosts = localhost
dbname = groupoffice
table = pa_mailboxes
select_field = quota
where_field = username
additional_conditions = and active = '1'

/etc/postfix/mysql_virtual_mailbox_maps.cf

user = groupoffice_database_user
password = groupoffice_database_password
hosts = localhost
dbname = groupoffice
table = pa_mailboxes
select_field = maildir
where_field = username
additional_conditions = and active = '1'

/etc/postfix/mysql_virtual_transports.cf

user = groupoffice_database_user
password = groupoffice_database_password
hosts = localhost
dbname = groupoffice
table = pa_domains
select_field = transport
where_field = domain
additional_conditions = and active = '1'

/etc/postfix/mysql_relay_domains_maps.cf

user = groupoffice_database_user
password = groupoffice_database_password
hosts = localhost
dbname = groupoffice
table = pa_domains
select_field = domain
where_field = domain
additional_conditions = and backupmx = '1' and active = '1'

Add the mail user:

# useradd -r -u 150 -g mail -d /home/vmail -s /sbin/nologin -c "Virtual Mailbox" vmail
# mkdir /home/vmail
# chmod 770 /home/vmail/
# chown vmail:mail /home/vmail/

Address mapping is done after content filtering with Amavis

# postconf -e 'receive_override_options = no_address_mappings'
# postconf -e 'content_filter=smtp-amavis:[127.0.0.1]:10024'

The line with receive_override_options overrides the main configuration value with receive_override_options = no_address_mappings. This way addresses are not mapped before the Amavis filter but are mapped after the content filter. If you don't do this it will cause address rewriting to be done both before and after the content filter. For most configurations this is not only unnecessary, but will cause duplicate mail to be delivered in configurations with virtual aliases of the form a -> a,b. Virtual rewriting must be turned off either before or after the content filter. How this is done is, again, documented in Postfix's FILTER_README file doc.

Optional group aliases

If you want to have mail aliases to send an email to an entire Group-Office user group you can change /etc/postfix/mysql_virtual_alias_maps.cf like this:

user = groupoffice-com
password = xxx
hosts = localhost
dbname = groupofficecom
query = select goto
 from pa_aliases
 where address='%s'
 and active='1'
 and goto not like '+%%'
 union
 SELECT
 gu.email
 FROM
 go_groups gg,
 go_users_groups gug,
 go_users gu,
 pa_aliases pa
 where pa.address='%s' and pa.active='1'
 and   pa.goto like '+%%'
 and   pa.goto = gg.name
 and   gg.id = gug.group_id
 and   gug.user_id=gu.id

Then you can create a user group with name "+examplegroup" and an alias that goes to "+examplegroup". This alias will reach all the members of that group.

Setup Amavis

Install the necessary packages

apt-get install amavisd-new clamav clamav-base clamav-daemon clamav-freshclam spamassassin spamc

Make sure master.cf contains these lines:

# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
smtp      inet  n       -       -       -       -       smtpd
smtp-amavis unix -      -       y     -       2  smtp
    -o smtp_data_done_timeout=1200
    -o smtp_send_xforward_command=yes
    -o disable_dns_lookups=yes
    -o max_use=20
127.0.0.1:10025 inet n  -       y     -       -  smtpd
    -o content_filter=
    -o local_recipient_maps=
    -o relay_recipient_maps=
    -o smtpd_restriction_classes=
    -o smtpd_delay_reject=no
    -o smtpd_client_restrictions=permit_mynetworks,reject
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o smtpd_data_restrictions=reject_unauth_pipelining
    -o smtpd_end_of_data_restrictions=
	-o smtpd_end_of_data_restrictions=
    -o mynetworks=127.0.0.0/8
    -o smtpd_error_sleep_time=0
    -o smtpd_soft_error_limit=1001
    -o smtpd_hard_error_limit=1000
    -o smtpd_client_connection_count_limit=0
    -o smtpd_client_connection_rate_limit=0
    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks

dovecot   unix  -       n       n       -       -       pipe
    flags=DRhu user=vmail:mail argv=/usr/lib/dovecot/deliver -d ${recipient}

# groupadd -g 5001 spamd
# useradd -u 5001 -g spamd -s /sbin/nologin -d /var/lib/spamassassin spamd
# mkdir /var/lib/spamassassin
# chown spamd:spamd /var/lib/spamassassin

To start the server in /etc/default/spamassassin set

ENABLED=1

You can tweak Spamassassin yourself in /etc/spamassassin/local.cf

Setup Dovecot

Dovecot is a fast IMAP deamon and it can also check user quota and filter Spam to the spam folder. Change / Add these values to /etc/dovecot/dovecot.conf:

#Enable IMAP and IMAPS
protocols = imap imaps 

#Listen on all IP addresses
listen = *

#configure the location of our virtual mailboxes
mail_location = maildir:/home/vmail/%d/%u

# Group to enable temporarily for privileged operations. Currently this is
# used only for creating mbox dotlock files when creation fails for INBOX.
# Typically this is set to "mail" to give access to /var/mail.
#mail_privileged_group =
mail_privileged_group = mail

# Grant access to these supplementary groups for mail processes. Typically
# these are used to set up access to shared mailboxes. Note that it may be
# dangerous to set these if users can create symlinks (e.g. if "mail" group is
# set here, ln -s /var/mail ~/mail/var could allow a user to delete others'
# mailboxes, or ln -s /secret/shared/box ~/mail/mybox would allow reading it).
mail_access_groups = mail

# Valid UID range for users, defaults to 500 and above. This is mostly
# to make sure that users can't log in as daemons or other system users.
# Note that denying root logins is hardcoded to dovecot binary and can't
# be done even if first_valid_uid is set to 0.
first_valid_uid = 150
last_valid_uid = 150

Inside the protocol imap { } section change:

mail_plugins = quota imap_quota

Inside the protocol lda { } section change:

postmaster_address = postmaster@intermesh.nl
mail_plugin_dir = /usr/lib/dovecot/modules/lda
mail_plugins = sieve quota
#the globalsieverc script will filter spam messages to the Spam folder 
global_script_path = /home/vmail/globalsieverc

Inside auth default { } section change:

auth default {
 mechanisms = plain
 passdb sql {
	 args = /etc/dovecot/dovecot-sql.conf
 }
 userdb sql {
	 args = /etc/dovecot/dovecot-sql.conf
 }
 user = nobody
 socket listen {
	 master {
	 path = /var/run/dovecot/auth-master
	 mode = 0660
	 user = vmail
	 group = mail
 }
 	client {
	 path = /var/spool/postfix/private/auth
	 mode = 0660
	 user = postfix
	 group = postfix
	}
}
plugin {
  quota = maildir:storage=512000
}

Also comment out the passdb pam section or you will get a very slow Dovecot connection!

/etc/dovecot/dovecot-sql.conf:

driver = mysql
connect = host=localhost dbname=groupoffice user=vmail_user password=vmail_user_pass
default_pass_scheme = PLAIN-MD5
user_query = SELECT '/home/vmail/%d/%n' AS home, 'maildir:/home/vmail/%d/%n' AS mail, 150 AS uid, 8 AS gid, CONCAT('*:storage=', quota) AS quota_rule FROM pa_mailboxes WHERE username = '%u' AND active = '1'
password_query = SELECT username AS user, password, '/home/vmail/%d/%n' AS userdb_home, 'maildir:/home/vmail/%d/%n' AS userdb_mail, 150 AS userdb_uid, 8 AS userdb_gid FROM pa_mailboxes WHERE username = '%u' AND active = '1'

Create globalsieverc script:

Edit /home/vmail/globalsieverc:

require "fileinto";
if exists "X-Spam-Flag" {
  fileinto "Spam";
}

Set correct permissions:

# chown vmail:mail -R /home/vmail/globalsieverc

Shared mailboxes

Create an empty shared mailboxes database:

chown vmail:mail /var/lib/dovecot
touch /var/lib/dovecot/shared-mailboxes.db
chown vmail:mail /var/lib/dovecot/shared-mailboxes.db

Add to dovecot.conf sections (The sections probably already exist. So add these values to the sections:

plugin {
  acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes.db
  acl = vfile
  #acl_anyone = allow
}

protocol imap {
  mail_plugins = acl imap_acl
}
protocol lda {
  mail_plugins = acl
}

# You need to create also a private namespace:
namespace private {
  separator = /
  prefix =
  #location defaults to mail_location.
  inbox = yes
}

namespace shared {
  separator = /
  prefix = shared/%%u/
  #location = maildir:/home/vmail/%%d/%%u:INDEX=~/shared/%%u
  location = maildir:%%h:INDEX=~/shared/%%u
  subscriptions = no
  list = children
  #list = yes
}

Now restart dovecot. Functionality to manage shared mailboxes is currenty in development. For now you can manage them with the imap-acl plugin in Thunderbird.

Enable the managesieve server

Group-Office 3.7 supports managesieve. With this protocol you can setup mail filtering rules directly on the mail server. To enable this add "managesieve" to the dovecot protocols:

protocols = imap imaps managesieve

protocol managesieve {
  # Specify an alternative address:port the daemon must listen on
  # (default: *:2000)
  #listen = localhost:2000

  managesieve_logout_format = bytes ( in=%i : out=%o )
}

Optionally you can specify these parameters in config.php if they are different then the default values displayed here:

$config['sieve_port']=2000;
$config['sieve_usetls']=false;

Now you can install the "sieve" module in Group-Office and the mail filter tab in the E-mail account settings will be replaced by the sieve filters.

Now it should work! Restart Postfix, Spamassassin and Dovecot and start testing!

Login to Group-Office and add a Domain and mailbox. Now go to the e-mail module and add an e-mail account:

E-mail -> Settings -> Accounts -> Add

Enter:

Host: localhost User: user@example.com Pass: The password you set

Now send a test mail!

Optionally install the serverclient

The serverclient module for Group-Office can automatically create new mailboxes when you create a new user. Install the module and add the following to the Group-Office config.php file:

Note: If you have multiple Group-Office installations and one to manage Postfix. You can add these values to: /etc/groupoffice/globalconfig.inc.php. All Group-Office installations will use these values then. You probably want to configure the serverclient_domains per installation in thier config.php files.

#GO will connect to this installation to add a mailbox. It is the full url to the Group-Office installation with the postfixadmin module installed.
$config['serverclient_server_url']='http://localhost/groupoffice/';
#The admin account of GO. Remember to change the password here too if you change it.
$config['serverclient_username']='administrator';
$config['serverclient_password']='somepass';

#comma separated list of mailbox domains
$config['serverclient_domains']='intermeshdev.nl';

#The email account properties that will be added for the user
$config['serverclient_mbroot'] = '';
$config['serverclient_use_ssl'] = '0';
$config['serverclient_novalidate_cert'] = '0';
$config['serverclient_type']='imap';
$config['serverclient_host']='localhost';
$config['serverclient_port']=143;
$config['serverclient_smtp_host']='localhost';
$config['serverclient_smtp_port']=25;
$config['serverclient_smtp_encryption']='';
$config['serverclient_smtp_username']='';
$config['serverclient_smtp_password']='';

The module will connect using curl functions so the php5 curl extension is required for this to work.

Note: Make sure everyone has usage/read permissions for the module for it to properly work. When you change a password in Group-Office the serverclient will update all accounts where the username field contains a domainname listed in the $config['serverclient_domains'] config option listed above.

Enabling TLS and SMTP authentication

Everything should work now on the webserver. But if you want to connect to the mailserver from the outside you can configure SMTP authentication.

Generate a certificate of Postfix TLS (Note that "Common Name (eg, YOUR name)" MUST match the name of the server):

mkdir -p /etc/ssl/mailserver/
cd /etc/ssl/mailserver/
openssl genrsa 1024 > mail-key.pem
chmod 400 mail-key.pem
openssl req -new -x509 -nodes -sha1 -days 365 -key mail-key.pem > mail-cert.pem

Configure SASL for Postfix:

postconf -e 'smtpd_sasl_auth_enable = yes'
postconf -e 'smtpd_sasl_security_options = noanonymous'
postconf -e 'broken_sasl_auth_clients = yes'
postconf -e 'smtpd_sasl_type = dovecot'
postconf -e 'smtpd_sasl_path = private/auth'

postconf -e 'smtpd_tls_cert_file = /etc/ssl/mailserver/mail-cert.pem'
postconf -e 'smtpd_tls_key_file = /etc/ssl/mailserver/mail-key.pem'
postconf -e 'smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache'
postconf -e 'smtpd_tls_security_level = may'
postconf -e 'smtpd_tls_loglevel = 0'
postconf -e 'tls_random_source = dev:/dev/urandom'

You must also add "permit_sasl_authenticated" to the "smtpd_recipient_restrictions" configuration option. Edit this option in /etc/postfix/main.cf and add it right after "permit_mynetworks".

Finally, you should enable this line in /etc/postfix/master.cf (Remove the # sign) :

smtps     inet  n       -       -       -       -       smtpd

Now you should be able to connect from the outside to the mailserver with TLS and a valid account.

On Ubuntu 12.04 lookup make sure this is present in /etc/dovecot/conf.d/10-master.conf

# Postfix smtp-auth
unix_listener /var/spool/postfix/private/auth {
    mode = 0666
    user = postfix
    group = postfix
}

Finally restart postfix and SMTP authentication with TLS should work! You can configure clients like this:

Usage statistics

To show usage statistics in the postfixadmin module run this script in a cronjob:

http://group-office.svn.sourceforge.net/viewvc/group-office/branches/groupoffice-3.5/debian-groupoffice-mailserver/usr/share/groupoffice-mailserver/

Links to other tutorials

More tutorials about Virtual Mail with postfix:

http://www.howtoforge.com/postfix_mysql_dovecot_dspam_clamav_postgrey_rbl_debian_etch

http://postfix.wiki.xs4all.nl/index.php?title=Virtual_Users_and_Domains_with_Courier-IMAP_and_MySQL

http://wiki.dovecot.org/HowTo/DovecotLDAPostfixAdminMySQL