Over the last four days I migrated my Android contacts and calendar to my own Linux server. Read how to setup the open source exchange server Zarafa under Debian and sync it with you Android smartphone. In detail, this how-to explains how to setup Zarafa 6.40 (Zarafa Collaboration Platform - Community Version) on Debian Lenny (5.0), working with Exim. You can also use every other Linux distribution, or mail MTA, since there are also binary packages for RHEL, Ubuntu, OpenSuse or SLES, and the source .tgz provided. The Zarafa howtos on the Website also integrate Postfix as mail server.
Installing Zarafa, Setting up the Zarafa Gateway (IMAP/POP Server), iCal/CalDav service, sabre-zarafa (a CardDAV Plugin), Zarafa server (Background Service), MySQL (zarafa stores Mail/Calendar in db), z-push (Active Sync Emulation - for pushing onto Android)
Note: I ran the install.sh, not the packages itself apt-get. the script will fetch and install dependencies with apt-get (like mysql, libxslt1.1, libpoppler3, .. (see the attached install.out for more). When the question about SQL password popped up, i fired up new terminal, and did the third step of this how to (Setup a MySQL DB).
# tar xvfz zcp-6.40.4-24200-debian-5.0-x86_64-opensource.tgz
# ./install.sh# mysql -u root -p
mysql> create database zarafa;
mysql> grant usage on *.* to zarafa@localhost identified by '<<mysecretpass>>';
mysql> grant all privileges on zarafa.* to zarafa@localhost;
mysql> exit;# openssl genrsa -out server.key 2048
<<enter password>>This will generate a RSA key of 2048 bits. You can also add –des3 or 1024 as a 3des key with 1024 bits. Now we can use this private key to generate a certificate signing request (CSR). This CSR is then used, usually, for sending to an Certificate Authority, such as Thawte or Versisign, who verify your identity. We neglect that option and self-sign your CSR.
# openssl req -new -key server.key -out server.csr# openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crtYou most likely already have a .crt and .key file if you use any SSL connection with apache. You just need to generate the .csr.
For our Zarafa setup we need to combine the certificate (.crt) and the private key (.key) in a single file. Do so by typing:
# cat server.key > server_combined.pem
# cat server.crt >> server_combined.pemThis gives us the required Server certificate. This text is a note in the server.cfg:
# Required Server certificate, contains the certificate and the private key parts
server_ssl_key_file = /etc/zarafa/ssl/crt-combined.pemMake sure you first processed the .key, afterwards the .crt. The order is important
# cd /etc/zarafa/Here you setup your Zarafa install. You’ve got multiple config files. Mostly you have to enable ssl in every config and point it to the proper certs. Exemplary, I’ll show the difference for the ical.cfg. For all other changes see the attached diffs (.patch):
# whether unencrypted (non-ssl) connections can be made to the ical server
-ical_enable = yes
+ical_enable = no
# port on which the ical server listens on for unencrypted connections
ical_port = 8080
# whether ssl connections can be made to the ical server
-icals_enable = no
+icals_enable = yes
# port on which the ical server listens on for ssl connections
-icals_port = 8443
+icals_port = 8085
# default connection to the Zarafa server
# Please refer to the administrator manual or man page why HTTP is used rather than the UNIX socket.
@@ -55,9 +55,11 @@
# File with RSA key for SSL
# ssl_private_key_file = /etc/zarafa/ical/privkey.pem _(server.key from above) or use the combined_
+ssl_private_key_file = /etc/zarafa/ical/crt-combined.pem _(server_combined.pem from above)_
# File with certificate for SSL
# ssl_certificate_file = /etc/zarafa/ical/cert.pem _(server.crt from above) or use the combined_
+ssl_certificate_file = /etc/zarafa/ical/crt-combined.pem _(server_combined.pem from above)_server.cfg patch. Does everything in the background.
gateway.cfg patch. For IMAP/POP.
ical.cfg patch. iCal/CalDav service.
# zarafa-admin -c zab -p <<secret>> -e "johannes@somehost.de" -f "Johannes Meyer" -a 1The parameters are
-c: login name
-p: password
-e: email
-f: full name
-a: admin (0/1) (no/yes)
-P: if you want to get ask for password (not shown in bash history)You can already connect to the service via the webaccess. It’s available using this URL:
Your iCal and CalDav can be reached with the following:
or with https for ssl connections. Look inside the ical.cfg for the proper ports. As a note, I’m running this on a vserver (at Host Europe) and when I’ve set up my Thunderbird to use SSL, it didn’t work, without any error. The problem was that the default port 8443 was already used by my hosts Plesk. So, the connection was always redirected to the from service. To solve this, I simply switched to ical port to 8085. The Zarafa installer will modify your apache config with the following addition:
Alias /webaccess /usr/share/zarafa-webaccess
<Directory /usr/share/zarafa-webaccess>
DirectoryIndex index.php
Options -Indexes +FollowSymLinks
AllowOverride Options
Order allow,deny
Allow from all
</Directory>You just need to add a router and a transporter. As you know, order of routers is important in Exim. To let Exim know which marcos to use, you’ll need to update your update-exim4.conf.conf in /etc/exim4/ I’ve changed my
dc_localdelivery='maildir_home'to
dc_localdelivery='zarafa_pipe'and run a # update-exim4.conf Now, every mail is going into your new Zarafa database.
exim_zarafa patch
Use either this, or the one above. The advantage of this version is you don’t have to mess with your exim configuration, and it will work with any non-database based IMAP/POP3 server. For example dovecot or courier. My old .procmailrc filtered the messages into folders:
#now put my mail lists into mailboxes
:0:
* ^X-BeenThere: members@lists.somehostxxx.ccc.de
$MAILDIR/.ccc-mailing/Instead of putting everything into folders, you can additionally lead all mails to the zarafa-dagent. It handles your messages, and if you want, does sorting for you too:
#now put my mail lists into mailboxes
:0:
* ^X-BeenThere: members@lists.somehostxxx.ccc.de
| zarafa-dagent --folder 'ccc-mailing' userBecause I’m not a big fan of databases, php and all complex stuff, I’ve set up my procmail to send any new mails to the zarafa-dagent and still filter everything to my old Maildir structure. You cannot do multiple actions in one procmail condition directly, but you can duplicate mails, then use the second one as a new action.
#now put my mail lists into mailboxes
:0
* ^X-BeenThere: members@lists.somehostxxx.ccc.de
{
:0c :
$MAILDIR/.ccc-mailing/
:0 w
| zarafa-dagent --folder 'ccc-mailing' xen
}Here is a python script that imports your (dovecot delivered) mails (in Maildir format) into your new database. To use it, type:
# python dovecot-to-zarafa.py <path-to-maildir> <zarafausername>dovecot-to-zarafa.py
Note: It’s not that fast. ~2GB of mail in about an hour but at least it’s working.
# wget http://prdownload.berlios.de/z-push/z-push-1.4.3.tar.gz
# tar xvfz z-push-1.4.3.tar.gz
# mv z-push /var/wwwFor z-push you need to adjust your httpd config. Add an Alias to you apache host-site config (i.e. /etc/apache2/sites-available/default and /etc/apache2/sites-available/default-ssl (for ssl)):
# z-push
Alias /Microsoft-Server-ActiveSync /var/www/z-push/index.phpDon’t forget to do this for https. If you don’t, the new active sync account on your android might not connect and end with an expressionless error. On your android phone, you can now create a new Exchange account under Settings->Accounts & Sync Settings->Add Account->Microsoft Exchange. That will push new mail to your device and sync calendars and contacts. If you want to try if your z-push is indeed working, you can lead your browser to http://<ip>/Microsoft-Server-ActiveSync. There should be a pop up with a login mask. After the login you’ll see the z-push page and the message: “GET not supported”.
Download the plugin here and follow the install instruction. Nothing fancy there, except that you need to copy the php-mapi files to the sabre-zarafa/lib/mapi directory (you’ll find those under /usr/share/php/mapi/ in Debian). To use CardDAV with Thunderbird, download the Sogo-Connector. Now under the Address Book Window go to “File->New->New remote Address Book” and the correct URL is https://server/sabre-zarafa/addressbooks/zarafa-username/Contacts or in german https://server/sabre-zarafa/addressbooks/zarafa-username/Kontakte. So, the url is in fact language dependent.
Sabre-zarafa version 0.15 worked properly with my Zarafa 7.0(.5) installation. After I’ve upgraded to the 7.1(.2) branch, I kept getting errors. It seemed some values in the contacts (misc properties in the propertiesToVObject function) weren’t handled correctly. The error message was something similar to this (with changing offsets):
FATAL server - PHP error 8 in sabre-zarafa/vcard/VCardProducer.php:288 : Undefined offset: -1241513954I found this thread in the zarafa community forums and tried to apply the changes. For me, the patches didn’t work (on Linux), even if after trying to remove the carrige return as stated there. Now, I’ve uploaded a 0.15 version with the patchset 2 from the thread above. You can download it here or download my single patchfile here (contains patches 0000-0015). This version works for me with the current Zarafa release (version 7.1.2) using Thunderbird 17 with Sogo Connector.
The Sogo Connector seems to not work properly anymore. It only synchronizes initially, even if i manually trigger it from the drop-down menu, nothing happens to the local synchronized addressbook. It does create new contacts on the server for me, but does not fetch any new ones to the local side. When I remove the account, and reapply it, everything is there. I’ve tested eM Client for the first time (Windows Client with native CalDav/CardDav support), everything works like a charm here with the patched sabre-zarafa (15pl2 - Update 2012-12-20 on Zarafa 7.1). Bidirectional contact- and addressbook-sync. So I think the patch does work, just not the Sogo Connector.
The author of the patches took over development of the sabre-zarafa and merged it with a more recent sabreDav. Here is the link to the github project. The newest version (v 0.18) finally works flawless for me. Also note the install instructions on the github site. You need to fetch a recent version of sabreDAV (1.8.x) now manually, since it doesn’t come pre-bundled anymore. Also you need apache log4php and the php-mapi files as written above.
You reach your new services with:
For german users, you can change the language to german by editing the ZARAFA_LANG-option if your systems language is not set to German per default. In /etc/zarafa/userscripts/ properly adjust the *_common.sh scripts to
ZARAFA_LANG="de_DE.UTF-8"New users/companies/groups are now in german.
Simply follow the instructions from the Zarafa Upgrade Guide. Make sure that you properly upgrade your database after the installation of the new packages and before going online again. Use the zarafa-upgrade script found at /usr/share/doc/zarafa/zarafa7-upgrade. After that take a look at the Zarafa Alerts Page if you find any SQL errors in your server.log. I’ve also encountered several lines of out of range values in the server.log.
SQL Failed: Out of range value for column 'val_ulong'To fix these, update your z-push to an newer version. If you cannot login with your user using your mail client after the upgrade anymore, it may be that your user has not the sufficient rights to do so. With Version 7 comes a more granulated rights-management-system. In you gateway.log you’ll find errors as the following:
[ ] Starting worker process for IMAP request
[ ] IMAP not enabled for user 'user'
[ ] Client disconnectedHelp is found here. In short, you’ll have to add the following lines to your server.cfg
# In this list you can disable certain features for users. Normally all features are enabled for all users, making it
# possible through the user plugin to disable specific features for specific users.
# This list is space separated, and currently may contain the following features: imap, pop3
disabled_features =As an alternative, you can also set user-wide rights. Use these commands
zarafa-admin -u john --enable-feature imap
zarafa-admin -u john --disable-feature pop3to enable imap and disable pop3 for john. To check the john’s current rights, use
zarafa-admin --details john