Imapsync

Imapsync is a very powerfull tool to synchronize IMAP servers. It is written by Gilles LAMIRAL.

It can be used to :

• migrate an IMAP server
• backup an IMAP server
• consolidate serveral IMAP accounts

This page explain Howto install and use imapsync for an IMAP migration from courier-imap to Google Apps.

Imapsync is included in the Debian repository and can be install with the selection of the package “imapsync”.

To use the last features (like the support of Google Apps), a manual install is needed.
The INSTALL file included in the archive describe the installation, below is an installation example on Debian testing/squeeze (aug 2009) :

• download the last version on the official website http://www.linux-france.org/prj/imapsync/dist/
• untar the archive :

# tar -zxvf imapsync-1.286.tgz
imapsync-1.286/
imapsync-1.286/bugs/
imapsync-1.286/bugs/BUG_219_windows
imapsync-1.286/bugs/BUG_IMAPClient_3.xx
imapsync-1.286/README
imapsync-1.286/lsm.imapsync
imapsync-1.286/freshmeat
imapsync-1.286/tools/
imapsync-1.286/tools/wonko_ruby_imapsync
imapsync-1.286/t/
imapsync-1.286/t/01_connect
...

• Install perl if needed, to check that perl is available :

# perl -v

This is perl, v5.10.0 built for x86_64-linux-gnu-thread-multi

Copyright 1987-2007, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

• Add the following Debian packages :
  • libmail-imapclient-perl
  • libterm-readkey-perl
  • libdate-manip-perl
  • libdigest-md5-perl

• Check that imapsync is ready to install with “perl -c imapsync”

~/imapsync-1.286# perl -c imapsync
imapsync syntax OK
~/imapsync-1.286#

• To see what will be done by the install, use “make -n install”

~/imapsync-1.286# make -n install
perl -c imapsync
pod2man imapsync > imapsync.1
install -D imapsync /usr/bin/imapsync
install -D imapsync.1 /usr/share/man/man1/imapsync.1
chmod 755 /usr/bin/imapsync
~/imapsync-1.286# 

• Install imapsync with “make install”

~/imapsync-1.286# make install
perl -c imapsync
imapsync syntax OK
pod2man imapsync > imapsync.1
install -D imapsync /usr/bin/imapsync
install -D imapsync.1 /usr/share/man/man1/imapsync.1
chmod 755 /usr/bin/imapsync
~/imapsync-1.286#

After the successful installation, imapsync in available :

~# imapsync -v
1.286
~#

Before start : To avoid data loss, please do some dry run. Please also note that a migration can last several hours, it's a good idea to start with a small mailbox.

Imapsync synchronize from an IMAP account 1 to an IMAP account 2. Each account need the following détails :

• the server name/IP with ”—host”
• the login name with ”—user”
• the password with ”—password”, or from a file with ”—passfile”

Simple IMAP synchronization example (emails are not deleted on the source server) :

imapsync -host1 192.168.10.1 --user1 matthieu --passfile1 pass1 --host2 imap.gmail.com --user2 test@bouthors.fr --password2 xxxxxx --dry

The ”—dry” option do a dry run, no action done.

Each IMAP server does not support the same features, so imapsync need some tunning.

For Google Apps / Gmail, the following options are required :

• ”—ssl2” : enable IMAPS on the destination side
• ”—useheader 'Message-Id' —skipsize” : imapsync use the size and the headers of emails to identify them. As Google change the headers, we need to make imapsync only look the “Message-Id” header
• ”—syncinternaldates” : synchronize dates

Example :

imapsync -host1 192.168.10.1 --user1 matthieu --passfile1 pass1 --host2 imap.gmail.com --user2 test@bouthors.fr --password2 xxxxxx --ssl2 --noauthmd5 --useheader 'Message-Id'  --skipsize --syncinternaldates --dry

Comment : ”—noauthmd5” disable the MD5 authentication, which wasn't well supported by my IMAP server (courier).

With Google Apps, Trash, Draft and Sent items folder are subfolders of [Gmail] folder.
During the synchronization, imapsync can change the folder name with the option ”—regextrans2”.

Example with Google Apps (french folder names) :

• Trash ⇒ [Gmail]/Corbeille : ”—regextrans2 's/Trash/\[Gmail\]\/Corbeille/'”
• Sent ⇒ [Gmail]/Messages envoyés : ”—regextrans2 's/Sent/\[Gmail\]\/Messages envoy&AOk-s/'”
• Drafts ⇒ [Gmail]/Brouillons : ”—regextrans2 's/Drafts/\[Gmail\]\/Brouillons/'”

Example :

imapsync -host1 192.168.10.1 --user1 matthieu --passfile1 pass1 --host2 imap.gmail.com --user2 test@bouthors.fr --password2 xxxxxx --ssl2 --noauthmd5 --useheader 'Message-Id'  --skipsize --syncinternaldates --regextrans2 's/Trash/\[Gmail\]\/Corbeille/' --regextrans2 's/Sent/\[Gmail\]\/Messages envoy&AOk-s/' --regextrans2 's/Drafts/\[Gmail\]\/Brouillons/' --dry --justfolders

Comment : The option ”—justfolders” is very usefull to test “regextrans”.

A migration is a synchronisation with deletion.
With IMAP, deletion is done in 2 steps :

• deletion : an email is tagged “deleted”
• expunge : emails tagged “deleted” are removed

With imapsync we can specify what action to realize with 2 options :

• deletion on the source is done with ”—delete”
• expunge on the source is done with ”—expunge1”

Example :

imapsync -host1 192.168.10.1 --user1 matthieu --passfile1 pass1 --host2 imap.gmail.com --user2 test@bouthors.fr --password2 xxxxxx --ssl2 --noauthmd5 --useheader 'Message-Id'  --skipsize --syncinternaldates --regextrans2 's/Trash/\[Gmail\]\/Corbeille/' --regextrans2 's/Sent/\[Gmail\]\/Messages envoy&AOk-s/' --regextrans2 's/Drafts/\[Gmail\]\/Brouillons/' --delete --expunge1 --dry 

When I migrated from courier to Google Apps, imapsync found some duplicated emails.
Most of them where real (email sent to 2 alias at the same time).
But few emails were considered duplicated because of an empty “Message-ID”.

The best way to migrate all emails is to delete emails after synchronization (with —delete) and manually verify remaining emails.

• http://www.linux-france.org/prj/imapsync/