Difference between revisions of "Migrate"

Latest revision as of 17:02, 31 December 2010

There are various tools for Migration. You can use ActiveSync or SyncML to synchronize the contacts and calendars or you can import calendar *.ics and CSV file for contacts.

Migrate IMAP accounts with a CSV file

To migrate users and e-mail through IMAP you can use the script below. You can use it to migrate from Exchange or any other server that supports IMAP. You need to create a CSV file with this format:

"Group in GO 0","Email address 1","First name 2","Middle name 3","Last name 4","Password 5","GroupOffice User name 6"

Then you need to adjust the variables at the top of the script and run it.

<?php

/**
 * Copyright Intermesh
 *
 * This file is part of Group-Office. You should have received a copy of the
 * Group-Office license along with Group-Office. See the file /LICENSE.TXT
 *
 * If you have questions write an e-mail to info@intermesh.nl
 *
 * @version $Id: imapsync.php 4569 2010-04-13 08:27:54Z mschering $
 * @copyright Copyright Intermesh
 * @author Merijn Schering <mschering@intermesh.nl>
 *
 */

/*
 * This script can be used to create users in Group-Office with a CSV file
 * and migrate the mail through IMAP.
 *
 * This only works if the postfixadmin module runs on the same installation
 * where the new Group-Office users will be created.
 *
 * The script was written for version 3.5.9
 */

//the domain for the mail
$maildomain='example.com';

//the new mail host
$local_host = 'localhost';

//the old mail host
$remote_host = 'mail.example.com';

//the path to the imapsync program
$imapsync = '/usr/bin/imapsync';

//CSV file in format:
//"Group in GO 0","Email address 1","First name 2","Middle name 3","Last name 4","Password 5","GroupOffice User name 6"
$csv_file='/home/mschering/Downloads/users.csv'


//end of variables section.



//otherwise log module will log all items as added.
define('NOLOG', true);

if (isset($argv[1])) {
	define('CONFIG_FILE', $argv[1]);
}

chdir(dirname(__FILE__));

require_once("../../Group-Office.php");
require_once($GO_MODULES->modules['postfixadmin']['class_path'].'postfixadmin.class.inc.php');
$postfixadmin = new postfixadmin();

require_once($GO_MODULES->modules['email']['class_path'].'email.class.inc.php');
$e = new email();

//make sure e-mail domain exists in database
$domain = $postfixadmin->get_domain_by_domain($maildomain);
if(!$domain)
{
	$domain['transport']='virtual';
	$domain['active']='1';
	$domain['domain']=$maildomain;
	$domain['user_id']=1;
	$domain['acl_id']=$GO_SECURITY->get_new_acl('domain');

	$domain_id=$postfixadmin->add_domain($domain);
}else
{
	$domain_id=$domain['id'];
}

$fp = fopen($csv_file, "r");
if (!$fp) {
	die('Could not read CSV file');
}

//headings, skip one row
$record = fgetcsv($fp, 4096, ',', '"');

while ($record = fgetcsv($fp, 4096, ',', '"')) {

	$remote_user = $record[1];
	$remote_pass = $record[5];
	$groupoffice_user = $record[6];
	$email = $record[1];

	//Check if the user exists in Group-Office and if it doesn't create it.
	$user = $GO_USERS->get_user_by_username($groupoffice_user);
	if(!$user){
		$user['first_name']=$record[2];
		$user['middle_name']=$record[3];
		$user['last_name']=$record[4];
		$user['email']=$email;
		$user['username']=$groupoffice_user;
		$user['password']=$remote_pass;
		$user['enabled']='1';

		$group = $GO_GROUPS->get_group_by_name($record[0]);
		if(!$group){
			$group_id=$GO_GROUPS->add_group(1, $record[0]);
		}else
		{
			$group_id=$group['id'];
		}

		$user_id = $GO_USERS->add_user($user, array($group_id),array($GO_CONFIG->group_everyone,$group_id));
	}else
	{
		$user_id = $user['id'];
	}
	
	$pa_user=$groupoffice_user.'@'.$maildomain;

	//Create a mailbox in postfixadmin
	$mailbox = $postfixadmin->get_mailbox_by_username($pa_user);
	if(!$mailbox){
		$mailbox['domain_id']=$domain_id;
		$mailbox['username']=$pa_user;
		$mailbox['password']=md5($remote_pass);
		$mailbox['maildir']=$maildomain.'/'.$groupoffice_user.'/';
		$mailbox['active']='1';
		$mailbox_id= $postfixadmin->add_mailbox($mailbox);

		$alias['domain_id']=$mailbox['domain_id'];
		$alias['address']=$mailbox['username'];
		$alias['active']=$mailbox['active'];
		$alias['goto']=$mailbox['username'];

		$postfixadmin->add_alias($alias);

		if($email!=$pa_user){
			$alias['goto']=$email;

			$postfixadmin->add_alias($alias);
		}
	}

	//create an e-mail account for the user
	$account = $e->get_account_by_username($pa_user, $user_id);
	if(!$account){
		$account['user_id']=$user_id;
		$account['mbroot'] = '';
		$account['use_ssl'] = 0;
		$account['type']='imap';
		$account['host']='localhost';
		$account['port']=143;
		$account['username']=$pa_user;
		$account['password']=$remote_pass;

		$account['smtp_host']='localhost';
		$account['smtp_port']=25;
		$account['smtp_encryption']='';
		$account['smtp_username']='';
		$account['smtp_password']='';
		$account['name']=String::format_name($record[4],$record[2],$record[3],'first_name');
		$account['email']=$email;
		$account['signature']='';

		$e->add_account($account);
}


	echo "Syncing " . $record[0] . "\n\n";

	//--skipsize --useheader Message-Id

	$cmd = $imapsync . ' --syncinternaldates --authmech1 LOGIN --authmech2 LOGIN ' .
					'--host1="' . $remote_host . '" --user1="' . $pa_user . '" --password1="' . $remote_pass . '" ' .
					'--user2="' . $remote_user . '" --host2="' . $local_host . '" --password2="' . $remote_pass . '"';

	echo $cmd . "\n\n";

	system($cmd);
}

Convert E-mail MSG files to EML files

Group-Office can display EML e-mails but not MSG files. So if you have MSG files you need to convert them. We used a perl script and a bash script to convert them all. You can find the tutorial here:

http://klo2k.wordpress.com/2009/12/27/convert-emails-from-outlook-msg-to-thunderbird-eml-format-using-msgconvert/

Migrate an e-mail domain from one Group-Office to another

You might have a need to transfer all mailboxes and aliases to another mailserver. If both mailservers run on the Group-Office mailserver solution with versiopn 3.6 or higher this is very easy. On the old server run:

php /usr/share/groupoffice-mailserver/export_domain.php --domain=example.com --targethost=newmailserver.example.com

Make sure you can connect with the root user with ssh to the new mailserver.

When this has succeeded you can run the following command to import the mailboxes:

php /usr/share/groupoffice-mailserver/import_domain.php --domain=example.com

After that you can adjust all mailboxes on the Group-Office database with this SQL command:

UPDATE em_accounts SET host='newmailserver.example.com' WHERE username LIKE '%@example.com';

Migrate entire linux server

Not really in the scope of Group-Office. But I used this script once to copy an entire Linux server to a new fresh debian installation. Change MIGRATEUSERS to 1 to copy all the users and groups with a UID greater then 500. Change the sources list to suit your needs. Make sure you have trailing slashes for directories otherwise you will sync them one level too deep.

It's also necsarry to create an SSH keypair if you don't have one yet:

ssh-keygen

Then copy it to the other server:

ssh-copy-id root@example.com

migrate.sh

#!/bin/bash
TARGET=root@example.com
MIGRATEUSERS=0
SOURCES="/etc/ssh/ssh_host_dsa_key.pub /etc/ssh/ssh_host_dsa_key /etc/ssh/ssh_host_rsa_key /etc/ssh/ssh_host_rsa_key.pub /root/passwd.mig /root/shadow.mig /root/group.mig /quota.user /quota.group /home/ /var/www/ /var/lib/mysql/ /vmail/ /usr/share/groupoffice/ /usr/share/www/ /usr/local/kringloopplanner/ /etc/postfix/ /etc/dovecot/ /etc/mailname /etc/apache2/sites-enabled/ /etc/apache2/ssl/ /etc/groupoffice/"

export UGIDLIMIT=500

awk -v LIMIT=$UGIDLIMIT -F: '($3>=LIMIT) && ($3!=65534)' /etc/passwd > /root/passwd.mig

awk -v LIMIT=$UGIDLIMIT -F: '($3>=LIMIT) && ($3!=65534)' /etc/group > /root/group.mig

awk -v LIMIT=$UGIDLIMIT -F: '($3>=LIMIT) && ($3!=65534) {print $1}' /etc/passwd | tee - |egrep -f - /etc/shadow > /root/shadow.mig


#the H option preserves hard links
for source in $SOURCES; do
	rsync -vaH -e "ssh -i /root/.ssh/id_rsa" $source $TARGET:$source
done

if [ $MIGRATEUSERS -eq 1 ]; then
	ssh -i /root/.ssh/id_rsa $TARGET "cat /root/passwd.mig >> /etc/passwd"
	ssh -i /root/.ssh/id_rsa $TARGET "cat /root/shadow.mig >> /etc/shadow"
	ssh -i /root/.ssh/id_rsa $TARGET "cat /root/group.mig >> /etc/group"
fi

The new server had MySQL 5.1 which uses a different table structure. Running the following command fixed that:

mysql_upgrade -h localhost -u root -p<password>