preserving flags for shared mailbox when migrating from cyrus to dovecot

[Jogi Hofmüller]

Dear all,

Despite all the progress we are making in our attempt to make the
migration as smooth and transparent for our users there are still some
important unsolved issues. One thing that I can't figure out is how to
preserve a user's flags that s/he has for mail in a shared mailbox.

So far we tried 'doveadm backup -R' as well as the cyrus2dovecot script
from here [1]. Both work well for individual mailboxes but do not
migrate subscriptions for shared mailboxes and -- more importantly --
flags for mail in a shared mailbox. Reconstructing subscriptions is
quite easy but I have no clue on how to reconstruct a user's flags.

Any hint is greatly appreciated!

[1] http://cyrus2dovecot.sw.fu-berlin.de/index.html
--
j.hofm?ller

mur.sat -- a space art project http://sat.mur.at/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140820/19bd33c8/attachment.sig>

[Timo Sirainen]

Post by Jogi Hofmüller
Despite all the progress we are making in our attempt to make the
migration as smooth and transparent for our users there are still some
important unsolved issues. One thing that I can't figure out is how to
preserve a user's flags that s/he has for mail in a shared mailbox.
So far we tried 'doveadm backup -R' as well as the cyrus2dovecot script
from here [1]. Both work well for individual mailboxes but do not
migrate subscriptions for shared mailboxes and -- more importantly --
flags for mail in a shared mailbox. Reconstructing subscriptions is
quite easy but I have no clue on how to reconstruct a user's flags.

I think you can first migrate the shared mailbox's mails as whatever user. Then use doveadm sync -1 (and maybe -m shared/foo to sync only the one mailbox) to migrate the message flags. You need to run that separately for all the users who have access to the shared mailbox. Of course Dovecot will also need to be configured properly to have private seen flags for users (only private seen flags are currently possible in Dovecot, I'm not sure how they are in Cyrus).

[Jogi Hofmüller]

Hi,

Post by Timo Sirainen
I think you can first migrate the shared mailbox's mails as whatever
user. Then use doveadm sync -1 (and maybe -m shared/foo to sync only
the one mailbox) to migrate the message flags.

Just to make sure I got you right:

The shared mailbox has been migrated to dovecot correctly. I call it
'sharedbox' for now.

The user's mail has also been migrated correctly. Let's call her/him
'user'.

Then I would run

doveadm sync -1 -u user -m shared.sharedbox imapc:

to sync the flags for user 'user'. Correct?

Post by Timo Sirainen
You need to run that
separately for all the users who have access to the shared mailbox.

Makes sense.

Post by Timo Sirainen
Of course Dovecot will also need to be configured properly to have
private seen flags for users (only private seen flags are currently
possible in Dovecot, I'm not sure how they are in Cyrus).

We have individual flags for shared mailboxes now and configured dovecot
accordingly (see dovecot -n output attached).

Thanks so far! I hope this will work!

Cheers,
--
J.Hofm?ller

Im ?brigen bin ich der Meinung, das Joanneum muss zerschlagen werden! -
Barbara Fischer
-------------- next part --------------
# 2.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 3.14-1-amd64 x86_64 Debian jessie/sid btrfs
auth_debug = yes
auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_ at +
auth_verbose = yes
disable_plaintext_auth = no
imapc_features = rfc822.size fetch-headers
imapc_host = hornet.mur.at
imapc_list_prefix = INBOX
imapc_master_user = cyrus
imapc_password = secret
imapc_port = 993
imapc_ssl = imaps
imapc_ssl_verify = no
imapc_user = %u
lmtp_proxy = yes
lmtp_save_to_detail_mailbox = yes
login_greeting = Dovecot is spitze!
mail_debug = yes
mail_location = maildir:/srv/vmail/%u/Maildir
mail_plugins = acl
mail_prefetch_count = 20
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave
namespace {
hidden = no
inbox = no
list = children
location = maildir:/srv/vmail/%%u/Maildir:INDEX=/srv/vmail/%u/shared/%%u:CONTROL=/srv/vmail/%u/shared/%%u:INDEXPVT=/srv/vmail/%u/shared/%%u
prefix = shared.%%u.
separator = .
subscriptions = yes
type = shared
}
namespace inbox {
hidden = no
ignore_on_failure = no
inbox = yes
list = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix = INBOX.
separator = .
subscriptions = yes
type = private
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
plugin {
acl = vfile:/etc/dovecot/global-acls:cache_secs=300
acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db
sieve = /srv/vmail/%u/sieve/.dovecot.sieve
sieve_default = /srv/vmail/sieve/default.sieve
sieve_dir = /srv/vmail/%u/sieve
sieve_global_dir = /srv/vmail/sieve/
}
protocols = " imap lmtp sieve pop3"
service auth {
unix_listener auth-userdb {
group = vmail
mode = 0777
user = vmail
}
}
service lmtp {
inet_listener lmtp {
address = 172.16.16.78
port = 24
}
user = vmail
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
inet_listener sieve_deprecated {
port = 2000
}
process_min_avail = 1
service_count = 1
vsz_limit = 64 M
}
service managesieve {
process_limit = 256
}
ssl_cert = </etc/dovecot/klee.pem
ssl_key = </etc/dovecot/private/klee.mur.at.pem
userdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
protocol lmtp {
mail_plugins = acl quota sieve
passdb {
args = /etc/dovecot/dovecot-ldap_for_lmtp.conf.ext
driver = ldap
name =
}
userdb {
args = /etc/dovecot/dovecot-ldap_for_lmtp.conf.ext
driver = ldap
name =
}
}
protocol imap {
mail_plugins = acl imap_acl
}
protocol sieve {
managesieve_implementation_string = Dovecot Pigeonhole
}
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140820/54679123/attachment.sig>

[Jogi Hofmüller]

Hi Timo, all,

Post by Timo Sirainen
I think you can first migrate the shared mailbox's mails as whatever
user. Then use doveadm sync -1 (and maybe -m shared/foo to sync only
the one mailbox) to migrate the message flags.

Well, I tried that in several different ways, all without any success.
Let's assume the user is called jane and she has a subscription for the
shared mailbox doe I tried the following:

doveadm sync -1 -R -u jane -m shared.doe imapc:

I also tried using the namespace switch:

doveadm sync -1 -R -u jane -n shared imapc:

again no success. The shared mailbox stays available and working but
the flags will not be synced to the state they had on the original
server. I also tried it without -R but that didn't get me anywhere
either and should be wrong anyways AFAICT.

Any further ideas anyone or should I prepare our shared mailbox users
that all their email will be unread after migration?

Cheers,
--
j.hofm?ller

We are all idiots with deadlines. - Mike West

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140910/e4294ae6/attachment.sig>

[Teemu Huovila]

This post might be inappropriate. Click to display it.

[Jogi Hofmüller]

Hi Teemu, all,

Post by Teemu Huovila
I looked at the dovecot -n output attached to your previous mail and
I think I spotted some issues.
(...)
The INDEX and INDEXPVT are identical, which means there is no private
index. Having the CONTROL defined is also questionable. I suggest you
location =
maildir:/srv/vmail/%%u/Maildir:INDEXPVT=/srv/vmail/%u/shared/%%u

Did that, and didn't change anything. I still have now way to sync
flags for shared mailboxes from the old mail server.

Post by Teemu Huovila
Also, to make subscriptions work sensibly, set the shared namespace
subscriptions = no and then add a placeholder namespace with an empty
namespace {
prefix =
hidden = yes
list = no
subscriptions=yes
}

Alright, thanks for the hint.

Cheers,
--
J.Hofm?ller

Im ?brigen bin ich der Meinung, das Joanneum muss zerschlagen werden! -
Barbara Fischer

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140910/e80baf51/attachment.sig>