Discussion:
My Dsync isn't replicating 'EXPUNGE' IMAP command
listas at adminlinux.com.br ()
2014-09-09 18:56:06 UTC
Permalink
Hi!

I have two Dovecot-2.2.13 servers with Dsync Replication: server1 and
server2.

When I execute 'EXPUNGE' on server1 side, the messages with FLAG
\Deleted are expunged only on server1. The message is still present in
Server2.

Could someone give me a hint reason for this ?
Thanks in advance!
--
Thiago Henrique

======================================================================
How I reproduce:

== ON SERVER1 ==

Client: $ telnet <IP server1> 143
Trying <IP server1>...
Connected to <IP server1>.
Escape character is '^]'.
Server: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID
ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] MAILBOX1 IMAP/POP3 server

Client: AUTH LOGIN <user> <pass>
Server: AUTH OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS
ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS
THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN
NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH
ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY
MOVE QUOTA] Logged in

Client: A002 SELECT "INBOX"
Server: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft nonjunk)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft nonjunk
\*)] Flags permitted.
* 2 EXISTS
* 0 RECENT
* OK [UNSEEN 1] First unseen.
* OK [UIDVALIDITY 1410284221] UIDs valid
* OK [UIDNEXT 16] Predicted next UID
* OK [HIGHESTMODSEQ 56] Highest
A002 OK [READ-WRITE] Select completed (0.000 secs).

Client: A003 FETCH 1:* (UID FLAGS)
Server: * 1 FETCH (UID 13 FLAGS (nonjunk))
* 2 FETCH (UID 15 FLAGS (nonjunk))
A003 OK Fetch completed.

Client: A005 STORE 1 +FLAGS \Deleted
Server: * 1 FETCH (FLAGS (\Deleted nonjunk))
A005 OK Store completed.

Client: A003 FETCH 1:* (UID FLAGS)
Server: * 1 FETCH (UID 13 FLAGS (\Deleted nonjunk))
* 2 FETCH (UID 15 FLAGS (nonjunk))
A003 OK Fetch completed.

Client: A006 EXPUNGE
Server: * 1 EXPUNGE
A006 OK Expunge completed.

Client: A007 FETCH 1:* (UID FLAGS)
Server: * 1 FETCH (UID 15 FLAGS (nonjunk))
A007 OK Fetch completed.

Client: A008 LOGOUT
Server: * BYE Logging out
A008 OK Logout completed.
Connection closed by foreign host.

== ON SERVER2 ==

Client: $ telnet <IP server2> 143
Trying <IP server2>...
Connected to <IP server2>.
Escape character is '^]'.
Server: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID
ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] MAILBOX2 IMAP/POP3 server

Client: AUTH LOGIN <user> <pass>
Server: AUTH OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS
ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS
THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN
NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH
ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY
MOVE QUOTA] Logged in

Client: A002 SELECT "INBOX"
Server: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft nonjunk)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft nonjunk
\*)] Flags permitted.
* 2 EXISTS
* 0 RECENT
* OK [UNSEEN 1] First unseen.
* OK [UIDVALIDITY 1410284221] UIDs valid
* OK [UIDNEXT 17] Predicted next UID
* OK [HIGHESTMODSEQ 61] Highest
A002 OK [READ-WRITE] Select completed (0.000 secs).

Client: A003 FETCH 1:* (UID FLAGS)
Server: * 1 FETCH (UID 13 FLAGS (\Deleted nonjunk))
* 2 FETCH (UID 15 FLAGS (nonjunk))
A003 OK Fetch completed.

Client: A008 LOGOUT
Server: * BYE Logging out
A008 OK Logout completed.
Connection closed by foreign host.


= My settings =

* My settins age identical on both server1 and server2, except by
'listen', 'address' and 'mail_replica' directives.

# doveconf -n
# 2.2.13 (6bb26098a45c): /etc/dovecot/dovecot.conf
# OS: Linux 3.13.0-30-generic x86_64 Ubuntu 14.04.1 LTS
auth_master_user_separator = *
auth_mechanisms = plain login
auth_username_chars =
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@=
auth_worker_max_count = 1
disable_plaintext_auth = no
doveadm_password = secret
doveadm_port = 12345
listen = server1
log_path = /var/log/dovecot.log
login_greeting = MAILBOX1 IMAP/POP3 server
mail_fsync = never
mail_gid = dovemail
mail_plugins = " notify replication quota"
mail_uid = dovemail
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
passdb {
args = /etc/dovecot/dovecot-sql-master.conf.ext
driver = sql
master = yes
pass = yes
}
plugin {
mail_replica = tcp:server2:12345
quota = dict:User quota::file:/var/lib/imap/%n/dovecot-quota
quota_rule = *:storage=1GB
}
protocols = imap pop3 lmtp
service aggregator {
fifo_listener replication-notify-fifo {
user = dovemail
}
unix_listener replication-notify {
user = dovemail
}
}
service auth-worker {
user = $default_internal_user
}
service auth {
client_limit = 1024
process_limit = 1
process_min_avail = 1
}
service doveadm {
inet_listener {
port = 12345
}
}
service imap {
process_limit = 1024
process_min_avail = 1
}
service lmtp {
inet_listener {
address = server1
port = 24
}
process_limit = 256
process_min_avail = 1
}
service pop3-login {
inet_listener pop3 {
address = server1
port = 110
}
}
service pop3 {
process_limit = 256
process_min_avail = 1
}
service replicator {
process_min_avail = 1
}
ssl = no
userdb {
driver = prefetch
}
userdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
verbose_proctitle = yes
protocol imap {
imap_client_workarounds = tb-extra-mailbox-sep
imap_idle_notify_interval = 2 mins
imap_max_line_length = 64 k
mail_max_userip_connections = 50
mail_plugins = " notify replication quota imap_quota"
}
protocol lmtp {
mail_fsync = optimized
mail_plugins = " notify replication sieve quota"
}
protocol pop3 {
mail_plugins = " notify replication quota"
}
listas at adminlinux.com.br ()
2014-09-10 17:19:28 UTC
Permalink
Dsync really is not working for EXPUNGE Imap command. This seems to be a
bug in version 2.2.13. Someone else was touched by it?

Thanks!
Thiago
Post by listas at adminlinux.com.br ()
Hi!
I have two Dovecot-2.2.13 servers with Dsync Replication: server1 and
server2.
When I execute 'EXPUNGE' on server1 side, the messages with FLAG
\Deleted are expunged only on server1. The message is still present in
Server2.
Could someone give me a hint reason for this ?
Thanks in advance!
--
Thiago Henrique
======================================================================
== ON SERVER1 ==
Client: $ telnet <IP server1> 143
Trying <IP server1>...
Connected to <IP server1>.
Escape character is '^]'.
Server: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR
LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] MAILBOX1 IMAP/POP3
server
Client: AUTH LOGIN <user> <pass>
Server: AUTH OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR
LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES
THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE
UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE
QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS
SPECIAL-USE BINARY MOVE QUOTA] Logged in
Client: A002 SELECT "INBOX"
Server: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft nonjunk)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft
nonjunk \*)] Flags permitted.
* 2 EXISTS
* 0 RECENT
* OK [UNSEEN 1] First unseen.
* OK [UIDVALIDITY 1410284221] UIDs valid
* OK [UIDNEXT 16] Predicted next UID
* OK [HIGHESTMODSEQ 56] Highest
A002 OK [READ-WRITE] Select completed (0.000 secs).
Client: A003 FETCH 1:* (UID FLAGS)
Server: * 1 FETCH (UID 13 FLAGS (nonjunk))
* 2 FETCH (UID 15 FLAGS (nonjunk))
A003 OK Fetch completed.
Client: A005 STORE 1 +FLAGS \Deleted
Server: * 1 FETCH (FLAGS (\Deleted nonjunk))
A005 OK Store completed.
Client: A003 FETCH 1:* (UID FLAGS)
Server: * 1 FETCH (UID 13 FLAGS (\Deleted nonjunk))
* 2 FETCH (UID 15 FLAGS (nonjunk))
A003 OK Fetch completed.
Client: A006 EXPUNGE
Server: * 1 EXPUNGE
A006 OK Expunge completed.
Client: A007 FETCH 1:* (UID FLAGS)
Server: * 1 FETCH (UID 15 FLAGS (nonjunk))
A007 OK Fetch completed.
Client: A008 LOGOUT
Server: * BYE Logging out
A008 OK Logout completed.
Connection closed by foreign host.
== ON SERVER2 ==
Client: $ telnet <IP server2> 143
Trying <IP server2>...
Connected to <IP server2>.
Escape character is '^]'.
Server: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR
LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] MAILBOX2 IMAP/POP3
server
Client: AUTH LOGIN <user> <pass>
Server: AUTH OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR
LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES
THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE
UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE
QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS
SPECIAL-USE BINARY MOVE QUOTA] Logged in
Client: A002 SELECT "INBOX"
Server: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft nonjunk)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft
nonjunk \*)] Flags permitted.
* 2 EXISTS
* 0 RECENT
* OK [UNSEEN 1] First unseen.
* OK [UIDVALIDITY 1410284221] UIDs valid
* OK [UIDNEXT 17] Predicted next UID
* OK [HIGHESTMODSEQ 61] Highest
A002 OK [READ-WRITE] Select completed (0.000 secs).
Client: A003 FETCH 1:* (UID FLAGS)
Server: * 1 FETCH (UID 13 FLAGS (\Deleted nonjunk))
* 2 FETCH (UID 15 FLAGS (nonjunk))
A003 OK Fetch completed.
Client: A008 LOGOUT
Server: * BYE Logging out
A008 OK Logout completed.
Connection closed by foreign host.
= My settings =
* My settins age identical on both server1 and server2, except by
'listen', 'address' and 'mail_replica' directives.
# doveconf -n
# 2.2.13 (6bb26098a45c): /etc/dovecot/dovecot.conf
# OS: Linux 3.13.0-30-generic x86_64 Ubuntu 14.04.1 LTS
auth_master_user_separator = *
auth_mechanisms = plain login
auth_username_chars =
auth_worker_max_count = 1
disable_plaintext_auth = no
doveadm_password = secret
doveadm_port = 12345
listen = server1
log_path = /var/log/dovecot.log
login_greeting = MAILBOX1 IMAP/POP3 server
mail_fsync = never
mail_gid = dovemail
mail_plugins = " notify replication quota"
mail_uid = dovemail
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
passdb {
args = /etc/dovecot/dovecot-sql-master.conf.ext
driver = sql
master = yes
pass = yes
}
plugin {
mail_replica = tcp:server2:12345
quota = dict:User quota::file:/var/lib/imap/%n/dovecot-quota
quota_rule = *:storage=1GB
}
protocols = imap pop3 lmtp
service aggregator {
fifo_listener replication-notify-fifo {
user = dovemail
}
unix_listener replication-notify {
user = dovemail
}
}
service auth-worker {
user = $default_internal_user
}
service auth {
client_limit = 1024
process_limit = 1
process_min_avail = 1
}
service doveadm {
inet_listener {
port = 12345
}
}
service imap {
process_limit = 1024
process_min_avail = 1
}
service lmtp {
inet_listener {
address = server1
port = 24
}
process_limit = 256
process_min_avail = 1
}
service pop3-login {
inet_listener pop3 {
address = server1
port = 110
}
}
service pop3 {
process_limit = 256
process_min_avail = 1
}
service replicator {
process_min_avail = 1
}
ssl = no
userdb {
driver = prefetch
}
userdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
verbose_proctitle = yes
protocol imap {
imap_client_workarounds = tb-extra-mailbox-sep
imap_idle_notify_interval = 2 mins
imap_max_line_length = 64 k
mail_max_userip_connections = 50
mail_plugins = " notify replication quota imap_quota"
}
protocol lmtp {
mail_fsync = optimized
mail_plugins = " notify replication sieve quota"
}
protocol pop3 {
mail_plugins = " notify replication quota"
}
Philipp
2014-09-11 13:44:43 UTC
Permalink
Post by listas at adminlinux.com.br ()
Dsync really is not working for EXPUNGE Imap command. This seems to
be a bug in version 2.2.13. Someone else was touched by it?
Just tried that with roundcube and 'doveadm expunge ...' on a fresh
2.2.13 with mail_replica:tcp; works as expected.

Didnt try the manual dance from 'listas'.
listas
2014-09-16 14:41:04 UTC
Permalink
I made a change in 'mail_plugins' directive, enabling 'notify' and
'replication' globally in dovecot.conf.

Now, in my tests, the second server sees the replication of the EXPUNGE
command only after running a SELECT command (or an click in the folder
on MUA).

Is this the expected behavior?

* I tested with IMAP talk (telnet) and Thunderbird.

Thanks!
--
Thiago Henrique

Loading...