[Dovecot] Help needed with plugin - Read Only access to IMAP mailbox

[Chris Moules]

System info:
# 1.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-5-686-bigmem i686 Debian squeeze/sid
..
mail_plugins: readonly
..

I have a requirement to have read-only to a mailbox. I have been researching through the wiki, the mailing list archives and
good old Google. There was a number of similar questions with no real definitive answer.

Option 1: ACL
This can work, but not if the mailbox(s) can change without you knowing how. I.E. a online read-only archive of someone else's
mailbox. There is no wild-card or recursive ACL options. Rsync style backups don't allow for easy creation of custom ACL files
per mailbox.

Option 2: Read-only filesystem (Linux)
This seems easy, but it's not.
1) how do you update your archive?
I tried with a bind mount, this does not let you set the 'ro' option *directly*. You must then use "-o remount,ro" to get your
read-only bind mount. This is a little messy (needs custom init scripts for mounting on boot), but do-able.

2) Dovecot needs write access to CONTROL and INDEX files.
This lead me to using the "CONTROL" and "INDEX" options on the mail_locaiton. Setting these to the original 'rw' mount and the
rest to my 'ro' bind mount. Again, messy but do-able.

3) Clients try to move messages from 'new' to 'cur' for Maildir.
Every time I opened a maildir I got an error per mail (dovecot log) about read-only filesysem as the client was trying to set
the Seen flag and dovecot was trying to move the 'new' mail to the 'cur' folder.
My test client, mutt, has a 'read_only' option when connecting. This eliminates this issue, but I cannot get everyone to use
this archive with a correctly configured mutt.

This sent me looking for 'read only' options in other clients. I could hardly find any.
I noted that mutt send an EXAMINE and not a SELECT to open the mailbox when in read_only mode and this set me thinking about
doing this in another way.

Option 3: Dovecot Plugin
I started looking into the dovecot plugins. Being more a systems administrator than developer I have cobbled together a working
plugin. It nearly does what I want, but not quite, and I don't know why.
NOTE: I no longer use the setup that I had to test Options 1 and 2.

I thought that this plugin can either be used server-wide or with the 'mail_plugins' userdb option.

The plugin forces the MAILBOX_OPEN_READONLY flag in a mailbox_open() call.
The plugin returns an error for all mailbox_create() calls.

Testing in mutt:
- I am unable to delete a mail
- I am unable to create a new folder
So far so good. However:
- I am able to move/copy a mail to an *existing* mailbox
(note, the move operation makes the copy but fails to set the 'delete' flag)

I do not understand why this works. I have been grep-ing through the source of dovecot and the plugins to find some answer, but
without luck.

Attached is the full source to my plugin and the Makefile (adapted from Johannes Berg's antispam plugin Makefile)

When finished, I will be happy to release this to the dovecot community under the GPL/LGPL or whatever.

Regards

Chris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: readonly_plugin.tgz
Type: application/x-gtar
Size: 1815 bytes
Desc: not available
Url : http://dovecot.org/pipermail/dovecot/attachments/20100825/d1bcddad/attachment.gtar

[Marcus Rueckert]

Post by Chris Moules
# 1.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-5-686-bigmem i686 Debian squeeze/sid
..
mail_plugins: readonly
..
I have a requirement to have read-only to a mailbox. I have been
researching through the wiki, the mailing list archives and good old
Google. There was a number of similar questions with no real
definitive answer.
Option 1: ACL
This can work, but not if the mailbox(s) can change without you
knowing how. I.E. a online read-only archive of someone else's
mailbox. There is no wild-card or recursive ACL options. Rsync style
backups don't allow for easy creation of custom ACL files per
mailbox.

acls are stored in dovecot-acls files either inside the mailbox or in
/etc/dovecot. so you can preserve them easily with rsync style backup.

Post by Chris Moules
2) Dovecot needs write access to CONTROL and INDEX files.
This lead me to using the "CONTROL" and "INDEX" options on the
mail_locaiton. Setting these to the original 'rw' mount and the rest
to my 'ro' bind mount. Again, messy but do-able.

Just for the record: you can configure CONTROL and INDEX seperately. see below.

my solution for a similar problem:

[[[
namespace public {
separator = /

# Mailboxes are visible under "shared/user at domain/"
# %%n, %%d and %%u are expanded to the destination user.
prefix = archive/

# Mail location for other users' mailboxes. Note that %variables and ~/
# expands to the logged in user's data. %%n, %%d, %%u and %%h expand to the
# destination user's data.
location = maildir:/srv/mail/archive:INDEX=/srv/mail/%u/shared/%%u:CONTROL=/srv/mail/%u/shared

# Use the default namespace for saving subscriptions.
subscriptions = yes

# List the shared/ namespace only if there are visible shared mailboxes.
list = children
}
]]]

only my mail archive user can deliver mails into that namespace (via ACL (p)).
all other users only have read permissions, as index/control are per user, each user can have their own
flags (like seen).

shouldnt this give you exactly what you want?

[Chris Moules]

Post by Marcus Rueckert

Post by Chris Moules
# 1.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-5-686-bigmem i686 Debian squeeze/sid
..
mail_plugins: readonly
..
I have a requirement to have read-only to a mailbox. I have been
researching through the wiki, the mailing list archives and good old
Google. There was a number of similar questions with no real
definitive answer.
Option 1: ACL
This can work, but not if the mailbox(s) can change without you
knowing how. I.E. a online read-only archive of someone else's
mailbox. There is no wild-card or recursive ACL options. Rsync style
backups don't allow for easy creation of custom ACL files per
mailbox.

acls are stored in dovecot-acls files either inside the mailbox or in
/etc/dovecot. so you can preserve them easily with rsync style backup.

Yes, I am aware of that. It is more the creation of ACL's on the *destination* that don't exist in the source.
Any new mailbox that would be created on the source server would need an ACL file created for it on the destination server after
being sync'ed.

Post by Marcus Rueckert

Post by Chris Moules
2) Dovecot needs write access to CONTROL and INDEX files.
This lead me to using the "CONTROL" and "INDEX" options on the
mail_locaiton. Setting these to the original 'rw' mount and the rest
to my 'ro' bind mount. Again, messy but do-able.

Just for the record: you can configure CONTROL and INDEX seperately. see below.

I thought that I stated that. I believe that I had set 'INDEX=MEMORY' and CONTROL=/home/vmail/%d/%u/Maildir
The home was set to the bind mount of /mail/vmail/%d/%u/
mail_locaiton was maildir:~/Maildir

Post by Marcus Rueckert
[[[
namespace public {
separator = /
# Mailboxes are visible under "shared/user at domain/"
# %%n, %%d and %%u are expanded to the destination user.
prefix = archive/
# Mail location for other users' mailboxes. Note that %variables and ~/
# expands to the logged in user's data. %%n, %%d, %%u and %%h expand to the
# destination user's data.
location = maildir:/srv/mail/archive:INDEX=/srv/mail/%u/shared/%%u:CONTROL=/srv/mail/%u/shared
# Use the default namespace for saving subscriptions.
subscriptions = yes
# List the shared/ namespace only if there are visible shared mailboxes.
list = children
}
]]]
only my mail archive user can deliver mails into that namespace (via ACL (p)).
all other users only have read permissions, as index/control are per user, each user can have their own
flags (like seen).
shouldnt this give you exactly what you want?

This seems to be solving a different problem to mine. I need, something like a mirror of accounts, on a separate server that
gives the user read-only access to the content.
The data is not public. I should only be accessible to the authorised user.
The input to this archive is the 'original' live maildir, so I do not have control over the creation of folders, etc. This
causes problems with dovecot ACL inheritance as the mailbox is not created via the dovecot server with the ACLs.

Regards

Chris

[Marcus Rueckert]

Post by Chris Moules

Post by Marcus Rueckert
[[[
namespace public {
separator = /
# Mailboxes are visible under "shared/user at domain/"
# %%n, %%d and %%u are expanded to the destination user.
prefix = archive/
# Mail location for other users' mailboxes. Note that %variables and ~/
# expands to the logged in user's data. %%n, %%d, %%u and %%h expand to the
# destination user's data.
location = maildir:/srv/mail/archive:INDEX=/srv/mail/%u/shared/%%u:CONTROL=/srv/mail/%u/shared
# Use the default namespace for saving subscriptions.
subscriptions = yes
# List the shared/ namespace only if there are visible shared mailboxes.
list = children
}
]]]
only my mail archive user can deliver mails into that namespace (via ACL (p)).
all other users only have read permissions, as index/control are per user, each user can have their own
flags (like seen).
shouldnt this give you exactly what you want?

This seems to be solving a different problem to mine. I need,
something like a mirror of accounts, on a separate server that gives
the user read-only access to the content.
The data is not public. I should only be accessible to the authorised user.
The input to this archive is the 'original' live maildir, so I do
not have control over the creation of folders, etc. This causes
problems with dovecot ACL inheritance as the mailbox is not created
via the dovecot server with the ACLs.

you can specify default ACLs in /etc/dovecot/acls?

i suggest playing around with mail_debug and see what ACL files it tries
to load.

and the name "public" for the namespace is confusing. it is not really
public. only people with ACL entries can read from it. (yes i tested
this)

but unlike shared namespaces it is not user specific (e.g.
"shared/foo at bar/INBOX")

darix

[Chris Moules]

Post by Marcus Rueckert

Post by Chris Moules
This seems to be solving a different problem to mine. I need,
something like a mirror of accounts, on a separate server that gives
the user read-only access to the content.
The data is not public. I should only be accessible to the authorised user.
The input to this archive is the 'original' live maildir, so I do
not have control over the creation of folders, etc. This causes
problems with dovecot ACL inheritance as the mailbox is not created
via the dovecot server with the ACLs.

Marcus,

thanks again for the reply.

Post by Marcus Rueckert
you can specify default ACLs in /etc/dovecot/acls?

I did try this. Again, the issue being that they are not inherited to sub-folders, so a ACL for the INBOX is not used for all
folders. You need a global ACL file named for each folder name. So if a client creates a folder called "My banana photo
collection" you would need a file "/etc/dovecot/acls/My banana photo collection" with something like "authenticated rl"

It is not possible to have a global ACL for every possible folder name.

Post by Marcus Rueckert
i suggest playing around with mail_debug and see what ACL files it tries
to load.
and the name "public" for the namespace is confusing. it is not really
public. only people with ACL entries can read from it. (yes i tested
this)
but unlike shared namespaces it is not user specific (e.g.
"shared/foo at bar/INBOX")
darix

I have experimented with the ACL options. It could be do-able but it seemed a *lot* harder to get that right than to have a
little plugin on a 'archive/recover' server.


Regards

Chris

[Marcus Rueckert]

Post by Chris Moules

Post by Marcus Rueckert
you can specify default ACLs in /etc/dovecot/acls?

I did try this. Again, the issue being that they are not inherited to
sub-folders, so a ACL for the INBOX is not used for all folders. You
need a global ACL file named for each folder name. So if a client
creates a folder called "My banana photo collection" you would need a
file "/etc/dovecot/acls/My banana photo collection" with something
like "authenticated rl"
It is not possible to have a global ACL for every possible folder name.

to quote http://wiki.dovecot.org/ACL :

[[[
Every time you create a new mailbox, it gets its ACLs from the parent
mailbox. If you're creating a root-level mailbox, it uses the
namespace's default ACLs. There is no actual inheritance, however: If
you modify parent's ACLs, the child's ACLs stay the same. There is
currently no support for ACL inheritance.

The default ACLs are read from "dovecot-acl" file in the namespace's
mail root directory (e.g. /var/public/Maildir).
]]]

darix

[Chris Moules]

Post by Marcus Rueckert

Post by Chris Moules

Post by Marcus Rueckert
you can specify default ACLs in /etc/dovecot/acls?

I did try this. Again, the issue being that they are not inherited to
sub-folders, so a ACL for the INBOX is not used for all folders. You
need a global ACL file named for each folder name. So if a client
creates a folder called "My banana photo collection" you would need a
file "/etc/dovecot/acls/My banana photo collection" with something
like "authenticated rl"
It is not possible to have a global ACL for every possible folder name.

[[[
Every time you create a new mailbox, it gets its ACLs from the parent
mailbox. If you're creating a root-level mailbox, it uses the
namespace's default ACLs. There is no actual inheritance, however: If
you modify parent's ACLs, the child's ACLs stay the same. There is
currently no support for ACL inheritance.
The default ACLs are read from "dovecot-acl" file in the namespace's
mail root directory (e.g. /var/public/Maildir).
]]]
darix

Marcus / darix,

I read the wiki ACL thoroughly. I believe that you are missing the point.

source server -rsync-> destination server
(Read/Write) (Read Only)

- I am _not_ doing everything though dovecot.
- Maildirs are being synced from one server to another (source -> destination).
- The 'new' mailbox (or folder as I have refered to them up until now) is created on the 'source' server (where ACLs are not
enabled).
- The 'destination' dovecot system has the Maildir changed underneath it, direct disk access (rsync). The ACL plugin has no
influence on it's creation, so no auto-created "dovecot-acl" file like the parent (or not).
- Global ACLs do not get inherited to the child mailboxes (I have not seen this written in black & white, my testing confirms
this however). In the wiki Global ACLs have a different write-up to their 'standard' counterpart and need the full name / hierarchy.

The fact that my ACL/read-only dovecot server does not have any control over the creation of the maildirs means that the sync
system would need to create a "dovecot-acl" file for all maildirs. This complicates the matter and leaves room for mistakes.

Through my research and testing I had the idea that using a dovecot plugin I could just tell the client that they only had read
access to the server. This would avoid then need to have over-complex ACLs that looked like they would not, elegantly, solve my
problem. The plugins did not seem over complex and I have been able to realize most of my need with very little code.

Regards

Chris

[Timo Sirainen]

Post by Chris Moules
Option 1: ACL
This can work, but not if the mailbox(s) can change without you knowing how. I.E. a online read-only archive of someone else's
mailbox. There is no wild-card or recursive ACL options. Rsync style backups don't allow for easy creation of custom ACL files
per mailbox.

I think you could pretty easily add support for "default ACL file" that
is used instead of the internal ACL defaults. I've been planning on
doing that at some point anyway. Maybe ~/Maildir/dovecot-acl-default or
something.

Post by Chris Moules
The plugin forces the MAILBOX_OPEN_READONLY flag in a mailbox_open() call.

Yeah .. this flag isn't enforced much really.. I think I should just
remove it.

[Chris Moules]

Post by Timo Sirainen

Post by Chris Moules
Option 1: ACL
This can work, but not if the mailbox(s) can change without you knowing how. I.E. a online read-only archive of someone else's
mailbox. There is no wild-card or recursive ACL options. Rsync style backups don't allow for easy creation of custom ACL files
per mailbox.

I think you could pretty easily add support for "default ACL file" that
is used instead of the internal ACL defaults. I've been planning on
doing that at some point anyway. Maybe ~/Maildir/dovecot-acl-default or
something.

So, that should be a patch to the current ACL plugin?
Any pointers on where to start with that? I only started on dovecot plugin programming yesterday. The ACL plugin seemed the most
complex so I avoided it for 'learning'.

Post by Timo Sirainen

Post by Chris Moules
The plugin forces the MAILBOX_OPEN_READONLY flag in a mailbox_open() call.

Yeah .. this flag isn't enforced much really.. I think I should just
remove it.

Well, that explains why it seemed to work, but not really.
An alternative to removing it could be to enforce it...

As a quick fix, I can combine my current plugin with my read only filesystem hack. At lease with this I only get a SERVERBUG
message when I try to copy or move a mail.

Regards

Chris

[Timo Sirainen]

Post by Chris Moules

Post by Timo Sirainen
I think you could pretty easily add support for "default ACL file" that
is used instead of the internal ACL defaults. I've been planning on
doing that at some point anyway. Maybe ~/Maildir/dovecot-acl-default or
something.

So, that should be a patch to the current ACL plugin?

Yes.

Post by Chris Moules
Any pointers on where to start with that? I only started on dovecot plugin programming yesterday. The ACL plugin seemed the most
complex so I avoided it for 'learning'.

Hmm.. Now that I look at the code, the default ACL handling is a bit
strange and I guess it needs some rethinking. But, I think for your
purpose you can do it very easily. acl-backend.c contains:

static const char *const owner_mailbox_rights[] = {
..

Simply change that list to what rights you want to have (probably
LOOKUP, READ).

Post by Chris Moules

Post by Timo Sirainen

Post by Chris Moules
The plugin forces the MAILBOX_OPEN_READONLY flag in a mailbox_open() call.

Yeah .. this flag isn't enforced much really.. I think I should just
remove it.

Well, that explains why it seemed to work, but not really.
An alternative to removing it could be to enforce it...

The reason why I didn't want to do that was because it wasn't entirely
clear what operations should be readonly and what shouldn't. For example
originally I was using READONLY whenever mailbox was opened with IMAP's
EXAMINE command instead of SELECT command. But it's still valid to save
a new message via APPEND command, because it doesn't care about what
mailbox is selected at the time. But Dovecot optimized this so that it
used the existing EXAMINEd readonly mailbox, which then failed saving.
Another possible fix would have been to simply open the same mailbox
again as readwrite, but that wasted CPU, memory and maybe disk I/O..

[Chris Moules]

Post by Timo Sirainen

Post by Chris Moules

Post by Timo Sirainen
I think you could pretty easily add support for "default ACL file" that
is used instead of the internal ACL defaults. I've been planning on
doing that at some point anyway. Maybe ~/Maildir/dovecot-acl-default or
something.

So, that should be a patch to the current ACL plugin?

Yes.

Post by Chris Moules
Any pointers on where to start with that? I only started on dovecot plugin programming yesterday. The ACL plugin seemed the most
complex so I avoided it for 'learning'.

Hmm.. Now that I look at the code, the default ACL handling is a bit
strange and I guess it needs some rethinking. But, I think for your
static const char *const owner_mailbox_rights[] = {
..
Simply change that list to what rights you want to have (probably
LOOKUP, READ).

Timo,

many thanks! A recompile with the below changes seems to do the trick!
This, obviously, is only good server-wide read-only, but that is what I need.

dovecot -n relevant settings:
mail_plugins: acl
plugin:
acl: vfile

No imap_acl as we do not want to advertise ACL support via IMAP.
Without the "acl = vfile" line in 'plugin' section, it does not work.

thanks again

Chris

Patch:
--- dovecot-1.2.13.orig//src/plugins/acl/acl-backend.c 2010-05-24 15:01:15.000000000 +0000
+++ dovecot-1.2.13/src/plugins/acl/acl-backend.c 2010-08-25 15:23:07.000000000 +0000
@@ -12,15 +12,6 @@
static const char *const owner_mailbox_rights[] = {
MAIL_ACL_LOOKUP,
MAIL_ACL_READ,
- MAIL_ACL_WRITE,
- MAIL_ACL_WRITE_SEEN,
- MAIL_ACL_WRITE_DELETED,
- MAIL_ACL_INSERT,
- MAIL_ACL_POST,
- MAIL_ACL_EXPUNGE,
- MAIL_ACL_CREATE,
- MAIL_ACL_DELETE,
- MAIL_ACL_ADMIN,
NULL
};