Discussion:
Creating a backup of incoming mail
LuKreme
2014-08-30 16:26:17 UTC
Permalink
This is my master.cf line for dovecot virtual users:

dovecot unix - n n - - pipe flags=DRhu user=vpopmail:vchkpw argv=/usr/local/libexec/dovecot/dovecot-lda -f ${sender} -d ${user}@${nexthop} -m ${extension}

I would like to also write every mail, before delivery to the user, to a backup location like /backup/imap/mm/dd

Possible?
--
'On whose authority?' demanded Wert. Trymon turned his grey eyes on him.
'Mine. I need no other.' --The Light Fantastic
Oscar del Rio
2014-08-30 16:33:54 UTC
Permalink
Post by LuKreme
I would like to also write every mail, before delivery to the user, to a backup location like /backup/imap/mm/dd
You could use a global "sieve_before" filter.

http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration
LuKreme
2014-09-01 16:46:58 UTC
Permalink
Post by Oscar del Rio
You could use a global "sieve_before" filter.
http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration
And, I get hampered on the very first step:

To use Sieve, you will first need to make sure you are using Dovecot LDA or LMTP for delivering incoming mail to users' mailboxes. Then, you need to enable the Pigeonhole Sieve plugin in your configuration:

protocol lda {
mail_plugins = $mail_plugins sieve
}

OK, so I am using dovecot-lda via postfix/master.cf, but in my configuration I have a section tagged protocols, not protocol.

Don't know if this is the same thing, or a slightly different thing. I don't know where/what $mail_plugins refers to, and there is no "plugins" section of my configuration file. In fact, the string "plug" doesn't appear in dovecot.conf (there are some commented lines for mail_plugins in conf.d/ files).

Then, sieve_global_path is deprecated, but despite being tagged as deprecated, the example config uses the sieve_global_path. Shouldn't the example use sieve_default?

Looks like further down is the sieve_before, but really, the documentation feels almost like it has set out with the intention of being as obtuse as possible.

If I understand it, and I am sure I do not, what I need to add in to dovecot.conf is something like this:

protocol lda {
mail_plugins = $mail_plugins sieve
}

(which is separate and unrelated to my "protocols = imap" existing line in dovecot.conf)

plugin {
sieve = ~/.dovecot.sieve
sieve_before = /var/lib/dovecot/sieve/archive.sieve
}

And then in archive.sieve I find out what the commands are to write a COPY of the mail to a location after getting the date info.

require ["variables","date","fileinto","mailbox"];

if currentdate :matches "year" "*" { set "year" "${1}"; }
if currentdate :matches "month" "*" { set "month" "${1}"; }

And a command like this, but not this, since i wan the backup stores in an absolute path outside the IMAP store and I obviously only want a copy of that message.

fileinto :create "INBOX.Lists.${year}.${month}.dovecot";
}
--
Did they get you to trade your heroes for ghosts? Hot ashes for trees?
Hot air for a cool breeze? Cold comfort for change?
Patrick De Zordo
2014-09-02 07:06:47 UTC
Permalink
Well, I'll give you some configuration hints..

1. don't forget to install sieve (eg. "apt-get install dovecot-sieve")..

2. Add to "postfix/main.cf" the following lines:
---8<--------------------------------------------------------------------------------
# Tell postfix to hand off mail to the definition for dovecot in master.cf
virtual_transport = dovecot
dovecot_destination_recipient_limit = 1
---8<--------------------------------------------------------------------------------

3. In "postfix/master.cf":
---8<--------------------------------------------------------------------------------
# Integration with Dovecot - hand mail over to it for local delivery, and
# run the process under the vmail user and mail group.
dovecot unix - n n - - pipe
flags=DRhu user=vmail:mail argv=/usr/lib/dovecot/dovecot-lda -d $(recipient)
---8<--------------------------------------------------------------------------------

4. add "$mail_plugins sieve" to "dovecot/conf.d/15-lda.conf"

That?s all! Then just provide the sieve scripts at location specified in "dovecot/conf.d/90-sieve.conf" and you are done!

Good luck!

Bye!

---------------------------------------------------------------------
Mit freundlichen Gr??en / Distinti saluti / Kind regards
De Zordo Patrick
patrick at spamreducer.eu
-----Urspr?ngliche Nachricht-----
Von: dovecot [mailto:dovecot-bounces at dovecot.org] Im Auftrag von
LuKreme
Gesendet: Montag, 1. September 2014 18:47
An: dovecot at dovecot.org
Betreff: Re: Creating a backup of incoming mail
Post by Oscar del Rio
You could use a global "sieve_before" filter.
http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration
To use Sieve, you will first need to make sure you are using Dovecot LDA or
LMTP for delivering incoming mail to users' mailboxes. Then, you need to
protocol lda {
mail_plugins = $mail_plugins sieve
}
OK, so I am using dovecot-lda via postfix/master.cf, but in my configuration I
have a section tagged protocols, not protocol.
Don't know if this is the same thing, or a slightly different thing. I don't know
where/what $mail_plugins refers to, and there is no "plugins" section of my
configuration file. In fact, the string "plug" doesn't appear in dovecot.conf
(there are some commented lines for mail_plugins in conf.d/ files).
Then, sieve_global_path is deprecated, but despite being tagged as
deprecated, the example config uses the sieve_global_path. Shouldn't the
example use sieve_default?
Looks like further down is the sieve_before, but really, the documentation
feels almost like it has set out with the intention of being as obtuse as
possible.
If I understand it, and I am sure I do not, what I need to add in to
protocol lda {
mail_plugins = $mail_plugins sieve
}
(which is separate and unrelated to my "protocols = imap" existing line in dovecot.conf)
plugin {
sieve = ~/.dovecot.sieve
sieve_before = /var/lib/dovecot/sieve/archive.sieve
}
And then in archive.sieve I find out what the commands are to write a COPY
of the mail to a location after getting the date info.
require ["variables","date","fileinto","mailbox"];
if currentdate :matches "year" "*" { set "year" "${1}"; } if currentdate
:matches "month" "*" { set "month" "${1}"; }
And a command like this, but not this, since i wan the backup stores in an
absolute path outside the IMAP store and I obviously only want a copy of
that message.
fileinto :create "INBOX.Lists.${year}.${month}.dovecot";
}
--
Did they get you to trade your heroes for ghosts? Hot ashes for trees?
Hot air for a cool breeze? Cold comfort for change?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6229 bytes
Desc: not available
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140902/c2e19d96/attachment-0001.p7s>
Patrick De Zordo
2014-09-01 07:33:52 UTC
Permalink
To backup all mail (incoming and outgoing), BCC all mails, you could do the following..

Add to your "/etc/postfix/main.cf" the following:
---8<-------------------------------------------------------------------------------------
# Auto-Backup all mails
transport_maps = hash:/etc/postfix/transport
backuplmtp_destination_recipient_limit = 1
lmtp_destination_recipient_limit = 1
recipient_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre
sender_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre
---8<-------------------------------------------------------------------------------------

Create a domain called "backup.local" in postfixadmin or whatever you use to edit your accounts..

Add every domain for which you like to activate incoming backup to " /etc/postfix/backup_bcc.pcre":
---8<-------------------------------------------------------------------------------------
/^(.*)@spamreducer\.eu$/ spamreducer.eu at backup.local
---8<-------------------------------------------------------------------------------------

Add to your "/etc/postfix/master.cf" the following:
---8<-------------------------------------------------------------------------------------
backuplmtp unix - - n - - lmtp
---8<-------------------------------------------------------------------------------------

Add to your "/etc/postfix/transport" the following:
---8<-------------------------------------------------------------------------------------
[backup.local] backuplmtp
---8<-------------------------------------------------------------------------------------

Don't forget to issue " postmap /etc/postfix/transport"..

For making folders based on "year/month" you have to use an appropriate sieve script.

Give me a feedback!
Bye!

---------------------------------------------------------------------
Mit freundlichen Gr??en / Distinti saluti / Kind regards
De Zordo Patrick
patrick at spamreducer.eu
-----Urspr?ngliche Nachricht-----
Von: dovecot [mailto:dovecot-bounces at dovecot.org] Im Auftrag von
LuKreme
Gesendet: Samstag, 30. August 2014 18:26
An: dovecot at dovecot.org
Betreff: Creating a backup of incoming mail
dovecot unix - n n - - pipe flags=DRhu user=vpopmail:vchkpw
argv=/usr/local/libexec/dovecot/dovecot-lda -f ${sender} -d
I would like to also write every mail, before delivery to the user, to a backup
location like /backup/imap/mm/dd
Possible?
--
'On whose authority?' demanded Wert. Trymon turned his grey eyes on him.
'Mine. I need no other.' --The Light Fantastic
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6229 bytes
Desc: not available
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140901/0c5825f7/attachment.p7s>
LuKreme
2014-09-01 16:20:42 UTC
Permalink
Post by Patrick De Zordo
To backup all mail (incoming and outgoing), BCC all mails, you could do the following..
---8<-------------------------------------------------------------------------------------
# Auto-Backup all mails
transport_maps = hash:/etc/postfix/transport
backuplmtp_destination_recipient_limit = 1
lmtp_destination_recipient_limit = 1
recipient_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre
sender_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre
---8<-------------------------------------------------------------------------------------
Create a domain called "backup.local" in postfixadmin or whatever you use to edit your accounts..
---8<-------------------------------------------------------------------------------------
---8<-------------------------------------------------------------------------------------
---8<-------------------------------------------------------------------------------------
backuplmtp unix - - n - - lmtp
---8<-------------------------------------------------------------------------------------
---8<-------------------------------------------------------------------------------------
[backup.local] backuplmtp
---8<-------------------------------------------------------------------------------------
Don't forget to issue " postmap /etc/postfix/transport"..
For making folders based on "year/month" you have to use an appropriate sieve script.
Give me a feedback!
Wow, that is a whole lot more work that what I did with procmail which was practically one line in the /usr/locale/etc/procmailrc right at the top.

YER=`date %Y`
MON=`date %m`

:0c
/backup/mail/$YER/$MON/

I'll start with Sieve (though I still haven't used it) and see what that can do, but this at least I can do, if sieve proves too annoying to get going.
--
Granny Weatherwax didn't hold with looking at the future, but now she
could feel the future looking at her. She didn't like its expression at
all.
Robert Schetterer
2014-09-02 07:26:36 UTC
Permalink
Post by LuKreme
Post by Patrick De Zordo
To backup all mail (incoming and outgoing), BCC all mails, you could do the following..
---8<-------------------------------------------------------------------------------------
# Auto-Backup all mails
transport_maps = hash:/etc/postfix/transport
backuplmtp_destination_recipient_limit = 1
lmtp_destination_recipient_limit = 1
recipient_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre
sender_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre
---8<-------------------------------------------------------------------------------------
Create a domain called "backup.local" in postfixadmin or whatever you use to edit your accounts..
---8<-------------------------------------------------------------------------------------
---8<-------------------------------------------------------------------------------------
---8<-------------------------------------------------------------------------------------
backuplmtp unix - - n - - lmtp
---8<-------------------------------------------------------------------------------------
---8<-------------------------------------------------------------------------------------
[backup.local] backuplmtp
---8<-------------------------------------------------------------------------------------
Don't forget to issue " postmap /etc/postfix/transport"..
For making folders based on "year/month" you have to use an appropriate sieve script.
Give me a feedback!
Wow, that is a whole lot more work that what I did with procmail which was practically one line in the /usr/locale/etc/procmailrc right at the top.
YER=`date %Y`
MON=`date %m`
:0c
/backup/mail/$YER/$MON/
see

https://sys4.de/de/blog/2013/02/07/mailarchiv-mit-dovecot-und-postfix-sortiert-nach-datum-mailadressen-und-ein-ausgehend-unterordnern/

sorry german


with listescape plugin enabled something like

require
["reject","variables","date","fileinto","mailbox","envelope","subaddress","regex","copy","include"];
# Extract date info
if currentdate :matches "year" "*" { set "year" "${1}"; }
if currentdate :matches "month" "*" { set "month" "${1}"; }
if currentdate :matches "day" "*" { set "day" "${1}"; }
if envelope :detail :matches "to" "*" { set :lower "to" "${1}"; }
if true {
if header :contains "Return-Path" "${to}@domain.de" { fileinto :create
"user-backup/${year}/${month}/${day}/${to}/out";}
else { fileinto :create "user-backup/${year}/${month}/${day}/${to}/in";}
stop;
}
Post by LuKreme
I'll start with Sieve (though I still haven't used it) and see what that can do, but this at least I can do, if sieve proves too annoying to get going.
Best Regards
MfG Robert Schetterer
--
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Franziskanerstra?e 15, 81669 M?nchen

Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
LuKreme
2014-09-02 22:14:54 UTC
Permalink
Post by Robert Schetterer
with listescape plugin enabled something like
require
["reject","variables","date","fileinto","mailbox","envelope","subaddress","regex","copy","include"];
# Extract date info
if currentdate :matches "year" "*" { set "year" "${1}"; }
if currentdate :matches "month" "*" { set "month" "${1}"; }
if currentdate :matches "day" "*" { set "day" "${1}"; }
if envelope :detail :matches "to" "*" { set :lower "to" "${1}"; }
if true {
"user-backup/${year}/${month}/${day}/${to}/out";}
else { fileinto :create "user-backup/${year}/${month}/${day}/${to}/in";}
stop;
}
Oh, thanks for that. I'd made very little progress in finding "fileinto :create"
--
'Pardon me for living, I'm sure.' NO-ONE GETS PARDONED FOR LIVING.
Will Yardley
2014-09-07 00:20:08 UTC
Permalink
Post by Patrick De Zordo
To backup all mail (incoming and outgoing), BCC all mails, you could do the following..
---8<-------------------------------------------------------------------------------------
# Auto-Backup all mails
transport_maps = hash:/etc/postfix/transport
backuplmtp_destination_recipient_limit = 1
lmtp_destination_recipient_limit = 1
recipient_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre
sender_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre
Why not just use $always_bcc?

w
LuKreme
2014-09-09 07:05:54 UTC
Permalink
Post by Will Yardley
Post by Patrick De Zordo
To backup all mail (incoming and outgoing), BCC all mails, you could do the following..
---8<-------------------------------------------------------------------------------------
# Auto-Backup all mails
transport_maps = hash:/etc/postfix/transport
backuplmtp_destination_recipient_limit = 1
lmtp_destination_recipient_limit = 1
recipient_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre
sender_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre
Why not just use $always_bcc?
Does that apply to INCOMING mail? (that's what I am trying to backup)
--
'I warn you, dragon, the human spirit is-' They never found out what it
was, or at least what he thought it was, although possibly in the dark
hours of a sleepless night some of them might have remembered the
subsequent events and formed a pretty good and gut-churning insight, to
whit, that one of the things sometimes forgotten about the human spirit
is that while it is, in the right conditions, noble and brave and
wonderful, it is also, when you get right down to it, only human.
Robert Schetterer
2014-09-09 08:06:58 UTC
Permalink
Post by LuKreme
Post by Will Yardley
Post by Patrick De Zordo
To backup all mail (incoming and outgoing), BCC all mails, you could do the following..
---8<-------------------------------------------------------------------------------------
# Auto-Backup all mails
transport_maps = hash:/etc/postfix/transport
backuplmtp_destination_recipient_limit = 1
lmtp_destination_recipient_limit = 1
recipient_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre
sender_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre
Why not just use $always_bcc?
Does that apply to INCOMING mail? (that's what I am trying to backup)
optional to

recipient_bcc_maps
sender_bcc_maps


you may use

http://www.postfix.org/postconf.5.html
...
always_bcc (default: empty)

Optional address that receives a "blind carbon copy" of each message
that is received by the Postfix mail system.

Note: with Postfix 2.3 and later the BCC address is added as if it
was specified with NOTIFY=NONE. The sender will not be notified when the
BCC address is undeliverable, as long as all down-stream software
implements RFC 3461.

Note: with Postfix 2.2 and earlier the sender will be notified when
the BCC address is undeliverable.

Note: automatic BCC recipients are produced only for new mail. To
avoid mailer loops, automatic BCC recipients are not generated after
Postfix forwards mail internally, or after Postfix generates mail itself.
...





Best Regards
MfG Robert Schetterer
--
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Franziskanerstra?e 15, 81669 M?nchen

Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
LuKreme
2014-09-10 00:25:32 UTC
Permalink
Post by Robert Schetterer
Post by LuKreme
Post by Will Yardley
Post by Patrick De Zordo
To backup all mail (incoming and outgoing), BCC all mails, you could
do the following..
---8<-------------------------------------------------------------------------------------
# Auto-Backup all mails
transport_maps = hash:/etc/postfix/transport
backuplmtp_destination_recipient_limit = 1
lmtp_destination_recipient_limit = 1
recipient_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre
sender_bcc_maps = pcre:/etc/postfix/backup_bcc.pcre
Why not just use $always_bcc?
Does that apply to INCOMING mail? (that's what I am trying to backup)
optional to
recipient_bcc_maps
sender_bcc_maps
Well, heck. I feel twice as dumb as I did before I read your message.

Thanks for that, I think. :)
--
I DID NOT SEE ELVIS Bart chalkboard Ep. 7G07
Loading...