Discussion:
Dovecot Sieve and Postfix header_checks Issue
Klaipedaville on Google
2014-09-26 11:35:15 UTC
Permalink
Hello List,

I tried to subscribe but it's taking forever for the confirmation email to arrive so I thought I would ask away by emailing directly. My apologies in advance should this question appear twice.

It may seem real simple to experts but I cannot really figure it out. I'll try to be concise:

Dovecot version is 2.1.7. Its dovecot ?n is real short one and follows right after my question.

I have my Postfix 2.9.6 properties set like this: header_checks = /etc/path/to/myfile. Then I have Dovecot Sieve also configured and working fine.

Now the trouble is that these two cannot be combined together can they? Dovecot Sieve and Postfix's header_checks? If I turn off header_checks in Postfix then Dovecot Sieve is working fine. If I vise versa turn off Dovecot Sieve then Postfix's header-checks are also working fine. However, when I keep them both turned on it also works but only partially and in this case Dovecot Sieve never bounces back any rejected massages remotely, the bounce happens only locally. That is I can see it in my logs that it was rejected locally but the message is never sent back to the sender saying something like, "Spam is rejected here". Thus the reject turns into a silent discard without telling the sender anything at all. Any ideas, help, advices how do I fix that? Would be really grateful for any suggestions / assistance at all. Many thanks in advance!

P.S. I would like to use both because Dovecot Sieve is a very powerful and great plugin that uses Spamassassin's added tags and headers based on which I create my rules. Postfix in its turn has a very neat feature of checking the headers via regexp but Postfix does not see / detect Spamassassin's added headers at all because I passed delivery and authentication (which is a way better than Postfix's) to dovecot deliver.

# 2.1.7: /etc/dovecot/dovecot.conf
# OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.6 ext4
auth_mechanisms = plain login cram-md5 scram-sha-1
auth_verbose = yes
hostname = WindTalker
info_log_path = /var/log/dovecot-sieve.log
log_path = /var/log/dovecot-sieve-errors.log
mail_location = maildir:/home/mvail/%d/%n
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave
passdb {
args = /etc/dovecot/passwd
driver = passwd-file
}
plugin {
sieve = ~/.dovecot.sieve
sieve_default = /etc/dovecot/default.sieve
sieve_dir = ~/sieve
}
pop3_uidl_format = %g
postmaster_address = postmaster at example.com
protocols = imap pop3 sieve
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
}
ssl_ca = </etc/ssl/ca.crt
ssl_cert = </etc/ssl//new.crt
ssl_key = </etc/ssl/new.key
userdb {
args = uid=vmail gid=vmail home=/home/vmail/%u
driver = static
}
protocol lda {
mail_debug = yes
mail_plugins = sieve
}

Regards,
Dennis.
Alex Crow
2014-09-26 11:47:21 UTC
Permalink
That would most likely be something in your header_checks that is
causing the bounce from Sieve to be rejected.

There is no reason why you cannot use both.
Post by Klaipedaville on Google
Hello List,
I tried to subscribe but it's taking forever for the confirmation email to arrive so I thought I would ask away by emailing directly. My apologies in advance should this question appear twice.
Dovecot version is 2.1.7. Its dovecot ?n is real short one and follows right after my question.
I have my Postfix 2.9.6 properties set like this: header_checks = /etc/path/to/myfile. Then I have Dovecot Sieve also configured and working fine.
Now the trouble is that these two cannot be combined together can they? Dovecot Sieve and Postfix's header_checks? If I turn off header_checks in Postfix then Dovecot Sieve is working fine. If I vise versa turn off Dovecot Sieve then Postfix's header-checks are also working fine. However, when I keep them both turned on it also works but only partially and in this case Dovecot Sieve never bounces back any rejected massages remotely, the bounce happens only locally. That is I can see it in my logs that it was rejected locally but the message is never sent back to the sender saying something like, "Spam is rejected here". Thus the reject turns into a silent discard without telling the sender anything at all. Any ideas, help, advices how do I fix that? Would be really grateful for any suggestions / assistance at all. Many thanks in advance!
P.S. I would like to use both because Dovecot Sieve is a very powerful and great plugin that uses Spamassassin's added tags and headers based on which I create my rules. Postfix in its turn has a very neat feature of checking the headers via regexp but Postfix does not see / detect Spamassassin's added headers at all because I passed delivery and authentication (which is a way better than Postfix's) to dovecot deliver.
# 2.1.7: /etc/dovecot/dovecot.conf
# OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.6 ext4
auth_mechanisms = plain login cram-md5 scram-sha-1
auth_verbose = yes
hostname = WindTalker
info_log_path = /var/log/dovecot-sieve.log
log_path = /var/log/dovecot-sieve-errors.log
mail_location = maildir:/home/mvail/%d/%n
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave
passdb {
args = /etc/dovecot/passwd
driver = passwd-file
}
plugin {
sieve = ~/.dovecot.sieve
sieve_default = /etc/dovecot/default.sieve
sieve_dir = ~/sieve
}
pop3_uidl_format = %g
postmaster_address = postmaster at example.com
protocols = imap pop3 sieve
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
}
ssl_ca = </etc/ssl/ca.crt
ssl_cert = </etc/ssl//new.crt
ssl_key = </etc/ssl/new.key
userdb {
args = uid=vmail gid=vmail home=/home/vmail/%u
driver = static
}
protocol lda {
mail_debug = yes
mail_plugins = sieve
}
Regards,
Dennis.
--
This message is intended only for the addressee and may contain
confidential information. Unless you are that person, you may not
disclose its contents or use it in any way and are requested to delete
the message along with any attachments and notify us immediately.
"Transact" is operated by Integrated Financial Arrangements plc. 29
Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608
5300. (Registered office: as above; Registered in England and Wales
under number: 3727592). Authorised and regulated by the Financial
Conduct Authority (entered on the Financial Services Register; no. 190856).
Klaipedaville on Google
2014-09-26 12:00:28 UTC
Permalink
Thank you for your suggestion, Alex.

However, my header_checks file has just 5 lines of regexp as follows:

/^Subject:.**{5}SPAM*{5}/ REJECT No spammers allowed here.
/^Subject:.*\*\*\*\*\*SPAM\*\*\*\*\*/ REJECT No spammers allowed.
/\s**{5}SPAM*{5}/ REJECT No spamming hullababballos allowed.
/^Subject:(.*)SPAM/ REJECT Spam is not allowed. DISCARD.
/^From:.*\@.*\.tw/ REJECT Sorry, Taiwanese mail is not allowed.

All the regexp are correct. The first and the third lines actually do the same thing and they are there for testing purposes only. Execution stops at the very first rule matched so there is no problem in having any rules repeated. I can't see how a few the most simplest regular expressions can cause the bounce from Sieve not to go through.. Do you see any problems in these lines above that could possibly be the cause of your suggestion?


From: Alex Crow
Sent: Friday, September 26, 2014 14:47
To: dovecot at dovecot.org
Subject: Re: Dovecot Sieve and Postfix header_checks Issue

That would most likely be something in your header_checks that is
causing the bounce from Sieve to be rejected.

There is no reason why you cannot use both.
Post by Klaipedaville on Google
Hello List,
I tried to subscribe but it's taking forever for the confirmation email to arrive so I thought I would ask away by emailing directly. My apologies in advance should this question appear twice.
Dovecot version is 2.1.7. Its dovecot ?n is real short one and follows right after my question.
I have my Postfix 2.9.6 properties set like this: header_checks = /etc/path/to/myfile. Then I have Dovecot Sieve also configured and working fine.
Now the trouble is that these two cannot be combined together can they? Dovecot Sieve and Postfix's header_checks? If I turn off header_checks in Postfix then Dovecot Sieve is working fine. If I vise versa turn off Dovecot Sieve then Postfix's header-checks are also working fine. However, when I keep them both turned on it also works but only partially and in this case Dovecot Sieve never bounces back any rejected massages remotely, the bounce happens only locally. That is I can see it in my logs that it was rejected locally but the message is never sent back to the sender saying something like, "Spam is rejected here". Thus the reject turns into a silent discard without telling the sender anything at all. Any ideas, help, advices how do I fix that? Would be really grateful for any suggestions / assistance at all. Many thanks in advance!
P.S. I would like to use both because Dovecot Sieve is a very powerful and great plugin that uses Spamassassin's added tags and headers based on which I create my rules. Postfix in its turn has a very neat feature of checking the headers via regexp but Postfix does not see / detect Spamassassin's added headers at all because I passed delivery and authentication (which is a way better than Postfix's) to dovecot deliver.
# 2.1.7: /etc/dovecot/dovecot.conf
# OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.6 ext4
auth_mechanisms = plain login cram-md5 scram-sha-1
auth_verbose = yes
hostname = WindTalker
info_log_path = /var/log/dovecot-sieve.log
log_path = /var/log/dovecot-sieve-errors.log
mail_location = maildir:/home/mvail/%d/%n
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave
passdb {
args = /etc/dovecot/passwd
driver = passwd-file
}
plugin {
sieve = ~/.dovecot.sieve
sieve_default = /etc/dovecot/default.sieve
sieve_dir = ~/sieve
}
pop3_uidl_format = %g
postmaster_address = postmaster at example.com
protocols = imap pop3 sieve
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
}
ssl_ca = </etc/ssl/ca.crt
ssl_cert = </etc/ssl//new.crt
ssl_key = </etc/ssl/new.key
userdb {
args = uid=vmail gid=vmail home=/home/vmail/%u
driver = static
}
protocol lda {
mail_debug = yes
mail_plugins = sieve
}
Regards,
Dennis.
--
This message is intended only for the addressee and may contain
confidential information. Unless you are that person, you may not
disclose its contents or use it in any way and are requested to delete
the message along with any attachments and notify us immediately.
"Transact" is operated by Integrated Financial Arrangements plc. 29
Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608
5300. (Registered office: as above; Registered in England and Wales
under number: 3727592). Authorised and regulated by the Financial
Conduct Authority (entered on the Financial Services Register; no. 190856).
Klaipedaville on Google
2014-09-26 13:10:40 UTC
Permalink
Hey! You are right Alex! Many thanks for pointing me to head over to the right direction!

It was a clash on rules for some reason. Now, I was also right that these two systems could not be used together because the rules declared in different systems to perform the same action (REJECT) cause the error I was having!

The following rule in default.sieve:

require ["reject"];
# rule: Reject on "x-spam-flag" header
if header :contains "X-Spam-Flag" "YES" {
reject "No spamming allowed here.";
stop;
}

and the following Postfix's regexp header_check rules on the subject field:

/^Subject:.**{5}SPAM*{5}/ REJECT No spammers allowed here.
/^Subject:.*\*\*\*\*\*SPAM\*\*\*\*\*/ REJECT No spammers allowed.
/\s**{5}SPAM*{5}/ REJECT No spamming hullababballos allowed.
/^Subject:(.*)SPAM/ REJECT Spam is not allowed. DISCARD.

were causing the Dovecot Sieve rejection bounce not to go through. The rules blocked the spam all right but rejection was turned into discard for some reason.

Now the question is how do I find out which regular expressions will be in conflict with default.sieve scripting rules? Default.sieve is set to block spam on the X-Spam-Flag header and header_checks is set to block spam on the subject field. I am still clueless why didn't these two "cooperate"? Was it just because they were "told" to perform the same action as per my previous guess? But the target to perform this same action on was different... Any more ideas anyone? Alex? Many thanks in advance for any input!


From: Klaipedaville on Google
Sent: Friday, September 26, 2014 15:00
To: Alex Crow ; dovecot at dovecot.org
Subject: Re: Dovecot Sieve and Postfix header_checks Issue

Thank you for your suggestion, Alex.

However, my header_checks file has just 5 lines of regexp as follows:

/^Subject:.**{5}SPAM*{5}/ REJECT No spammers allowed here.
/^Subject:.*\*\*\*\*\*SPAM\*\*\*\*\*/ REJECT No spammers allowed.
/\s**{5}SPAM*{5}/ REJECT No spamming hullababballos allowed.
/^Subject:(.*)SPAM/ REJECT Spam is not allowed. DISCARD.
/^From:.*\@.*\.tw/ REJECT Sorry, Taiwanese mail is not allowed.

All the regexp are correct. The first and the third lines actually do the same thing and they are there for testing purposes only. Execution stops at the very first rule matched so there is no problem in having any rules repeated. I can't see how a few the most simplest regular expressions can cause the bounce from Sieve not to go through.. Do you see any problems in these lines above that could possibly be the cause of your suggestion?


From: Alex Crow
Sent: Friday, September 26, 2014 14:47
To: dovecot at dovecot.org
Subject: Re: Dovecot Sieve and Postfix header_checks Issue

That would most likely be something in your header_checks that is
causing the bounce from Sieve to be rejected.

There is no reason why you cannot use both.
Post by Klaipedaville on Google
Hello List,
I tried to subscribe but it's taking forever for the confirmation email to arrive so I thought I would ask away by emailing directly. My apologies in advance should this question appear twice.
Dovecot version is 2.1.7. Its dovecot ?n is real short one and follows right after my question.
I have my Postfix 2.9.6 properties set like this: header_checks = /etc/path/to/myfile. Then I have Dovecot Sieve also configured and working fine.
Now the trouble is that these two cannot be combined together can they? Dovecot Sieve and Postfix's header_checks? If I turn off header_checks in Postfix then Dovecot Sieve is working fine. If I vise versa turn off Dovecot Sieve then Postfix's header-checks are also working fine. However, when I keep them both turned on it also works but only partially and in this case Dovecot Sieve never bounces back any rejected massages remotely, the bounce happens only locally. That is I can see it in my logs that it was rejected locally but the message is never sent back to the sender saying something like, "Spam is rejected here". Thus the reject turns into a silent discard without telling the sender anything at all. Any ideas, help, advices how do I fix that? Would be really grateful for any suggestions / assistance at all. Many thanks in advance!
P.S. I would like to use both because Dovecot Sieve is a very powerful and great plugin that uses Spamassassin's added tags and headers based on which I create my rules. Postfix in its turn has a very neat feature of checking the headers via regexp but Postfix does not see / detect Spamassassin's added headers at all because I passed delivery and authentication (which is a way better than Postfix's) to dovecot deliver.
# 2.1.7: /etc/dovecot/dovecot.conf
# OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.6 ext4
auth_mechanisms = plain login cram-md5 scram-sha-1
auth_verbose = yes
hostname = WindTalker
info_log_path = /var/log/dovecot-sieve.log
log_path = /var/log/dovecot-sieve-errors.log
mail_location = maildir:/home/mvail/%d/%n
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave
passdb {
args = /etc/dovecot/passwd
driver = passwd-file
}
plugin {
sieve = ~/.dovecot.sieve
sieve_default = /etc/dovecot/default.sieve
sieve_dir = ~/sieve
}
pop3_uidl_format = %g
postmaster_address = postmaster at example.com
protocols = imap pop3 sieve
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
}
ssl_ca = </etc/ssl/ca.crt
ssl_cert = </etc/ssl//new.crt
ssl_key = </etc/ssl/new.key
userdb {
args = uid=vmail gid=vmail home=/home/vmail/%u
driver = static
}
protocol lda {
mail_debug = yes
mail_plugins = sieve
}
Regards,
Dennis.
--
This message is intended only for the addressee and may contain
confidential information. Unless you are that person, you may not
disclose its contents or use it in any way and are requested to delete
the message along with any attachments and notify us immediately.
"Transact" is operated by Integrated Financial Arrangements plc. 29
Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608
5300. (Registered office: as above; Registered in England and Wales
under number: 3727592). Authorised and regulated by the Financial
Conduct Authority (entered on the Financial Services Register; no. 190856).
Alex Crow
2014-09-26 13:34:54 UTC
Permalink
Post by Klaipedaville on Google
Hey! You are right Alex! Many thanks for pointing me to head over to the right direction!
It was a clash on rules for some reason. Now, I was also right that
these two systems could not be used together because the rules
declared in different systems to perform the same action (REJECT)
cause the error I was having!
require ["reject"];
# rule: Reject on "x-spam-flag" header
if header :contains "X-Spam-Flag" "YES" {
reject "No spamming allowed here.";
stop;
}
/^Subject:.**{5}SPAM*{5}/ REJECT No spammers allowed here.
/^Subject:.*\*\*\*\*\*SPAM\*\*\*\*\*/ REJECT No spammers allowed.
/\s**{5}SPAM*{5}/ REJECT No spamming
hullababballos allowed.
I think it may be this one above. From the postfix manuals:

"By default, matching is case-insensitive, and newlines are not treated
as special characters. The behavior is controlled by flags, which are
toggled by appending one or more of the following characters after the
pattern: *i* (default: on) Toggles the case sensitivity flag. By
default, matching is case insensitive."

And it looks like * needs escaping there too (if you're trying to match
exactly 5 asterisks, you should probably do "\*{5}" not just *{5}.
Post by Klaipedaville on Google
/^Subject:(.*)SPAM/ REJECT Spam is not allowed. DISCARD.
were causing the Dovecot Sieve rejection bounce not to go through. The
rules blocked the spam all right but rejection was turned into discard
for some reason.
Now the question is how do I find out which regular expressions will
be in conflict with default.sieve scripting rules?
That's just about learning Posix Regex syntax.
Post by Klaipedaville on Google
Default.sieve is set to block spam on the X-Spam-Flag header and
header_checks is set to block spam on the subject field. I am still
clueless why didn't these two "cooperate"? Was it just because they
were "told" to perform the same action as per my previous guess?
I'm almost 100% sure that that regex also matched the bounce from your
sieve rules. There is no mysterious interaction between header_checks
and sieve rules, it's just your pattern was too liberal in the former.
Post by Klaipedaville on Google
But the target to perform this same action on was different... Any
more ideas anyone? Alex? Many thanks in advance for any input!
I think if you tune that header_checks file correctly you should have no
more issues.

Thanks

Alex
--
This message is intended only for the addressee and may contain
confidential information. Unless you are that person, you may not
disclose its contents or use it in any way and are requested to delete
the message along with any attachments and notify us immediately.
"Transact" is operated by Integrated Financial Arrangements plc. 29
Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608
5300. (Registered office: as above; Registered in England and Wales
under number: 3727592). Authorised and regulated by the Financial
Conduct Authority (entered on the Financial Services Register; no. 190856).
Klaipedaville on Google
2014-09-26 14:27:25 UTC
Permalink
Post by Klaipedaville on Google
/^Subject:.**{5}SPAM*{5}/ REJECT No spammers allowed here.
/^Subject:.*\*\*\*\*\*SPAM\*\*\*\*\*/ REJECT No spammers allowed.
/\s**{5}SPAM*{5}/ REJECT No spamming
hullababballos allowed.
I think it may be this one above. From the postfix manuals"By default, matching is case-insensitive, and newlines are not treated as special characters. The behavior is controlled by flags, which are toggled by appending one or more of the following characters after the pattern: *i* (default: on) Toggles the case sensitivity flag. By default, matching is case insensitive."
Case insensitive is declared by putting this /i at the end of a rule.
Postfix has nothing to do with regular expressions (regexp) and regexp is not controlled by postfix. There should be a regexp library available on the server where you are using regexp. It?s like PHP, or tml, or js, or css, it cannot be controlled by postfix.
Post by Klaipedaville on Google
And it looks like * needs escaping there too (if you're trying to match exactly 5 asterisks, you should probably do "\*{5}" not just *{5}.
Yes, the escape character in front \*{5} to match 5 asterisks is the correct one. You are right. I am an expert on regexp and this (incorrect one) was there just for testing purposes because there was a problem with the library on the server so I had this bad rule over there to follow up on error in logs. The library has been fixed by now and as I said earlier execution stops on the first rule matched but does not really do any harm if there is a mistake in the rule, in this 'mistake' case the rule is simply skipped.
Post by Klaipedaville on Google
/^Subject:(.*)SPAM/ REJECT Spam is not allowed. DISCARD.
were causing the Dovecot Sieve rejection bounce not to go through. The rules blocked the spam all right but rejection was turned into discard for some reason. Now the question is how do I find out which regular >expressions will be in conflict with default.sieve scripting rules?
That's just about learning Posix Regex syntax.
All the rules are 100% correct as there is a very simple and useful tool in postfix to check if regexp is correct. The tool can be used even by people who don't have a foggiest idea how regexp work. All you have to do is to type on a command line this postmap -q "Subject: *****SPAM***** blablablabla" regexp:/etc/postfix/header_checks or this postmap -q "X-Spam-Flag: YES" regexp:/etc/postfix/header_checks and it will tell you if your rule is correct or not. It is bullet and fool proof system with 100% guarantee. All these rules have been checked like that despite the fact that I know for a fact that they are correct by my own knowledge and experience.
Post by Klaipedaville on Google
I'm almost 100% sure that that regex also matched the bounce from your sieve rules. There is no mysterious interaction between header_checks and sieve rules, it's just your pattern was too liberal in the former.
No, no. The regex could not have matched the bounce from my own rules because it would be silly to send a test message from the same server that would loop back and block myself by my own rules. I sent test messages from another server's accounts. Plus, there is a difference. Header_checks in Postfix use only customized rules that do not involve any Spamassassin's added headers. Now in my case only Dovecot Sieve goes through Spamassassin headers because as mentioned earlier I passed delivery from Postfix to dovecot LDA in my configuration. That's why everything that has Spamassassin's headers and tags added has to be configured via default.sieve scripting and everything else (that do not get Spamassassin's headers added) may use header_checks of Postfix. I have just figured that out by runnning quite a few different and simple tests.
Post by Klaipedaville on Google
I think if you tune that header_checks file correctly you should have no more issues.
The header_check rules are fine tuned to their best.

Anyway, I am thankful for your suggestion as it pointed me to the right direction. Then I picked it up and simply followed onwards by elaborating and building on top which led me to a solved problem Thank you.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: wlEmoticon-openmouthedsmile[1].png
Type: image/png
Size: 1053 bytes
Desc: not available
URL: <Loading Image...>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wlEmoticon-smile[1].png
Type: image/png
Size: 1046 bytes
Desc: not available
URL: <Loading Image...>
Alex Crow
2014-09-26 14:49:25 UTC
Permalink
Post by Klaipedaville on Google
Post by Klaipedaville on Google
/^Subject:.**{5}SPAM*{5}/ REJECT No spammers allowed here.
/^Subject:.*\*\*\*\*\*SPAM\*\*\*\*\*/ REJECT No spammers allowed.
/\s**{5}SPAM*{5}/ REJECT No spamming
hullababballos allowed.
I think it may be this one above. From the postfix manuals"By default, matching is case-insensitive, and newlines are not treated as special characters. The behavior is controlled by flags, which are toggled by appending one or more of the following characters after the pattern: *i* (default: on) Toggles the case sensitivity flag. By default, matching is case insensitive."
Case insensitive is declared by putting this /i at the end of a rule.
Postfix has nothing to do with regular expressions (regexp) and regexp is not controlled by postfix. There should be a regexp library available on the server where you are using regexp. It?s like PHP, or tml, or js, or css, it cannot be controlled by postfix.
So why does it state in man 5 regexp_table that such tables are *case
insensitive* by default and the /i actually toggles that? Are you saying
that man page is wrong? I'd be surprised as I don't think I've yet come
across an occasion where postfix man pages are incorrect!
Post by Klaipedaville on Google
Post by Klaipedaville on Google
And it looks like * needs escaping there too (if you're trying to match exactly 5 asterisks, you should probably do "\*{5}" not just *{5}.
Yes, the escape character in front \*{5} to match 5 asterisks is the correct one. You are right. I am an expert on regexp and this (incorrect one) was there just for testing purposes because there was a problem with the library on the server so I had this bad rule over there to follow up on error in logs. The library has been fixed by now and as I said earlier execution stops on the first rule matched but does not really do any harm if there is a mistake in the rule, in this 'mistake' case the rule is simply skipped.
Post by Klaipedaville on Google
/^Subject:(.*)SPAM/ REJECT Spam is not allowed. DISCARD.
were causing the Dovecot Sieve rejection bounce not to go through. The rules blocked the spam all right but rejection was turned into discard for some reason. Now the question is how do I find out which regular >expressions will be in conflict with default.sieve scripting rules?
That's just about learning Posix Regex syntax.
All the rules are 100% correct as there is a very simple and useful tool in postfix to check if regexp is correct. The tool can be used even by people who don't have a foggiest idea how regexp work. All you have to do is to type on a command line this postmap -q "Subject: *****SPAM***** blablablabla" regexp:/etc/postfix/header_checks or this postmap -q "X-Spam-Flag: YES" regexp:/etc/postfix/header_checks and it will tell you if your rule is correct or not. It is bullet and fool proof system with 100% guarantee. All these rules have been checked like that despite the fact that I know for a fact that they are correct by my own knowledge and experience.
Post by Klaipedaville on Google
I'm almost 100% sure that that regex also matched the bounce from your sieve rules. There is no mysterious interaction between header_checks and sieve rules, it's just your pattern was too liberal in the former.
No, no. The regex could not have matched the bounce from my own rules because it would be silly to send a test message from the same server that would loop back and block myself by my own rules. I sent test messages from another server's accounts. Plus, there is a difference. Header_checks in Postfix use only customized rules that do not involve any Spamassassin's added headers. Now in my case only Dovecot Sieve goes through Spamassassin headers because as mentioned earlier I passed delivery from Postfix to dovecot LDA in my configuration. That's why everything that has Spamassassin's headers and tags added has to be configured via default.sieve scripting and everything else (that do not get Spamassassin's headers added) may use header_checks of Postfix. I have just figured that out by runnning quite a few different and simple tests.
Post by Klaipedaville on Google
I think if you tune that header_checks file correctly you should have no more issues.
The header_check rules are fine tuned to their best.
Anyway, I am thankful for your suggestion as it pointed me to the right direction. Then I picked it up and simply followed onwards by elaborating and building on top which led me to a solved problem Thank you.
So if the regexes were all correct, then:

a) what was your actual problem once you identified it

and

b) for the benefit of the list, how did you actually solve it?

Alex
--
This message is intended only for the addressee and may contain
confidential information. Unless you are that person, you may not
disclose its contents or use it in any way and are requested to delete
the message along with any attachments and notify us immediately.
"Transact" is operated by Integrated Financial Arrangements plc. 29
Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608
5300. (Registered office: as above; Registered in England and Wales
under number: 3727592). Authorised and regulated by the Financial
Conduct Authority (entered on the Financial Services Register; no. 190856).
Klaipedaville on Google
2014-09-26 15:00:34 UTC
Permalink
So why does it state in man 5 regexp_table that such tables are *case insensitive* by default and the /i actually toggles that? Are you saying that man page is wrong? I'd be surprised as I don't think I've yet come
across an occasion where postfix man pages are incorrect!
I am not saying that the Postfix man is incorrect. It is 100% case insensitive and SPAM (upper case) and spam (lower case) would mean absolutely the same. Therefore my SPAM (upper case) regexp was correct.
a) what was your actual problem once you identified it
and
b) for the benefit of the list, how did you actually solve it?
Alex
I explained it all in my previous post.
Alex Crow
2014-09-26 15:14:47 UTC
Permalink
Post by Alex Crow
Post by Alex Crow
So why does it state in man 5 regexp_table that such tables are *case
insensitive* by default and the /i actually toggles that? Are you
saying that man page is wrong? I'd be surprised as I don't think I've
yet come
Post by Alex Crow
across an occasion where postfix man pages are incorrect!
I am not saying that the Postfix man is incorrect. It is 100% case
insensitive and SPAM (upper case) and spam (lower case) would mean
absolutely the same. Therefore my SPAM (upper case) regexp was correct.
Post by Alex Crow
a) what was your actual problem once you identified it
and
b) for the benefit of the list, how did you actually solve it?
Alex
I explained it all in my previous post.
I realise it's probably because of the use of the reject action, which
presumably inserts the text "No spamming allowed here." into the subject
of the bounce.

However what also concerns me is that sending MDN's back to the envelope
sender of SPAM messages is very likely to cause your server to be the
origin of "Joe-Job" spam. The sieve RFCs state that if using reject
(therefore sending MDNs) you should also be able to check for forged
envelope senders and in that case do not send bounces. In real life it's
considered best to just not bother sending anything back at all, and
either discard, or block at the SMTP level using RBLs.

Cheers

Alex
Post by Alex Crow
--
This message has been scanned for viruses and
dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is
believed to be clean.
--
This message is intended only for the addressee and may contain
confidential information. Unless you are that person, you may not
disclose its contents or use it in any way and are requested to delete
the message along with any attachments and notify us immediately.
"Transact" is operated by Integrated Financial Arrangements plc. 29
Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608
5300. (Registered office: as above; Registered in England and Wales
under number: 3727592). Authorised and regulated by the Financial
Conduct Authority (entered on the Financial Services Register; no. 190856).
Klaipedaville on Google
2014-09-26 15:44:50 UTC
Permalink
I realise it's probably because of the use of the reject action, which presumably inserts the text "No spamming allowed here." into the subject of the bounce.
However what also concerns me is that sending MDN's back to the envelope sender of SPAM messages is very likely to cause your server to be the origin of "Joe-Job" spam. The sieve RFCs state that if using reject (therefore >sending MDNs) you should also be able to check for forged envelope senders and in that case do not send bounces. In real life it's considered best to just not bother sending anything back at all, and either discard, or block at >the SMTP level using RBLs.
Cheers
Alex
Whatever's the case the backscatter you're talking about has its own ways and methods to be fought with. There are countries for example Germany where it is prohibited by law to discard any email messages silently. You must reject them so that the senders would be aware what is going on. I was told that by one German admin. I am not sure if this is really true but it has some logic on one hand and it is completely groundless on the other hand because you cannot substitute a live person by a machine. Let's say if the machine (computer / server) confirmed reception of the email it does not necessarily mean that any person on the other end also received and read that email. But this is just demagogy that has very vogue legal / law aspects involved.
Reindl Harald
2014-09-26 15:57:23 UTC
Permalink
Post by Klaipedaville on Google
There are countries for example Germany where it is prohibited by law to discard
any email messages silently. You must reject them so that the senders would be
aware what is going on. I was told that by one German admin. I am not sure if
this is really true but it has some logic on one hand
it is true and besides the german legal letter below you violate a
second law at the same time - that is why you have to run a spamfilter
*before queue* and sa-milter exists - in case you reject a message
the sending server is responsible for a bounce

in case you accept and silently drop it you have a unacceptable
configuration - independent of laws - in case of a important and
time critical mail i need to know it was rejected and so can call
the person by phone or try to remove something which triggered
a false positive
____________________________________________________

there are two important rules for mail:

a) if you accept it you have to deliver it
b) if you can't deliver it you must not accept it

if you follow b) you don't become a backscatter

http://www.postfix.org/MILTER_README.html
____________________________________________________


? 303a StGB - Datenver?nderung (1) Wer rechtswidrig Daten
(? 202a Abs. 2) l?scht, unterdr?ckt, unbrauchbar macht oder
ver?ndert, wird mit Freiheitsstrafe bis zu zwei Jahren oder
mit Geldstrafe bestraft

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140926/1a75655f/attachment.sig>
Klaipedaville on Google
2014-09-26 16:18:25 UTC
Permalink
it is true and besides the german legal letter below you violate a second law at the same time - that is why you have to run a spamfilter *before queue* and sa-milter exists - in case you reject a message
the sending server is responsible for a bounce
in case you accept and silently drop it you have a unacceptable configuration - independent of laws - in case of a important and time critical mail i need to know it was rejected and so can call the person by phone or try to >remove something which triggered a false positive
____________________________________________________
a) if you accept it you have to deliver it
b) if you can't deliver it you must not accept it
if you follow b) you don't become a backscatter
http://www.postfix.org/MILTER_README.html
Yep, that's correct about point b). Plus, I also personally agree that rejecting is better, than having it silently discarded. Good thing is that I do not reside in Germany so I am safe on that

In fact, postfix before-queue content filter has limits in the amount of mail that a site can handle: http://www.postfix.org/SMTPD_PROXY_README.html therefore legal part of this in Germany limits all German people on resources as every single person is forced to run before-queue filters which in my opinion has its own disadvantages on top of everything else and despite the laws as well.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wlEmoticon-smile[1].png
Type: image/png
Size: 1046 bytes
Desc: not available
URL: <Loading Image...>
Reindl Harald
2014-09-26 16:40:36 UTC
Permalink
Post by Klaipedaville on Google
it is true and besides the german legal letter below you violate a second law at the same time - that is why you have to run a spamfilter *before queue* and sa-milter exists - in case you reject a message
the sending server is responsible for a bounce
in case you accept and silently drop it you have a unacceptable configuration - independent of laws - in case of a important and time critical mail i need to know it was rejected and so can call the person by phone or try to >remove something which triggered a false positive
____________________________________________________
a) if you accept it you have to deliver it
b) if you can't deliver it you must not accept it
if you follow b) you don't become a backscatter
http://www.postfix.org/MILTER_README.html
Yep, that's correct about point b). Plus, I also personally agree that rejecting is better, than having it silently discarded. Good thing is that I do not reside in Germany so I am safe on that
In fact, postfix before-queue content filter has limits in the amount of mail that a site can handle: http://www.postfix.org/SMTPD_PROXY_README.html therefore legal part of this in Germany limits all German people on resources as every single person is forced to run before-queue filters which in my opinion has its own disadvantages on top of everything else and despite the laws as well.
that's no problem because with RBL weighting and postscreen you reject 95%
of the crap before it ever touchs smtpd or even the contentfilter

that stats below are about a maillog starting with Sep 18 19:50:39
for some hundrest domains and currently 2000 valid RCPT, if the
contentfilter has to handle most of your incoming flow you made
a mistake by not reject earlier with "cheaper" methods
___________________________________________________

that part never touched the contentfilter

Invalid User: 3074
Disallowed User: 2
Reject Postscreen: 148451
Reject Postfix: 6225
Blacklist: 144880
Pregreet: 5143
Protocol Error: 1607
Helo: 190
Sender Blocked: 200
Sender Invalid: 229
Sender Spoofed: 847
PTR Missing: 1347
PTR Generic: 319
___________________________________________________

Connections: 185848
Delivered: 20293
Invalid User: 3074
Disallowed User: 2
Reject Postscreen: 148451
Reject Postfix: 6225
Reject Temporary: 507
Blacklist: 144880
Pregreet: 5143
Protocol Error: 1607
Spamfilter: 2297
Virus: 115
Helo: 190
Subject: 42
Attachment: 0
Sender Blocked: 200
Sender Invalid: 229
Sender Spoofed: 847
PTR Missing: 1347
PTR Generic: 319

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140926/201bece9/attachment.sig>
Klaipedaville on Google
2014-09-27 10:01:01 UTC
Permalink
that's no problem because with RBL weighting and postscreen you reject 95% of the crap before it ever touchs smtpd or even the contentfilter that stats below are about a maillog starting with Sep 18 19:50:39
for some hundrest domains and currently 2000 valid RCPT, if the contentfilter has to handle most of your incoming flow you made a mistake by not reject earlier with "cheaper" methods
___________________________________________________
that part never touched the contentfilter
Invalid User: 3074
Disallowed User: 2
Reject Postscreen: 148451
Reject Postfix: 6225
Blacklist: 144880
Pregreet: 5143
Protocol Error: 1607
Helo: 190
Sender Blocked: 200
Sender Invalid: 229
Sender Spoofed: 847
PTR Missing: 1347
PTR Generic: 319
___________________________________________________
Connections: 185848
Delivered: 20293
Invalid User: 3074
Disallowed User: 2
Reject Postscreen: 148451
Reject Postfix: 6225
Reject Temporary: 507
Blacklist: 144880
Pregreet: 5143
Protocol Error: 1607
Spamfilter: 2297
Virus: 115
Helo: 190
Subject: 42
Attachment: 0
Sender Blocked: 200
Sender Invalid: 229
Sender Spoofed: 847
PTR Missing: 1347
PTR Generic: 319
That sounds interesting. Since you are German and most probably are an expert on before-queue filtering could you advise if it is actually possible to use both before-queue and after-queue filtering? Since it is the Dovecot list I would be curious to know how to do it Dovecot Sieve way. That is my Postfix passes it over to Dovecot for delivery but all the milters work with Postfix directly and my passing delivery over to Dovecot should theoretically simply disable any before-queue filters. I ran a few tests and it did not work for the reason I have just mentioned, because it is piped to dovecot.
Reindl Harald
2014-09-27 11:16:40 UTC
Permalink
Post by Klaipedaville on Google
that's no problem because with RBL weighting and postscreen you reject 95% of the crap before it ever touchs smtpd or even the contentfilter that stats below are about a maillog starting with Sep 18 19:50:39
for some hundrest domains and currently 2000 valid RCPT, if the contentfilter has to handle most of your incoming flow you made a mistake by not reject earlier with "cheaper" methods
That sounds interesting. Since you are German and most probably
are an expert on before-queue filtering could you advise if it
is actually possible to use both before-queue and after-queue
filtering?
surely but how does that make sense?
Post by Klaipedaville on Google
Since it is the Dovecot list I would be curious to know
how to do it Dovecot Sieve way.
doing *what*?
Post by Klaipedaville on Google
That is my Postfix passes it over to Dovecot for delivery but all the milters
work with Postfix directly and my passing delivery over to Dovecot should
theoretically simply disable any before-queue filters
that's just impossible

you can't control a before-queue filter that way
because, well, he is before-queue and has no idea
what later happens with that message

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140927/6b7b6fc2/attachment.sig>
Klaipedaville on Google
2014-09-27 13:04:09 UTC
Permalink
...could you advise if it is actually possible to use both before-queue and after-queue filtering?
Reindl
surely but how does that make sense?
It makes because it will use two filters, not just one. It will filter before queue first and then anything that may be missed or let through on purpose will be filtered after queue by the second filter.
Since it is the Dovecot list I would be curious to know how to do it Dovecot Sieve way.
doing *what*?
Making two filters work together (dovecot after-queue and postfix before-queue).
That is my Postfix passes it over to Dovecot for delivery but all the milters work with Postfix directly and my passing delivery over to Dovecot should theoretically simply disable any before-queue filters
that's just impossible
It is technically and theoretically impossible as far as I understand but I thought that I also could be wrong at some point and there were ways to arrange it somehow... that?s why I asked away.
you can't control a before-queue filter that way because, well, he is before-queue and has no idea what later happens with that message.
I wouldn?t care what the before-queue would be up to as long as they would "cooperate" together with the after-queue one. That is I would create different rules for "before" and for "after" filters making sure that the rules do not clash, conflict, or contradict with each other in any way. But then again my delivery is done by either Postfix with filters or Dovecot with filters and I am unable to use both. That's actually how my current settings on this server I was setting up work. Postfix uses its regexp header_checks without any filters and it works just fine, then it is passed over to Dovecot for delivery where Spamassassin filters kick in and do their job on top. Thus I have two "spam stoppers" so to speak. The advantage is that I can use both regexp / pcre in Postfix and sieve scripting in Dovecot Sieve.
Reindl Harald
2014-09-27 13:14:24 UTC
Permalink
Post by Klaipedaville on Google
...could you advise if it is actually possible to use both before-queue and after-queue filtering?
Reindl
surely but how does that make sense?
It makes because it will use two filters, not just one.
It will filter before queue first and then anything that may be missed
it don't

having sa-milter, clamav-milter, postscreen, PTR/HELO checks
is all before queue there is not much left....
Post by Klaipedaville on Google
or let through on purpose will be filtered after queue by the second filter
that's not possible except the milter itself
has whitelists which are triggered

you need to realize that if a reject before queue was
triggered there is nothing happing after because there
just exists no queue at all
Post by Klaipedaville on Google
Since it is the Dovecot list I would be curious to know how to do it Dovecot Sieve way.
doing *what*?
Making two filters work together (dovecot after-queue and postfix before-queue).
it's not the job of dovecot at all
Post by Klaipedaville on Google
That is my Postfix passes it over to Dovecot for delivery but all the milters work with Postfix directly and my passing delivery over to Dovecot should theoretically simply disable any before-queue filters
that's just impossible
It is technically and theoretically impossible
technically

you can't even have a DNSWL before a milter

you only can have other thins rejecting on meta-data and stop
the milter overhead because a decision is already made
Post by Klaipedaville on Google
you can't control a before-queue filter that way because, well, he is before-queue and has no idea what later happens with that message.
I wouldn?t care what the before-queue would be up to as long as they
would "cooperate" together with the after-queue
there is no "cooperate" - if the milter rejects it's rejected
if a PTR check rejectes it is rejected
no layer after that will get in touch with the message

that's the idea of having several layers and try to
get the cheaper ones in front - frankly after postscreen
if correctly used the layers after only face 5-10% of all
incoming mail


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140927/fc9c3a82/attachment.sig>
Alex Crow
2014-09-26 16:29:33 UTC
Permalink
Post by Klaipedaville on Google
Whatever's the case the backscatter you're talking about has its own
ways and methods to be fought with. There are countries for example
Germany where it is prohibited by law to discard any email messages
silently. You must reject them so that the senders would be aware what
is going on. I was told that by one German admin. I am not sure if
this is really true but it has some logic on one hand and it is
completely groundless on the other hand because you cannot substitute
a live person by a machine. Let's say if the machine (computer /
server) confirmed reception of the email it does not necessarily mean
that any person on the other end also received and read that email.
But this is just demagogy that has very vogue legal / law aspects
involved.
One *very* convincing argument not to send an *email* response (reject
at SMTP is fine) is that it is very likely indeed you'll end up on an
RBL yourself for doing this. It happened to us when we were still
bouncing (probably about 8-10 years ago). It was the main reason we stopped.

Reindl,
I respecfully disagree with (a) at least for the UK. It may be the case
in Germany but I'll be damned if I'm going to give up on my Mailscanner
- tuned over the years enough that we've never had a legit mail get canned.

Anyway, that's enough for me otherwise this is going to turn into a
flamewar rather than informational.
Reindl Harald
2014-09-26 17:00:09 UTC
Permalink
Post by Alex Crow
Reindl,
I respecfully disagree with (a) at least for the UK. It may be the case in Germany but I'll be damned if I'm going
to give up on my Mailscanner - tuned over the years enough that we've never had a legit mail get canned.
I respecfully disagree - why can't it run before-queue
as spamassassin does with sa-milter and in combination
with a wise postscreen/postfix setup faces only 5%
of all mail sicne the rest got washed away before?
Post by Alex Crow
Anyway, that's enough for me otherwise this is going to turn into a flamewar rather than informational
well, i would not want to be in danger one of my
mails got silently dropped and was a false positive

frankly, i was in that situation because using a
"Barracuda Spamfirewall" and expecting a sane
behavior (reject in case of spam) and it took
years to find out it was not the case and needed
to be replaced because *in fact* it silently
dropped way too much of my own messages as well
as from other users too

you just don't know about false positives because
not every mail is missed by somebody which don't
imply it was junk

frankly while replace that commercial mails i had
mail-discussions where i personally did not miss
a few messages until the other side talked about
a message in the same thread i never saw

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140926/91a780c8/attachment.sig>
Klaipedaville on Google
2014-09-27 09:48:11 UTC
Permalink
One *very* convincing argument not to send an *email* response (reject at SMTP is fine) is that it is very likely indeed you'll end up on an RBL yourself for doing this. It happened to us when we were still bouncing (probably >about 8-10 years ago). It was the main reason we stopped.
Reindl,
I respecfully disagree with (a) at least for the UK. It may be the case in Germany but I'll be damned if I'm going to give up on my Mailscanner - tuned over the years enough that we've never had a legit mail get canned.
Anyway, that's enough for me otherwise this is going to turn into a flamewar rather than informational.
It is really convincing but if you do it right I guess you should be safe .

Well, bouncing it all bluntly like shooting from a machine gun absolutely everywhere at random will certainly put you on RBL. How about checking it first if the sender's email address really exists and only then either bounce back (it the address exists) or do not bounce it back if it doesn't. You were the one to tell me in your previous messages to comply with RFC first that is to check for forged envelope senders first. That?s why I have been bouncing it back (rejecting) for the past 4 years and have never ever hit any RBLs at all.
Reindl Harald
2014-09-26 15:17:51 UTC
Permalink
Post by Klaipedaville on Google
Post by Klaipedaville on Google
/^Subject:.**{5}SPAM*{5}/ REJECT No spammers allowed here.
/^Subject:.*\*\*\*\*\*SPAM\*\*\*\*\*/ REJECT No spammers allowed.
/\s**{5}SPAM*{5}/ REJECT No spamming
hullababballos allowed.
I think it may be this one above. From the postfix manuals"By default, matching is case-insensitive, and
newlines are not treated as special characters. The behavior is controlled by flags, which are toggled by
appending one or more of the following characters after the pattern: *i* (default: on) Toggles the case
sensitivity flag. By default, matching is case insensitive."
Case insensitive is declared by putting this /i at the end of a rule.
Postfix has nothing to do with regular expressions (regexp) and regexp is not controlled by postfix. There should
be a regexp library available on the server where you are using regexp. It?s like PHP, or tml, or js, or css, it
cannot be controlled by postfix
this is just unqualified junk - surely it *has a lot* to do
with it because it can and do add the flag as default

frankly every script can add i after / as default
So why does it state in man 5 regexp_table that such tables are *case insensitive* by default and the /i actually
toggles that? Are you saying that man page is wrong? I'd be surprised as I don't think I've yet come across an
occasion where postfix man pages are incorrect!
they are *not* case-insensitive and it takes 5 seconds to verify that


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140926/3e85529b/attachment.sig>
Noel
2014-09-26 14:51:52 UTC
Permalink
Post by Klaipedaville on Google
Post by Klaipedaville on Google
/^Subject:.**{5}SPAM*{5}/ REJECT No spammers allowed here.
/^Subject:.*\*\*\*\*\*SPAM\*\*\*\*\*/ REJECT No spammers allowed.
/\s**{5}SPAM*{5}/ REJECT No spamming
hullababballos allowed.
I think it may be this one above. From the postfix manuals"By default, matching is case-insensitive, and newlines are not treated as special characters. The behavior is controlled by flags, which are toggled by appending one or more of the following characters after the pattern: *i* (default: on) Toggles the case sensitivity flag. By default, matching is case insensitive."
Case insensitive is declared by putting this /i at the end of a rule.
Postfix has nothing to do with regular expressions (regexp) and regexp is not controlled by postfix.
Not true. Postfix regexp (and pcre) matches are case insensitive by
default, adding the /i flag makes them case sensitive. This should
be quite clear in the postfix docs quoted above. This documented
default behavior may be different from other software you're
familiar with.

You're welcome to discuss postfix header checks further on the
postfix-users list.



-- Noel Jones
Klaipedaville on Google
2014-09-26 15:10:55 UTC
Permalink
Not true. Postfix regexp (and pcre) matches are case insensitive by default, adding the /i flag makes them case sensitive. This should be quite clear in the postfix docs quoted above. This documented
default behavior may be different from other software you're familiar with.
You're welcome to discuss postfix header checks further on the postfix-users list.
-- Noel Jones
I am not here to argue on details. It's just getting side tracked. I meant in general. A couple of 100 tests that I ran trying to solve this issue included both case sensitive and case insensitive and all the other possible and impossible options I could think of. Perhaps I missed that little detail, but postfix-user list members and Postfix creator Mr. Wetsie Venema in person said that the regexp were correct. I have discussed it there before as well.
Joseph Tam
2014-09-26 23:21:32 UTC
Permalink
...
Can't speak about the other issues you are having, but is this regexp
pattern what you want? Unless Postfix PCRE automatically right-anchors
these regexps, aren't you rejecting mail from someone at mail.twinpeaks.org,
or even twitter notifications (from *@bounce.twitter.com).

Joseph Tam <jtam.home at gmail.com>
Klaipedaville on Google
2014-09-27 10:49:09 UTC
Permalink
...
Can't speak about the other issues you are having, but is this regexp pattern what you want? Unless Postfix PCRE automatically right-anchors these regexps, aren't you rejecting mail from someone at mail.twinpeaks.org,
I am not sure I even understand what you were trying to say. It's either because you have no idea how to write any regexp / pcre rules or I did not understand your wording. The rule means:
/ means a regular expression. Any regular expression is enclosed in forward slashes. For example /Josepsh/ is already the simplest rule in regexp and pcre.
^ it is called caret and indicates the beginning of a line / expression
From: is the field I refer to
. means any character
* matches zero or any number of occurrences of the previous character
\ it's called escape that treats any character after it as a regular character (item), not regexp
tw means two letters at the end of an email address for example mail at example.tw
What it basically does it blocks any email address that ends on .tw
Plus, if you refer to the post I wrote it clearly says that running the following on your command line postmap ?q "From: mail at example.tw? regexp:/etc/postfix/header_checks will tell you if the rule is correct or not. This is a bullet / fool proof method. The file called header_checks has to exist of course and it has to contain the rule mentioned above. Plus, I also said in my post that regexp were not the problem in my case at all as it literally takes only 5 seconds and has a couple of dozen various methods to check if the rules are correct or not despite the fact that they come from "stone age" computing like I do myself as well. Honestly speaking it was funny to see how so many people started picking on regexpes in the first place without actually having any clear understanding about how they work.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: wlEmoticon-smile[1].png
Type: image/png
Size: 1046 bytes
Desc: not available
URL: <Loading Image...>
Alex Crow
2014-09-27 12:56:34 UTC
Permalink
Post by Klaipedaville on Google
...
Can't speak about the other issues you are having, but is this regexp pattern what you want? Unless Postfix PCRE automatically right-anchors these regexps, aren't you rejecting mail from someone at mail.twinpeaks.org,
I am not sure I even understand what you were trying to say. It's either because you have no idea how to write any regexp / pcre rules
I don't understand why you seem to think there is some kind of magical
interaction between postfix and dovecot. Once postfix passes to the
Dovecot LDA that's the end of its responsibility. Likewise dovecot has
no inside knowledge of that Postfix does.

Certainly in grep that /would/ match ,eg, foo at bar.twister.com because as
Joseph said the .tw is not anchored to the end of the pattern. Now
whether this applies in headerchecks is not for this list.

if it was

^From:.*\@.*\.tw$

it would not.

And again according to the man page, $ is usable:

"/^(.*)-outgoing@(.*)$/"

You seem to think that you are the worlds greatest regexp expert and to
be frank it comes off as a bit arrogant.

Alex
Klaipedaville on Google
2014-09-27 13:16:22 UTC
Permalink
Post by Alex Crow
if it was
it would not.
$ is optional and it only means the end of expression, the rule works either with or without it in the problem I was trying to solve.
This is again an option ($), not a must, the rule would be valid either way.
Post by Alex Crow
You seem to think that you are the worlds greatest regexp expert and to
be frank it comes off as a bit arrogant.
Alex
I am not the world greatest regexp expert but definitely not the worst one. My rules work the way I want and need them to work. Period. I have neither time no desire to prove that 2+2=4. I will not clutter this list speaking off Dovecot issues any more as I feel sorry for time wasted for list readers
Reindl Harald
2014-09-27 13:28:05 UTC
Permalink
Post by Klaipedaville on Google
Post by Alex Crow
if it was
it would not.
$ is optional and it only means the end of expression, the rule works either with or without it in the problem I was trying to solve.
This is again an option ($), not a must, the rule would be valid either way.
you are a ignorant fool
____________________________________________________________________________

[root at srv-rhsoft:~]$ postmap -q "From: bla at bla.tw" regexp:/home/harry/Desktop/bla.cf
REJECT

[root at srv-rhsoft:~]$ postmap -q "From: bla at bla.twitter" regexp:/home/harry/Desktop/bla.cf
REJECT

[root at srv-rhsoft:~]$ cat /home/harry/Desktop/bla.cf
/^From:.*\@.*\.tw/ REJECT
____________________________________________________________________________

[root at srv-rhsoft:~]$ postmap -q "From: bla at bla.tw" regexp:/home/harry/Desktop/bla.cf
REJECT

[root at srv-rhsoft:~]$ postmap -q "From: bla at bla.twitter" regexp:/home/harry/Desktop/bla.cf

[root at srv-rhsoft:~]$ cat /home/harry/Desktop/bla.cf
/^From:.*\@.*\.tw$/ REJECT
____________________________________________________________________________
Post by Klaipedaville on Google
Post by Alex Crow
You seem to think that you are the worlds greatest regexp expert and to
be frank it comes off as a bit arrogant.
knowledge often comes arrogant to ignorant people
Post by Klaipedaville on Google
I am not the world greatest regexp expert but definitely not the worst one
you are the worst one because you argue instead realize your error
Post by Klaipedaville on Google
My rules work the way I want and need them to work. Period.
no they don't damned
Post by Klaipedaville on Google
I have neither time no desire to prove that 2+2=4
no, but you telling us it's 5
Post by Klaipedaville on Google
I will not clutter this list speaking off Dovecot issues any more as I feel sorry for time wasted for list readers
please do so

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140927/a01b1a0f/attachment-0001.sig>
Alex Crow
2014-09-27 13:34:57 UTC
Permalink
Wasnt that productive?

I'm hoping the hubris will lead to some self-realisation later but I
doubt it.

Also felt like he was testing us, posting regexes for us to look at and
then when we pointed out the errors in them suddenly declaring they were
deliberate errors for testing!

Alex
Post by Reindl Harald
Post by Klaipedaville on Google
Post by Alex Crow
if it was
it would not.
$ is optional and it only means the end of expression, the rule works either with or without it in the problem I was trying to solve.
This is again an option ($), not a must, the rule would be valid either way.
you are a ignorant fool
____________________________________________________________________________
[root at srv-rhsoft:~]$ postmap -q "From: bla at bla.tw" regexp:/home/harry/Desktop/bla.cf
REJECT
[root at srv-rhsoft:~]$ postmap -q "From: bla at bla.twitter" regexp:/home/harry/Desktop/bla.cf
REJECT
[root at srv-rhsoft:~]$ cat /home/harry/Desktop/bla.cf
____________________________________________________________________________
[root at srv-rhsoft:~]$ postmap -q "From: bla at bla.tw" regexp:/home/harry/Desktop/bla.cf
REJECT
[root at srv-rhsoft:~]$ postmap -q "From: bla at bla.twitter" regexp:/home/harry/Desktop/bla.cf
[root at srv-rhsoft:~]$ cat /home/harry/Desktop/bla.cf
____________________________________________________________________________
Post by Klaipedaville on Google
Post by Alex Crow
You seem to think that you are the worlds greatest regexp expert and to
be frank it comes off as a bit arrogant.
knowledge often comes arrogant to ignorant people
Post by Klaipedaville on Google
I am not the world greatest regexp expert but definitely not the worst one
you are the worst one because you argue instead realize your error
Post by Klaipedaville on Google
My rules work the way I want and need them to work. Period.
no they don't damned
Post by Klaipedaville on Google
I have neither time no desire to prove that 2+2=4
no, but you telling us it's 5
Post by Klaipedaville on Google
I will not clutter this list speaking off Dovecot issues any more as I feel sorry for time wasted for list readers
please do so
Reindl Harald
2014-09-27 13:43:47 UTC
Permalink
Post by Alex Crow
Wasnt that productive?
i would call it funny :-)
Post by Alex Crow
I'm hoping the hubris will lead to some self-realisation later but I doubt it
forget it
Post by Alex Crow
Also felt like he was testing us, posting regexes for us to look
at and then when we pointed out the errors in them suddenly
declaring they were deliberate errors for testing!
i doubt

looking at the subject mixing two complete different layers of a
mail system points out clueless from the very beginning and every
time i say the truth which is "do not run a mailserver without
qualification and real understanding" i am the bad ass everywhere

such people are responsible for a lot of spam out there because
filter out customer-ranges with potential zombies is easy, but
mailservers maintained by people lacking the knowledge how to
verify that the configuration does what they think is hard

there are two ways to run a mailserver

* knowing what you are doing
* don't do it
Post by Alex Crow
Post by Reindl Harald
Post by Klaipedaville on Google
Post by Alex Crow
if it was
it would not.
$ is optional and it only means the end of expression, the rule works either with or without it in the problem I
was trying to solve.
This is again an option ($), not a must, the rule would be valid either way.
you are a ignorant fool
____________________________________________________________________________
[root at srv-rhsoft:~]$ postmap -q "From: bla at bla.tw" regexp:/home/harry/Desktop/bla.cf
REJECT
[root at srv-rhsoft:~]$ postmap -q "From: bla at bla.twitter" regexp:/home/harry/Desktop/bla.cf
REJECT
[root at srv-rhsoft:~]$ cat /home/harry/Desktop/bla.cf
____________________________________________________________________________
[root at srv-rhsoft:~]$ postmap -q "From: bla at bla.tw" regexp:/home/harry/Desktop/bla.cf
REJECT
[root at srv-rhsoft:~]$ postmap -q "From: bla at bla.twitter" regexp:/home/harry/Desktop/bla.cf
[root at srv-rhsoft:~]$ cat /home/harry/Desktop/bla.cf
____________________________________________________________________________
Post by Klaipedaville on Google
Post by Alex Crow
You seem to think that you are the worlds greatest regexp expert and to
be frank it comes off as a bit arrogant.
knowledge often comes arrogant to ignorant people
Post by Klaipedaville on Google
I am not the world greatest regexp expert but definitely not the worst one
you are the worst one because you argue instead realize your error
Post by Klaipedaville on Google
My rules work the way I want and need them to work. Period.
no they don't damned
Post by Klaipedaville on Google
I have neither time no desire to prove that 2+2=4
no, but you telling us it's 5
Post by Klaipedaville on Google
I will not clutter this list speaking off Dovecot issues any more as I feel sorry for time wasted for list readers
please do so
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140927/5e21ece5/attachment.sig>
Reindl Harald
2014-09-27 13:18:09 UTC
Permalink
Post by Klaipedaville on Google
...
Can't speak about the other issues you are having, but is this regexp pattern what you want? Unless Postfix PCRE automatically right-anchors these regexps, aren't you rejecting mail from someone at mail.twinpeaks.org,
I am not sure I even understand what you were trying to say.
It's either because you have no idea how to write any regexp
the only one having no clue are you (bseides wrong mailing-list)

/^From:.*\@.*\.tw/ is bullshit and matches twitter.com and so on
/^From:.*\@.*\.tw$/ at least would make sure it *ends with* tw

frankly your rule matchs *anything* having a @ and .tw



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://dovecot.org/pipermail/dovecot/attachments/20140927/5284ee07/attachment.sig>
Loading...