[Dovecot] Thunderbird slow in talking with dovecot IMAP AND to sendmail

[Linda Walsh]

I'm trying to find out what's causing this slowdown -- it's INTOLERABLE....

over 1 minute and less than 1% done. (400MB file)...

After trying 3 times, I gave up and logged in using X to the server and
ran Tbird from there....

Mail sent out in < 1 minute, though the copy to dovecot took about 50%
longer.

So...

I looked at the network trace.

and everyfrackin' body was using 4K packet sizes (at the application
level!, the window size on TCP was over 64K...but no one was using
it)....especially galling with my network's MTU at 9K, BTW, because
small packets are really bad on a 1Gb network.

sendmail -- 4K,
dovecot /ssl, 4K...

wazzup .. is t-bird forcing this, or is there some ssl requirement?

but it can't just be ssl -- as it's talking to sendmail on port 25
unencrypted (it's a local
net anyway)...

I could see the entire binary going out in text form...

1 line at a time...a "C" line in sendmail, with lens of 4096...is that
some max?

I don't see it in the sendmail.cf files...wanted to see if anyone knew
of dovecot restrictions that
might limit packets to 4k, before I lamblasted the thunderbird people
for another act of mindless
stupidity (the first being when they decided to cache all your IMAP
store on every local client
in the client's ROAMING profile...*brilliant*!!!...

sides, if I wanted it on local I would have set 'store on local', but
in TB3, they know better
and change that for me...

Something about them being too stupid to use indexing and searching on
an imap server? Maybe I
just imagined hearing that...

[Charles Marcus]

Post by Linda Walsh
and everyfrackin' body was using 4K packet sizes (at the application
level!, the window size on TCP was over 64K...but no one was using
it)....especially galling with my network's MTU at 9K, BTW, because
small packets are really bad on a 1Gb network.
sendmail -- 4K,
dovecot /ssl, 4K...
wazzup .. is t-bird forcing this,

If I'm not mistaken, yes, this is (or could be) a TBird problem... I
can't find the bug report where this was discussed, but I distinctly
remember one of the devs commenting on this 4k packet size issue.
Apparently it was an intentional change, but he couldn't figure out why.

Fyi, it was discussed in one of the IMAP performance bugs...

[Linda Walsh]

Post by Charles Marcus

Post by Linda Walsh
and everyfrackin' body was using 4K packet sizes (at the application
level!, the window size on TCP was over 64K...but no one was using
it)....especially galling with my network's MTU at 9K, BTW, because
small packets are really bad on a 1Gb network.
sendmail -- 4K,
dovecot /ssl, 4K...
wazzup .. is t-bird forcing this,

If I'm not mistaken, yes, this is (or could be) a TBird problem... I
can't find the bug report where this was discussed, but I distinctly
remember one of the devs commenting on this 4k packet size issue.
Apparently it was an intentional change, but he couldn't figure out why.
Fyi, it was discussed in one of the IMAP performance bugs...

---
Thanks for the lead...will check it out.

The problem with the Tbird (and FF) is that design for home users with
dialup connections, so if you have a home network and run IMAP @home,
all their tuning goes out the window -- and they don't make it configurable.

I had to go to a 9K packet size on 1Gb ethernet to get close to full
bandwitch usage (and then it is a large
effort with a windows client)...and that's down at layer 2? FF IMAP is
at layer 5? ... the latency is insane at that point.

Alot of companies aren't real bright when it comes to storing files
locally -- instead of 'local' they almost
always use the 'roaming' profile...Cuprits: TB at 4G, Adobe at 2.5G, XBMC
~1-2G. Adobe's great -- most of
that 2.5G are the product helpfiles which you don't get when you install
-- they are d/led later and thus stored in your roaming profile. Each
user gets their own copy of the help material...

Of course good thing they got rid of customer input for product design
and got rid of 'usability studies'...
those things always caused problems. Like MS removing the start bar in
Win8 cause users don't want it?
Huh? or Cocacola switching to 'newCoke, then having to revert due to
outcry...because Coke drinkers
didn't want another pepsi knockoff.

Baka!

[Linda Walsh]

Post by Charles Marcus

Post by Linda Walsh
and everyfrackin' body was using 4K packet sizes (at the application
level!, the window size on TCP was over 64K...but no one was using
it)....especially galling with my network's MTU at 9K, BTW, because
small packets are really bad on a 1Gb network.
sendmail -- 4K,
dovecot /ssl, 4K...
wazzup .. is t-bird forcing this,

If I'm not mistaken, yes, this is (or could be) a TBird problem... I
can't find the bug report where this was discussed, but I distinctly
remember one of the devs commenting on this 4k packet size issue.
Apparently it was an intentional change, but he couldn't figure out why.
Fyi, it was discussed in one of the IMAP performance bugs...

---
Thanks for the lead...will check it out.

The problem with the Tbird (and FF) is that design for home users with
dialup connections, so if you have a home network and run IMAP @home,
all their tuning goes out the window -- and they don't make it configurable.

I had to go to a 9K packet size on 1Gb ethernet to get close to full
bandwitch usage (and then it is a large
effort with a windows client)...and that's down at layer 2? FF IMAP is
at layer 5? ... the latency is insane at that point.

Alot of companies aren't real bright when it comes to storing files
locally -- instead of 'local' they almost
always use the 'roaming' profile...Cuprits: TB at 4G, Adobe at 2.5G, XBMC
~1-2G. Adobe's great -- most of
that 2.5G are the product helpfiles which you don't get when you install
-- they are d/led later and thus stored in your roaming profile. Each
user gets their own copy of the help material...

Of course good thing they got rid of customer input for product design
and got rid of 'usability studies'...
those things always caused problems. Like MS removing the start bar in
Win8 cause users don't want it?
Huh? or Cocacola switching to 'newCoke, then having to revert due to
outcry...because Coke drinkers
didn't want another pepsi knockoff.

Baka!

[Linda Walsh]

Post by Charles Marcus

Post by Linda Walsh
and everyfrackin' body was using 4K packet sizes (at the application
level!, the window size on TCP was over 64K...but no one was using
it)....especially galling with my network's MTU at 9K, BTW, because
small packets are really bad on a 1Gb network.
sendmail -- 4K,
dovecot /ssl, 4K...
wazzup .. is t-bird forcing this,

If I'm not mistaken, yes, this is (or could be) a TBird problem... I
can't find the bug report where this was discussed, but I distinctly
remember one of the devs commenting on this 4k packet size issue.
Apparently it was an intentional change, but he couldn't figure out why.
Fyi, it was discussed in one of the IMAP performance bugs...

---
Thanks for the lead...will check it out.

The problem with the Tbird (and FF) is that design for home users with
dialup connections, so if you have a home network and run IMAP @home,
all their tuning goes out the window -- and they don't make it configurable.

I had to go to a 9K packet size on 1Gb ethernet to get close to full
bandwitch usage (and then it is a large
effort with a windows client)...and that's down at layer 2? FF IMAP is
at layer 5? ... the latency is insane at that point.

Alot of companies aren't real bright when it comes to storing files
locally -- instead of 'local' they almost
always use the 'roaming' profile...Cuprits: TB at 4G, Adobe at 2.5G, XBMC
~1-2G. Adobe's great -- most of
that 2.5G are the product helpfiles which you don't get when you install
-- they are d/led later and thus stored in your roaming profile. Each
user gets their own copy of the help material...

Of course good thing they got rid of customer input for product design
and got rid of 'usability studies'...
those things always caused problems. Like MS removing the start bar in
Win8 cause users don't want it?
Huh? or Cocacola switching to 'newCoke, then having to revert due to
outcry...because Coke drinkers
didn't want another pepsi knockoff.

Baka!

[Linda Walsh]

I'm trying to find out what's causing this slowdown -- it's INTOLERABLE....

over 1 minute and less than 1% done. (400MB file)...

After trying 3 times, I gave up and logged in using X to the server and
ran Tbird from there....

Mail sent out in < 1 minute, though the copy to dovecot took about 50%
longer.

So...

I looked at the network trace.

and everyfrackin' body was using 4K packet sizes (at the application
level!, the window size on TCP was over 64K...but no one was using
it)....especially galling with my network's MTU at 9K, BTW, because
small packets are really bad on a 1Gb network.

sendmail -- 4K,
dovecot /ssl, 4K...

wazzup .. is t-bird forcing this, or is there some ssl requirement?

but it can't just be ssl -- as it's talking to sendmail on port 25
unencrypted (it's a local
net anyway)...

I could see the entire binary going out in text form...

1 line at a time...a "C" line in sendmail, with lens of 4096...is that
some max?

I don't see it in the sendmail.cf files...wanted to see if anyone knew
of dovecot restrictions that
might limit packets to 4k, before I lamblasted the thunderbird people
for another act of mindless
stupidity (the first being when they decided to cache all your IMAP
store on every local client
in the client's ROAMING profile...*brilliant*!!!...

sides, if I wanted it on local I would have set 'store on local', but
in TB3, they know better
and change that for me...

Something about them being too stupid to use indexing and searching on
an imap server? Maybe I
just imagined hearing that...

[Charles Marcus]

Post by Linda Walsh
and everyfrackin' body was using 4K packet sizes (at the application
level!, the window size on TCP was over 64K...but no one was using
it)....especially galling with my network's MTU at 9K, BTW, because
small packets are really bad on a 1Gb network.
sendmail -- 4K,
dovecot /ssl, 4K...
wazzup .. is t-bird forcing this,

If I'm not mistaken, yes, this is (or could be) a TBird problem... I
can't find the bug report where this was discussed, but I distinctly
remember one of the devs commenting on this 4k packet size issue.
Apparently it was an intentional change, but he couldn't figure out why.

Fyi, it was discussed in one of the IMAP performance bugs...

[Linda Walsh]

I'm trying to find out what's causing this slowdown -- it's INTOLERABLE....

over 1 minute and less than 1% done. (400MB file)...

After trying 3 times, I gave up and logged in using X to the server and
ran Tbird from there....

Mail sent out in < 1 minute, though the copy to dovecot took about 50%
longer.

So...

I looked at the network trace.

and everyfrackin' body was using 4K packet sizes (at the application
level!, the window size on TCP was over 64K...but no one was using
it)....especially galling with my network's MTU at 9K, BTW, because
small packets are really bad on a 1Gb network.

sendmail -- 4K,
dovecot /ssl, 4K...

wazzup .. is t-bird forcing this, or is there some ssl requirement?

but it can't just be ssl -- as it's talking to sendmail on port 25
unencrypted (it's a local
net anyway)...

I could see the entire binary going out in text form...

1 line at a time...a "C" line in sendmail, with lens of 4096...is that
some max?

I don't see it in the sendmail.cf files...wanted to see if anyone knew
of dovecot restrictions that
might limit packets to 4k, before I lamblasted the thunderbird people
for another act of mindless
stupidity (the first being when they decided to cache all your IMAP
store on every local client
in the client's ROAMING profile...*brilliant*!!!...

sides, if I wanted it on local I would have set 'store on local', but
in TB3, they know better
and change that for me...

Something about them being too stupid to use indexing and searching on
an imap server? Maybe I
just imagined hearing that...

[Charles Marcus]

Post by Linda Walsh
and everyfrackin' body was using 4K packet sizes (at the application
level!, the window size on TCP was over 64K...but no one was using
it)....especially galling with my network's MTU at 9K, BTW, because
small packets are really bad on a 1Gb network.
sendmail -- 4K,
dovecot /ssl, 4K...
wazzup .. is t-bird forcing this,

If I'm not mistaken, yes, this is (or could be) a TBird problem... I
can't find the bug report where this was discussed, but I distinctly
remember one of the devs commenting on this 4k packet size issue.
Apparently it was an intentional change, but he couldn't figure out why.

Fyi, it was discussed in one of the IMAP performance bugs...

[Ed W]

Post by Linda Walsh
I'm trying to find out what's causing this slowdown -- it's
INTOLERABLE....
over 1 minute and less than 1% done. (400MB file)...
After trying 3 times, I gave up and logged in using X to the server
and ran Tbird from there....
Mail sent out in < 1 minute, though the copy to dovecot took about 50%
longer.
So...
I looked at the network trace.
and everyfrackin' body was using 4K packet sizes (at the application
level!, the window size on TCP was over 64K...but no one was using
it)....especially galling with my network's MTU at 9K, BTW, because
small packets are really bad on a 1Gb network.

Although larger packets might be helpful, I don't see that you shouldn't
be getting much faster speed without it? Even the 64K window, whilst it
looks too small, might be ok if your ping times are very low?

Something else is limiting your performance I think?

Ed W

[Ed W]

Post by Linda Walsh
I'm trying to find out what's causing this slowdown -- it's
INTOLERABLE....
over 1 minute and less than 1% done. (400MB file)...
After trying 3 times, I gave up and logged in using X to the server
and ran Tbird from there....
Mail sent out in < 1 minute, though the copy to dovecot took about 50%
longer.
So...
I looked at the network trace.
and everyfrackin' body was using 4K packet sizes (at the application
level!, the window size on TCP was over 64K...but no one was using
it)....especially galling with my network's MTU at 9K, BTW, because
small packets are really bad on a 1Gb network.

Although larger packets might be helpful, I don't see that you shouldn't
be getting much faster speed without it? Even the 64K window, whilst it
looks too small, might be ok if your ping times are very low?

Something else is limiting your performance I think?

Ed W

[Ed W]

Post by Linda Walsh
I'm trying to find out what's causing this slowdown -- it's
INTOLERABLE....
over 1 minute and less than 1% done. (400MB file)...
After trying 3 times, I gave up and logged in using X to the server
and ran Tbird from there....
Mail sent out in < 1 minute, though the copy to dovecot took about 50%
longer.
So...
I looked at the network trace.
and everyfrackin' body was using 4K packet sizes (at the application
level!, the window size on TCP was over 64K...but no one was using
it)....especially galling with my network's MTU at 9K, BTW, because
small packets are really bad on a 1Gb network.

Although larger packets might be helpful, I don't see that you shouldn't
be getting much faster speed without it? Even the 64K window, whilst it
looks too small, might be ok if your ping times are very low?

Something else is limiting your performance I think?

Ed W