Wednesday, November 10, 2010

bounce processing for yahoo and new ip address

Found a lot of delays from yahoo mail servers, deferred connections and so on. At first I thought it was just due to the new ip address - found that it was listed in Spamhaus, and sent in a request for removal from Spamhaus.
Contacted yahoo using their bulk mail contact form. Their reply said
After looking into the issue, it appears that your emails may have become de-prioritized due to a number of potential issues with your mailings. Please visit the following URL for information on improving list maintenance:
http://help.yahoo.com/l/us/yahoo/mail/postmaster/errors/postmaster-23.html
http://help.yahoo.com/l/us/yahoo/mail/postmaster/basics/basics-55.html
http://help.yahoo.com/l/us/yahoo/mail/postmaster/errors/421-ts03.html
The last point was about too many non-existent email addresses - and found lots of such errors in mail.log

Digging deeper, this is what I replied to yahoo:
Going through your checklist, and checking out our bounce processing, found that the bounces from yahoo were not being processed! The reason being that yahoo mail servers don't send the bounce messages, our mailer daemon has to send the bounce message, and this was not being sent to user noreply - user did not exist! This meant that more than 100 - 200 bad yahoo accounts have been sent mail from our server for the past several months! We're weeding out the bad ids - done some manually, rest will be processed by the bounce processing mechanism of phplist over the next week.
Actually there were around 400 bad mails of yahoo domains and a few from other domains. Unsubscribed the yahoo ones manually by digging through the bounces one by one using the gmail interface. The others will be caught by the bounce processing in due course.

For getting the bounces to the right place, I just had to create a noreply user, and put the following lines in a file /home/noreply/.procmailrc
MAILDIR=/var/spool/mail/noreply #The location of mail directory
:0
! our.bounces.email.id@gmail.com


After I sent the email to yahoo, they replied, saying in part,
Yahoo! Mail accepts a maximum of 20 messages per SMTP connection. We encourage you to cap the number of messages you send to Yahoo! Mail to fall within this per-connection limit.

My reply to that was:
We're running postfix > 2.3 which does not have a direct way to limit number of mails sent per connection.
http://www.postfix.org/CONNECTION_CACHE_README.html#limitations

Hopefully our server will reconnect OK with yahoo servers on dropping connections - our logs don't show any problems with that. We're still in the process of weeding out bad addresses. Once we finish that, hopefully, we won't irritate the yahoo MX servers as much!


In addition, I have now put
smtp_defer_if_no_mx_address_found = yes
in the /etc/postfix/main.cf - Default was to bounce mail if MX record not found, i.e. if DNS fails. Now changed to wait and retry for upto 5 days or whatever message age time is set.

No comments:

Post a Comment