Hotmail email deliverability problems and how to fix it!

For our latest project – I’m in TV, we used VPSLand to host the website. We were using VPSLand for the first time, and after a few initial hiccups, we thought we had everything in place. Or did we?

About a week from launching the first private beta version of the website, we realised that our emails to hotmail accounts were vanishing into thin air! This was extremely weird – because hotmail would accept the delivery of our emails and then just eat them up.

Emails would not get delivered to the members – nor would we get a bounce email back.
Hotmail would just quietly eat up all the emails that we threw at it – and not even give us any indication. No wonder as none of the users who signed up with a hotmail account were able to verify their email addresses.

We were extremely concerned with this issue and were at our wits’ end as to what was going wrong. This had not happened before. We had to finally, temporarily suspended hotmail signups for a week until we figured out what is wrong …

We spent an entire week on researching this issue. We tried permutations and combinations of email headers, what encoding to use, sending purely text and purely html emails, validating or SPF records and revalidating them again. Trying various permutations of our SPF records — but nothing worked. We were back to square one.

Then finally, frustruated – I decide to write an email to the hotmail support staff and see what happens.
I was hardly expecting a reply back – when they actually did. I was pleasantly surprised.
A few more email exchanges later – they told us that they were blocking our IP for no apparent reason at all! Hotmail does that.
Any new IP that tries to send it email – it blocks it assuming its spam.

Finally, after letting them know of our intentions, they agreed to open up a small window to allow our emails through. They would then monitor these emails for reports of spam – and if they found nothing suspicious, they would open up access permanently.

Phew! It finally worked!
After a painful week of researching, all it took was to ask them to do it manually — add us to a temporary white list.

However, for those of you who face the same problems, here is what you need to do first before hotmail will give you that small window of opportunity. Also, the application matters.
We were actually not spamming our users. All emails sent were either verification emails, password reminders or friend notifications.

So, your mileage might vary.

Make sure your SPF records are in place and are accurate.
If you don’t know what SPF records are, check out the following links to get you started:

1. Information about SPF and why it is important:
   http://en.wikipedia.org/wiki/Sender_Policy_Framework
2. SPF Record generator tool by Microsoft
   http://www.microsoft.com/mscorp/safety/content/technologies/senderid/wizard/
3. Validate your SPF records:
   http://www.kitterman.com/spf/validate.html
4. Sender ID validator from ReturnPath
   http://senderid.returnpath.net/how.php

Make sure your server is actually sending mail without problems to other providers.
In our tests, GMail was a peach – and accepted everything we threw at it. Mostly in the Inbox and sometimes in the SPAM. But it did accept it and never bounced any of our emails.
Yahoo too accepted all our mail – but put it in the Spam folder. The easy way to get out of this mess is to get as many people as you can to mark your email as NOT SPAM and also add your sender email ID to their contacts list.

A cool mail relay testing tool that we used was:
http://www.abuse.net/relay.html

Make sure you are following the Microsoft Postmaster Best Practices.
These can be found here:
http://postmaster.live.com/Guidelines.aspx

Finally, if you are doing more or less everything mentioned above, and your email still refuses to go – write to the postmaster.
https://support.msn.com/eform.aspx?productKey=edfsmsbl&ct=eformts&scrx=1

This is a long form but needs to be filled.
Once you have done this, a hotmail support person will get back to you generally within 24 hours and help you from then on.

In our case, we had to make some corrections to our SPF records according to their policies and when they were satisfied, they let us through.

I hope this helps and provides some insight for all those stuck with a similar problem.
Please do note however, that this is not always guaranteed to work. It worked for us and I am posting this from my experience.

Can’t have your cake and eat it too!

If you have gone through any of these project bidding and freelancing sites like RentACoder, Guru, etc., you will definitely come across hundreds of projects in which the clients have the following requirement:

The application must be of very good quality and very cheap also.

This is generally followed by a line which states that if “you deliver timely, good quality software which is also cheap, we will give you more projects in the future”.

I thought to myself – cheap and good quality… How does this possibly work?
(In my opinion, quality never comes cheap!)

There can possibly be two reasons for people posting this:

1. People posting on such sites have extremely low expectations of the end product.
   If their definition of good quality is something which plain works, then I guess this point holds true. There are lots of companies out there doing really shoddy work – believe me – I have seen (20+ developers) software sweatshops which produce such low quality applications that some kids learning a new technology, hashing out their first application over a weekend would do a far better job.
2. People posting have truly no clue …
   In this case, the people really want good quality software – but they have no clue on how much it actually costs to develop it. These people would be the most disappointed of the lot because more often than not, some desperate company or freelancers choose to do the project in the measly amount they’re paying and what they get in the end is really shoddy software (which eventually drives them to the category 1 people above).

Writing excellent software is a decently intensive process with thousands of things going into it.
At a birds-eye level, you need to do your research well, work up a good architecture, code well and finally test it thoroughly.

Moral of the story is that you cannot have your cake and eat it too!
If you want to get something done cheaply, you either have to cut down on the features (a highly recommended option) or settle for something which you need to cross your fingers hoping that it does not crash and burn.