You appear to have done everything right, uninstalled Windows Live Essentials, set Windows defaults, I'm not at all sure what to suggest next.
How about browser settings? Which browser are you using?
If IE, then Tools > Internet Options > Programs tab and make sure defaults are set under Internet Programs. In the new window you will see not only 'Set your default programs' but also 'Set program access and computer defaults'.
If some other browser search through their Options. For example Firefox Options has an Applications tab.
I'm a bit surprised that Windows Messenger pops up as the default. That is not an email program.