Backport of 0ed7d155635da9f79d4dd67e4889087d3673c6da from master
Should be unneeded with Python 2.7 and up. Added some unicode_literals along the way.