Backport of 778ce245dd466bce1b19f89e52cf9ed8f1b46513 from master
Should be unneeded with Python 2.7 and up. Added some unicode_literals along the way.