Unicode bloopers! HTML should not be visible on a web page!
By Mike Banks Valentine

Have you noticed how many web sites are showing unicode in the visible page text in place of punctuation marks? I’ve seen it on major newspaper sites and even on monstrous portals like Yahoo. I don’t know whether this comes from ignorance of the site maintainer, bad content management software, cutting and pasting directly from word processing software, or all of the above – but it can be incredibly annoying.

Reading becomes nearly impossible when you find the unicode HTML string for apostrophe ‘ or quote ” marks inserted within, or bracketing a word. If you haven't seen it here’s an “example” for everyone's mutual annoyance!

Some sites are using unicode characters for periods and question marks. While it is possible to use unicode to represent all of the alphabet, numbers, punctuation and most symbols – it just isn’t necessary to do so. Beyond that, those who are still using older browser versions will see that code on the page, instead of the characters intended to display. I’ve posted a complete chart of unicode characters and the HTML equivalent chart below.

If you’d like to know more about Unicode visit the authority site

I would encourage those who maintain their web site or those of clients to reference the proper use of those characters in your code on your web pages. If your web page software is displaying this code visibly, your software is outdated or incompatible with current standards.

Sometimes software is the enemy when updating pages. If you cut and paste text from a Microsoft Word document into a non Microsoft web page editor, it will sometimes use proprietary code from that word processing software attempting to represent it as text or simply paste unreadable gibberish into the document. It’s better to work in the Notepad plain text editor, rather than Word, if you will be pasting that text into a web page in any software other than another Microsoft product.

Microsoft has built all of it’s software to recognize the proprietary code from each of it’s other products. Outlook Express, to Internet Explorer, to Word, to Excel in each of the others within the family of Microsoft products. But as soon as you attempt to cut and paste text from a Microsoft product to a web page or any non Microsoft software, you will suddenly get strange characters appearing in the text of pages that don’t recognize the Microsoft proprietary code.

It’s not just a Microsoft problem, it also occurs when cutting and pasting from other proprietary word processing software product into web pages or email products NOT displayed in the same code base.

Unicode can be useful in web pages when the @ symbol displayed unicode equivalent of @ is posted to a web page to stop the spambot harvesting software from gathering useful email addresses from web pages. This trick is not always effective but does stop some harvesting software from recognizing those unicode symbols as valid email addresses.

Further discussion on using unicode to stop spam harvesting software is here.

I spend quite a bit of time in plain text editors like NotePad writing documents that will be transferred to a web page. The transfer of formatting simply won’t work from Word documents to a web page. Some word processing software offers the command “Save As HTML” but not all work flawlessly. Some products don’t present these web pages properly in all browsers.

Even worse is the amount of code clutter created when a Word document is saved as a web page using the “Save As HTML” command in that software (Word). I won’t burden you with showing that clutter here. Dreamweaver web design software actually has a command labeled “Clean up Word HTML” to rid the code of extraneous and unnecessary clutter created by Word during that “Save as HTML” command.

We’ve made available a very simple Text to HTML Converter for Web Publishing that allows simple conversions of plain text documents to HTML without adding unicode characters to visible text.
This great tool automates these normally tedious tasks for web publishing:

Adds mailto: to email addresses
Inserts paragraph tags
Allows ordered lists or bullets
Adds clickable hyperlinks to URL’s
Sets headings to bold text
Allows links to open in new window
All of this without inserting unicode characters into that text as some content management software does.

Web content creators need to know – and actually write HTML sometimes. I hope the Unicode chart below will help toward that end.

Just be sure that you are inserting that code into the HTML and NOT into the visible page text by becoming familiar with your software.