Comments for Why do I get odd characters instead of quotes in my documents?

Comment from connie on 2012-01-28

2012-01-28T21:36:03Z

@Scott
Does a new article end up with the characters, or just the restored database? Since you can find the characters in the database, you could do a search and replace for them all. But if new documents are having the same error, you are going to have to go deeper.

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Scott on 2012-01-28

2012-01-28T20:51:55Z

Thanks Leo for this explanation. Now (I think) I understand what's going on. My client has a website that is Joomla (a Content Management System, or CMS) based, which stores page content in an SQL database (in this case, MySQL). They switched over to a new hosting provider back in November 2011, and their content (about 1400 articles, newsletters, ezines, etc) was displaying just fine - until last weekend. Someone inadvertantly deleted the domain and the files, directories, etc. and they had to be restored. After they were restored, and Joomla re-installed, suddenly these strange characters started popping up in almost every single article - yup, you got it, the â€™ in place of every single apostrophe. So I'm trying to figure out how to help them resolve this issue, without going in to over 1400 documents and editing/modifying them all manually. I first checked Joomla, and it's got its encoding character set as UTF-8. Then I checked MySQL, thinking it must be set for some different character encoding set, but no, it's actually UTF-8 also! But when I look into the MySQL database and run search queries against the entire database for the ' â€™ ' designation, sure enough, it finds that exact string, right throughout the database. Hundreds and hundreds of places.

So, I'm trying to figure out how the data got there like that in the first place, and it must be that, PRIOR to the "restore", it was being stored and displayed, encoded and decoded, as ISO-8859-1 because it was being displayed properly. So now what can I do to fix the problem?!? Anyone?!? HELP?!?

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Jaxsun on 2011-06-29

2011-06-29T13:49:08Z

Leo's info is correct but I finally found the solution. Open WORD, go to TOOLS then AutoCorrectOptions. Select the AutoFormat Tab and de-select "Plain Text Wordmail documents" under Always Autoformat. This stops reformatting of a plain text email message!!

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Jonathan on 2011-06-19

2011-06-19T16:51:12Z

Thanks for your answer, Leo. That was very informative.

And for those who complained, the question did not ask how to fix the problem, it asked why the characters were appearing.

If you are seeing the â€™ characters, chances are the fault is with the source, not your browser. The source was probably written using something like Word that uses "smart quotes" and when converted to HTML those smart quotes were simply converted into what the coding conversion indicated should be used. In the case of a smart apostrophe, it is converted to the â€™ characters.

When creating HTML code, be sure to use a text-based editor or convert it into plain text before saving it as HTML.

As far as what shows up on someone else's web page, if they have a comment page or a means to contact them, bring it to their attention in a courteous and humble manner, and perhaps they will fix the problem.

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Mike on 2011-04-26

2011-04-26T19:13:03Z

First, thank you, Leo. I'd read this article before which provided a basic understanding. Now I need to address it, so the extended explanation gives me a direction.

Second, Dick, your automated response reminds ME of another automated response: "Give a man a fish, and you feed him for a day. Teach a man to fish, and you feed him for a lifetime."

Lazy people want a singular, simplistic answer to their singular local problem. Since it's never singular, and it's never simplistic, and the audience SURE isn't local, Leo's site is a learning experience that helps deal with future problems encountered that are related to an earlier one.

Since my issue can't be resolved with one simple answer, I've learned how to apply the general principle to dealing with it. And if I still can't, at least now I'm armed with more information to search for an answer that does apply to my particular situation.

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Ragini on 2011-02-22

2011-02-22T13:36:03Z

Hi,

I am facing problem with apostrophe symbol that is copied from MS -WORD. Once its been stored in the database apostrophe symbol is stored as "?" mark.Please help in selecting the proper encoding type that supports MS-WORD apostrophe symbol.Thanks Ragini

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Zamboni Tony on 2011-01-25

2011-01-25T22:49:20Z

What I don't get is why a program isn't smart enough to see â€™ and say to itself "Gee - THAT isn't very common, is it? NO! What else could it be? AH - it's a '. Or a smart version of that. WHATEVER - I *do* know that â€™ isn't what the person typing me wanted to have down, so I'll just do a quick conversion."

Now - HOW HARD WOULD THAT BE?

â€™â€™â€™â€™â€™â€™â€™â€™â€™â€™â€™

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Anna Toschlog on 2010-10-04

2010-10-05T00:46:02Z

I read all of your reply, but I still do not understand how to resolve this problem. I am now having this problem when I forward an email that did not have this problem.
Thank you kindly for any more help you could offer.

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Ganymede on 2010-09-28

2010-09-28T23:08:54Z

This reminds me of the old saw, "When you ask him what time it is, he tells you how to build a watch."

All we need is to know which settings to change.

This detailed information is only good for annoying people at cocktail parties and showing off for family at Thanksgiving.

KISS.

It's not simple. Sorry my explanation disappoints. There is no magic setting. If you take the time to understand the explanation you'll see why.

29-Sep-2010

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Dick Victor on 2010-07-08

2010-07-09T05:15:50Z

Very helpful article which enabled me to understand and resolve the problem for my wife who is still using Eudora 7.

For those still using "old" Eudora, there's a nice free plugin that will convert received UTF-8 emails to ISO-8859-1 which is what Eudora understands. You can even put a button for the plugin in the button bar so it's easy to use when you hit a message with odd characters. It's at:

http://www.windharp.de/software/utf8iso.htm

Dick

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from George Jensen on 2009-09-30

2009-10-01T00:51:47Z

When I received an email (in Eudora) with those odd characters in it, I copied it into MS word - the correct characters appeared. Then I copied it back into Eudora and the weird characters were gone!

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from James on 2009-09-16

2009-09-16T16:11:30Z

It's worse than that.

'Unicode' is an encoding of characters as integers, which it calls code points.

UTF-8 is a method of encoding each code point as a variable number of (8-bit) bytes, at least one and possibly as many as three.

UCS-2 is actually a subset of Unicode, which encodes the first 64,000 code points (there are more!) as two bytes.

Where UTF-8 needs only one byte, UCS-2 wastes one byte; however, the fixed character width of UCS-2 is easier to process for such apps as Word.

ISO-8859-1 is a one-byte encoding that happens to be identical to UTF-8, for the first 128 characters only. If an interpreter thinks it's looking at 8859-1, it goes wrong when it sees a byte with the top bit set (i.e. a character beyond 127). And vice versa of course.

If you want more, there's an extensive article on Unicode at Wikipedia. It may not help much. I have the problem with Thunderbird, which allows me to specify the character encoding, though it will Auto-Detect. I suspect the problem lies at the sender, for example, by pasting USC-2 text into a UTF-8 message.

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Robin Clay on 2009-09-16

2009-09-16T11:25:34Z

Not helped, of course, by SOME people not knowing when (and when not) to use an apostrophe !

"It's" - as shown a cuppla times on this page - Acksherly is the abbreviation for "it is". The possesive "its" has no apostrophe.

It's also my Achilles heel. Fortunately I have about three people that pounce every time I get it wrong.

16-Sep-2009

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Michael Smith on 2009-09-15

2009-09-16T04:36:00Z

I think we've all been seeing these more lately, and your explanation was about as clear as it could be, I guess. It was also the first I've ever seen where anybody even attempted to detail it out - and now I see why. Good job!

One thing, though - you didn't tell us what you thought would be the best "fix". What should savvy PC folks be working towards?

Good question, and I'm not sure there is a simple fix. Ideally, I suppose, we'd all juse use a single character encoding, like perhaps Unicode, all the time. Today guess I'd be happy if everyone just settled on UTF-8, but because of all the different combinations of systems, tools and legacy documents it's not very likely.

16-Sep-2009

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Chris on 2009-09-15

2009-09-16T01:41:57Z

There's another type of cause for these unanticipated character swaps that database developers are accustomed to dealing with. Each database system has it's own set of special characters, which need to be 'escaped' with other special characters or sequences whenever they are used in a text representation - in order to prevent them from being interpreted as instructions. When working with multiple database types, it's easy to use the wrong escape sequence. Also, escaping can be overlooked, which, technically is a 'bug'. These are just some additional reasons quote characters get messed up - especially on mass-produced interactive web pages.

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Dan on 2009-09-15

2009-09-15T23:11:55Z

I rarely get this in Emails. However, I come across this very often, while navigating the web. If an site has SOME, for example, Japanese type characters included, I get the little boxes with numbers. Even if I use Google's, "translate this page" function there will still be "little boxes w/ numbers in them" Can be really frustrating.

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Sue on 2009-09-15

2009-09-15T20:30:14Z

UTF-8 - I need a lesson on this too. Many thanks

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Don Taber on 2009-09-15

2009-09-15T18:36:41Z

The first clear explanation I've seen for why I get the strange "a Euro-sign TM" characters -- which I see a lot. Thanks.

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Shawn on 2009-09-15

2009-09-15T17:30:41Z

Outlook is culprit #3, since it can be configured (maybe the default config) to use Word as the email editor.

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.

Comment from Greg Bulmash on 2009-09-13

2009-09-14T06:59:39Z

I just liked to blame Word, because MSFT is an easy target. Thanks for the detailed explanation.

]]>

A comment on: Why do I get odd characters instead of quotes in my documents?

Tech Questions? Get Answers! - Ask Leo! ... by Leo Notenboom
Leo's Answers Newsletter - Ask Leo! in your inbox every week.