Foreign Wikipedia accent issues with French

[braddockcg]

The Wikipedia ZIM file encodings are not consistent (sometimes UTF-8, sometimes Latin, sometimes god knows what) and this is causing problems.

A field report from Haiti:
"PS a few of the most advanced kids played with and searched thru article titles with IIAB's French Wikipedia on their XO-1s, which works but French accents are quite badly mangled (each accented character shows up as 2 "random" characters, rendering the text almost unreadable in many cases. Apparently regardless whether the XO's set to English, French or Haitian Creole"

[braddockcg]

"For example the very main page of http://schoolserver/iiab/zim/iframe/wikipedia_fr_all_09_2012 shows in both schools here in Haiti as:

Wikipédia est une encyclopédie multilingue, universelle et librement diffusable.

Which should appear as:

Wikipédia est une encyclopédie multilingue, universelle et librement diffusable.

[braddockcg]

It is due to inconsistent language encodings in the wikipedia dumps from the Kiwix project. We need to hack around it (and have indeed tried, but obviously not well enough).