utf8 support and dir syntax #27

wants to merge 11 commits into


None yet
3 participants


this was just soo spot on for what I need I would like to contribute this back.

Utf8: to_yaml just does not always work correctly. We have äöü here in finland and they come out garbled.

dir syntax: For a similar task I have used this syntax and find it much easier to use. You can cherrypick, I've made 2 commit in case you don't like this

Torsten Rüger added some commits Nov 23, 2011

Sorry about the lack of testing: So I now find out that ya2yaml does not play nice with your chunking code.
I would be tempted to ditch the chunking (as it's only on write anyway) or make it explicit (by storing schema seperately and having table.CHUNK.yml files)
Both are unfortunately to big for me just now, so just be careful or remove the ya2yaml stuff

duffyjp commented Mar 12, 2012

I'm having the same problem with UTF-8 data. I'm dumping out of a mysql db, and the data.yml will have binary fields for any utf8 text. Doing a load into an sqlite I just get mojibake.

I tried Torsten's "better utf8" patch above (with ya2yaml), but there was no change for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment