Code Improvements and split by apostrophe #3
Sorry for the delay with this and thanks for making these changes. Just so that I understand your code correctly, why does the split not occur when reading from the dictionary file? Is that just an efficiency thing or is there a specific compatibility reason for it?
Splitting on apostrophes should occur after they have been normalized from curly quotes, otherwise some will slip through. I also think splitting on apostrophes should be behind a flag, defaulting to off. It would be weird for English and probably other languages too, so it should only be on for people who need that behavior.
In italian we don't use curly apostrophes so the risk of that there isn't.
I can add a flag without problems to enable the split by a parameter is not a problem.
About the dictionary file I didn't used in my case because the readme of the CV scraper don't mention it.