Language Independence

[ianmilligan1]

I wanted to move the conversation from #647 so policy discussions don't get wrapped up with an individual lesson.

@arojascastro noted below:

I may be wrong but the point is that Geoparser only recognizes named entities in English; I have never used it but I checked the website and I did not see any way to change the language parameter or whatever. So if anyone translates this lesson, s/he will have to use the text in English - because it will not work with texts in Spanish.

What do you think? @vgayolrs @mariajoafana

On the other hand, maybe it would be good pracrice to accept the publication of lessons that takes into consideration how dependant the content is on language.

Examples:

• is it possible to use texts in different language other than English? Then it may be accepted
• can you use resources that are transnational (for instance Europeana)? Fine!
• are you using tools that are available in different languages? Perfect!

In my opinion, if the PH is committed to open access and free tecnologies, it may be good idea to extend its committment to multilingual tecnologies as well... of course a first step could be tecnologies that are localisation friendly...

[ianmilligan1]

My own two cents is that I think it's worth making a consideration, but I don't think we want to have yet another blanket policy.

The Geoparser is a good example: by default it focuses on English, but a research team could tweak different elements to make it work in different languages. It would take effort and time, and it's not realistically feasible for tool projects to implement all of this out of the box on the resources that most DH projects have.

For what it's worth, my work for example focuses on English-language tools out of the box, mostly because (a) that's our expertise; (b) our pipeline of tools; and (c) our local users. We make it so that it's extendable down the road for other linguistic communities, but in some cases that's going to require people to do a bit of their own legwork on the text classifier front. With these complex tools, language independence isn't something you're going to easily have out of the box, especially given interdependent processing steps.

[arojascastro]

Yup, I agree mostly, I think you could take into account when reviewing proposals - just as one factor amongst others.

However, your scenario applies well to programming languages, tools and methods mostly.

What about the reuse of information already published by third parties? I think a great amount of work can be on this second direction: to use resources that are transnational and multilingual. They already exist. So for instance, instead of mining the Internet Archive, why not mining Europeana?

At the moment all PH contents, examples and resources are focused on the English culture, which of course makes total sense because authors are English or American, but the audience... is it English or American only?

I do not think it would take a lot of effort to look for examples and resources that are not that dependent on language -- that are more diverse. We cannot limit diversity to the composition of our team board... it must be reflected on content as well. I insist this is feasible for all authors.

[acrymble]

It could be something that someone mentions at the review stage if a lesson looks like it will be heavily focused on English material. Maybe someone with experience translating could just say something like: 'is there a way this could be altered so that we could more easily translate it into Spanish and other languages?'

[mdlincoln]

@arojascastro Does this guideline need to be written down in our editorial and/or reviewer guidelines?

[arojascastro]

In my opinion, yes. But maybe we should vote? Or see more views? It is up to you (English team) really.

[mdlincoln]

I think it makes sense as a guideline - something that editors and reviewers should keep in mind, much like they do all the other traits of lessons that we try to cultivate. @arojascastro would you like to draft some additions to those pages and submit it as a PR? Then we can all take a look at those.

[arojascastro]

Yes, @mdlincoln I'll take this. I need some days, but I am back on tracks and working for the PHes fully (I had to focus on teaching).

[arojascastro]

I found these resources by the W3C about internationalization and localization: https://www.w3.org/International/getting-started/index

After reading, I drafted this text. Of course, my English can be improved, do not hesitate to suggest improvements concerning language and contents.

Internationalization and localization

As the World Wide Consortium (W3C) reminds us, internationalization "is the design and development of a product, application or document content that enables easy localization for target audiences that vary in culture, region, or language". Since 2017 The Programming Historian has become an international and multilingual website that provides lessons for English and Spanish audiences (February 2018). For this reason, we encourage authors to adopt an international outlook while writing their lessons. Key aspects to be considered include but are not limited to:

• Images, texts and datasets should be as cosmopolitan as possible accordingly with our global audience. We recommend using international resources well known all around the world.
• Examples, representations, icons, symbols and colors are important. We encourage seeking cases that embrace diversity and inclusivity and that are not offensive to other cultures - a trivial thumb up gesture or emoji may be misunderstood in Australia, Greece or Middle East.
• Do not expect that everybody knows referents originated in a specific cultural context. Give the full name of organizations and the abbreviated form in parenthesis. For people, places or events, some explanations or links to Wikipedia may be necessary.
• Use methods and tools that can be used, adapted or trained in a different language other than the language of your tutorial.
• Use methods and tools whose web interface and documentation are available in several languages.
• Support different character sets. If dealing with text processing, consider that most languages make use of non Latin characters as well as accents and various other diacritics.
• Be aware of cultural differences related to display capabilities, time and date formats, and personal names.
• Provide information and links to further resources and bibliographical references in several languages.

These key aspects are desiderable in order to make the localization process easier. Our editors will assist authors and reviewers and give appropriate feedback in case of doubt.

[arojascastro]

You may also be interested in this Internationalization Checklist by Windows: https://msdn.microsoft.com/en-us/library/windows/desktop/ee845046(v=vs.85).aspx

There are more tecnical stuff that you may want to include @mdlincoln @acrymble ...