Implement Azerbaijani support #212

saymantech-co · 2021-10-18T12:46:43Z

Implement Azerbaijani support(az_AZ)

Type of PR

Feature implementation

‌Testing

pytest test/test_format_az.py
pytest test/test_parse_az.py

devs-mycroft · 2021-10-18T12:47:13Z

Hello, @saymantech-co, thank you for helping with the Mycroft project! We welcome everyone
into the community and greatly appreciate your help as we work to build an AI
for Everyone.

To protect yourself, the project, and users of Mycroft technologies we require
a Contributor Licensing Agreement (CLA) before accepting any code
contribution. This agreement makes it crystal clear that along with your
code you are offering a license to use it within the confines of this project.
You retain ownership of the code, this is just a license.

Please visit https://mycroft.ai/cla to initiate this one-time signing. Thank
you!

saymantech-co · 2021-10-19T05:59:58Z

Hello, @saymantech-co, thank you for helping with the Mycroft project! We welcome everyone into the community and greatly appreciate your help as we work to build an AI for Everyone.

To protect yourself, the project, and users of Mycroft technologies we require a Contributor Licensing Agreement (CLA) before accepting any code contribution. This agreement makes it crystal clear that along with your code you are offering a license to use it within the confines of this project. You retain ownership of the code, this is just a license.

Please visit https://mycroft.ai/cla to initiate this one-time signing. Thank you!

Signed the cla.

siathon · 2021-11-05T20:52:09Z

@krisgesling For the date time stuff to work, the "az_AZ" locale needs to be generated in the OS, that's why the tests failed. and it can't be done from inside the code, it requires root access. Is this acceptable or should I find another way to handle the date time stuff?

krisgesling · 2021-11-24T05:56:30Z

Sorry I missed the notification for your last message.

My first reaction is that we should expect the distribution packaging this to ensure the correct locales are available, and yes we can add that to the CI job here too.

I'm not at all familiar with the locale module so not sure what side effects we might be creating by switching the locale back and forth. There are also warnings of it not being threadsafe.

Would it be sufficient for us to install language-pack-az?

krisgesling · 2021-11-24T06:06:14Z

#216 looks like it generated the locale without issue, so we can try it but I'm very interested to hear from anyone who's worked with locale before. Or if there are other options we can go with.

krisgesling

Hey sorry it's taken a while to get to this.

It's always a challenge reviewing extensive code in another language! But it looks pretty great.

There are a few little questions below but nothing of concern. The only thing is whether we really need to set the locale or not.

lingua_franca/lang/common_data_az.py

lingua_franca/res/text/az-az/date_time.json

krisgesling · 2021-11-24T06:22:52Z

test/test_format_az.py

+        # value is platform dependent so better not use in tests?
+        # self.assertEqual(
+        #    pronounce_number(sys.float_info.min), "iki nöqtə iki iki times "
+        #                                          "on to the power of "
+        #                                          "mənfi üç yüz "
+        #                                          "və səkkiz")
+        # self.assertEqual(
+        #    pronounce_number(sys.float_info.max), "bir nöqtə yeddi doqquz "
+        #                                          "times on to the power of"
+        #                                          " üç yüz və səkkiz")


I'd agree with the comment - this can be deleted unless you see specific value in it?

yeah, most of these comments are the result of using the english files as template. I'll review and remove them.

krisgesling · 2021-11-24T06:24:01Z

test/test_format_az.py

+        self.assertEqual(
+            pronounce_number(1000000000000, short_scale=True),
+            "bir trilyon")
+        # TODO maybe beautify this


Is this relevant to the AZ test or just cut and pasted from the other file?

No, sorry I used english files as template and edited them, and seems i have forgotten to remove some of non relevant comments. I'll review and remove them.

test/test_format_az.py

krisgesling · 2021-11-24T06:29:19Z

test/test_parse_az.py

@@ -0,0 +1,429 @@
+#
+# Copyright 2017 Mycroft AI Inc.


Would be good to use the current year in each of these license headers.

Ok, I will fix them.

siathon · 2021-11-25T11:34:53Z

Sorry I missed the notification for your last message.

My first reaction is that we should expect the distribution packaging this to ensure the correct locales are available, and yes we can add that to the CI job here too.

I'm not at all familiar with the locale module so not sure what side effects we might be creating by switching the locale back and forth. There are also warnings of it not being threadsafe.

Would it be sufficient for us to install language-pack-az?

The reason for switching the locale back and forth was when running pytest, after running AZ tests, because of changing the locale in AZ tests, all other language tests failed, So in AZ, I returned the locale to the previous locale.

if that is not necessary and only passing test_parse_az and test_format_az is enough, the switching back and forth is not required and can be removed.

krisgesling · 2021-11-30T07:02:37Z

With the AZ language pack installed and all the locale setting removed from parse_az.py. The following 4 test cases fail:

        testExtract("3 avqustda anama zəng etməyi xatırlat",
                    "2017-08-03 00:00:00", "anama zəng etməyi xatırlat")
        testExtract("iyulun 4 də atəşfəşanlıq al",
                    "2017-07-04 00:00:00", "atəşfəşanlıq al")
        testExtract("dekabr 3",
                    "2017-12-03 00:00:00", "")
        testExtract("5 iyun 2017 ci il axşamı anama zəng etməyi xatırlat",
                    "2017-06-05 19:00:00", "anama zəng etməyi xatırlat")

Is that what you're seeing also?

Are these the only test cases that would rely on the locale getting set or does it point to something else going on?

siathon · 2021-11-30T13:12:48Z

With the AZ language pack installed and all the locale setting removed from parse_az.py. The following 4 test cases fail:

        testExtract("3 avqustda anama zəng etməyi xatırlat",
                    "2017-08-03 00:00:00", "anama zəng etməyi xatırlat")
        testExtract("iyulun 4 də atəşfəşanlıq al",
                    "2017-07-04 00:00:00", "atəşfəşanlıq al")
        testExtract("dekabr 3",
                    "2017-12-03 00:00:00", "")
        testExtract("5 iyun 2017 ci il axşamı anama zəng etməyi xatırlat",
                    "2017-06-05 19:00:00", "anama zəng etməyi xatırlat")

Is that what you're seeing also?

Are these the only test cases that would rely on the locale getting set or does it point to something else going on?

yeah, These are the cases that rely on locale being set.

fix license year

siathon · 2021-12-21T06:06:20Z

@krisgesling is using the locale setting alright or should I try another way to handle these cases?

krisgesling · 2021-12-23T06:07:46Z

I'd really like it if we could find another way. But I don't have a suggestion for what that is sorry.

siathon · 2021-12-25T07:58:11Z

@krisgesling, Done, no locale setting required for parsing datetime.
575299c

MycroftAI#212

extract datetime without setting locale (cherry picked from commit 575299c) remove extra comments and imports fix license year (cherry picked from commit d449931) improve az-az duration extract (cherry picked from commit 6ee8412) Implemented Azerbaijani support (cherry picked from commit 0439a80)

extract datetime without setting locale (cherry picked from commit 575299c) remove extra comments and imports fix license year (cherry picked from commit d449931) improve az-az duration extract (cherry picked from commit 6ee8412) Implemented Azerbaijani support (cherry picked from commit 0439a80) Co-authored-by: Siavash Mollayi <siavash.mollayi@gmail.com>

Implemented Azerbaijani support

0439a80

devs-mycroft added the CLA: Needed Need signed CLA from https://mycroft.ai/cla label Oct 18, 2021

devs-mycroft added CLA: Yes Contributor License Agreement exists (see https://github.com/MycroftAI/contributors) and removed CLA: Needed Need signed CLA from https://mycroft.ai/cla labels Oct 19, 2021

improve az-az duration extract

6ee8412

saymantech-co force-pushed the master branch 2 times, most recently from 1dac5d9 to 6ee8412 Compare November 2, 2021 10:39

krisgesling mentioned this pull request Nov 24, 2021

Add Azerbaijani language pack to Github Actions system #216

Open

2 tasks

krisgesling reviewed Nov 24, 2021

View reviewed changes

krisgesling mentioned this pull request Nov 25, 2021

Remove random commented out code #217

Merged

2 tasks

remove extra comments and imports

d449931

fix license year

saymantech-co force-pushed the master branch from 988b2ae to d449931 Compare November 30, 2021 13:18

extract datetime without setting locale

575299c

NeonJarbas pushed a commit to NeonJarbas/ovos-lingua-franca that referenced this pull request Jan 19, 2022

port/azerbeijan

078b7d4

MycroftAI#212

NeonJarbas mentioned this pull request Jan 19, 2022

port/azerbeijan OpenVoiceOS/ovos-lingua-franca#4

Merged

NeonJarbas pushed a commit to NeonJarbas/ovos-lingua-franca that referenced this pull request Feb 22, 2022

port/azerbeijan

c22a463

MycroftAI#212

NeonJarbas pushed a commit to NeonJarbas/ovos-lingua-franca that referenced this pull request Apr 11, 2022

port/azerbeijan

b32159a

MycroftAI#212

siathon force-pushed the master branch from 879e79c to 575299c Compare August 14, 2023 11:29

improve farsi

d1225ac

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Azerbaijani support #212

Implement Azerbaijani support #212

saymantech-co commented Oct 18, 2021 •

edited

devs-mycroft commented Oct 18, 2021

saymantech-co commented Oct 19, 2021

siathon commented Nov 5, 2021 •

edited

krisgesling commented Nov 24, 2021

krisgesling commented Nov 24, 2021

krisgesling left a comment

krisgesling Nov 24, 2021

siathon Nov 25, 2021

krisgesling Nov 24, 2021

siathon Nov 25, 2021 •

edited

krisgesling Nov 24, 2021

siathon Nov 25, 2021

siathon commented Nov 25, 2021 •

edited

krisgesling commented Nov 30, 2021

siathon commented Nov 30, 2021

siathon commented Dec 21, 2021

krisgesling commented Dec 23, 2021

siathon commented Dec 25, 2021 •

edited

Implement Azerbaijani support #212

Are you sure you want to change the base?

Implement Azerbaijani support #212

Conversation

saymantech-co commented Oct 18, 2021 • edited

devs-mycroft commented Oct 18, 2021

saymantech-co commented Oct 19, 2021

siathon commented Nov 5, 2021 • edited

krisgesling commented Nov 24, 2021

krisgesling commented Nov 24, 2021

krisgesling left a comment

Choose a reason for hiding this comment

krisgesling Nov 24, 2021

Choose a reason for hiding this comment

siathon Nov 25, 2021

Choose a reason for hiding this comment

krisgesling Nov 24, 2021

Choose a reason for hiding this comment

siathon Nov 25, 2021 • edited

Choose a reason for hiding this comment

krisgesling Nov 24, 2021

Choose a reason for hiding this comment

siathon Nov 25, 2021

Choose a reason for hiding this comment

siathon commented Nov 25, 2021 • edited

krisgesling commented Nov 30, 2021

siathon commented Nov 30, 2021

siathon commented Dec 21, 2021

krisgesling commented Dec 23, 2021

siathon commented Dec 25, 2021 • edited

saymantech-co commented Oct 18, 2021 •

edited

siathon commented Nov 5, 2021 •

edited

siathon Nov 25, 2021 •

edited

siathon commented Nov 25, 2021 •

edited

siathon commented Dec 25, 2021 •

edited