feat(deinflect): Add logic to handle all inflections of the copula だ

[maawisul]

• Adds a new type of deinflection for the copula, だ.
• Adds mappings for all common conjugations to deinflect.dat.
  • See http://www.japaneseprofessor.com/reference/grammar/conjugations-of-the-japanese-copula/
• For handling polite negative じゃないです update general adjective handling to recognize polite です addition.

Fixes #89

[codecov]

Codecov Report

Merging #988 (4b57766) into main (69c7fe7) will increase coverage by 0.15%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #988      +/-   ##
==========================================
+ Coverage   78.97%   79.12%   +0.15%     
==========================================
  Files           7        7              
  Lines        3001     3004       +3     
  Branches      174      184      +10     
==========================================
+ Hits         2370     2377       +7     
+ Misses        628      622       -6     
- Partials        3        5       +2     

Impacted Files	Coverage Δ
extension/data.ts	82.43% <100.00%> (+0.43%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 69c7fe7...4b57766. Read the comment docs.

[melink14]

I think the change to make it conjugate as a verb might be too broad since it will cause it to match other verb endings as well:

I wasn't thinking it through when I thought we could reuse the existing す verb logic... 🙇🏾

Since です is so irregular I think the complete fix would be to add a new type for cop in deinflect.dat and then add entries for the cases which only apply to this type of word. (https://www.tanoshiijapanese.com/dictionary/conjugation_details.cfm?entry_id=103243&element_id=128958&conjugation_type_id=5 is the best list I found but we don't need to add them all now.)

To add an entry to deinflect.dat is faily straight forward. It starts with a set of reasons and then each entry has the form of:
<inflected_form> <deinflected_form> <type_mask> <index_into_reasons>

The first two are encoding the logic we want and the last is a 0 based index into the reasons at the top of the file.

The type mask is opaque but it looks to be two values OR'd together (type of inflected word and type of deinflected word) as was clarified in this fork: https://gist.github.com/lynn/07f7ce3c314223d2aca19ec2bb0540cd#file-patch-deinflect-py-L7

I think if we add a new base mask for です then we can easily add a line to deinflect.dat and update your branch a bit to make things work specifically for this case!

What do you think?

[maawisul]

I agree! I did think that the case of です and its conjugation is pretty irregular and need to be differentiated from any other verbs or adjectives. However, I am not sure on how to come up with a new typemask and would need some guidance on that.

[melink14]

No problem! I have some ideas about that.

For the type of the inflected word, I think 0x0080 (OTHER) makes sense in this case (though if we added forms like ではない it would be different). For BASE we'll need a new type in the upper half of the mask (imagine BASE_DESU) and since BASE_SURU is 0x1000 we can increment it to 0x2000 and be safe. (This will convert to a type of 32 in data.ts).

> node -p "0x80|0x2000"
8320

So I think a type mask of 8320 should work well for でした　ー＞　です

Let me know if you have any other questions.

[melink14]

BTW, I was reading http://www.japaneseprofessor.com/reference/grammar/conjugations-of-the-japanese-copula/ which made me realize the honorific forms are also lost in rikaikun.

I think we'll be able to add several new lines for various copula conjugations once the base logic is in. You could do it at the same time or just focus on でした depending on your time/want.

[maawisul]

Thanks! I will try working on it as much as I can submit another PR when I'm satisfied with my change.

[tentagul]

that works.

[melink14]

Thanks! I will try working on it as much as I can submit another PR when I'm satisfied with my change.

For now, I'll close this PR then and feel free to re-open when ready, or let me know if you have any more questions on the issue.

[maawisul]

I think I am satisfied with my changes and ready to re-open. How do I re-open a pull request?

[maawisul]

I added entries and type(past negative) to deinflect.dat so that rikaikun now recognizes:
でした
じゃありません
じゃありませんでした
でございません / ではございません
でございませんでした / ではございません

I am not sure on what to do with ございます on its own as I am not sure if it is still considered a copula or not　

[melink14]

Thanks for the update! Looks like we're pretty close.

I think maybe things became complicated because the dictionary contains many congugated forms of the copula which you tried to work with directly and thus created congugation mappings to intermediate inflections.

It makes sense given that the dictionary contains things like じゃない even though that's the negative form of だ but I think we should maintain consistency and conjugate back to the base form in all cases. This means that we'll show two entries for some words but that's okay in my mind because sometimes these short common phrases have idiomatic meaning as well as the basic grammar meaning.

I think not relying on the dictionary containing certain common inflections is also more robust since they might remove the inflected forms in a future update.

In that case, it makes sense to add mappings back to だ since it's the polite affirmative form of the copula. We can start with the mappings that don't have any other entry in the dictionary or fill out the entire table. I'll leave a comment with what I mean inline.

[melink14]

This looks pretty good with some small comments.

We should also add some tests for the data.ts changes. There aren't tests for that file yet but it should be fairly straightforward to do in the same manner of other files like background.ts.

That being said, it can be tricky writing the first tests for a class, so let me know if you want me to add a commit with some tests this time around.

[melink14]

I think it should be basically done now though it looks like I need to increase the timeout for the new data tests (loading the data is slower in github than on my local computer...)

[melink14]

@all-contributors Add @maawisul for code

[allcontributors]

@melink14

I've put up a pull request to add @maawisul! 🎉

[melink14]

🎉 This PR is included in version 2.4.0 🎉

The release is available on:

• Chrome Web Store
• GitHub release

Your semantic-release bot 📦🚀