New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple dictionaries not searched for inexact queries #495

Closed
timrae opened this Issue May 5, 2015 · 9 comments

Comments

Projects
None yet
2 participants
@timrae

timrae commented May 5, 2015

Example: Download and enable the proper names dictionary
新木場駅: works by itself
新木場駅で: fails

Also, words in kana and deinflection don't seem to work for inexact queries:
新木場駅でキミを見ました

@mvysny

This comment has been minimized.

Show comment
Hide comment
@mvysny

mvysny May 6, 2015

Owner

Thanks, fixed in Aedict 3.18

Owner

mvysny commented May 6, 2015

Thanks, fixed in Aedict 3.18

@mvysny mvysny closed this May 6, 2015

@mvysny mvysny added the bug label May 6, 2015

@timrae

This comment has been minimized.

Show comment
Hide comment
@timrae

timrae May 8, 2015

There seems to have been a regression with this change... now none of the examples I gave are working.

新木場駅 -> doesn't find the entry in the names dictionary
見ました -> Doesn't deinflect

I think you need some unit tests for checking the search algorithm

timrae commented May 8, 2015

There seems to have been a regression with this change... now none of the examples I gave are working.

新木場駅 -> doesn't find the entry in the names dictionary
見ました -> Doesn't deinflect

I think you need some unit tests for checking the search algorithm

@mvysny mvysny reopened this May 8, 2015

@mvysny

This comment has been minimized.

Show comment
Hide comment
@mvysny

mvysny May 8, 2015

Owner

It is currently quite hard to write unit tests when two or more dictionaries are used at the same time, but I will try. Let me look into why 新木場駅 is not found in the names dictionary when sentence analysis is performed.
Regarding the 見ました not being deinflected correctly - yes, this is a known issue and it's not easy to fix. What the deinflection algorithm misses is the information whether 見 is v1 or v5 and what the exact reading is in this particular case. Can you please open a separate bug for this?

Owner

mvysny commented May 8, 2015

It is currently quite hard to write unit tests when two or more dictionaries are used at the same time, but I will try. Let me look into why 新木場駅 is not found in the names dictionary when sentence analysis is performed.
Regarding the 見ました not being deinflected correctly - yes, this is a known issue and it's not easy to fix. What the deinflection algorithm misses is the information whether 見 is v1 or v5 and what the exact reading is in this particular case. Can you please open a separate bug for this?

@mvysny

This comment has been minimized.

Show comment
Hide comment
@mvysny

mvysny May 8, 2015

Owner

I have tried the example sentence 新木場駅でキミを見ました and it is analyzed correctly into:
新木場駅: Shinkiba Station
で:
キミ: Kimi
を:
見る: to see (okay, this is fixed in yet-to-be-released Aedict 3.19, you will probably get 見 and ました or something similar).
My point is that it works for me and Shinkiba Station gets resolved properly. Have you configured Aedict to search in the JP Proper Names dictionary? You can do that in Settings or in Advanced search dialog (by pressing the upper-right "cog wheel", then touch "Dictionary" and make sure that both JMDict and JP Proper Names are selected.

Owner

mvysny commented May 8, 2015

I have tried the example sentence 新木場駅でキミを見ました and it is analyzed correctly into:
新木場駅: Shinkiba Station
で:
キミ: Kimi
を:
見る: to see (okay, this is fixed in yet-to-be-released Aedict 3.19, you will probably get 見 and ました or something similar).
My point is that it works for me and Shinkiba Station gets resolved properly. Have you configured Aedict to search in the JP Proper Names dictionary? You can do that in Settings or in Advanced search dialog (by pressing the upper-right "cog wheel", then touch "Dictionary" and make sure that both JMDict and JP Proper Names are selected.

@timrae

This comment has been minimized.

Show comment
Hide comment
@timrae

timrae May 8, 2015

I just tried again and it's working now. I did have the names dict enabled,
but since I didn't take a screenshot it's possible that I made a mistake
entering the kanji. I'll let you know if I experience it again.

By the way, is there any room for performance improvements? The search on
that example sentence takes almost 30s on my phone.
On 09/05/2015 6:28 am, "Martin Vysny" notifications@github.com wrote:

I have tried the example sentence 新木場駅でキミを見ました and it is analyzed
correctly into:
新木場駅: Shinkiba Station
で:
キミ: Kimi
を:
見る: to see (okay, this is fixed in yet-to-be-released Aedict 3.19, you
will probably get 見 and ました or something similar).
My point is that it works for me and Shinkiba Station gets resolved
properly. Have you configured Aedict to search in the JP Proper Names
dictionary? You can do that in Settings or in Advanced search dialog (by
pressing the upper-right "cog wheel", then touch "Dictionary" and make sure
that both JMDict and JP Proper Names are selected.


Reply to this email directly or view it on GitHub
#495 (comment).

timrae commented May 8, 2015

I just tried again and it's working now. I did have the names dict enabled,
but since I didn't take a screenshot it's possible that I made a mistake
entering the kanji. I'll let you know if I experience it again.

By the way, is there any room for performance improvements? The search on
that example sentence takes almost 30s on my phone.
On 09/05/2015 6:28 am, "Martin Vysny" notifications@github.com wrote:

I have tried the example sentence 新木場駅でキミを見ました and it is analyzed
correctly into:
新木場駅: Shinkiba Station
で:
キミ: Kimi
を:
見る: to see (okay, this is fixed in yet-to-be-released Aedict 3.19, you
will probably get 見 and ました or something similar).
My point is that it works for me and Shinkiba Station gets resolved
properly. Have you configured Aedict to search in the JP Proper Names
dictionary? You can do that in Settings or in Advanced search dialog (by
pressing the upper-right "cog wheel", then touch "Dictionary" and make sure
that both JMDict and JP Proper Names are selected.


Reply to this email directly or view it on GitHub
#495 (comment).

@mvysny

This comment has been minimized.

Show comment
Hide comment
@mvysny

mvysny May 9, 2015

Owner

Thanks, just let me know if it happens again.
On my phone the analysis of 新木場駅でキミを見ました takes less than 1 second (OnePlus One). Perhaps it is time to buy a new phone? :) Currently I don't have much time to improve the algorithm, perhaps it can be improved but it will still be slow as it makes tons of requests to the database. Which phone do you have?

Owner

mvysny commented May 9, 2015

Thanks, just let me know if it happens again.
On my phone the analysis of 新木場駅でキミを見ました takes less than 1 second (OnePlus One). Perhaps it is time to buy a new phone? :) Currently I don't have much time to improve the algorithm, perhaps it can be improved but it will still be slow as it makes tons of requests to the database. Which phone do you have?

@mvysny mvysny closed this May 9, 2015

@timrae

This comment has been minimized.

Show comment
Hide comment
@timrae

timrae May 9, 2015

I have an Xperia ZR, it's not super slow, but OK I'll give it a try on a different phone sometime

Chipset: Qualcomm Snapdragon APQ8064
CPU: Quad-core 1.5 GHz Krait
GPU: Adreno 320
RAM: 2 GB

timrae commented May 9, 2015

I have an Xperia ZR, it's not super slow, but OK I'll give it a try on a different phone sometime

Chipset: Qualcomm Snapdragon APQ8064
CPU: Quad-core 1.5 GHz Krait
GPU: Adreno 320
RAM: 2 GB

@mvysny

This comment has been minimized.

Show comment
Hide comment
@mvysny

mvysny May 9, 2015

Owner

The specs of that phone are quite good actually, Aedict should definitely work okay on ZR. Hmm.. just a wild guess, perhaps you have the dictionaries on a SDCard and the dictionary lookups are therefore slower? On my phone there is no sdcard so I have dictionaries on the internal memory.

Owner

mvysny commented May 9, 2015

The specs of that phone are quite good actually, Aedict should definitely work okay on ZR. Hmm.. just a wild guess, perhaps you have the dictionaries on a SDCard and the dictionary lookups are therefore slower? On my phone there is no sdcard so I have dictionaries on the internal memory.

@timrae

This comment has been minimized.

Show comment
Hide comment
@timrae

timrae May 9, 2015

Nope that's not the issue but I'll do a bit more testing
On 09/05/2015 5:33 pm, "Martin Vysny" notifications@github.com wrote:

The specs of that phone are quite good actually, Aedict should definitely
work okay on ZR. Hmm.. just a wild guess, perhaps you have the dictionaries
on a SDCard and the dictionary lookups are therefore slower? On my phone
there is no sdcard so I have dictionaries on the internal memory.


Reply to this email directly or view it on GitHub
#495 (comment).

timrae commented May 9, 2015

Nope that's not the issue but I'll do a bit more testing
On 09/05/2015 5:33 pm, "Martin Vysny" notifications@github.com wrote:

The specs of that phone are quite good actually, Aedict should definitely
work okay on ZR. Hmm.. just a wild guess, perhaps you have the dictionaries
on a SDCard and the dictionary lookups are therefore slower? On my phone
there is no sdcard so I have dictionaries on the internal memory.


Reply to this email directly or view it on GitHub
#495 (comment).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment