Autosuggestions Data-type isn't accessible from the CLI

[axif0]

Terms

• I have searched all open bug reports
• I agree to follow Scribe-Data's Code of Conduct

Behavior

Currently in get.py, there is no logic set for Autosuggestions.

As in process_wiki.py function called gen_autosuggestions already defined. Can we can that in get.py ? If yes, I want to work on this issue.

[andrewtavis]

Yes, this would be great, @axif0! But let's finish the current issues first :)

[andrewtavis]

Discussing this a bit with @axif0 right now. We don't really think that the way that we're doing autosuggestions is something that would be applicable to other communities. Also, the long term plan is that we would actually not be using Wikipedia based calculated autosuggestions for this, but rather include a small LLM model in the language packs that would provide context based autosuggestions. Because of this, the work for this issue could potentially be to simply remove autosuggestions as an option for the CLI, and we will then use the notebook as needed in the Scribe community.

CC @wkyoshida and @mhmohona

[Collins-Webdev]

Hello @andrewtavis 👋🏼, I hope you are doing well !
I am eager to work on this bug.
Can you asign me to it please 🙏🏽?

[Collins-Webdev]

Hello @andrewtavis 👋🏼,
I just made a PR about this issue here : #462
Please review it and let me know if I should modify something 😊.

[andrewtavis]

For this one I think we're going to need to do some work to make it so that when someone passes -dt autosuggestions it will run the Wikipedia based processes. This would be great and would make the update of Scribe-iOS much easier. Basically:

• We should get rid of gen_autosuggestions.ipynb and move all of its functionality into a function that the CLI calls
• We should only accept one language with -dt autosuggestions as we'll need to download a Wikipedia dump
• In a similar way that we get down the Wikidata dump and parse for the Lexeme data, we'd then also get the autosuggestions for this
• We'll also need to remake the words to ignore/skip to make sure that these are not being included in the end autosuggestions
  • These are in an older version of the language metadata file, and maybe we can just have one large list somewhere that's not in the metadata file 🤔

[andrewtavis]

Do you have any further thoughts on the suggestions above, @axif0? This should just be hooking the wikipedia directory processes into the CLI, but might need some tweaks along the way.