What's new in 0.24.0

[nicolay-r]

Main feature is support of the custom documents

Backend Updates and Extended Schemata

• Switch to the AREkit==0.24.0 (be14ffc)

Quality of the sampled data

According to the #52 experiments:

• Support Non-span based translation via GoogleTranslate engine #70
• Disable term splitting for LLM and BERT #71

Dynamic prompting support

When analysing NEREL-bio outputs, it was found that some labels and relations are better to describe more precisely with prompts

• Support SQLite provider  #67
• Dynamic prompting -- Relation and Entity types parameter for filtering and precise prompting #66

Simplify new sources registration

There were few feedbacks on bad API with labels and that whole project might be crashed because of the incompletely registered source

• Dynamic import for sources #62
• Label scaler implementation could be simplified [NEREL registration backlog] #58
• Structurization #59

Other / Minor updates

• Save label value (int, non-int) #68
• Inform empty split [user experience backlog] #69
• Fold type selection #42
• doc_ids argument support which allows to select the specific doc_ids for processing (90488c2)
• [output_dir] customization removed (234ae21)
• RuSentRel consider the predefined split and foldings for train and test parts separately #72
• dest_lang is optional parameter (by default is src_lang) (234ae21)
• Simple entity masking option support #74
• 🔧 Use translator from AREkit with googletrans engine (AREkit related uptade and ARElight) #75
• Compose a complete instruction for AREkit-ss usage #76

[nicolay-r]

🥳