This is a monorepo containing all the language tools for CAI (Conversational Artificial Intelligence) project. Every single language can be found as an independant package, so you can install only the languages that you need.
Each language has it's own package, this is an example of how to install the english one. Run this in your project folder:
$ npm install @caijs/lang-en
Language | Locale | Package | Github |
---|---|---|---|
English | en | @caijs/lang-en | Link |
Each language has it's own package, this is an example of how to use the english one. You can tokenize and stem a sentence. It will normalize the sentence, tokenize it and return the stems.
const { tokenizeAndStem } = require('@caijs/lang-en');
const stems = tokenizeAndStem('what else is developing your enterprise');
console.log(stems); // ['what', 'els', 'is', 'develop', 'your', 'enterpris']
Each language has it's own package, this is an example of how to use the english one. You can normalize a sentence, it will pass the sentece to lower case and replace special characters with the equivalent characters.
const { normalize } = require('@caijs/lang-en');
const normalized = normalize('What döès youR Compañy develop');
console.log(normalized); // what does your company develop
Each language has it's own package, this is an example of how to use the english one. It tokenizes a sentence, without normalizing it. Split the sentence into tokens
const { tokenize } = require('@caijs/lang-en');
const tokens = tokenize('If you\'re here, then enter');
console.log(tokens); // ['If', 'you', 'are', 'here', 'then', 'enter']
Each language has it's own package, this is an example of how to use the english one. It stems a word, without normalizing it.
const { stem } = require('@caijs/lang-en');
const stemmed = stem('enterprise');
console.log(stemmed); // enterpris