Skip to content
NeuralTokenizer is a very simple to use text tokenizer which uses neural networks from the SimpleDNN library.
Kotlin
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
examples
src/main
.gitignore
.travis.yml
LICENSE.txt
README.md
pom.xml

README.md

NeuralTokenizer GitHub version Build Status

NeuralTokenizer is a very simple to use text tokenizer which uses neural networks from the SimpleDNN library.

NeuralTokenizer is part of KotlinNLP.

Getting Started

Import with Maven

<dependency>
    <groupId>com.kotlinnlp</groupId>
    <artifactId>neuraltokenizer</artifactId>
    <version>0.4.5</version>
</dependency>

Examples

Try some examples of usage of NeuralTokenizer running the files in the examples folder.

To run the examples you need datasets of test and training that you can find here

Model Serialization

The neural model is all contained into a single class which provides simple dump() and load() methods to serialize it and afterwards load it.

License

This software is released under the terms of the Mozilla Public License, v. 2.0

Contributions

We greatly appreciate any bug reports and contributions, which can be made by filing an issue or making a pull request through the github page.

You can’t perform that action at this time.