Skip to content
NLP DNN Toolkit - Building Your NLP DNN Models Like Playing Lego
Branch: master
Clone or download
adolphk-yk and ljshou Modify sequence labeling tutorial (#69)
* add sequence labeling in tutorial

* add tag scheme convert script

* add paper link

* modify sequence labeling tutorial

* modify senquence labeling tutorial
Latest commit 47008bb Jun 22, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
autotest update auto_test May 30, 2019
block_zoo add CRF related block and logic (#67) Jun 19, 2019
core Use lazy loading instead of one-off loading (#28) May 10, 2019
dataset add CRF related block and logic (#67) Jun 19, 2019
docs add dataset and modification to Tutorial.md Apr 22, 2019
losses add CRF related block and logic (#67) Jun 19, 2019
metrics update sequence tagging task workflow (#62) Jun 5, 2019
model_visualizer change visualizer readme.md Apr 22, 2019
model_zoo add CRF related block and logic (#67) Jun 19, 2019
optimizers Code commit Apr 20, 2019
preparation Code commit Apr 20, 2019
tools Add Sequence labeling in Tutorial and tag scheme convert script (#65) Jun 17, 2019
utils add CRF related block and logic (#67) Jun 19, 2019
.gitignore Fix concat in interaction layer. (#57) Jun 14, 2019
.nojekyll change visualizer readme.md Apr 22, 2019
Contributing.md Update Contributing.md README (#58) May 28, 2019
LICENSE Initial commit Apr 15, 2019
LearningMachine.py add CRF related block and logic (#67) Jun 19, 2019
Model.py add CRF related block and logic (#67) Jun 19, 2019
ModelConf.py add CRF related block and logic (#67) Jun 19, 2019
NOTICE.txt Code commit Apr 20, 2019
README.md change logo Jun 3, 2019
README_zh_CN.md change logo Jun 3, 2019
Tutorial.md Modify sequence labeling tutorial (#69) Jun 22, 2019
Tutorial_zh_CN.md Modify sequence labeling tutorial (#69) Jun 22, 2019
autotest.sh chmod 775 autotest.sh, chmod 775 dataset/get_glove.sh (#61) May 29, 2019
predict.py Reorg logical flow in train (#37) May 22, 2019
problem.py add CRF related block and logic (#67) Jun 19, 2019
requirements.txt remove torch from requirements.txt (#32) May 9, 2019
settings.py update sequence tagging task workflow (#62) Jun 5, 2019
test.py update sequence tagging task workflow (#62) Jun 5, 2019
train.py update sequence tagging task workflow (#62) Jun 5, 2019

README.md

Building Your NLP DNN Models Like Playing Lego

language python pytorch license

简体中文

Tutorial 中文教程

Table of Contents

Overview

NeuronBlocks is a NLP deep learning modeling toolkit that helps engineers/researchers to build end-to-end pipelines for neural network model training for NLP tasks. The main goal of this toolkit is to minimize developing cost for NLP deep neural network model building, including both training and inference stages.

NeuronBlocks consists of two major components: Block Zoo and Model Zoo.

  • In Block Zoo, we provide commonly used neural network components as building blocks for model architecture design.
  • In Model Zoo, we provide a suite of NLP models for common NLP tasks, in the form of JSON configuration files.

Language Supported

  • English
  • Chinese

NLP Tasks Supported

  • Sentence Classification
  • Sentiment Analysis
  • Question Answering Matching
  • Textual Entailment
  • Slot tagging
  • Machine Reading Comprehension
  • Knowledge Distillation for Model Compression
  • More on-going

Toolkit Usage

Users can either pick existing models (config files) in Model Zoo to start model training or create new models by leveraging neural network blocks in Block Zoo just like playing with Lego.

Get Started in 60 Seconds

Installation

Note: NeuronBlocks requires Python 3.6 and above.

  1. Clone this project.

    git clone https://github.com/Microsoft/NeuronBlocks
  2. Install Python packages in requirements.txt by the following command.

    pip install -r requirements.txt
  3. Install PyTorch (NeuronBlocks supports PyTorch 0.4.1 and above).

    For Linux, run the following command:

    pip install "torch>=0.4.1"

    For Windows, we suggest you to install PyTorch via Conda by following the instruction of PyTorch.

Quick Start

Get started by trying the given examples. Both Linux/Windows, GPU/CPU are supported. For Windows, we suggest you to use PowerShell instead of CMD.

Tips: in the following instruction, PROJECTROOT denotes the root directory of this project.

# train
cd PROJECT_ROOT
python train.py --conf_path=model_zoo/demo/conf.json

# test
python test.py --conf_path=model_zoo/demo/conf.json

# predict
python predict.py --conf_path=model_zoo/demo/conf.json

For more details, please refer to Tutorial.md and Code documentation.

Who should consider using NeuronBlocks

Engineers or researchers who face the following challenges when using neural network models to address NLP problems:

  • Many frameworks to choose and high framework studying cost.
  • Heavy coding cost. A lot of details make it hard to debug.
  • Fast Model Architecture Evolution. It is difficult for engineers to understand the mathematical principles behind them.
  • Model Code optimization requires deep expertise.
  • Model Platform Compatibility Requirement. It requires extra coding work for the model to run on different platforms, such as Linux/Windows, GPU/CPU.

The advantages of leveraging NeuronBlocks for NLP neural network model training includes:

  • Model Building: for model building and parameter tuning, users only need to write simple JSON config files, which greatly minimize the effort of implementing new ideas.

  • Model Sharing It is super easy to share models just through JSON files, instead of nasty codes. For different models or tasks, our users only need to maintain one single centralized source code base.

  • Code Reusability: Common blocks can be easily shared across various models or tasks, reducing duplicate coding work.

  • Platform Flexibility: NeuronBlocks can run both on Linux and Windows machines, using both CPU and GPU. It also supports training on GPU platforms like Philly and PAI.

    CPU inferenceSingle-GPU inferenceMulti-GPU inference
    CPU train
    Single-GPU train
    Multi-GPU train
  • Model Visualization: A model visualizer is provided for visualization and configure correctness checking, which helps users to visualize the model architecture easily during debugging.

  • Extensibility: NeuronBlocks is extensible, allowing users to contribute new blocks or contributing novel models (JSON files).

Contribute

NeuronBlocks operates in an open model. It is designed and developed by STCA NLP Group, Microsoft. Contributions from academia and industry are also highly welcome. For more details, please refer to Contributing.md.

Ongoing Work and Call for Contributions

Anyone who are familiar with are highly encouraged to contribute code.

  • Knowledge Distillation for Model Compression. Knowledge distillation for heavy models such as BERT, OpenAI Transformer. Teacher-Student based knowledge distillation is one common method for model compression.
  • Multi-Lingual Support
  • NER Model Support
  • Multi-Task Training Support

Reference

NeuronBlocks -- Building Your NLP DNN Models Like Playing Lego, at https://arxiv.org/abs/1904.09535.

@article{gong2019neuronblocks,
  title={NeuronBlocks--Building Your NLP DNN Models Like Playing Lego},
  author={Gong, Ming and Shou, Linjun and Lin, Wutao and Sang, Zhijie and Yan, Quanjia and Yang, Ze and Jiang, Daxin},
  journal={arXiv preprint arXiv:1904.09535},
  year={2019}
}

Related Project

  • OpenPAI is an open source platform that provides complete AI model training and resource management capabilities, it is easy to extend and supports on-premise, cloud and hybrid environments in various scale.
  • Samples for AI:  a deep learning samples and projects collection. It contains a lot of classic deep learning algorithms and applications with different frameworks, which is a good entry for the beginners to get started with deep learning.

License

Copyright (c) Microsoft Corporation. All rights reserved.

Licensed under the MIT License.

Contact

If you have any questions, please contact NeuronBlocks@microsoft.com

If you have wechat, you can also add the following account:

You can’t perform that action at this time.