soochem / transformer-impl Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

Re-implementation of "Attention is All You Need" (NIPS 2017)

0 stars 0 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 7 Commits
src		src
test		test
util		util
.gitignore		.gitignore
README.md		README.md
__init__.py		__init__.py
main.py		main.py
requirements.txt		requirements.txt

Repository files navigation

Transformer Implementation

0. Preparation

Python 3.6+
torch>=1.6.0+cu101
sentencepiece>=0.1.94
numpy>=1.18.5
progressbar>=2.5
lxml>=4.6.1

1. Data Preprocessing

Process

web => (urllib) => tar 
    => (tarfile) => xml 
    => (lxml) => txt
    => (setencepiece) => embedding vector

About

Re-implementation of "Attention is All You Need" (NIPS 2017)

Report repository

Releases

No releases published

Packages

No packages published

Languages