Final project of Statistical Learning Course.
Here is our's information:
ID | Name | Contact |
---|---|---|
20120057 | Trần Ngọc Đô | |
20120024 | Huỳnh Minh Tuấn |
This project is a small application of auto-text summarization using Transformer architecture. The codebase includes 2 parts
Languages:
- Python
Frameworks/Libraries:
- Django - Build RESTful API
- SpaceOCR - Process OCR Task
- Pytorch - Build model and inference
- HuggingFace - Using for Transformer architecture
Languages:
- Javascript
- Typescript
- CSS and HTML (markup language)
Frameworks/Libraries:
- VueJS - Web structure
- ElementPlus - UI supports
- Axios - API communicating
- Vite - Optimization
# setup backend
$user cd server
$user python manage.py runserver <host>:<port>
# setup frontend
$user cd web
$user yarn dev --port <web_port>
Hyper-parameters that were used to train the model:
Parameter | Value |
---|---|
No. Epoch | 3 |
Learning rate | 1e-5 (First two epochs), 5e-6 (Last epoch) |
Optimizer | AdamW |
Layers | Full |
The pre-trained model and weight are now available: here. Use this code snippet for inference:
from transformers import T5ForConditionalGeneration, AutoTokenizer
model = T5ForConditionalGeneration.from_pretrained('ndtran/t5-small_cnn-daily-mail')
model.eval()
tokenizer = AutoTokenizer.from_pretrained('t5-small')
generated_ids = model.generate(
tokenizer('summarize: ' + input('Enter your input'), return_tensors = 'pt').input_ids,
do_sample = True,
max_length = 256,
top_k = 1,
temperature = 0.8
)
output = tokenizer.decode(generated_ids.squeeze(), skip_special_tokens = True)
Or use this instance app: Huggingface Space
Feel free to open any issues.