notebooks

TensorRT Demo with BERT

To run the demo Jupyter notebooks in this folder, follow the instructions in the TRT setup guide to build and launch the docker container. Then, use your browswer to open the Jupyter lab interface at <host_name>:8888/lab using the password provided in the terminal.

Notebook list:

• BERT-TRT-FP16.ipynb: Step by step walkthrough for building BERT TensorRT FP16 engine.
• BERT-TRT-INT8-QAT-sparse.ipynb: Step by step walkthrough for building BERT TensorRT INT8 engine.
• Q-and-A.ipynb: GUI for Q&A with BERT.
• benchmark.ipynb: GUI for benchmarking TensorRT engines.