How to apply BERT to a custom classification model?

[Scarlettg]

Let's say I'm trying to load a pretrained BERT model from a checkpoint, add layers after it and build the model with Model.compile():

What does the input have to look like if BERT is the first "layer"? Would be nice to have a simple example.

[CyberZHG]

See extract demo.

There are two inputs with shape (batch_size, 512). For token input you should load vocab.txt just like the demo, then map words to indices. The segment input is a zero array if you don't need to compare two parts of the text input.