SmartBERT v1.0

This is a pre-trained language model designed to generate embeddings for smart contract code.

1. The embedding dimension is 768.
2. Trained on over 40,000+ smart contracts.
3. Input should be function-level code snippets extracted from smart contracts.
4. Output is the vector representations of these functions.

Notice:

• SmartBERT was initially trained on a dataset: data.json.
• Subsequently, SmartBERT was trained on an extended dataset of 2000+ vulnerable smart contracts available at: https://github.com/Messi-Q/Smart-Contract-Dataset (Resource 3).

Powered by the RoBERTa model.