Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions nlp/gpt_j/popxl/finetuning.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -969,6 +969,22 @@
"print(out)\n",
"# [{'generated_text': ' contradiction'}]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "0fd3bd49",
"metadata": {},
"source": [
"## Conclusion\n",
"This notebook has demonstrated how easy it is to perform Fine-Tuning on GPT-J on the Graphcore IPU for a text entailment task. While not as powerful as larger models for free text-generation, medium-size auto-regressive models GPT-J can still be successfully fine-tuned to handle a range of NLP downstream tasks such as question answering, sentiment analysis, and named entity recognition. In fact, for these kind of tasks you don't need GPT-3 175B sized models. GPT-J at 6B has very good language understanding and is ideally suited & highly efficient for most of these scenarios.\n",
"\n",
"In this example we performed fine-tuning on GPT-J as a Causal Language Model (CLM) for Text Entailment on GLUE MNLI dataset.\n",
"\n",
"You can easily adapt this example to do your custom fine-tuning on several downstream tasks, such as question answering, named entity recognition, sentiment analysis, & text classification in general – by preparing your data accordingly.\n",
"\n",
"Overall, this notebook showcases the potential for GPT-J to be used effectively and efficiently for Fine-Tuning. Next, find out how GPT-J can be used effectively and efficiently on several downstream tasks after a simple fine-tuning with our Text generation on IPU using GPT-J – Inference notebook, GPTJ-generative-inference.ipynb."
]
}
],
"metadata": {
Expand Down