Skip to content

Commit

Permalink
Move llm-question-answering to generate api (#2062)
Browse files Browse the repository at this point in the history
CVS-142802

---------

Co-authored-by: Ekaterina Aidova <ekaterina.aidova@intel.com>
  • Loading branch information
sbalandi and eaidova committed Jun 19, 2024
1 parent 0467fe0 commit 63b5d2c
Show file tree
Hide file tree
Showing 3 changed files with 185 additions and 297 deletions.
1 change: 1 addition & 0 deletions .ci/spellcheck/.pyspelling.wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -399,6 +399,7 @@ llm
LLM
LLMs
LMS
LLMPipeline
logits
LogSoftmax
LoRA
Expand Down
6 changes: 3 additions & 3 deletions notebooks/llm-question-answering/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

LLM stands for “Large Language Model”, which refers to a type of artificial intelligence model that is designed to understand and generate human-like text based on the input it receives. LLMs are trained on large datasets of text to learn patterns, grammar, and semantic relationships, allowing them to generate coherent and contextually relevant responses. One core capability of Large Language Models (LLMs) is to follow natural language instructions. Instruction-following models are capable of generating text in response to prompts and are often used for tasks like writing assistance, chatbots, and content generation.

In this tutorial, we consider how to run an instruction-following text generation pipeline using popular LLMs and OpenVINO. We will use pre-trained models from the [Hugging Face Transformers](https://huggingface.co/docs/transformers/index) library. To simplify the user experience, the [Hugging Face Optimum Intel](https://huggingface.co/docs/optimum/intel/index) library converts the models to OpenVINO™ IR format.
In this tutorial, we consider how to run an instruction-following text generation pipeline using popular LLMs and OpenVINO. We will use pre-trained models from the [Hugging Face Transformers](https://huggingface.co/docs/transformers/index) library and convert them to OpenVINO™ IR format using The [Hugging Face Optimum Intel](https://huggingface.co/docs/optimum/intel/index) library. To simplify the user experience, we will use [OpenVINO Generate API](https://github.com/openvinotoolkit/openvino.genai) for generation of instruction-following inference pipeline.


The tutorial supports different models, you can select one from provided options to compare quality of open source LLM solutions.
Expand All @@ -20,9 +20,9 @@ The available options are:
The tutorial consists of the following steps:

- Install prerequisites
- Download and convert the model from a public source using the [OpenVINO integration with Hugging Face Optimum](https://huggingface.co/blog/openvino).
- Download the model from Hugging Face and convert it to OpenVINO IR via [optimum-intel CLI](https://github.com/huggingface/optimum-intel)
- Compress model weights to INT8 and INT4 with [OpenVINO NNCF](https://github.com/openvinotoolkit/nncf)
- Create an instruction-following inference pipeline
- Create an instruction-following inference pipeline with [OpenVINO Generate API](https://github.com/openvinotoolkit/openvino.genai/blob/master/src/README.md)
- Run instruction-following pipeline

The image below illustrates the provided user instruction and model answer examples.
Expand Down
Loading

0 comments on commit 63b5d2c

Please sign in to comment.