- fix 404s due to openvino link structure change
- 2023.3 -> 2024 where neccessary
- spelling fixes

CVS-135106
---------

Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>

* validate class and execute method existence, extend pyovms.Tensor constructor, fix finalize not called issue, print with flush in demos

Fixed bugs in capi benchmark app, documented and created demo showcasing benchmark app features

---------

Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>

* Resolve python node todo's

CVS-132111

* Dump to file flags

smart building depending on the content
parallel tests execution
build performance optimization

By having a verbose flag, it creates ~67,000 lines of messages in the build
logs just for unpacking the boost tar file. This makes it challenging to
audit the build process.

CVS-135755

CVS-135936

CVS-132446

…change (#2393)

CVS-136795

* Allow flag injection to pugixml

This commit contains a patch that adds the variables for the CXX and linker
flags to the CMakeLists.txt file. It then uses the patch during build so that
later we can inject build flags on the cmake command.

* exclude header check
* fix dockerfile sequence
* set ubi as the default base image

---------

Co-authored-by: Steve Grubb <ausearch.1@gmail.com>

* Add string output demo

…MediaPipe stream info (#2395)

* Add support of _contents fields in KServe request input for mediapipe for all deserialization paths

---------

Co-authored-by: atobisze <adrian.tobiszewski@intel.com>

* Fixing references

* Fix internal link

* Expose compiler flags

By default, cmake hides the build process. For auditing purposes, the build
flags need to be exposed. This adds VERBOSE=1 to make commands to get the
desired verbosity.

* Use -DCMAKE_VERBOSE_MAKEFILE=ON

Previous patch was using VERBOSE=1 which does the trick. But for consistency,
let's use -DCMAKE_VERBOSE_MAKEFILE=ON wherever possible.

---------

Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>

cpprestsdk has a bug where it tries to override LD_FLAGS. With cmake, this is
not the right thing to do. You have to use a targeted CMAKE_XXX_LINKER_FLAGS
variable where XXX is what you want to affect. Fortunately this repo patches
the same line and we can correct this design mistake. The patch changes
LD_FLAGS to CMAKE_SHARED_LINKER_FLAGS since we are building a shared object.
You can now see that the flag originally trying to be patched in is visible
in the linking of this part of the build.

Co-authored-by: Dariusz Trawinski <Dariusz.Trawinski@intel.com>

…ponse (#2373)

Co-authored-by: Dariusz Trawinski <Dariusz.Trawinski@intel.com>

* Add unit tests for KFS REST Mediapipe support

* Add deserialization of json string input to OVMS pytensor

*Fix uninitialized value issue in pytensor
*Fix minor leak in Mediapipe fork

Changes in Mediapipe fork:
openvinotoolkit/mediapipe#72

ID:CVS-136076

CVS-138030

The amount of LOC changed is significantly smaller than reported by GitHub due to the fact that a lot of the code is moved from mediapipegraphexecutor.cpp to kfs_graph_executor_impl.cpp.

What really has changed:

* MediapipeGraphExecutor:: infer & inferStream are templatized and do not rely on gRPC KServe types at all
* Added gRPC KServe specialization for MP graph execution (in kfs_graph_executor_impl.h/cpp)
* Two types of packet holders are no longer needed - if pasing the ownership to packet is not needed, just simply passing shared_ptr with no-op destructor.
* Previous unit tests are re-used.

In next PRs: specialization for HTTP types will be added

…2397)

* Add option to summarize benchmark results

* Add sample results summary with option -ps

Command argument to view summary of benchmarking results

* Additional metrics to results summary

* print_summary -ps usage and sample output

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Katarzyna Papis <katarzyna.papis@intel.com>

---------

Co-authored-by: Ravi Panchumarthy <ravi.panchumarthy@intel.com>
Co-authored-by: Katarzyna Papis <katarzyna.papis@intel.com>

Deprecated in previous release, removing now. Use python based LLM demos.

+ removing example tokenizer custom node, since we have tokenizer OV models now.

CVS-136786

Changing the demos and docs since the models were removed from TFHub and added to Kaggle.

CVS-139265

Co-authored-by: ngrozae <104074686+ngrozae@users.noreply.github.com>
Co-authored-by: Damian Kalinowski <damian.kalinowski@intel.com>

* universal_and_benchmark_documentation_updates

* no proxy update

* update benchmark proxy

* add version to ubuntu tag

* revert ubuntu changes

* added localhost

* review

* dockerfile for gradio
* monitoring changes in the documents scope
* preinstall nltk modules
* default security context set to ovms account
* improvements in rag demo

CVS-138032

Implementation of /v3/chat/completions endpoint and forwarding the HTTP message to MediaPipe graph.
The data is std::string now, to be adjusted in following tasks (CVS-139240/CVS-140684).

* CVS-137992_fix_deadline_exceeded_dg2

* add retry for get_model_metadata_request

* add get_model_metadata function

* fix test names

* increase timeout for GetModelStatus

https://jira.devtools.intel.com/browse/CVS-139240
Implementation of chat completion request conversion to HttpPayload struct.

* Fix ovms status to http status conversion

* add-version-to-ubuntu-os

* fix ovms_pkg link

* BASE_OS_DISTRO

* ovms_pkg os

* updates

* DIST_OS added

* adjust nginx build

* fix nginx

* Update Makefile

Co-authored-by: ngrozae <104074686+ngrozae@users.noreply.github.com>

* Update Makefile

Co-authored-by: ngrozae <104074686+ngrozae@users.noreply.github.com>

---------

Co-authored-by: ngrozae <104074686+ngrozae@users.noreply.github.com>

CVS-139231/CVS-139233

This introduces LLM calculator that accepts HTTP OpenAI /v3/chat/completion requests and produces compliant responses.
Working in both - unary and streaming modes.
Bunch of parameters are still marked as TODO, but should be enough to perform benchmarks.

Minimal demo description how to run.

* Add scheduler config in graph options
* Fix centos stream-8

---------

Co-authored-by: Miłosz Żeglarski <milosz.zeglarski@intel.com>
Co-authored-by: ngrozae <104074686+ngrozae@users.noreply.github.com>

CVS-142768

Forwards beam search and multinomial sampling parameters to CB library - this enables returning more than 1 completion for beam search (only for unary)
Adds profiling traces (minitrace)

* Add UTs for llm request conversion

* fix tbb handling for ubuntu20

…2464)

There is an issue (or feature?) that adding generated token to the token cache produces shorter message than previous without newly generated one.

TextStreamer did not expect such behavior.

The fix ignores such event and makes the generation wait for the next tokens.

+ reducing number of response chunks by adding requirement so that chunk needs to include space in order to send cache to the client