Skip to content

Commit

Permalink
Add streaming support for openai v0 (#951)
Browse files Browse the repository at this point in the history
* Mark instrumentation points for SDK (#1009)

* Mark instrumentation points for SDK

* Remove duplicated assertion

* Fixup: assert attribute not function

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Prefix conversation id with llm (#1012)

* Change conversation_id->llm.conversation_id

* Fixup formatting

* Add support for Meta Llama2. (#1010)

* Add support for Llama2.

* Fixup: lint errors

* [Mega-Linter] Apply linters fixes

* Trigger tests

---------

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>
Co-authored-by: hmstepanek <hmstepanek@users.noreply.github.com>

* Add bedrock feedback into preview (#1030)

* Add AWS Bedrock testing infrastructure

* Squashed commit of the following:

commit 2834663
Author: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Date:   Mon Oct 9 17:42:05 2023 -0700

    OpenAI Mock Backend (#929)

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Pin flask version for flask restx tests. (#931)

    * Ignore new redis methods. (#932)

    Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>

    * Remove approved paths

    * Update CI Image (#930)

    * Update available python versions in CI

    * Update makefile with overrides

    * Fix default branch detection for arm builds

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Only get package version once (#928)

    * Only get package version once

    * Add disconnect method

    * Add disconnect method

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add datalib dependency for embedding testing.

    * Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Remove approved paths

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Add datalib dependency for embedding testing.

    ---------

    Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
    Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: TimPansino <TimPansino@users.noreply.github.com>
    Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

commit db63d45
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Mon Oct 2 15:31:38 2023 -0700

    Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* Squashed commit of the following:

commit 182c7a8
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Fri Oct 13 10:12:55 2023 -0700

    Add request/ response IDs.

commit f6d13f8
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Thu Oct 12 13:23:39 2023 -0700

    Test cleanup.

commit d057663
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Tue Oct 10 10:23:00 2023 -0700

    Remove commented code.

commit dd29433
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Tue Oct 10 10:19:01 2023 -0700

    Add openai sync instrumentation.

commit 2834663
Author: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Date:   Mon Oct 9 17:42:05 2023 -0700

    OpenAI Mock Backend (#929)

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Pin flask version for flask restx tests. (#931)

    * Ignore new redis methods. (#932)

    Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>

    * Remove approved paths

    * Update CI Image (#930)

    * Update available python versions in CI

    * Update makefile with overrides

    * Fix default branch detection for arm builds

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Only get package version once (#928)

    * Only get package version once

    * Add disconnect method

    * Add disconnect method

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add datalib dependency for embedding testing.

    * Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Remove approved paths

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Add datalib dependency for embedding testing.

    ---------

    Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
    Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: TimPansino <TimPansino@users.noreply.github.com>
    Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

commit db63d45
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Mon Oct 2 15:31:38 2023 -0700

    Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* TEMP

* Bedrock titan extraction nearly complete

* Bedrock Testing Infrastructure (#937)

* Add AWS Bedrock testing infrastructure

* Cache Package Version Lookups (#946)

* Cache _get_package_version

* Add Python 2.7 support to get_package_version caching

* [Mega-Linter] Apply linters fixes

* Bump tests

---------

Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com>
Co-authored-by: TimPansino <TimPansino@users.noreply.github.com>

* Fix Redis Generator Methods (#947)

* Fix scan_iter for redis

* Replace generator methods

* Update instance info instrumentation

* Remove mistake from uninstrumented methods

* Add skip condition to asyncio generator tests

* Add skip condition to asyncio generator tests

---------

Co-authored-by: Lalleh Rafeei <lrafeei@newrelic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Automatic RPM System Updates (#948)

* Checkout old action

* Adding RPM action

* Add dry run

* Incorporating action into workflow

* Wire secret into custom action

* Enable action

* Correct action name

* Fix syntax

* Fix quoting issues

* Drop pre-verification. Does not work on python

* Fix merge artifact

* Remove OpenAI references

---------

Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com>
Co-authored-by: TimPansino <TimPansino@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <lrafeei@newrelic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Cleaning up titan bedrock implementation

* TEMP

* Tests for bedrock passing

Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com>

* Cleaned up titan testing

Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com>
Co-authored-by: Hannah Stepanek <hmstepanek@users.noreply.github.com>

* Parametrized bedrock testing

* Add support for AI21-J2 models

* Change to dynamic no conversation id events

* Drop all openai refs

* [Mega-Linter] Apply linters fixes

* Adding response_id and response_model

* Apply suggestions from code review

* Remove unused import

* Bedrock Sync Chat Completion Instrumentation (#953)

* Add AWS Bedrock testing infrastructure

* Squashed commit of the following:

commit 2834663
Author: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Date:   Mon Oct 9 17:42:05 2023 -0700

    OpenAI Mock Backend (#929)

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Pin flask version for flask restx tests. (#931)

    * Ignore new redis methods. (#932)

    Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>

    * Remove approved paths

    * Update CI Image (#930)

    * Update available python versions in CI

    * Update makefile with overrides

    * Fix default branch detection for arm builds

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Only get package version once (#928)

    * Only get package version once

    * Add disconnect method

    * Add disconnect method

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add datalib dependency for embedding testing.

    * Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Remove approved paths

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Add datalib dependency for embedding testing.

    ---------

    Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
    Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: TimPansino <TimPansino@users.noreply.github.com>
    Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

commit db63d45
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Mon Oct 2 15:31:38 2023 -0700

    Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* Squashed commit of the following:

commit 182c7a8
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Fri Oct 13 10:12:55 2023 -0700

    Add request/ response IDs.

commit f6d13f8
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Thu Oct 12 13:23:39 2023 -0700

    Test cleanup.

commit d057663
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Tue Oct 10 10:23:00 2023 -0700

    Remove commented code.

commit dd29433
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Tue Oct 10 10:19:01 2023 -0700

    Add openai sync instrumentation.

commit 2834663
Author: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Date:   Mon Oct 9 17:42:05 2023 -0700

    OpenAI Mock Backend (#929)

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Pin flask version for flask restx tests. (#931)

    * Ignore new redis methods. (#932)

    Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>

    * Remove approved paths

    * Update CI Image (#930)

    * Update available python versions in CI

    * Update makefile with overrides

    * Fix default branch detection for arm builds

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Only get package version once (#928)

    * Only get package version once

    * Add disconnect method

    * Add disconnect method

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add datalib dependency for embedding testing.

    * Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Remove approved paths

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Add datalib dependency for embedding testing.

    ---------

    Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
    Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: TimPansino <TimPansino@users.noreply.github.com>
    Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

commit db63d45
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Mon Oct 2 15:31:38 2023 -0700

    Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* Cache Package Version Lookups (#946)

* Cache _get_package_version

* Add Python 2.7 support to get_package_version caching

* [Mega-Linter] Apply linters fixes

* Bump tests

---------

Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com>
Co-authored-by: TimPansino <TimPansino@users.noreply.github.com>

* Fix Redis Generator Methods (#947)

* Fix scan_iter for redis

* Replace generator methods

* Update instance info instrumentation

* Remove mistake from uninstrumented methods

* Add skip condition to asyncio generator tests

* Add skip condition to asyncio generator tests

---------

Co-authored-by: Lalleh Rafeei <lrafeei@newrelic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* TEMP

* Automatic RPM System Updates (#948)

* Checkout old action

* Adding RPM action

* Add dry run

* Incorporating action into workflow

* Wire secret into custom action

* Enable action

* Correct action name

* Fix syntax

* Fix quoting issues

* Drop pre-verification. Does not work on python

* Fix merge artifact

* Bedrock titan extraction nearly complete

* Cleaning up titan bedrock implementation

* TEMP

* Tests for bedrock passing

Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com>

* Cleaned up titan testing

Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com>
Co-authored-by: Hannah Stepanek <hmstepanek@users.noreply.github.com>

* Parametrized bedrock testing

* Add support for AI21-J2 models

* Change to dynamic no conversation id events

* Drop all openai refs

* [Mega-Linter] Apply linters fixes

* Adding response_id and response_model

* Drop python 3.7 tests for Hypercorn (#954)

* Apply suggestions from code review

* Remove unused import

---------

Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com>
Co-authored-by: TimPansino <TimPansino@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <lrafeei@newrelic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com>
Co-authored-by: Hannah Stepanek <hmstepanek@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>

* Initial feedback commit for botocore

* Feature bedrock cohere instrumentation (#955)

* Add AWS Bedrock testing infrastructure

* Squashed commit of the following:

commit 2834663
Author: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Date:   Mon Oct 9 17:42:05 2023 -0700

    OpenAI Mock Backend (#929)

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Pin flask version for flask restx tests. (#931)

    * Ignore new redis methods. (#932)

    Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>

    * Remove approved paths

    * Update CI Image (#930)

    * Update available python versions in CI

    * Update makefile with overrides

    * Fix default branch detection for arm builds

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Only get package version once (#928)

    * Only get package version once

    * Add disconnect method

    * Add disconnect method

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add datalib dependency for embedding testing.

    * Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Remove approved paths

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Add datalib dependency for embedding testing.

    ---------

    Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
    Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: TimPansino <TimPansino@users.noreply.github.com>
    Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

commit db63d45
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Mon Oct 2 15:31:38 2023 -0700

    Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* Squashed commit of the following:

commit 182c7a8
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Fri Oct 13 10:12:55 2023 -0700

    Add request/ response IDs.

commit f6d13f8
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Thu Oct 12 13:23:39 2023 -0700

    Test cleanup.

commit d057663
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Tue Oct 10 10:23:00 2023 -0700

    Remove commented code.

commit dd29433
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Tue Oct 10 10:19:01 2023 -0700

    Add openai sync instrumentation.

commit 2834663
Author: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Date:   Mon Oct 9 17:42:05 2023 -0700

    OpenAI Mock Backend (#929)

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Pin flask version for flask restx tests. (#931)

    * Ignore new redis methods. (#932)

    Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>

    * Remove approved paths

    * Update CI Image (#930)

    * Update available python versions in CI

    * Update makefile with overrides

    * Fix default branch detection for arm builds

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Only get package version once (#928)

    * Only get package version once

    * Add disconnect method

    * Add disconnect method

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add datalib dependency for embedding testing.

    * Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Remove approved paths

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Add datalib dependency for embedding testing.

    ---------

    Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
    Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: TimPansino <TimPansino@users.noreply.github.com>
    Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

commit db63d45
Author: Uma Annamalai <uannamalai@newrelic.com>
Date:   Mon Oct 2 15:31:38 2023 -0700

    Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* TEMP

* Bedrock titan extraction nearly complete

* Cleaning up titan bedrock implementation

* TEMP

* Tests for bedrock passing

Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com>

* Cleaned up titan testing

Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com>
Co-authored-by: Hannah Stepanek <hmstepanek@users.noreply.github.com>

* Parametrized bedrock testing

* Add support for AI21-J2 models

* Change to dynamic no conversation id events

* Add cohere model

* Remove openai instrumentation from this branch

* Remove OpenAI from newrelic/config.py

---------

Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
Co-authored-by: Tim Pansino <timpansino@gmail.com>
Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com>
Co-authored-by: Hannah Stepanek <hmstepanek@users.noreply.github.com>

* Bedrock feedback w/ testing for titan and jurassic models

* AWS Bedrock Embedding Instrumentation (#957)

* AWS Bedrock embedding instrumentation

* Correct symbol name

* Add support for bedrock claude (#960)

Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>

* Fix merge conflicts

* Combine Botocore Tests (#959)

* Initial file migration

* Enable DT on all span tests

* Add pytest skip for older botocore versions

* Fixup: app name merge conflict

---------

Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Add to and move feedback tests

* Handle 0.32.0.post1 version in tests (#963)

* Remove response_id dependency in bedrock

* Change API name

* Update moto

* Bedrock Error Tracing (#966)

* Cache Package Version Lookups (#946)

* Cache _get_package_version

* Add Python 2.7 support to get_package_version caching

* [Mega-Linter] Apply linters fixes

* Bump tests

---------

Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com>
Co-authored-by: TimPansino <TimPansino@users.noreply.github.com>

* Fix Redis Generator Methods (#947)

* Fix scan_iter for redis

* Replace generator methods

* Update instance info instrumentation

* Remove mistake from uninstrumented methods

* Add skip condition to asyncio generator tests

* Add skip condition to asyncio generator tests

---------

Co-authored-by: Lalleh Rafeei <lrafeei@newrelic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Automatic RPM System Updates (#948)

* Checkout old action

* Adding RPM action

* Add dry run

* Incorporating action into workflow

* Wire secret into custom action

* Enable action

* Correct action name

* Fix syntax

* Fix quoting issues

* Drop pre-verification. Does not work on python

* Fix merge artifact

* Drop python 3.7 tests for Hypercorn (#954)

* Fix pyenv installation for devcontainer (#936)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Remove duplicate kafka import hook (#956)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Initial bedrock error tracing commit

* Handle 0.32.0.post1 version in tests (#963)

* Add status code to mock bedrock server

* Updating error response recording logic

* Work on bedrock errror tracing

* Chat completion error tracing

* Adding embedding error tracing

* Delete comment

* Update moto

---------

Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com>
Co-authored-by: TimPansino <TimPansino@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <lrafeei@newrelic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>
Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Change ids to match other tests

* move message_ids declaration outside for loop

* Add comment to tox.ini

* Drop py27 from memcache testing.

* Drop pypy27 from memcache testing.

* Update flaskrestx testing #1004

* Remove tastypie 0.14.3 testing

* Remove tastypie 0.14.3 testing

* Remove python 3.12 support (for now)

* Remove untouched files from diff list

---------

Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
Co-authored-by: Tim Pansino <timpansino@gmail.com>
Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com>
Co-authored-by: TimPansino <TimPansino@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com>
Co-authored-by: Hannah Stepanek <hmstepanek@users.noreply.github.com>
Co-authored-by: Hannah Stepanek <hstepanek@newrelic.com>

* Fix instrumentation for openai 1.8.0

* Add LLM attribute to transactions. (#1050)

* Add LLM attr to transactions.

* Remove newlines.

* Add llm attribute to transaction event default attrs list.

* Linting.

* Remove imports for moto on py37. (#1053)

* Update botocore tests. (#1051)

* Update moto decorators in tests.

* Remove py27 botocore dependencies.

* Drop testing for Python 3.7

* Add support for streaming in openai

* Add support for streaming errors

* Support async generators

* Add support for error during streaming

* Ignore v1 tests in v0

* Refactor generator wrappers

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Uma Annamalai <uannamalai@newrelic.com>
Co-authored-by: hmstepanek <hmstepanek@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <84813886+lrafeei@users.noreply.github.com>
Co-authored-by: Tim Pansino <timpansino@gmail.com>
Co-authored-by: Timothy Pansino <11214426+TimPansino@users.noreply.github.com>
Co-authored-by: SlavaSkvortsov <29122694+SlavaSkvortsov@users.noreply.github.com>
Co-authored-by: TimPansino <TimPansino@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <lrafeei@users.noreply.github.com>
  • Loading branch information
10 people committed Feb 23, 2024
1 parent 3bb98d2 commit 12212d3
Show file tree
Hide file tree
Showing 8 changed files with 2,024 additions and 180 deletions.
6 changes: 6 additions & 0 deletions newrelic/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -2062,6 +2062,12 @@ def _process_module_builtin_defaults():
"newrelic.hooks.mlmodel_openai",
"instrument_openai_util",
)
_process_module_definition(
"openai.api_resources.abstract.engine_api_resource",
"newrelic.hooks.mlmodel_openai",
"instrument_openai_api_resources_abstract_engine_api_resource",
)

_process_module_definition(
"openai.resources.chat.completions",
"newrelic.hooks.mlmodel_openai",
Expand Down
677 changes: 517 additions & 160 deletions newrelic/hooks/mlmodel_openai.py

Large diffs are not rendered by default.

1 change: 0 additions & 1 deletion tests/external_botocore/test_botocore_sqs.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@
BOTOCORE_VERSION = get_package_version_tuple("botocore")

url = "sqs.us-east-1.amazonaws.com"

if BOTOCORE_VERSION < (1, 29, 0):
url = "queue.amazonaws.com"

Expand Down
334 changes: 332 additions & 2 deletions tests/mlmodel_openai/_mock_external_openai_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,325 @@
# created by an external call.
# 3) This app runs on a separate thread meaning it won't block the test app.

STREAMED_RESPONSES = {
"Stream parsing error.": [
{
"Content-Type": "text/event-stream",
"openai-model": "gpt-3.5-turbo-0613",
"openai-organization": "new-relic-nkmd8b",
"openai-processing-ms": "516",
"openai-version": "2020-10-01",
"x-ratelimit-limit-requests": "200",
"x-ratelimit-limit-tokens": "40000",
"x-ratelimit-remaining-requests": "199",
"x-ratelimit-remaining-tokens": "39940",
"x-ratelimit-reset-requests": "7m12s",
"x-ratelimit-reset-tokens": "90ms",
"x-request-id": "49dbbffbd3c3f4612aa48def69059ccd",
},
200,
[
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [
{"index": 0, "delta": {"role": "assistant", "content": ""}, "logprobs": None, "finish_reason": None}
],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": "212"}, "logprobs": None, "finish_reason": None}],
},
],
],
"Invalid API key.": [
{"Content-Type": "application/json; charset=utf-8", "x-request-id": "4f8f61a7d0401e42a6760ea2ca2049f6"},
401,
{
"error": {
"message": "Incorrect API key provided: DEADBEEF. You can find your API key at https://platform.openai.com/account/api-keys.",
"type": "invalid_request_error",
"param": None,
"code": "invalid_api_key",
}
},
],
"Model does not exist.": [
{
"Content-Type": "application/json; charset=utf-8",
"x-request-id": "cfdf51fb795362ae578c12a21796262c",
},
404,
{
"error": {
"message": "The model `does-not-exist` does not exist",
"type": "invalid_request_error",
"param": None,
"code": "model_not_found",
}
},
],
"You are a scientist.": [
{
"Content-Type": "text/event-stream",
"openai-model": "gpt-3.5-turbo-0613",
"openai-organization": "new-relic-nkmd8b",
"openai-processing-ms": "516",
"openai-version": "2020-10-01",
"x-ratelimit-limit-requests": "200",
"x-ratelimit-limit-tokens": "40000",
"x-ratelimit-remaining-requests": "199",
"x-ratelimit-remaining-tokens": "39940",
"x-ratelimit-reset-requests": "7m12s",
"x-ratelimit-reset-tokens": "90ms",
"x-request-id": "49dbbffbd3c3f4612aa48def69059ccd",
},
200,
[
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [
{"index": 0, "delta": {"role": "assistant", "content": ""}, "logprobs": None, "finish_reason": None}
],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": "212"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " degrees"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " Fahrenheit"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " is"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " equal"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " to"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " "}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": "100"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " degrees"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " Celsius"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": "."}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {}, "logprobs": None, "finish_reason": "stop"}],
},
],
],
}

RESPONSES_V1 = {
"You are a scientist.": [
{
"Content-Type": "text/event-stream",
"openai-model": "gpt-3.5-turbo-0613",
"openai-organization": "foobar-jtbczk",
"openai-processing-ms": "516",
"openai-version": "2020-10-01",
"x-ratelimit-limit-requests": "200",
"x-ratelimit-limit-tokens": "40000",
"x-ratelimit-remaining-requests": "196",
"x-ratelimit-remaining-tokens": "39880",
"x-ratelimit-reset-requests": "23m5.129s",
"x-ratelimit-reset-tokens": "180ms",
"x-request-id": "5c53c9b80af57a1c9b38568f01dcde7f",
},
200,
[
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [
{"index": 0, "delta": {"role": "assistant", "content": ""}, "logprobs": None, "finish_reason": None}
],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": "212"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " degrees"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " Fahrenheit"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " is"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " equal"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " to"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " "}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": "100"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " degrees"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": " Celsius"}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {"content": "."}, "logprobs": None, "finish_reason": None}],
},
{
"id": "chatcmpl-87sb95K4EF2nuJRcTs43Tm9ntTemv",
"object": "chat.completion.chunk",
"created": 1706565311,
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [{"index": 0, "delta": {}, "logprobs": None, "finish_reason": "stop"}],
},
],
]
}
RESPONSES_V1 = {
"You are a scientist.": [
{
Expand Down Expand Up @@ -380,7 +699,7 @@ def simple_get(openai_version, extract_shortened_prompt):
def _simple_get(self):
content_len = int(self.headers.get("content-length"))
content = json.loads(self.rfile.read(content_len).decode("utf-8"))

stream = content.get("stream", False)
prompt = extract_shortened_prompt(content)
if not prompt:
self.send_response(500)
Expand All @@ -392,6 +711,8 @@ def _simple_get(self):

if openai_version < (1, 0):
mocked_responses = RESPONSES
if stream:
mocked_responses = STREAMED_RESPONSES
else:
mocked_responses = RESPONSES_V1

Expand All @@ -414,7 +735,16 @@ def _simple_get(self):
self.end_headers()

# Send response body
self.wfile.write(json.dumps(response).encode("utf-8"))
if stream and status_code < 400:
for resp in response:
data = json.dumps(resp).encode("utf-8")
if prompt == "Stream parsing error.":
# Force a parsing error by writing an invalid streamed response.
self.wfile.write(b"data: %s" % data)
else:
self.wfile.write(b"data: %s\n\n" % data)
else:
self.wfile.write(json.dumps(response).encode("utf-8"))
return

return _simple_get
Expand Down
Loading

0 comments on commit 12212d3

Please sign in to comment.