Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add text generation task type #569

Merged
merged 162 commits into from
Jul 26, 2024
Merged

Add text generation task type #569

merged 162 commits into from
Jul 26, 2024

Commits on Apr 19, 2024

  1. copy of Nicks draft for llm guided metric integration tests

    b.nativi committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    1ad92be View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2024

  1. updates to llm guided integration test

    b.nativi committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    e5cc8ad View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    73b09d4 View commit details
    Browse the repository at this point in the history

  3. typo in llm guided integration test

    b.nativi committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    f03fba5 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. LLM_EVALUATION task type, evaluate_llm_output(), first draft of test_… 

    …llm_evaluation()
    b.nativi committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    283eb6e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    0cc5f11 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2024

  1. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    0615a8b View commit details
    Browse the repository at this point in the history

  2. llm_url and llm_api_key added in multiple places in client, api and i… 

    …ntegration tests
    b.nativi committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    660ee71 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. finalize integration test data llm_evaluation

    b.nativi committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    36a9a17 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    d7b53b4 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2024

  1. llm_evaluation functional tests

    b.nativi committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    d2f2ebe View commit details
    Browse the repository at this point in the history

  2. all llm evaluation metrics added

    b.nativi committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    c054b47 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. commit with all tests except new tests passing

    b.nativi committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    0ac0f4c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    af90c95 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. first draft of coherence working

    b.nativi committed May 3, 2024
    Configuration menu
    Copy the full SHA
    aa9c007 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed May 3, 2024
    Configuration menu
    Copy the full SHA
    e1ae78c View commit details
    Browse the repository at this point in the history

Commits on May 6, 2024

  1. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed May 6, 2024
    Configuration menu
    Copy the full SHA
    b0657d1 View commit details
    Browse the repository at this point in the history

  2. updated metric specifications to metrics_to_return

    b.nativi committed May 6, 2024
    Configuration menu
    Copy the full SHA
    631f3f9 View commit details
    Browse the repository at this point in the history

  3. add OpenAIClient class

    b.nativi committed May 6, 2024
    Configuration menu
    Copy the full SHA
    a761d6c View commit details
    Browse the repository at this point in the history

  4. fix on client side to match with metrics_to_return in EvaluationParam… 

    …eters on api side
    b.nativi committed May 6, 2024
    Configuration menu
    Copy the full SHA
    b041eb1 View commit details
    Browse the repository at this point in the history

  5. fix type error

    b.nativi committed May 6, 2024
    Configuration menu
    Copy the full SHA
    7b3d62e View commit details
    Browse the repository at this point in the history

  6. add mocking for coherence

    b.nativi committed May 6, 2024
    Configuration menu
    Copy the full SHA
    99d880b View commit details
    Browse the repository at this point in the history

Commits on May 8, 2024

  1. add test for openai api connection

    b.nativi committed May 8, 2024
    Configuration menu
    Copy the full SHA
    52acfa4 View commit details
    Browse the repository at this point in the history

  2. remove Metric from metric names, improved error handling in coherence

    b.nativi committed May 8, 2024
    Configuration menu
    Copy the full SHA
    36eb7dd View commit details
    Browse the repository at this point in the history

  3. mock openai api connection

    b.nativi committed May 8, 2024
    Configuration menu
    Copy the full SHA
    15c4030 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed May 8, 2024
    Configuration menu
    Copy the full SHA
    c9a29cb View commit details
    Browse the repository at this point in the history

  5. use grouper_id = label.key for llm evaluation

    b.nativi committed May 8, 2024
    Configuration menu
    Copy the full SHA
    72aaa7e View commit details
    Browse the repository at this point in the history

  6. change CoherenceMetric from using label_key to using label

    b.nativi committed May 8, 2024
    Configuration menu
    Copy the full SHA
    d31a26c View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed May 8, 2024
    Configuration menu
    Copy the full SHA
    cbe79ec View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed May 10, 2024
    Configuration menu
    Copy the full SHA
    ddcb7e3 View commit details
    Browse the repository at this point in the history

Commits on May 15, 2024

  1. text generation examples

    b.nativi committed May 15, 2024
    Configuration menu
    Copy the full SHA
    ffef625 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed May 15, 2024
    Configuration menu
    Copy the full SHA
    5a4fee8 View commit details
    Browse the repository at this point in the history

  3. rename misnamed qa folder to text-generation

    b.nativi committed May 15, 2024
    Configuration menu
    Copy the full SHA
    95b25dc View commit details
    Browse the repository at this point in the history

  4. remove .txt from example

    b.nativi committed May 15, 2024
    Configuration menu
    Copy the full SHA
    544b401 View commit details
    Browse the repository at this point in the history

Commits on May 16, 2024

  1. add all three use case examples for text generation

    b.nativi committed May 16, 2024
    Configuration menu
    Copy the full SHA
    8a3a159 View commit details
    Browse the repository at this point in the history

Commits on May 17, 2024

  1. update text generation examples

    b.nativi committed May 17, 2024
    Configuration menu
    Copy the full SHA
    061065c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed May 17, 2024
    Configuration menu
    Copy the full SHA
    63d10a8 View commit details
    Browse the repository at this point in the history

  3. rename llm_evaluation to text_generation

    b.nativi committed May 17, 2024
    Configuration menu
    Copy the full SHA
    c7f923a View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. update datum and annotation to work with text gen, functional tests p… 

    …assing
    b.nativi committed May 22, 2024
    Configuration menu
    Copy the full SHA
    92abc09 View commit details
    Browse the repository at this point in the history

  2. add client side unit tests for Annotation and Datum

    b.nativi committed May 22, 2024
    Configuration menu
    Copy the full SHA
    c927e3f View commit details
    Browse the repository at this point in the history

  3. fix github checks error for Context

    b.nativi committed May 22, 2024
    Configuration menu
    Copy the full SHA
    9224ce7 View commit details
    Browse the repository at this point in the history

  4. attempt number two to fix github checks issue with Context

    b.nativi committed May 22, 2024
    Configuration menu
    Copy the full SHA
    0e701f4 View commit details
    Browse the repository at this point in the history

  5. remove unused create_datums_with_dataset_names() method

    b.nativi committed May 22, 2024
    Configuration menu
    Copy the full SHA
    168867f View commit details
    Browse the repository at this point in the history

  6. rename QAG to Summarization metric

    b.nativi committed May 22, 2024
    Configuration menu
    Copy the full SHA
    092bf7a View commit details
    Browse the repository at this point in the history

Commits on May 23, 2024

  1. add LLMClient parent class

    b.nativi committed May 23, 2024
    Configuration menu
    Copy the full SHA
    59fca69 View commit details
    Browse the repository at this point in the history

  2. add MistralValorClient

    b.nativi committed May 23, 2024
    Configuration menu
    Copy the full SHA
    45118d3 View commit details
    Browse the repository at this point in the history

  3. add mistralai client dependency

    b.nativi committed May 23, 2024
    Configuration menu
    Copy the full SHA
    9ea2b82 View commit details
    Browse the repository at this point in the history

Commits on May 24, 2024

  1. add support for mistral client in valor_api text_generation.py

    b.nativi committed May 24, 2024
    Configuration menu
    Copy the full SHA
    de9911b View commit details
    Browse the repository at this point in the history

  2. fix type error for EvaluationParameters.llm_api_params

    b.nativi committed May 24, 2024
    Configuration menu
    Copy the full SHA
    8fd9ada View commit details
    Browse the repository at this point in the history

Commits on May 29, 2024

  1. temporarily comment out client text gen test until ready

    b.nativi committed May 29, 2024
    Configuration menu
    Copy the full SHA
    db85df5 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2024

  1. Merge branch 'main' into add_llm_guided_metrics

    @ntlind
    ntlind committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    c57a464 View commit details
    Browse the repository at this point in the history

  2. fix all tests except test_text_generation.py

    @ntlind
    ntlind committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    ae39f0a View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2024

  1. Merge branch 'main' into add_llm_guided_metrics

    @ntlind
    ntlind committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    b1f8405 View commit details
    Browse the repository at this point in the history

Commits on Jun 17, 2024

  1. Merge branch 'main' into add_llm_guided_metrics

    @ntlind
    ntlind committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    191cee2 View commit details
    Browse the repository at this point in the history

  2. fix text generation functional tests

    b.nativi committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    bd4f92a View commit details
    Browse the repository at this point in the history

  3. add metric_params to EvaluationParameters in api and client

    b.nativi committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    70ecee7 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    3616e5e View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. Merge branch 'main' into add_llm_guided_metrics

    @ntlind
    ntlind committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    de34b88 View commit details
    Browse the repository at this point in the history

  2. text gen metric names added to client

    b.nativi committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    334442b View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'refs/remotes/origin/add_llm_guided_metr… 

    …ics' into add_llm_guided_metrics
    b.nativi committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    b4f7fb5 View commit details
    Browse the repository at this point in the history

  4. fix Filter issues in text generation

    b.nativi committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    6c11315 View commit details
    Browse the repository at this point in the history

  5. fix filter issue for text generation in coretypes

    b.nativi committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    118dafe View commit details
    Browse the repository at this point in the history

  6. remove label_id from db_mapping of text gen metrics

    b.nativi committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    091df90 View commit details
    Browse the repository at this point in the history

  7. isort pre-commit error in evaluation.py

    b.nativi committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    53d51f4 View commit details
    Browse the repository at this point in the history

  8. text gen integration test passing

    b.nativi committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    dd476c1 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. mock client integration test for text generation

    b.nativi committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    9ee7c50 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. cleanup text generation code

    b.nativi committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    dd7188a View commit details
    Browse the repository at this point in the history

  2. add minor PR improvements

    b.nativi committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    34520b3 View commit details
    Browse the repository at this point in the history

  3. Add text comparison metrics to Ben's PR (#632)

    @ntlind
    ntlind committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    de9be2e View commit details
    Browse the repository at this point in the history

  4. remove a few todos in llm_call.py

    b.nativi committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    aa19435 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'add_llm_guided_metrics' of github.com:Striveworks/valor… 

    … into add_llm_guided_metrics
    b.nativi committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    cbcbcc8 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. move llm_client and fix pre-commit config and import issue

    @ntlind
    ntlind committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    9bef346 View commit details
    Browse the repository at this point in the history

  2. part 1 address Nicks comments

    b.nativi committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    543bea8 View commit details
    Browse the repository at this point in the history

  3. rename llm_call to llm_clients

    b.nativi committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    70b44fa View commit details
    Browse the repository at this point in the history

  4. Increase code coverage to 90% (#633)

    @ntlind
    ntlind committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    faee691 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. add answer relevance metric to api

    b.nativi committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    91b2f6a View commit details
    Browse the repository at this point in the history

  2. add answer relevance to integration test

    b.nativi committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    155d343 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into add_llm_guided_metrics

    @ntlind
    ntlind committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    a17d78d View commit details
    Browse the repository at this point in the history

  4. additional test coverage for answer relevance metric

    b.nativi committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    7c32c64 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'add_llm_guided_metrics' of github.com:Striveworks/valor… 

    … into add_llm_guided_metrics
    b.nativi committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    2b87c9d View commit details
    Browse the repository at this point in the history

  6. test mistral and invalid clients in text_generation.py

    b.nativi committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    eb8d171 View commit details
    Browse the repository at this point in the history

  7. remove unimplemented metrics

    b.nativi committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    58167d3 View commit details
    Browse the repository at this point in the history

  8. Add docs (#634)

    @ntlind
    ntlind committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    93aed41 View commit details
    Browse the repository at this point in the history

  9. cleanup of text_generation.py, validation for AnswerRelevance and Coh… 

    …erence metrics
    b.nativi committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    5f3adbe View commit details
    Browse the repository at this point in the history

  10. partial support for BLEU and ROUGE metrics in client

    b.nativi committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    58c13cf View commit details
    Browse the repository at this point in the history

  11. cut down example notebook for text generation

    b.nativi committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    efd4be5 View commit details
    Browse the repository at this point in the history

  12. test BLEU and ROUGE metrics in client, test invalid api key

    b.nativi committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    ca5c273 View commit details
    Browse the repository at this point in the history

  13. Coherence and AnswerRelevance added to the docs

    b.nativi committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    3a6e20b View commit details
    Browse the repository at this point in the history

  14. fix client.connect() test for llm guided metrics

    b.nativi committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    0d0ec90 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    e9b0655 View commit details
    Browse the repository at this point in the history

  2. VALOR_VERSION -> VERSION

    @ntlind
    ntlind committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    7b99188 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. cleanup for answer relevance llm response error handling, testing for… 

    … that and for bleu weights validation
    b.nativi committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    da79792 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'add_llm_guided_metrics' of github.com:Striveworks/valor… 

    … into add_llm_guided_metrics
    b.nativi committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    4c251a7 View commit details
    Browse the repository at this point in the history

  3. unit tests for AnswerRelevance and Coherence

    b.nativi committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    c372803 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. MockLLMClient added and used in integration tests, BLEU and ROUGE add… 

    …ed to integration test
    b.nativi committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    d7d6366 View commit details
    Browse the repository at this point in the history

  2. remove EvaluationParameters.metric_params

    b.nativi committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    52162e5 View commit details
    Browse the repository at this point in the history

  3. add ROUGE parameters

    b.nativi committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    966c044 View commit details
    Browse the repository at this point in the history

  4. remove asserts from text gen production code

    b.nativi committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    7e4fb48 View commit details
    Browse the repository at this point in the history

  5. remove Text and Context

    b.nativi committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    30b5dde View commit details
    Browse the repository at this point in the history

  6. fix pre-commit issue in test_text_generation.py

    b.nativi committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    ffa66c8 View commit details
    Browse the repository at this point in the history

  7. add integration tests for openai and mistral external api calls

    b.nativi committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    a39fc91 View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. testing external tests 1

    b.nativi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    b95f1cf View commit details
    Browse the repository at this point in the history

  2. testing external tests 2

    b.nativi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    51f8a16 View commit details
    Browse the repository at this point in the history

  3. testing external tests 3

    b.nativi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    6e8b001 View commit details
    Browse the repository at this point in the history

  4. testing external tests 3

    b.nativi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    47d4594 View commit details
    Browse the repository at this point in the history

  5. testing external tests 5

    b.nativi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    913b4e8 View commit details
    Browse the repository at this point in the history

  6. testing external tests failing

    b.nativi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    40a4690 View commit details
    Browse the repository at this point in the history

  7. secrets in tests-and-coverage.yml

    b.nativi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    c7eb936 View commit details
    Browse the repository at this point in the history

  8. add manual api key specification in llm_api_params

    b.nativi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    5cbfd7a View commit details
    Browse the repository at this point in the history

  9. improve code coverage on valor_api text_generation.py

    b.nativi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    e9ccbad View commit details
    Browse the repository at this point in the history

  10. test that external integrations tests dont run on other branch pushes

    b.nativi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    1a384fb View commit details
    Browse the repository at this point in the history

  11. external integration tests fail when no OPENAI_API_KEY is set

    b.nativi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    95864e9 View commit details
    Browse the repository at this point in the history

  12. the normal integration tests cant access the external API keys

    b.nativi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    fc06423 View commit details
    Browse the repository at this point in the history

  13. rename files, integration tests should pass

    b.nativi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    a7e9742 View commit details
    Browse the repository at this point in the history

  14. code coverage and cleanup

    b.nativi committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    ac5ae43 View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2024

  1. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    5206e15 View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2024

  1. improved validation for llm messages

    b.nativi committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    ee24191 View commit details
    Browse the repository at this point in the history

  2. merge with main

    b.nativi committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    3451558 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2024

  1. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    d1f0c05 View commit details
    Browse the repository at this point in the history

  2. metric_params for evaluate_text_generation() on client side, other co… 

    …de coverage improvements
    b.nativi committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    bfaa830 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    696dcbb View commit details
    Browse the repository at this point in the history

  4. fix in test_groundtruth.py

    b.nativi committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    903bf1e View commit details
    Browse the repository at this point in the history

  5. update metric docs

    b.nativi committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    b8b9864 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. add Context class for Annotation.context

    b.nativi committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    e9b088c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    0498612 View commit details
    Browse the repository at this point in the history

  3. example notebook working for RAG and summarization

    b.nativi committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    8232bbd View commit details
    Browse the repository at this point in the history

  4. merge with main

    b.nativi committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    096b1fe View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2024

  1. add ROUGEType to enums, cleanup tests, address some comments

    b.nativi committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    aec4b63 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    4461286 View commit details
    Browse the repository at this point in the history

  3. addressing comments

    b.nativi committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    28e08b7 View commit details
    Browse the repository at this point in the history

  4. fix outdated code in coretypes.py where datasets was optional

    b.nativi committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    3f99c62 View commit details
    Browse the repository at this point in the history

  5. fix tex gen metrics didnt work if no groundtruth labels

    b.nativi committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    420b244 View commit details
    Browse the repository at this point in the history

  6. text gen example notebook working

    b.nativi committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    530cca1 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. update external test mistral model to latest

    b.nativi committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    2579971 View commit details
    Browse the repository at this point in the history

  2. merge with main

    b.nativi committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    637ac23 View commit details
    Browse the repository at this point in the history

  3. address comments with Nick

    b.nativi committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    f344d77 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    67f27d6 View commit details
    Browse the repository at this point in the history

  5. improvements to text gen integration tests

    b.nativi committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    f3cabc6 View commit details
    Browse the repository at this point in the history

  6. improvements to example notebook, test_text_generation.py

    b.nativi committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    ba5eb41 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2024

  1. fix filters for text generation

    b.nativi committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    7cdaa37 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2024

  1. merge with main, datum filtering is no longer functional for text gen… 

    …eration
    b.nativi committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    0375bae View commit details
    Browse the repository at this point in the history

  2. add datum_filtering back into text generation metrics

    b.nativi committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    e7d2179 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. try removing slim for alpine

    b.nativi committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    d87c05e View commit details
    Browse the repository at this point in the history

  2. switch back to slim

    b.nativi committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    e7823b0 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. revert valor_api/backend/core/evaluation.py

    b.nativi committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    1acda02 View commit details
    Browse the repository at this point in the history

  2. comment out all text gen tests

    b.nativi committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    53b87fc View commit details
    Browse the repository at this point in the history

  3. revert valor_api/backend/models.py and migrations

    b.nativi committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    4014a62 View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2024

  1. Revert "revert valor_api/backend/models.py and migrations" 

    This reverts commit 4014a62.
    b.nativi committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    02cbd52 View commit details
    Browse the repository at this point in the history

  2. Revert "revert valor_api/backend/core/evaluation.py" 

    This reverts commit 1acda02.
    b.nativi committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    bbe0458 View commit details
    Browse the repository at this point in the history

  3. set object detection benchmark timeout at 60

    b.nativi committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    e13a1fd View commit details
    Browse the repository at this point in the history

  4. uncomment text gen tests

    b.nativi committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    88e8e0c View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into add_llm_guided_metrics

    b.nativi committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    e601217 View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. Add more llm guided metrics (#677) 

    Co-authored-by: b.nativi <b.nativi@lambda002.america.striveworks.us>
    @bnativi
    bnativi and b.nativi committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    be74b2a View commit details
    Browse the repository at this point in the history

  2. merge with main

    b.nativi committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    03ddb6b View commit details
    Browse the repository at this point in the history

  3. set object detection benchmark timeout back to the same time as main

    b.nativi committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    fcc277b View commit details
    Browse the repository at this point in the history

  4. reverted filter preparation

    @czaloom
    czaloom committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    5527262 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'add_llm_guided_metrics' of github.com:Striveworks/valor… 

    … into add_llm_guided_metrics
    @czaloom
    czaloom committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    4d3e1c3 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. code coverage

    b.nativi committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    6ea9d01 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'text_comparison_llm_guided_metrics' into add_llm_guided… 

    …_metrics
    b.nativi committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    67442ca View commit details
    Browse the repository at this point in the history

  3. revert removal of _format_context()

    b.nativi committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    3755cac View commit details
    Browse the repository at this point in the history

  4. Revert "Merge branch 'add_llm_guided_metrics' of github.com:Strivewor… 

    …ks/valor into add_llm_guided_metrics"

This reverts commit 4d3e1c3, reversing
changes made to 5527262.
    b.nativi committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    0baa229 View commit details
    Browse the repository at this point in the history

  5. revert to last stable branch

    b.nativi committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    efbbe20 View commit details
    Browse the repository at this point in the history