Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open Inference Protocol Implementation. #2609

Merged
merged 11 commits into from
Jan 24, 2024
Merged

Commits on Jan 18, 2024

  1. Open Inference Protocol Implementation.

    Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com>
    andyi2it committed Jan 18, 2024
    Configuration menu
    Copy the full SHA
    9db2e30 View commit details
    Browse the repository at this point in the history
  2. added worker status check.

    Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com>
    andyi2it committed Jan 18, 2024
    Configuration menu
    Copy the full SHA
    8b0a37c View commit details
    Browse the repository at this point in the history
  3. torchserve version updated with server metadata.

    Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com>
    andyi2it committed Jan 18, 2024
    Configuration menu
    Copy the full SHA
    f1120a6 View commit details
    Browse the repository at this point in the history
  4. Added OIP e2e test in CI/CD.

        1. Allocated ISVC resource to avoid pod running timeout.
        2. Configured environment variable INFERENCE_PROTOCOL as 'oip'.
        3. Increased `max_wait_time` for pod running.
        4. Deleted ISVC once the test has been passed.
    
    Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com>
    andyi2it committed Jan 18, 2024
    Configuration menu
    Copy the full SHA
    379334d View commit details
    Browse the repository at this point in the history
  5. Addressed review feedback:

        1. Updated environment variable to `TS_OPEN_INFERENCE_PROTOCOL`.
        2. Added logic to read the variable `ts_open_inference_protocol=true` from the property file to determine if OIP is enabled or not.
        3. Implemented extra check for OIP `ModelInferResponse` in GRPC responses.
        4. Utilized local path for the proto file in test_mnist.sh.
    
    Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com>
    andyi2it committed Jan 18, 2024
    Configuration menu
    Copy the full SHA
    c48c214 View commit details
    Browse the repository at this point in the history
  6. Changes from review feedback:

        1. Modified `server live`, `server health`, `model ready` check method.
        2. Added tests `server live`, `server health`, `model ready` for grpc
        3. Added tests `server live`, `server health`, for http
    
    Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com>
    andyi2it committed Jan 18, 2024
    Configuration menu
    Copy the full SHA
    6e1c4d2 View commit details
    Browse the repository at this point in the history
  7. Added a new worker command to handle the KServe OIP inference request.

    Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com>
    andyi2it committed Jan 18, 2024
    Configuration menu
    Copy the full SHA
    c9f1e40 View commit details
    Browse the repository at this point in the history
  8. Added OIPPREDICT worker command in Job to allow oip requests.

    Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com>
    andyi2it committed Jan 18, 2024
    Configuration menu
    Copy the full SHA
    d5975e0 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2024

  1. Format changes.

    Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com>
    andyi2it committed Jan 19, 2024
    Configuration menu
    Copy the full SHA
    08bab0f View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2024

  1. Configuration menu
    Copy the full SHA
    4aa5e91 View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2024

  1. Configuration menu
    Copy the full SHA
    8de21e4 View commit details
    Browse the repository at this point in the history