This repository was archived by the owner on Jul 4, 2025. It is now read-only.
fix: check model status before inferencing #1864
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe Your Changes
This pull request introduces several changes to the
InferenceServiceclass and related methods to improve model handling and logging. The most important changes include adding logic to handle model loading, updating logging levels, and modifying the storage of saved models.Improvements to model handling and storage:
engine/services/inference_service.cc: Added logic to check if a model is loaded and start loading it if not. This includes retrieving the model status and initiating the model loading process if necessary.engine/services/inference_service.cc: Removed redundant code for retrieving themodel_idinside a nested block and consolidated it at the beginning of the function.engine/services/inference_service.cc: Added logic to save models in thesaved_models_map when they are loaded, ensuring they can be reused later.engine/services/inference_service.h: Introduced a newSavedModeltype and anunordered_mapto store saved models, facilitating efficient model retrieval.Logging improvements:
engine/services/inference_service.cc: Changed the logging level fromCTL_INFtoCTL_DBGfor the JSON body inference message to reduce log verbosity.Fixes Issues
Self Checklist