[QQC-2474] Expose quality mode through SDK create_project method #1224
Conversation
richardsun0713
force-pushed
the
rsun/QQC-2474
branch
from
19f0a09 to
49da169
Compare
richardsun0713
force-pushed
the
rsun/QQC-2474
branch
from
49da169 to
9569243
Compare
| queue_mode (Optional[QueueMode]): The queue mode to use | ||
| auto_audit_percentage (Optional[float]): The percentage of data rows that will require more than 1 label | ||
| auto_audit_number_of_labels (Optional[float]): Number of labels required for data rows selected for multiple labeling (auto_audit_percentage) | ||
| quality_mode (Optional[QualityMode]): The quality mode to use (e.g. Benchmark, Consensus). Defaults to |
There was a problem hiding this comment.
what happens to existing projects where these attributes are set and, say, we do smth like client.get_project(). I suppose it would work but the attributes will be missing? From the client point of view, is there any migration needs to happen, i.e. if auto_audit_percentage and auto_audit_number_of_labels => set quality mode etcv
There was a problem hiding this comment.
Also if the quality mode default to Benchmark, shouldn't we set it here?
There was a problem hiding this comment.
auto_audit_percentage and auto_audit_number_of_labels are still in the Project model but have no meaning anymore - it has been replaced with equivalents at the batch level (specified during batch creation). The migration has already been performed on the backend, this change simply removes the old obsolete data from the client model
| """ | ||
| media_type = kwargs.get("media_type") | ||
|
|
||
| auto_audit_percentage = kwargs.get("auto_audit_percentage") |
There was a problem hiding this comment.
if these attributes are removed from the model, why are we dealing with them here?
There was a problem hiding this comment.
This is here to help present a more readable message that can inform users how to migrate to the new usage model
| queue_mode (Optional[QueueMode]): The queue mode to use | ||
| auto_audit_percentage (Optional[float]): The percentage of data rows that will require more than 1 label | ||
| auto_audit_number_of_labels (Optional[float]): Number of labels required for data rows selected for multiple labeling (auto_audit_percentage) | ||
| quality_mode (Optional[QualityMode]): The quality mode to use (e.g. Benchmark, Consensus). Defaults to |
There was a problem hiding this comment.
Also if the quality mode default to Benchmark, shouldn't we set it here?
| logger.warning( | ||
| "QueueMode.Dataset will eventually be deprecated, and is no longer " | ||
| "recommended for new projects. Prefer QueueMode.Batch instead.") | ||
| quality_mode = kwargs.get("quality_mode") |
There was a problem hiding this comment.
Looks like dupe lines, since we do the same line 684-..
There was a problem hiding this comment.
These aren't duplicate lines; one exists to inform the user that a default has been selected, and the other exists to translate the given quality_mode to something our backend can interpret
labelbox/client.py
Outdated
| data = kwargs | ||
| data.pop("quality_mode", None) | ||
| if quality_mode is None or quality_mode is QualityMode.Benchmark: | ||
| data["auto_audit_number_of_labels"] = 1 |
There was a problem hiding this comment.
suggest making these numbers constants
labelbox/schema/quality_mode.py
Outdated
| from enum import Enum | ||
|
|
||
|
|
||
| class QualityMode(str): |
There was a problem hiding this comment.
Did you mean QualityMode(str, Enum) ?
auto_audit_percentageandauto_audit_number_of_labelsparameters fromlabelbox.client.Client.create_projectand replaces them with thequality_modeenum