Conversation
…rier worker tasks The background tracker update had a quadratic delay problem (progressive delay formula created ~54h total sleep for 3.9k trackers, exceeding the 2h crontab). This replaces it with a two-tier architecture: - Periodic dispatcher groups trackers by carrier and enqueues per-carrier Huey sub-tasks - Per-carrier worker processes batches of 10 with flat delays and incremental saves - Carrier isolation prevents slow carriers from blocking others - Results saved after each batch instead of after all batches complete Scaling: 10k trackers across 5 carriers completes in ~10 minutes vs days before.
Remove legacy backward-compatibility functions (create_request_batches, fetch_tracking_info, save_tracing_records, save_updated_trackers) and unused type aliases. Add Huey lock_task to the dispatcher to prevent duplicate runs when overdue periodic tasks queue up at worker startup.
Fix Cache.get() writing None to system cache instead of actual value when syncing from shallow cache. Reduce DHL Parcel DE buffer_minutes from 30 to 5 to prevent tokens from being considered immediately expired when expires_in <= 1800s.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
| try: | ||
| rate_sheet.add_zone(zone_dict) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"zone": str(e)}) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
This autofix suggestion was applied.
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 19 days ago
In general, to fix information exposure via exceptions you should avoid sending raw exception messages or stack traces back to the client. Instead, log the detailed error on the server (including stack trace if needed) and return a generic or pre-defined, non-sensitive error message to the user. This maintains debuggability while preventing leakage of internal implementation details.
For this specific code, the best fix without altering existing functionality semantics is:
- Keep treating a
ValueErrorfromrate_sheet.add_zoneas a validation problem and still raiseexceptions.ValidationErrorso the client continues to see a validation error on thezonefield. - Replace
str(e)with a generic, controlled error message (for example “Invalid zone data.”) that does not depend on the exception text. - Log the original exception and stack trace using the existing
loggerimported at the top of the file, similar to how it’s done inInstanceConfigMutation.mutate. This preserves diagnostic information for developers without exposing it to users.
Concretely, in modules/admin/karrio/server/admin/schemas/base/mutations.py, inside AddSharedZoneMutation.mutate, change the except ValueError as e: block to call logger.error with exc_info=True, then raise exceptions.ValidationError with a safe, static message for the zone key. No new imports are needed because logger is already imported in this file and exceptions is already imported in this function.
| @@ -567,7 +567,8 @@ | ||
| try: | ||
| rate_sheet.add_zone(zone_dict) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"zone": str(e)}) | ||
| logger.error("Failed to add shared zone", error=str(e), exc_info=True) | ||
| raise exceptions.ValidationError({"zone": "Invalid zone data."}) | ||
|
|
||
| return AddSharedZoneMutation(rate_sheet=rate_sheet) | ||
|
|
| try: | ||
| rate_sheet.update_zone(input["zone_id"], zone_dict) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"zone_id": str(e)}) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 19 days ago
To fix the issue, we should avoid passing the raw exception message str(e) directly into the ValidationError that is returned to the client. Instead, we should generate a controlled, user-safe message that does not depend on the internal exception text. We can still log the original exception server-side to preserve debugging value.
The best minimal fix without changing existing behavior patterns is:
- Replace
raise exceptions.ValidationError({"zone_id": str(e)})with aValidationErrorthat uses a generic, safe message such as"Invalid zone"or"Invalid zone ID". - Optionally, log the original exception
ewith the existinglogger(already imported) for server-side diagnostics.
Concretely, in modules/admin/karrio/server/admin/schemas/base/mutations.py:
- Around line 494 in
UpdateSharedZoneMutation.mutate, change the except block to raiseValidationErrorwith a generic string, and log the exception. - Similarly, apply the same pattern to the closely related
AddSharedZoneMutationandDeleteSharedZoneMutationblocks (lines 468 and 518) so that all three mutations treatValueErrorconsistently and safely.
No new imports are required; logger and exceptions are already available in this file.
| @@ -465,7 +465,8 @@ | ||
| try: | ||
| rate_sheet.add_zone(zone_dict) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"zone": str(e)}) | ||
| logger.exception("Failed to add shared zone") | ||
| raise exceptions.ValidationError({"zone": "Invalid zone data."}) | ||
|
|
||
| return AddSharedZoneMutation(rate_sheet=rate_sheet) | ||
|
|
||
| @@ -491,7 +492,8 @@ | ||
| try: | ||
| rate_sheet.update_zone(input["zone_id"], zone_dict) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"zone_id": str(e)}) | ||
| logger.exception("Failed to update shared zone") | ||
| raise exceptions.ValidationError({"zone_id": "Invalid zone ID or data."}) | ||
|
|
||
| return UpdateSharedZoneMutation(rate_sheet=rate_sheet) | ||
|
|
||
| @@ -515,7 +517,8 @@ | ||
| try: | ||
| rate_sheet.remove_zone(input["zone_id"]) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"zone_id": str(e)}) | ||
| logger.exception("Failed to delete shared zone") | ||
| raise exceptions.ValidationError({"zone_id": "Invalid zone ID."}) | ||
|
|
||
| return DeleteSharedZoneMutation(rate_sheet=rate_sheet) | ||
|
|
| try: | ||
| rate_sheet.remove_zone(input["zone_id"]) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"zone_id": str(e)}) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 19 days ago
In general, to fix this kind of problem you should avoid returning raw exception text (or any data derived directly from exception objects) to the client. Instead, log the exception on the server (including stack trace if useful) and return a generic, user-safe error message or a controlled, pre-defined set of messages.
For this specific case in DeleteSharedZoneMutation.mutate, the best fix is to replace {"zone_id": str(e)} with a generic message such as {"zone_id": "Invalid zone id"} or, if you need to preserve some context, a message that does not expose internals (for example, “Unable to delete zone with the given id.”). Since the file already imports logger and uses it earlier, a robust fix also logs the exception before raising the ValidationError, so developers still see the underlying details in server logs while clients only see the generic message.
Concretely:
- In
modules/admin/karrio/server/admin/schemas/base/mutations.py, withinDeleteSharedZoneMutation.mutate, update theexcept ValueError as e:block (lines 517–518). - Add a call to
logger.errorsimilar to the earlier pattern inInstanceConfigMutation, passingexc_info=True. - Change the
ValidationErrorconstruction to use a fixed, generic string instead ofstr(e).
No additional imports are needed because logger and exceptions are already available in this file/scope.
| @@ -515,7 +515,8 @@ | ||
| try: | ||
| rate_sheet.remove_zone(input["zone_id"]) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"zone_id": str(e)}) | ||
| logger.error("Failed to delete shared zone", error=str(e), exc_info=True) | ||
| raise exceptions.ValidationError({"zone_id": "Invalid zone id"}) | ||
|
|
||
| return DeleteSharedZoneMutation(rate_sheet=rate_sheet) | ||
|
|
| try: | ||
| rate_sheet.add_surcharge(surcharge_dict) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"surcharge": str(e)}) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 19 days ago
To fix the problem, do not propagate the raw exception message str(e) back to the client. Instead, log the full exception (including stack trace) server-side and return a generic, non-sensitive error message to the client. This keeps debugging information available to developers while avoiding leaking internal details.
Concretely, in AddSharedSurchargeMutation.mutate, replace raise exceptions.ValidationError({"surcharge": str(e)}) with a logger.error(...) call capturing the exception and then raise a ValidationError with a generic message such as "Invalid surcharge configuration.". We already have logger imported at the top of the file, so no new imports are needed. All changes are limited to modules/admin/karrio/server/admin/schemas/base/mutations.py, specifically the AddSharedSurchargeMutation block around lines 546–550. No behavior changes are made beyond message content: callers still get a ValidationError on the surcharge field, but without the internal exception text.
| @@ -546,7 +546,14 @@ | ||
| try: | ||
| rate_sheet.add_surcharge(surcharge_dict) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"surcharge": str(e)}) | ||
| logger.error( | ||
| "Failed to add shared surcharge", | ||
| extra={"rate_sheet_id": input.get("rate_sheet_id"), "error": str(e)}, | ||
| exc_info=True, | ||
| ) | ||
| raise exceptions.ValidationError( | ||
| {"surcharge": "Invalid surcharge configuration."} | ||
| ) | ||
|
|
||
| return AddSharedSurchargeMutation(rate_sheet=rate_sheet) | ||
|
|
| try: | ||
| rate_sheet.update_surcharge(input["surcharge_id"], surcharge_dict) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"surcharge_id": str(e)}) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 19 days ago
To fix this issue, avoid returning the raw ValueError message (str(e)) directly to the client. Instead, either (a) map known error conditions to controlled, non-sensitive messages, or (b) return a generic validation error and log the detailed exception server-side. This ensures that no internal implementation details or potentially sensitive data in exception messages are exposed to external users, while still allowing developers to investigate errors.
The simplest, least intrusive fix here is to replace {"surcharge": str(e)} and {"surcharge_id": str(e)} with generic messages, and log the exception using the existing logger import already present at the top of the file. Concretely:
- In
AddSharedSurchargeMutation.mutate, inside theexcept ValueError as e:block, log the exception and raiseValidationErrorwith a generic message for thesurchargefield. - In
UpdateSharedSurchargeMutation.mutate, log the exception and raiseValidationErrorwith a generic message forsurcharge_id. - In
DeleteSharedSurchargeMutation.mutate, do the same.
We don’t need new imports: logger and exceptions are already available. Functionality from the client’s perspective remains that an error is returned when the operation fails, but the exact internal exception message is no longer exposed; instead, a safe, descriptive message is returned and the detailed error is preserved in server logs.
| @@ -546,7 +546,10 @@ | ||
| try: | ||
| rate_sheet.add_surcharge(surcharge_dict) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"surcharge": str(e)}) | ||
| logger.exception("Failed to add shared surcharge to rate sheet %s", rate_sheet.id) | ||
| raise exceptions.ValidationError( | ||
| {"surcharge": "Invalid surcharge configuration."} | ||
| ) | ||
|
|
||
| return AddSharedSurchargeMutation(rate_sheet=rate_sheet) | ||
|
|
||
| @@ -572,7 +575,14 @@ | ||
| try: | ||
| rate_sheet.update_surcharge(input["surcharge_id"], surcharge_dict) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"surcharge_id": str(e)}) | ||
| logger.exception( | ||
| "Failed to update shared surcharge %s on rate sheet %s", | ||
| input.get("surcharge_id"), | ||
| rate_sheet.id, | ||
| ) | ||
| raise exceptions.ValidationError( | ||
| {"surcharge_id": "Invalid surcharge identifier or configuration."} | ||
| ) | ||
|
|
||
| return UpdateSharedSurchargeMutation(rate_sheet=rate_sheet) | ||
|
|
||
| @@ -596,7 +606,14 @@ | ||
| try: | ||
| rate_sheet.remove_surcharge(input["surcharge_id"]) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"surcharge_id": str(e)}) | ||
| logger.exception( | ||
| "Failed to delete shared surcharge %s from rate sheet %s", | ||
| input.get("surcharge_id"), | ||
| rate_sheet.id, | ||
| ) | ||
| raise exceptions.ValidationError( | ||
| {"surcharge_id": "Invalid surcharge identifier."} | ||
| ) | ||
|
|
||
| return DeleteSharedSurchargeMutation(rate_sheet=rate_sheet) | ||
|
|
| try: | ||
| rate_sheet.remove_surcharge(input["surcharge_id"]) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"surcharge_id": str(e)}) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 19 days ago
In general, to avoid information exposure through exceptions, you should not pass raw exception messages directly to client-facing error objects. Instead, log the original exception (and stack trace if useful) on the server, and return a generic, user-safe error message (e.g., "Invalid surcharge ID"). This keeps detailed diagnostic information for developers while preventing disclosure of internal details.
In this file, the problematic pattern appears in the DeleteSharedSurchargeMutation.mutate method:
try:
rate_sheet.remove_surcharge(input["surcharge_id"])
except ValueError as e:
raise exceptions.ValidationError({"surcharge_id": str(e)})The best targeted fix without changing functionality is to replace str(e) with a generic, controlled message. Because the error is attached to the "surcharge_id" field, a suitable message is something like "Invalid surcharge_id" or "Invalid surcharge ID" or a short phrase that doesn’t include internal data. To keep the code simple and avoid extra dependencies, we can just hard-code a clear but generic string. If you want to preserve diagnostics, you can separately log e using the already-imported logger, but that is optional from the security perspective.
Concretely:
- In
DeleteSharedSurchargeMutation.mutate, change theexcept ValueErrorblock to raiseValidationErrorwith a generic message instead ofstr(e), e.g.:
except ValueError as e:
logger.warning("Failed to remove surcharge %s from rate_sheet %s: %s",
input.get("surcharge_id"), input.get("rate_sheet_id"), e)
raise exceptions.ValidationError({"surcharge_id": "Invalid surcharge ID"})However, since the instructions emphasize minimal functional change, I will only swap the exposed message and not add logging unless necessary. The imports already include logger, so no new imports are needed.
No other blocks in the provided snippet are flagged, so we only adjust that one line (and its immediate except block) in DeleteSharedSurchargeMutation.
| @@ -595,8 +595,9 @@ | ||
|
|
||
| try: | ||
| rate_sheet.remove_surcharge(input["surcharge_id"]) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"surcharge_id": str(e)}) | ||
| except ValueError: | ||
| # Return a generic validation message to avoid exposing internal error details | ||
| raise exceptions.ValidationError({"surcharge_id": "Invalid surcharge ID"}) | ||
|
|
||
| return DeleteSharedSurchargeMutation(rate_sheet=rate_sheet) | ||
|
|
| try: | ||
| rate_sheet.batch_update_surcharges(surcharges) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"surcharges": str(e)}) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 19 days ago
In general, to fix this problem you should avoid passing raw exception messages from internal code directly to the client. Instead, log the detailed error on the server (including the exception and stack trace if needed), and return either a generic message or a constrained, safe message that doesn’t reveal internal implementation details.
For this specific case, the best minimal fix is to:
- Log the full exception server-side when
batch_update_surchargesraisesValueError. - Replace
str(e)in the API response with a generic, client-safe message such as"Invalid surcharges data."or a similarly neutral message. This preserves the fact that the client gets a validation error on thesurchargesfield but removes exposure of internal exception text.
Concretely, in modules/admin/karrio/server/admin/schemas/base/mutations.py, in the BatchUpdateSurchargesMutation.mutate method:
- Update the
except ValueError as e:block to:- Log the error via the already-imported
logger(fromkarrio.server.core.logging). - Raise
exceptions.ValidationError({"surcharges": "<generic message>"})without usingstr(e).
- Log the error via the already-imported
No new methods or imports are needed; logger is already available at the top of the file and exceptions is imported. The rest of the logic (performing the update and returning the mutation with rate_sheet) remains unchanged.
| @@ -624,7 +624,14 @@ | ||
| try: | ||
| rate_sheet.batch_update_surcharges(surcharges) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"surcharges": str(e)}) | ||
| logger.error( | ||
| "Failed to batch update surcharges", | ||
| extra={"error": str(e), "rate_sheet_id": input.get("rate_sheet_id")}, | ||
| exc_info=True, | ||
| ) | ||
| raise exceptions.ValidationError( | ||
| {"surcharges": "Invalid surcharges data."} | ||
| ) | ||
|
|
||
| return BatchUpdateSurchargesMutation(rate_sheet=rate_sheet) | ||
|
|
| rate_data=rate_data | ||
| ) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"rate": str(e)}) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 19 days ago
In general, to fix this kind of issue, you should avoid exposing raw exception messages to the client. Instead, map internal exceptions to either (a) a generic error message that does not reveal implementation details, or (b) a curated set of safe, user-facing messages. The original exception (including full stack trace) should be logged on the server side for debugging, but not returned in the API response.
For this file specifically, the best minimal-impact fix is:
- In both
UpdateServiceRateMutation.mutateandBatchUpdateServiceRatesMutation.mutate, catchValueErroras now, but:- Log the exception server-side using the already imported
logger. - Replace
str(e)in theValidationErrorpayload with a generic, user-safe message that doesn’t depend on the exception text (for example, “Invalid rate data.” and “One or more rate updates are invalid.”).
- Log the exception server-side using the already imported
- This preserves the behavior of returning a 400-style validation error in the GraphQL/DRF layer, keeps logging for debugging, and removes the direct exposure of exception text.
Concretely:
- In
modules/admin/karrio/server/admin/schemas/base/mutations.py, around lines 660–667, change theexcept ValueErrorblock to log the exception and raiseValidationErrorwith a fixed message instead ofstr(e). - Do the same in the
BatchUpdateServiceRatesMutationblock around lines 693–696.
No new imports are needed; logger is already imported at line 22, and exceptions is already imported at the top and locally in the methods.
| @@ -664,7 +664,8 @@ | ||
| rate_data=rate_data | ||
| ) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"rate": str(e)}) | ||
| logger.exception("Failed to update service rate for rate sheet %s", rate_sheet.id) | ||
| raise exceptions.ValidationError({"rate": "Invalid rate data."}) | ||
|
|
||
| return UpdateServiceRateMutation(rate_sheet=rate_sheet) | ||
|
|
||
| @@ -693,7 +694,8 @@ | ||
| try: | ||
| rate_sheet.batch_update_service_rates(updates) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"rates": str(e)}) | ||
| logger.exception("Failed to batch update service rates for rate sheet %s", rate_sheet.id) | ||
| raise exceptions.ValidationError({"rates": "One or more rate updates are invalid."}) | ||
|
|
||
| return BatchUpdateServiceRatesMutation(rate_sheet=rate_sheet) | ||
|
|
| try: | ||
| rate_sheet.batch_update_service_rates(updates) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"rates": str(e)}) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 19 days ago
In general, the fix is to avoid returning raw exception messages to the external client and instead send a generic, safe error message while logging the detailed exception (including stack trace) on the server. This maintains debuggability without exposing internal information such as implementation details or sensitive data.
For this specific code, the single best fix with minimal behavioral change is:
- Catch
ValueErroraseas it already does. - Log the full exception (including stack trace) using the existing
loggerwithexc_info=Trueor similar. - Raise a
ValidationErrorwith a generic message for the"rates"field (for example,"Invalid rate configuration."), rather than usingstr(e).
Concretely, in modules/admin/karrio/server/admin/schemas/base/mutations.py, inside BatchUpdateServiceRatesMutation.mutate, replace:
try:
rate_sheet.batch_update_service_rates(updates)
except ValueError as e:
raise exceptions.ValidationError({"rates": str(e)})with something like:
try:
rate_sheet.batch_update_service_rates(updates)
except ValueError as e:
logger.error(
"Failed to batch update service rates",
error=str(e),
exc_info=True,
)
raise exceptions.ValidationError(
{"rates": "Invalid rate configuration."}
)This uses the already-imported logger (line 22) so no new imports are needed. The external API still reports a validation error on the "rates" field, but without leaking the internal ValueError message.
| @@ -693,7 +693,14 @@ | ||
| try: | ||
| rate_sheet.batch_update_service_rates(updates) | ||
| except ValueError as e: | ||
| raise exceptions.ValidationError({"rates": str(e)}) | ||
| logger.error( | ||
| "Failed to batch update service rates", | ||
| error=str(e), | ||
| exc_info=True, | ||
| ) | ||
| raise exceptions.ValidationError( | ||
| {"rates": "Invalid rate configuration."} | ||
| ) | ||
|
|
||
| return BatchUpdateServiceRatesMutation(rate_sheet=rate_sheet) | ||
|
|
313db4c to
3731d6d
Compare
3731d6d to
0137783
Compare
Karrio 2026.1.9
Changes
Feat
Fix
Refactor
Chore