Skip to content

fix: add diagnostic logging for Hawk query string ordering#262

Merged
layertwo merged 1 commit intomainlinefrom
debug/hawk-query-string-ordering
Mar 2, 2026
Merged

fix: add diagnostic logging for Hawk query string ordering#262
layertwo merged 1 commit intomainlinefrom
debug/hawk-query-string-ordering

Conversation

@layertwo
Copy link
Owner

@layertwo layertwo commented Mar 2, 2026

Summary

  • Log reconstructed query string param order and full server URL on Hawk MAC mismatch
  • Diagnoses storage 401 failures on GET /storage/prefs?newer=...&full=1&limit=1000

Context

Storage sync fails with 401 on the prefs collection but succeeds for clients, meta/global, crypto/keys, etc. All use the same Hawk token.

Hypothesis: extract_hawk_request_params reconstructs the query string from event.query_string_parameters (a dict), which loses the original parameter ordering from the HTTP request. API Gateway REST API v1 may serialize dict keys alphabetically:

Request Client order Server order (if alphabetized) MAC match?
clients?full=1&limit=1000 full,limit full,limit Yes → 200
prefs?newer=...&full=1&limit=1000 newer,full,limit full,limit,newer No → 401

What to look for in CloudWatch

  • "Hawk query string reconstruction" — check param_key_order on prefs requests
  • "HAWK MAC mismatch" — confirms which URL mohawk tried to verify

Test plan

  • Deploy to prod
  • Trigger a sync from Firefox
  • Check CloudWatch logs for the prefs request's param_key_order
  • If confirmed, follow up with a fix (HTTP API v2 for rawQueryString, or alternative)

Log the reconstructed query string parameter order and full server URL
on MAC mismatch to diagnose storage/prefs 401 failures. Hypothesis:
API Gateway REST API v1 reorders query params alphabetically in the
event dict, causing Hawk MAC verification to fail when the client's
original param order differs (e.g. newer,full,limit → full,limit,newer).
@github-actions
Copy link

github-actions bot commented Mar 2, 2026

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  lambda/src/services
  hawk_service.py
  lambda/src/shared
  utils.py
Project Total  

This report was generated by python-coverage-comment-action

@github-actions
Copy link

github-actions bot commented Mar 2, 2026

Diff for stage: DefaultStage

Diff for stack: GitHubOidcStack - 0 to add, 1 to update, 0 to destroy 🟡

Details
Resources
[~] Custom::AWSCDKOpenIdConnectProvider GitHubOidcProvider7EBF861F
 ├─ [~] CodeHash
 │   ├─ [-] 62fa02efcaa700e1c247e1d3cc2aa0cd07a0808a9a3e3d2230e51f57a02233fb
 │   └─ [+] d75c48c9f82cde63e9bf414df335e84e8ac24f11eb34889be255b702aec71e50
 └─ [~] RejectUnauthorized
[~] AWS::Lambda::Function CustomAWSCDKOpenIdConnectProviderCustomResourceProviderHandlerF2C543E0
 ├─ [~] Code
 │   └─ [~] .S3Key:
 │       ├─ [-] 62fa02efcaa700e1c247e1d3cc2aa0cd07a0808a9a3e3d2230e51f57a02233fb.zip
 │       └─ [+] d75c48c9f82cde63e9bf414df335e84e8ac24f11eb34889be255b702aec71e50.zip
 └─ [~] Metadata
     └─ [~] .aws:asset:path:
         ├─ [-] asset.62fa02efcaa700e1c247e1d3cc2aa0cd07a0808a9a3e3d2230e51f57a02233fb
         └─ [+] asset.d75c48c9f82cde63e9bf414df335e84e8ac24f11eb34889be255b702aec71e50

Diff for stack: Service-prod - 0 to add, 0 to update, 0 to destroy

Details
Resources
[~] AWS::Lambda::Function AuthApiHandlerED50ACFA
 ├─ [~] Code
 │   └─ [~] .S3Key:
 │       ├─ [-] ecd5fbf25a5e0425e7c7ddd001835de68f1cb7b7e2943adf0d7ea190efe9dbc9.zip
 │       └─ [+] 3f1fc4e35179c6e033b76937a0d25db8f3170704e4ff47ece2f2d3603615ba03.zip
 └─ [~] Metadata
     └─ [~] .aws:asset:path:
         ├─ [-] asset.ecd5fbf25a5e0425e7c7ddd001835de68f1cb7b7e2943adf0d7ea190efe9dbc9
         └─ [+] asset.3f1fc4e35179c6e033b76937a0d25db8f3170704e4ff47ece2f2d3603615ba03
[~] AWS::Lambda::Function TokenApiHandler2E66DB25
 ├─ [~] Code
 │   └─ [~] .S3Key:
 │       ├─ [-] ecd5fbf25a5e0425e7c7ddd001835de68f1cb7b7e2943adf0d7ea190efe9dbc9.zip
 │       └─ [+] 3f1fc4e35179c6e033b76937a0d25db8f3170704e4ff47ece2f2d3603615ba03.zip
 └─ [~] Metadata
     └─ [~] .aws:asset:path:
         ├─ [-] asset.ecd5fbf25a5e0425e7c7ddd001835de68f1cb7b7e2943adf0d7ea190efe9dbc9
         └─ [+] asset.3f1fc4e35179c6e033b76937a0d25db8f3170704e4ff47ece2f2d3603615ba03
[~] AWS::Lambda::Function ProfileApiHandler9B65A298
 ├─ [~] Code
 │   └─ [~] .S3Key:
 │       ├─ [-] ecd5fbf25a5e0425e7c7ddd001835de68f1cb7b7e2943adf0d7ea190efe9dbc9.zip
 │       └─ [+] 3f1fc4e35179c6e033b76937a0d25db8f3170704e4ff47ece2f2d3603615ba03.zip
 └─ [~] Metadata
     └─ [~] .aws:asset:path:
         ├─ [-] asset.ecd5fbf25a5e0425e7c7ddd001835de68f1cb7b7e2943adf0d7ea190efe9dbc9
         └─ [+] asset.3f1fc4e35179c6e033b76937a0d25db8f3170704e4ff47ece2f2d3603615ba03
[~] AWS::Lambda::Function ApiHandler5E7490E8
 ├─ [~] Code
 │   └─ [~] .S3Key:
 │       ├─ [-] ecd5fbf25a5e0425e7c7ddd001835de68f1cb7b7e2943adf0d7ea190efe9dbc9.zip
 │       └─ [+] 3f1fc4e35179c6e033b76937a0d25db8f3170704e4ff47ece2f2d3603615ba03.zip
 └─ [~] Metadata
     └─ [~] .aws:asset:path:
         ├─ [-] asset.ecd5fbf25a5e0425e7c7ddd001835de68f1cb7b7e2943adf0d7ea190efe9dbc9
         └─ [+] asset.3f1fc4e35179c6e033b76937a0d25db8f3170704e4ff47ece2f2d3603615ba03

No Changes for stack: Monitoring-prod ✅
No Changes for stack: Frontend-prod ✅

Generated for commit e1bcf93 at 2026-03-02T23:04:50.990Z

@layertwo layertwo merged commit ed341c9 into mainline Mar 2, 2026
10 checks passed
layertwo added a commit that referenced this pull request Mar 3, 2026
API Gateway REST API v1 alphabetizes query parameters in the event
dict, breaking Hawk MAC verification when the client's original
parameter order differs (e.g. newer,full,limit → full,limit,newer).

Fix: before calling mohawk.Receiver, pre-compute the Hawk MAC for
each permutation of query parameters to find the client's original
ordering. Only call mohawk once with the correct URL, preserving
nonce replay protection. Removes diagnostic logging from PR #262.
layertwo added a commit that referenced this pull request Mar 3, 2026
API Gateway REST API v1 alphabetizes query parameters in the event
dict, breaking Hawk MAC verification when the client's original
parameter order differs (e.g. newer,full,limit → full,limit,newer).

Fix: before calling mohawk.Receiver, pre-compute the Hawk MAC for
each permutation of query parameters to find the client's original
ordering. Only call mohawk once with the correct URL, preserving
nonce replay protection. Removes diagnostic logging from PR #262.
@layertwo layertwo deleted the debug/hawk-query-string-ordering branch March 3, 2026 02:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant