Skip to content

Fix Content-MD5 in FIPS environments - #48241

Merged
weirongw23-msft merged 4 commits into
Azure:mainfrom
JensWehner:fix/storage-fips-content-md5
Jul 28, 2026
Merged

Fix Content-MD5 in FIPS environments#48241
weirongw23-msft merged 4 commits into
Azure:mainfrom
JensWehner:fix/storage-fips-content-md5

Conversation

@JensWehner

Copy link
Copy Markdown
Contributor

Content-MD5 uses hashlib.md5() and fails under FIPS policy. Mark the protocol checksum as non-security use and add a regression test.

Copilot AI review requested due to automatic review settings July 24, 2026 13:35
@github-actions github-actions Bot added Community Contribution Community members are working on the issue customer-reported Issues that are reported by GitHub users external to the Azure organization. Storage Storage Service (Queues, Blobs, Files) labels Jul 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution @JensWehner! We will review the pull request and get back to you soon.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
8 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates Blob Content-MD5 calculation for FIPS environments and adds regression coverage.

Changes:

  • Marks MD5 as non-security usage.
  • Adds a FIPS-policy regression test.
  • Sibling Storage packages remain unaddressed.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
tests/test_content_validation.py Tests non-security MD5 usage.
_shared/validation.py Passes usedforsecurity=False to MD5.

@jalauzon-msft

Copy link
Copy Markdown
Member

@JensWehner, thanks for the contribution! We are good to take this change but please address the comment from Copilot and bring this change to all four of our base packages. While it may not be used in all packages, we like to keep the _shared folder in sync across all packages. You do not need to port the test to other packages if you do not want though.

Copilot AI review requested due to automatic review settings July 26, 2026 20:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Copilot AI review requested due to automatic review settings July 26, 2026 20:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

@github-actions

This comment has been minimized.

Copilot AI review requested due to automatic review settings July 27, 2026 06:24
@JensWehner
JensWehner force-pushed the fix/storage-fips-content-md5 branch from e949264 to 11a7c3e Compare July 27, 2026 06:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

@github-actions

Copy link
Copy Markdown
Contributor
[Pilot] PR Pipeline Failure Analysis

A CI pipeline failed on this pull request. Here is an automated analysis of what went wrong and how to get the build green.

What failed

The mypy validation stage failed for 2 packages in the sdk/eventhub area. Both failures share the same root cause — a type error in vendored storage blob code:

  • azure-eventhub-checkpointstoreblob — mypy FAIL(1)
  • azure-eventhub-checkpointstoreblob-aio — mypy FAIL(1)

The storage packages (azure-storage-blob, azure-storage-file-datalake, azure-storage-file-share, azure-storage-queue) all passed mypy.

Error (identical in both packages):

azure/eventhub/extensions/checkpointstoreblob[aio]/_vendor/storage/blob/_models.py:603:
  error: Incompatible types in assignment
    (expression has type "Any | None", variable has type "str")  [assignment]

Build: https://dev.azure.com/azure-sdk/public/_build/results?buildId=6618602

Recommended next steps

  • Fix the type error at line 603 of _vendor/storage/blob/_models.py in both packages:
    • sdk/eventhub/azure-eventhub-checkpointstoreblob/azure/eventhub/extensions/checkpointstoreblob/_vendor/storage/blob/_models.py
    • sdk/eventhub/azure-eventhub-checkpointstoreblob-aio/azure/eventhub/extensions/checkpointstoreblobaio/_vendor/storage/blob/_models.py
  • The variable is typed as str but the expression can be Any | None. Add a type guard/cast (e.g. cast(str, ...)) or update the vendored code to allow Optional[str].
  • Since this is vendored code, the fix may also need to be applied upstream in azure-storage-blob and then re-vendored — or suppressed with a # type: ignore[assignment] comment if the vendored copy is intentionally frozen.
  • See the CI troubleshooting guide: https://aka.ms/ci-fix
  • Push new commits to address the failures; this comment updates automatically on the next failing run.
Raw pipeline analysis (azsdk ci analyze)
Failed Tasks
### Errors:
azure/eventhub/extensions/checkpointstoreblob/_vendor/storage/blob/_models.py:603:
  error: Incompatible types in assignment (expression has type "Any | None", variable has type "str")  [assignment]
Found 1 error in 1 file (checked 75 source files)
mypy check completed with exit code 1

azure/eventhub/extensions/checkpointstoreblobaio/_vendor/storage/blob/_models.py:603:
  error: Incompatible types in assignment (expression has type "Any | None", variable has type "str")  [assignment]
Found 1 error in 1 file (checked 85 source files)
mypy check completed with exit code 1

=== SUMMARY ===
PACKAGE                                                                 CHECK  STATUS  DURATION(s)
/mnt/vss/_work/1/s/sdk/storage/azure-storage-blob                       mypy  OK             26.73
/mnt/vss/_work/1/s/sdk/storage/azure-storage-file-datalake              mypy  OK             28.32
/mnt/vss/_work/1/s/sdk/storage/azure-storage-file-share                 mypy  OK             16.42
/mnt/vss/_work/1/s/sdk/storage/azure-storage-queue                      mypy  OK             15.06
/mnt/vss/_work/1/s/sdk/eventhub/azure-eventhub-checkpointstoreblob      mypy  FAIL(1)        23.34
/mnt/vss/_work/1/s/sdk/eventhub/azure-eventhub-checkpointstoreblob-aio  mypy  FAIL(1)        26.17

Total checks: 6 | Failed: 2 | Worst exit code: 1

Pipeline: https://dev.azure.com/azure-sdk/public/_build/results?buildId=6618602

Copilot detected the failing pipeline and generated the analysis above. To have it attempt a fix automatically, reply with @copilot please fix the failing pipeline on this PR.

Generated by Pipeline Analysis - Next Steps · 47.7 AIC · ⌖ 6.37 AIC · ⊞ 6.6K ·

Copilot AI review requested due to automatic review settings July 27, 2026 21:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings July 28, 2026 07:45
@JensWehner
JensWehner force-pushed the fix/storage-fips-content-md5 branch from 2620977 to 1429cae Compare July 28, 2026 07:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

sdk/storage/azure-storage-queue/azure/storage/queue/_shared/validation.py:80

  • The regression test added in this PR imports only Blob's validation module, so it does not exercise this separate Queue implementation. A future omission or regression here would still pass that test; add the same FIPS simulation against azure.storage.queue._shared.validation.calculate_content_md5.
    md5 = hashlib.md5(usedforsecurity=False)  # nosec

sdk/storage/azure-storage-file-share/azure/storage/fileshare/_shared/validation.py:80

  • The new regression test executes only Blob's copy of this helper, not the File Share module changed here. Please add equivalent coverage in File Share's existing tests/test_content_validation.py so its independently shipped implementation is verified under the simulated FIPS policy.
    md5 = hashlib.md5(usedforsecurity=False)  # nosec

sdk/storage/azure-storage-file-datalake/azure/storage/filedatalake/_shared/validation.py:80

  • The Blob-only regression test does not execute this independently shipped Data Lake helper. Add the equivalent FIPS simulation to Data Lake's existing tests/test_content_validation.py; otherwise this package can regress to plain hashlib.md5() without any package test failing.
    md5 = hashlib.md5(usedforsecurity=False)  # nosec

@weirongw23-msft
weirongw23-msft merged commit 8dbd2e4 into Azure:main Jul 28, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Community Contribution Community members are working on the issue customer-reported Issues that are reported by GitHub users external to the Azure organization. Storage Storage Service (Queues, Blobs, Files)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants