Skip to content

Fix #2971: declare dropbox-source's delete, and match the azure blob container exactly - #2974

Merged
oscerd merged 1 commit into
apache:mainfrom
oscerd:ci-issue-2971
Aug 24, 2026
Merged

Fix #2971: declare dropbox-source's delete, and match the azure blob container exactly#2974
oscerd merged 1 commit into
apache:mainfrom
oscerd:ci-issue-2971

Conversation

@oscerd

@oscerd oscerd commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Fixes #2971

1. dropbox-source — the delete is now declared and switchable

The Kamelet deleted every file it consumed via an unconditional dropbox:del, with no property controlling it and no mention in the description ("Consume Files"). An operator reading the schema had no way to learn the folder gets emptied.

-          - toD: "dropbox:del?...&remotePath=${exchangeProperty.dropboxFileName}"
+          - choice:
+              when:
+                - simple: '{{deleteAfterRead}}'
+                  steps:
+                    - toD: "dropbox:del?...&remotePath=${exchangeProperty.dropboxFileName}"

The default is true, not false — deliberately. My first pass at this issue proposed false to match azure-storage-blob-source, which was wrong: dropbox-source polls with timer + dropbox:search and has no idempotent consumer or repository. The delete is what stops the next poll re-delivering the same files, so defaulting it off would turn every period tick into a re-delivery of the whole folder.

So this is a no-op at runtime. What changes is that the destruction is now visible in the schema and in tooling, and an operator who has added their own idempotency can switch it off. Making false safe would additionally need an idempotent consumer — noted in the issue as separate work.

The gate uses {{deleteAfterRead}} rather than the ${properties:deleteAfterRead:true} form the sibling Kamelets use. That is not cosmetic: ${properties:...} does not register as a use, so the validator rejects the property as declared-but-unused, and the existing Kamelets work around that by being listed in verifyUsedParams's exclusion list in script/validator/validator.go. Adding dropbox-source there would disable the unused-parameter check for every property in the file. Verified both branches with camel run: default → delete runs, deleteAfterRead=false → skipped.

Side note for a maintainer: that exclusion list is why #2958's dead queueURL went unnoticed — aws-sqs-source and aws-s3-event-based-source are both on it. Once #2965 lands, those two may be removable from the list.

2. azure-storage-blob-event-based-source — exact container match

- ... && ${exchangeProperty.azure-storage-blob-subject} contains "{{containerName}}"
+ ... && ${exchangeProperty.azure-storage-blob-subject} startsWith "/blobServices/default/containers/{{containerName}}/blobs/"

The subject comes from the Event Grid event's $.subject, shaped /blobServices/default/containers/<container>/blobs/<path>, where <path> is chosen by whoever uploaded the blob. contains therefore matched on the blob path too. The google-storage-event-based-source sibling already uses strict equality.

Verified with camel run:

subject configured container result
/blobServices/default/containers/mycontainer/blobs/report.csv mycontainer matches
/blobServices/default/containers/other/blobs/mycontainer/evil.csv mycontainer no match
/blobServices/default/containers/mycontainer-staging/blobs/x.csv mycontainer no match

The last two both matched under contains.

Verification

  • script/validator reports no errors
  • mvn verify passes
  • Both fixes exercised with camel run as described; no live Dropbox or Azure endpoint involved

Claude Code on behalf of Andrea Cosentino

… blob container exactly

dropbox-source deleted every file it consumed with no property controlling
it and no mention in the description. Declare a deleteAfterRead property
and gate the dropbox:del step on it. The default is true, not false: the
Kamelet polls with a timer and keeps no record of what it has read, so
turning the delete off makes every poll re-deliver the whole folder. The
behaviour is unchanged; it is now visible in the schema and switchable.

The gate uses {{deleteAfterRead}} rather than ${properties:...} so the
property counts as used, avoiding another entry in the validator's
unused-parameter exclusion list.

azure-storage-blob-event-based-source guarded its fetch with
"subject contains containerName". The Event Grid subject is
/blobServices/default/containers/<container>/blobs/<path>, and <path> is
chosen by the uploader, so the guard also matched on the blob path and on
containers whose name merely has the configured one as a prefix. Match the
container segment exactly instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@oscerd
oscerd requested review from christophd and davsclaus August 24, 2026 11:18
@oscerd
oscerd merged commit abf96a8 into apache:main Aug 24, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Source Kamelets whose behaviour is not described by their schema (dropbox-source delete, azure blob container guard)

2 participants