Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use regular expressions for list of metadata to persist and transfer #728

Closed
jnioche opened this issue May 21, 2019 · 1 comment
Closed

Use regular expressions for list of metadata to persist and transfer #728

jnioche opened this issue May 21, 2019 · 1 comment

Comments

@jnioche
Copy link
Contributor

jnioche commented May 21, 2019

This would allow defining generic patterns e.g. xxx_s or yyy_p so that any matching key gets store or persisted without having to list them in the config explicitly.
@jnioche jnioche added this to the 1.18 milestone Nov 25, 2020
@jnioche jnioche removed this from the 1.18 milestone Mar 26, 2021
michaeldinzinger added a commit to michaeldinzinger/storm-crawler that referenced this issue Nov 4, 2023
@michaeldinzinger
Issue apache#728: Adding asterisk for metadata transfer
374e0bb 
Signed-off-by: Michael Dinzinger <michael.dinzinger@uni-passau.de>
jnioche pushed a commit that referenced this issue Nov 13, 2023
@michaeldinzinger
Issue #728: Adding asterisk for metadata transfer (#1117)
4d3340f 
* Issue #728: Adding asterisk for metadata transfer

Signed-off-by: Michael Dinzinger <michael.dinzinger@uni-passau.de>

* Add method keySet(prefix) and metadata.copy

Signed-off-by: Michael Dinzinger <michael.dinzinger@uni-passau.de>

* Add comments to crawler-default.yaml and add test cases for metadata.persist + transfer

Signed-off-by: Michael Dinzinger <michael.dinzinger@uni-passau.de>

---------

Signed-off-by: Michael Dinzinger <michael.dinzinger@uni-passau.de>
@jnioche
Copy link
Contributor Author

jnioche commented Nov 13, 2023

Overlaps with #728 which is probably sufficient as an approach

@jnioche jnioche closed this as completed Nov 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jnioche