-
Notifications
You must be signed in to change notification settings - Fork 519
Closed
Labels
t-toolingIssues with this label are in the ownership of the tooling team.Issues with this label are in the ownership of the tooling team.
Description
Description
We currently construct parts of the unique_key using the pipe (|) as a separator.
For example, a standard (extended) unique_key looks like this:
unique_key='GET|e3b0c442|e3b0c442|https://crawlee.dev/'
However, for the always_enqueue feature, we append a random alphanumeric suffix using an underscore (_) instead of a pipe:
if always_enqueue:
unique_key = f'{unique_key}_{crypto_random_object_id()}'Resulting in:
unique_key='GET|e3b0c442|e3b0c442|https://crawlee.dev/_7VnDVJgE3TrjbDl86'
Proposed change
Use the pipe (|) consistently as a separator throughout all unique_key constructions, including the always_enqueue case.
Note
This change affects the resulting unique_key format and may be breaking, so it should be addressed as part of the next major release (v2.0).
Metadata
Metadata
Assignees
Labels
t-toolingIssues with this label are in the ownership of the tooling team.Issues with this label are in the ownership of the tooling team.