Convert blueskyposts to LAVA @artifact_processor (+ feed bug fix)#850
Merged
Conversation
Converts the Bluesky posts JSON-cache parser to the decorated pattern. Fixes a bug where the feed loop always read data['feed'][0]['post'] (index 0) instead of the current loop item, so every feed entry was a duplicate of the first post and collapsed to a single row. Now iterates each entry correctly (verified: distinct feed posts yield distinct rows). Consolidates the two identical-shape reports (feed and posts) into one table with a 'Collection' column. ISO timestamps become aware UTC datetimes. Replaces the bare except and reads each cache file once.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Converts
blueskyposts.py(Bluesky feed/posts http-cache) to the decorated@artifact_processorpattern.Bug fix: the feed loop read
data['feed'][0]['post'](hard-coded index 0) insidefor items in value, so every feed entry was a copy of the first post and deduped down to a single row. Now reads each entry's ownpost. Verified with a 2-post fixture: yields 2 distinct rows (was 1).Other
feed+posts) into one table with aCollectioncolumn.datetime;langslist joined to a string.except:removed; each cache file read once. Reserved-word audit clean.Own artifact; author and dates (2024-11-20) preserved.