-
Notifications
You must be signed in to change notification settings - Fork 0
Input and Data Structure
Sebastian Göttel edited this page Feb 28, 2025
·
1 revision
RedTEI works with Pushshift data, so the following explanations refer to the Pushshift structure as input. Pushshift archives are provided in .zst format and contain NDJSON files. NDJSON (Newline Delimited JSON) is a format where each line is a valid JSON object. In the context of Pushshift archives, each JSON object represents a single Reddit comment.
Each JSON object (comment) contains various keys providing information about the comment:
- author: Username of the author.
- author_fullname: Unique ID of the author.
- body: The actual content of the comment.
- created_utc: Time of creation of the comment in UTC, specified as a Unix timestamp.
- id: Unique ID of the comment.
- link_id: ID of the parent main post (thread) to which the comment belongs. Used to group all comments of a thread.
-
parent_id: ID of the direct ancestor (parent) of the comment. Starts with
t1_if the comment is a reply to another comment; starts witht3_if it is a reply to the main post (thread). - permalink: Permanent link (URL) to the comment on Reddit.
- retrieved_on / retrieved_utc: Time of retrieval and archiving of the comment from Reddit, specified as a Unix timestamp.
- score: Net score of the comment (number of upvotes minus downvotes).
-
subreddit: Name of the subreddit in which the comment was written (e.g.,
r/de). - subreddit_id: Unique ID of the subreddit.