Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process posts without media files #1569

Open
TestPolygon opened this issue May 20, 2021 · 3 comments
Open

Process posts without media files #1569

TestPolygon opened this issue May 20, 2021 · 3 comments

Comments

@TestPolygon
Copy link

TestPolygon commented May 20, 2021

I think it would be useful to have a flag to process posts that have no media files, instead of skipping them, in order to run postprocessors, or just write a JSON with --write-metadata.

In order to have ability to save some text information that was posted without attached image/video files.

@Twi-Hard
Copy link

This is super important to me. I hope something like this can happen.:)

@mikf
Copy link
Owner

mikf commented May 22, 2021

"postprocessors": [{
    "name": "metadata",
    "event": "post",
    "filename": "{post_id}.json"
}]

is how this should be generally possible at the moment (with a fitting filename for each site).

  • "event": "post" to write metadata for each post. The default "event": "file", which triggers for each individual file, doesn't work at all for posts without files or duplicates most data for posts with more than one file.
  • "filename": "{post_id}.json" because there is no default filename format for posts, only individual files.

(for Twitter in particular: 724ca61)

@TestPolygon
Copy link
Author

TestPolygon commented Jun 15, 2021

What's about to add support for mastodon if it would be not difficult? Currently the program queries only posts with media files.


Also I don't like how it works with --download-archive. This post postprocessor runs every time while it can. While I would prefer rerun it only with --no-skip (--no-download) option.
Maybe also to save in the history archive also entries of the posts if there is the post postprocessor in the config file? Or it would have some problems with archive_fmt?

For example, I run a command (with --download-archive): the program has downloaded all media and post postprocessor created the related files with meta information.
Next I move the folder to other place.
Rerun the command.
Files are not redownloaded as expected, but post postprocessor created the new files while I expected the skipping of it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants