Can it keep track of what it has already downloaded?

[Romanmir]

This should be able to track what it last downloaded and then download only everything after that.

In this way, it could be "cron"-able and used as some sort of a regular backup of your data from reddit.

[uxdxdev]

It seems the reddit api uses before and after options that take a coded Id in listings, e.g. for comments the Id of the comment is prefixed with t1, such as t1_e3ykxbc.

To do this I'd probably need to save a metadata file locally that stores the Ids of the latest data, then subsequent runs of the tool would first read from the metadata file if it exists and use the Ids to fetch the latest content.

Also would need a new CLI option to enable this feature, options might be:

• -p, --persist
• -c, --continue
• -c, --cron
• -c, --config <filename>
• -l, --latest

Any ideas?

[Romanmir]

I like either "cron" or "latest". You could also use "-r, --resume".

[uxdxdev]

I went with an --only-latest flag for this feature, its available now.

Orca now generates an orca.config.js file in the CWD.

• when data is downloaded the latest entry for each data type is recorded, and the data is written to the file system
• if using the --only-latest flag the last entry is used when fetching data from the Reddit API, the latest data will overwrite the files previously written to the file system with only the latest data.
• if not using the --only-latest all data is downloaded and the data files are overwritten on the file system

So now just run the below command in a cron job to get only the latest data since the last download.

npx @mortond/orca --access-token=70162531-eWBggyup_FAKE_Usdf1cz7u-G9pM_dhrVf3g --only-latest

Make sure to copy or process the data files written to the file system before running Orca multiple times.