Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implements command to change a job location. #739

Merged
merged 7 commits into from Jan 23, 2023

Conversation

trevorshannon
Copy link
Contributor

@trevorshannon trevorshannon commented Dec 19, 2022

This PR adds a new command line option --change-location to change the location (i.e. url, command, or navigate) of a job while preserving history for that job. The tool ensures that the new location string is unique within the list of jobs prior to changing anything.

Note that in the (admittedly contrived) scenario of having abandoned entries in a redis db that use the user-provided new location (perhaps from old runs that were never garbage-collected) then those abandoned entries will be clobbered by the --change-location option. I think that should be ok, but I'm sure I could find a way to preserve those entries if needed.

I did some manual testing to verify functionality.

Save the starting version of a simple urls YAML:

➜  urlwatch git:(feat/update-url)  cp ~/urls.yaml ~/urls_init.yaml
➜  urlwatch git:(feat/update-url) cat ~/urls_init.yaml 
command: date
kind: shell
---
kind: url
url: https://www.example.org
---
kind: url
url: https://www.wikipedia.org
user_visible_url: https://en.wikipedia.org
---
kind: browser
navigate: https://www.google.com

Change location of all 4 jobs:

➜  urlwatch git:(feat/update-url)  urlwatch --urls ~/urls.yaml --change-location 1 "date -v1m"                                
Moving location of <shell command='date'> to "date -v1m"
Saving updated list to '/Users/trevorshannon/urls.yaml'
➜  urlwatch git:(feat/update-url)  urlwatch --urls ~/urls.yaml --change-location https://www.example.org https://www.example.net
Moving location of <url url='https://www.example.org'> to "https://www.example.net"
Saving updated list to '/Users/trevorshannon/urls.yaml'
➜  urlwatch git:(feat/update-url)  urlwatch --urls ~/urls.yaml --change-location 3 https://es.wikipedia.org                     
Moving location of <url url='https://www.wikipedia.org' user_visible_url='https://en.wikipedia.org'> to "https://es.wikipedia.org"
Saving updated list to '/Users/trevorshannon/urls.yaml'
➜  urlwatch git:(feat/update-url)  urlwatch --urls ~/urls.yaml --change-location 4 https://www.google.co.uk
Moving location of <browser navigate='https://www.google.com'> to "https://www.google.co.uk"
Saving updated list to '/Users/trevorshannon/urls.yaml'

Compare the updated urls YAML to the initial reference version. Appropriate key values are updated according to job type:

➜  urlwatch git:(feat/update-url)  diff ~/urls_init.yaml ~/urls.yaml   
1c1
< command: date
---
> command: date -v1m
5c5
< url: https://www.example.org
---
> url: https://www.example.net
9c9
< user_visible_url: https://en.wikipedia.org
---
> user_visible_url: https://es.wikipedia.org
12c12
< navigate: https://www.google.com
---
> navigate: https://www.google.co.uk

Existing history is moved with --change-location:

➜  urlwatch git:(feat/update-url) urlwatch --urls ~/urls.yaml --dump-history 1
==============================
2022-12-20 13:08
------------------------------
Tue Dec 20 13:08:59 EST 2022

============================== 

==============================
2022-12-20 13:18
------------------------------
Tue Dec 20 13:18:54 EST 2022

============================== 

2 historic snapshot(s) available
➜  urlwatch git:(feat/update-url) urlwatch --urls ~/urls.yaml --change-location date "date -v1m"
Moving location of <shell command='date'> to "date -v1m"
Moved 2 snapshots of "date" to "date -v1m"
Saving updated list to '/Users/trevorshannon/urls.yaml'
➜  urlwatch git:(feat/update-url) urlwatch --urls ~/urls.yaml                                   
===========================================================================
01. CHANGED: date -v1m
===========================================================================

---------------------------------------------------------------------------
CHANGED: date -v1m
---------------------------------------------------------------------------
--- @	Tue, 20 Dec 2022 13:18:54 -0500
+++ @	Tue, 20 Dec 2022 13:20:33 -0500
@@ -1 +1 @@
-Tue Dec 20 13:18:54 EST 2022
+Thu Jan 20 13:20:33 EST 2022
---------------------------------------------------------------------------

➜  urlwatch git:(feat/update-url) urlwatch --urls ~/urls.yaml --dump-history 1                  
==============================
2022-12-20 13:08
------------------------------
Tue Dec 20 13:08:59 EST 2022

============================== 

==============================
2022-12-20 13:18
------------------------------
Tue Dec 20 13:18:54 EST 2022

============================== 

==============================
2022-12-20 13:20
------------------------------
Thu Jan 20 13:20:33 EST 2022

============================== 

3 historic snapshot(s) available

I confirmed the same behavior with --cache redis://127.0.0.1:6379

Fixes #300

@trevorshannon trevorshannon force-pushed the feat/update-url branch 2 times, most recently from dd10f89 to c12769a Compare December 20, 2022 15:25
@trevorshannon trevorshannon marked this pull request as ready for review December 20, 2022 20:11
Copy link
Owner

@thp thp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments.

lib/urlwatch/jobs.py Outdated Show resolved Hide resolved
lib/urlwatch/command.py Outdated Show resolved Hide resolved
lib/urlwatch/command.py Outdated Show resolved Hide resolved
lib/urlwatch/command.py Outdated Show resolved Hide resolved
Copy link
Owner

@thp thp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's go with your suggestion from this comment.

If a job has a user_visible_url set, that key's value is left unchanged by a --change-location run.
@trevorshannon
Copy link
Contributor Author

Updated per our discussion. I ran the same manual tests documented in the initial PR description, plus verified that no history gets moved if a job has user_visible_url set when using --change-location (because the guid does not change in that case)

The thing to note below is the absence of a string like "Moved {n} snapshots of...":

➜  urlwatch git:(feat/update-url) ✗ urlwatch --urls ~/urls.yaml --change-location 2 https://www.wikipedia.org
Moving location of <url url='https://www.example.org' user_visible_url='https://www.google.com'> to "https://www.wikipedia.org"
Saving updated list to '/home/trevor/urls.yaml'

lib/urlwatch/command.py Outdated Show resolved Hide resolved
lib/urlwatch/command.py Outdated Show resolved Hide resolved
Copy link
Owner

@thp thp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just two small style/wording comments. Can you also add an entry to CHANGELOG.md in the "Added" category describing the new feature? (+ PR number + your name as contributor, like the other examples already in the file)

After that, I think this is finally ready to merge. Thanks for pushing this PR forward, I think it turned out quite nice.

@trevorshannon
Copy link
Contributor Author

Ok, all set. Thanks for the feedback!

@thp thp merged commit 08ddce5 into thp:master Jan 23, 2023
@thp
Copy link
Owner

thp commented Jan 23, 2023

Merged, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow changing URLs/commands while keeping change history
2 participants