Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error in update_wikidata should not crash the bot #156

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Error in update_wikidata should not crash the bot #156

wants to merge 1 commit into from

Conversation

simon04
Copy link
Contributor

@simon04 simon04 commented Sep 29, 2022

Fixes #139.

@konstin
Copy link
Owner

konstin commented Sep 30, 2022

I'm afraid that's not a solution; We can't just silently swallow all errors. What needs to be done is a check for the specific error from #139 as specifically as possible, which also logs the offending entries so we can work on fixing the spam filter and/or talk to projects about their websites doing something bad

@simon04
Copy link
Contributor Author

simon04 commented Sep 30, 2022 via email

@konstin
Copy link
Owner

konstin commented Oct 1, 2022

no, except Exception and just continuing isn't appropriate error handling.

A failure on one Wikidata item/project probably shouldn't crash the bot all together, when it could resume with the next one.

There shouldn't be any random failures on single items! We want to know when something fails, because it most likely means a bug, and i don't remember any spurious failures prior to the spam filter thing

I'd like to offer my help in maintaining and running this bot. Besides this issue, what would be the most relevant issues/features to look into?

What needs to be done is:

@simon04
Copy link
Contributor Author

simon04 commented Oct 1, 2022

I cannot reproduce the bug on Q107387130 mentioned on https://www.wikidata.org/wiki/Special:Log/spamblacklist/Github-wiki-bot since apparently the website of the GitHub repository has been changed from anti-captcha.com to antigate.com

> python main.py --filter antigate
# Querying Projects
11757 projects were found by the sparql query
1 projects remained after filtering
1 projects were found
# Processing projects
## antigate: http://www.wikidata.org/entity/Q107387130
There are 3 stable releases
Added '1.3.1'
Added '1.3.4'
Added '1.4.0'
Setting preferred rank for 1.4.0
# Finished successfully

@konstin
Copy link
Owner

konstin commented Oct 2, 2022

I tried doing a new run, but now i get different crash:

$ .venv/bin/python ./main.py
[...]
## Q109616408: http://www.wikidata.org/entity/Q109616408
There are 23 stable releases
Setting normal rank for 1.3.1
Password for user Github-wiki-bot on wikidata:wikidata (no characters will be shown): 
Failed to update http://www.wikidata.org/entity/Q109616408: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "./main.py", line 844, in <module>
    main()                                     
  File "./main.py", line 838, in main
    raise e                                    
  File "./main.py", line 835, in main
    update_wikidata(properties, edit_group_hash)
  File "./main.py", line 722, in update_wikidata
    set_claim_rank(claim, latest_version, release, edit_group_hash)
  File "./main.py", line 587, in set_claim_rank 
    claim.changeRank("normal", summary=get_summary(edit_group_hash))
  File "github-wikidata-bot/.venv/lib/python3.8/site-packages/pywikibot/page/__init__.py", line 4666, in changeRank
    return self.repo.save_claim(self, **kwargs) 
  File "github-wikidata-bot/.venv/lib/python3.8/site-packages/pywikibot/site/_decorators.py", line 92, in callee
    return fn(self, *args, **kwargs)
  File "github-wikidata-bot/.venv/lib/python3.8/site-packages/pywikibot/site/_datasite.py", line 427, in save_claim
    'token': self.tokens['edit'],
  File "github-wikidata-bot/.venv/lib/python3.8/site-packages/pywikibot/site/_tokenwallet.py", line 58, in __getitem__
    key = self.site.validate_tokens([key])[0]
  File "github-wikidata-bot/.venv/lib/python3.8/site-packages/pywikibot/site/_apisite.py", line 1309, in validate_tokens
    types_wiki_old = self._paraminfo.parameter('query+info',
TypeError: 'NoneType' object is not subscriptable

No idea where that is coming from the password is correct. I tried updating pywikibot and all dependencies respectively but that didn't help either

@simon04 simon04 mentioned this pull request Oct 2, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Spam filter crashes the bot
2 participants
@simon04 @konstin