Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Added personal access token support to GitHub Collector #2145

Merged
merged 1 commit into from
Jul 25, 2022
Merged

Conversation

waldbauer-certat
Copy link
Contributor

As github basic authentication has been marked as deprecated by
GitHub, we now implemented the Personal Access Token authentication
method.

Fixes #1549

@codecov-commenter
Copy link

codecov-commenter commented Feb 1, 2022

Codecov Report

Merging #2145 (ed819e9) into develop (79cae29) will decrease coverage by 0.11%.
The diff coverage is 60.00%.

❗ Current head ed819e9 differs from pull request most recent head 2a4beaf. Consider uploading reports for the commit 2a4beaf to get more accurate results

@@             Coverage Diff             @@
##           develop    #2145      +/-   ##
===========================================
- Coverage    76.34%   76.23%   -0.12%     
===========================================
  Files          441      441              
  Lines        23654    23628      -26     
  Branches      3739     3437     -302     
===========================================
- Hits         18059    18012      -47     
- Misses        4858     4879      +21     
  Partials       737      737              
Impacted Files Coverage Δ
...tests/bots/collectors/github_api/test_collector.py 100.00% <ø> (ø)
...ots/collectors/github_api/_collector_github_api.py 75.86% <60.00%> (-3.55%) ⬇️
intelmq/bots/experts/reverse_dns/expert.py 83.87% <0.00%> (-11.30%) ⬇️
intelmq/bots/experts/mcafee/expert_mar.py 35.55% <0.00%> (-8.89%) ⬇️
intelmq/bots/parsers/dshield/parser_domain.py 95.45% <0.00%> (-4.55%) ⬇️
intelmq/bots/collectors/opendxl/collector.py 30.15% <0.00%> (-3.18%) ⬇️
intelmq/bots/experts/modify/expert.py 94.66% <0.00%> (-2.67%) ⬇️
intelmq/bots/parsers/dshield/parser_block.py 87.50% <0.00%> (-2.50%) ⬇️
intelmq/bots/collectors/tcp/collector.py 84.31% <0.00%> (-1.97%) ⬇️
intelmq/bots/parsers/generic/parser_csv.py 85.18% <0.00%> (-1.86%) ⬇️
... and 14 more

Copy link
Member

@sebix sebix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a note to the NEWS file on the parameter change

https://developer.github.com/changes/2020-02-14-deprecating-password-auth/#removal says:

All password authentication will return a status code of 401 starting:

November 13, 2020 at 16:00 UTC

So I guess the old auth does no longer work? If that's the case, please remove the old method, it's of no use.

the feeds.yml needs an update as well: https://intelmq.readthedocs.io/en/latest/user/feeds.html#id123

CHANGELOG.md Outdated Show resolved Hide resolved
docs/user/bots.rst Outdated Show resolved Hide resolved
@sebix sebix self-assigned this Feb 1, 2022
@sebix sebix added bug Indicates an unexpected problem or unintended behavior component: bots labels Feb 1, 2022
@sebix sebix added this to the 3.1.0 milestone Feb 1, 2022
@waldbauer-certat
Copy link
Contributor Author

Please add a note to the NEWS file on the parameter change

https://developer.github.com/changes/2020-02-14-deprecating-password-auth/#removal says:

All password authentication will return a status code of 401 starting:

November 13, 2020 at 16:00 UTC

So I guess the old auth does no longer work? If that's the case, please remove the old method, it's of no use.

the feeds.yml needs an update as well: https://intelmq.readthedocs.io/en/latest/user/feeds.html#id123

Feeds will be auto build, no? Because its gitignored https://github.com/certtools/intelmq/blob/develop/.gitignore#L30

@sebix
Copy link
Member

sebix commented Feb 2, 2022

So I guess the old auth does no longer work? If that's the case, please remove the old method, it's of no use.
the feeds.yml needs an update as well: https://intelmq.readthedocs.io/en/latest/user/feeds.html#id123

Feeds will be auto build, no? Because its gitignored https://github.com/certtools/intelmq/blob/develop/.gitignore#L30

That page is generated from intelmq/etc/feeds.yml

intelmq/tests/bots/collectors/github_api/test_collector.py Outdated Show resolved Hide resolved
intelmq/etc/feeds.yaml Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
@sebix
Copy link
Member

sebix commented Jul 6, 2022

And:

==================================== ERRORS ====================================
______ ERROR at setup of test_collector_should_fail_with_bad_credentials _______
file /home/runner/work/intelmq/intelmq/intelmq/tests/bots/collectors/github_api/test_collector.py, line 96
  @patch('intelmq.bots.collectors.github_api.collector_github_contents_api.requests.get')
  def test_collector_should_fail_with_bad_credentials(self, requests_get_mock):
E       fixture 'requests_get_mock' not found
>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, cov, doctest_namespace, monkeypatch, no_cover, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, requests_mock, time_machine, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.
/home/runner/work/intelmq/intelmq/intelmq/tests/bots/collectors/github_api/test_collector.py:96
---------- coverage: platform linux, python 3.7.13-final-0 -----------
Coverage XML written to file coverage.xml
=========================== short test summary info ============================
ERROR intelmq/tests/bots/collectors/github_api/test_collector.py::test_collector_should_fail_with_bad_credentials

@waldbauer-certat waldbauer-certat force-pushed the enh-1549 branch 2 times, most recently from ed819e9 to 2a4beaf Compare July 7, 2022 13:37
As github basic authentication has been marked as deprecated by
GitHub, we now implemented the Personal Access Token authentication
method.

Fixes #1549

Signed-off-by: Sebastian Waldbauer <waldbauer@cert.at>
@@ -95,7 +92,6 @@ def print_requests_get_parameters(url, *args, **kwargs):
main_mock = MagicMock(content=EXAMPLE_CONTENT_STR)
return main_mock


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you are keen on codestyle and "Quality of Life": Before a class statement there should be two empty lines ;)

@sebix sebix merged commit 28ccf11 into develop Jul 25, 2022
@sebix sebix deleted the enh-1549 branch July 25, 2022 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Indicates an unexpected problem or unintended behavior component: bots
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GitHub API collector: support for personal access token
3 participants