add egg updater script which fixes pytest initialization issue #672

Niraj-Kamdar · 2020-05-14T22:01:23Z

No description provided.

add update_egg to cli

Niraj-Kamdar · 2020-05-14T22:02:16Z

Fixes issue #605

cve_bin_tool/cli.py

terriko · 2020-05-14T22:47:02Z

Also, refactoring nitpicks aside, thanks for doing the research to get this done! I'm really glad to see a solution to what looks like such a weird bug.

cve_bin_tool/cli.py

pdxjohnny

Nice!!! Great work finding the fix for this. Very crafty

codecov-io · 2020-05-15T15:46:24Z

Codecov Report

Merging #672 into master will decrease coverage by 8.69%.
The diff coverage is 82.35%.

@@            Coverage Diff             @@
##           master     #672      +/-   ##
==========================================
- Coverage   89.70%   81.01%   -8.70%     
==========================================
  Files          66       67       +1     
  Lines        2118     2165      +47     
  Branches      282      283       +1     
==========================================
- Hits         1900     1754     -146     
- Misses        152      341     +189     
- Partials       66       70       +4

Flag	Coverage Δ
#longtests	`81.01% <82.35%> (-8.70%)`	⬇️

Impacted Files	Coverage Δ
cve_bin_tool/cli.py	`77.73% <82.35%> (-6.24%)`	⬇️
cve_bin_tool/cvedb.py	`52.15% <0.00%> (-30.57%)`	⬇️
test/test_json.py	`61.76% <0.00%> (-29.42%)`	⬇️
cve_bin_tool/checkers/xml2.py	`61.36% <0.00%> (-27.28%)`	⬇️
cve_bin_tool/OutputEngine.py	`73.10% <0.00%> (-24.54%)`	⬇️
test/test_cli.py	`82.05% <0.00%> (-17.95%)`	⬇️
test/test_scanner.py	`83.11% <0.00%> (-9.10%)`	⬇️
cve_bin_tool/extractor.py	`63.63% <0.00%> (-2.73%)`	⬇️
cve_bin_tool/checkers/__init__.py	`100.00% <0.00%> (ø)`
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 367926b...46a1b5e. Read the comment docs.

Niraj-Kamdar · 2020-05-15T15:55:36Z

Nice!!! Great work finding the fix for this. Very crafty

Thank you. Always up for challenging problems.

egg_updater.py

cve_bin_tool/cli.py

pdxjohnny

Looking good (after these two minor changes)

Co-authored-by: John Andersen <johnandersenpdx@gmail.com>

cve_bin_tool/cli.py

pdxjohnny · 2020-05-16T16:27:54Z

egg_updater.py

+def update_egg():
+    with StringIO() as f:
+        sys.stdout = f
+        dist = Distribution(


We should probably do a try: os.chdir to os.path.join(os.path.dirname(__file__), "..") and then a finally: os.chdir(wherever_os.getcwd()_reported_before_first_chdir). Since I'd guess that this fails if the user is not in the root of the source tree

Yes this will work.

I have tried it, it's not working

What's not working?

In the above example: os.path.join(os.path.dirname(__file__), "..") should be os.path.dirname(__file__)

I have put egg_updater inside cve_bin_tool and save cwd by cwd = os.getcwd() and then changed diectory to os.chdir(os.path.join(os.path.dirname(__file__), "..")) and after execution of script, I have reverted back to the previous cwd but it is giving same error as we get without applying chdir hack.

(venv) C:\Users\Root\Documents\cve-bin-tool>python -m cve_bin_tool.cli build cve_bin_tool - WARNING - ********************************************** Warning: this utility was developed for Linux. You may need to install additional utilities to use it on other operating systems. ********************************************** cve_bin_tool.CVEDB - INFO - Using cached CVE data (<24h old). Use -u now to update immediately. cve_bin_tool.Scanner - INFO - Updating egg_info C:\Users\Root\Documents\cve-bin-tool C:\Users\Root\Documents\cve-bin-tool\cve_bin_tool Traceback (most recent call last): File "C:\Users\Root\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "C:\Users\Root\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\Root\Documents\cve-bin-tool\cve_bin_tool\cli.py", line 587, in <module> sys.exit(main()) File "C:\Users\Root\Documents\cve-bin-tool\cve_bin_tool\cli.py", line 471, in main scanner = Scanner(cvedb) File "C:\Users\Root\Documents\cve-bin-tool\cve_bin_tool\cli.py", line 72, in __init__ egg_updater.update_egg() File "C:\Users\Root\Documents\cve-bin-tool\cve_bin_tool\egg_updater.py", line 35, in update_egg "checkers", FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\Root\\Documents\\cve-bin-tool\\cve_bin_tool\\cve_bin_tool\\c heckers'

It's solved.

cve_bin_tool/cli.py

update cli.py

Niraj-Kamdar · 2020-05-18T21:36:12Z

My alternative didn't work. We can't exclude a single script from the package using find_packages's exclude option. It was working before because root-level script automatically get converted into an package by python but since now we have put egg_updater script inside cve_bin_tool package, egg_updater won't get promoted to the package. So, for now developer has to install cve_bin_tool using pip install -e command to get benefit of automatic egg updater.

pdxjohnny · 2020-05-19T15:50:52Z

cve_bin_tool/cli.py

+    spec.loader.exec_module(egg_updater)
+else:
+    egg_updater = None
+DEVELOP = True


This block isn't necessary now that we've moved the module into the library, we can import with from . import egg_updater

I have done it.

pdxjohnny

That's a bummer about exclude_files :( Just one thing left I think

codecov-commenter · 2020-05-19T16:04:22Z

Codecov Report

Merging #672 into master will decrease coverage by 0.38%.
The diff coverage is 61.29%.

@@            Coverage Diff             @@
##           master     #672      +/-   ##
==========================================
- Coverage   80.95%   80.56%   -0.39%     
==========================================
  Files          67       68       +1     
  Lines        2168     2187      +19     
  Branches      281      284       +3     
==========================================
+ Hits         1755     1762       +7     
- Misses        346      357      +11     
- Partials       67       68       +1

Flag	Coverage Δ
#longtests	`80.56% <61.29%> (-0.39%)`	⬇️

Impacted Files	Coverage Δ
cve_bin_tool/egg_updater.py	`43.75% <43.75%> (ø)`
cve_bin_tool/cli.py	`77.58% <80.00%> (-0.82%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3ebc7da...06b09a1. Read the comment docs.

pdxjohnny · 2020-05-19T18:16:46Z

Awesome!! Thank you!!

add egg updater script

c7e319a

add update_egg to cli

Niraj-Kamdar changed the title ~~add egg updater script which fixes pytest initialization issue #605~~ add egg updater script which fixes pytest initialization issue May 14, 2020

terriko reviewed May 14, 2020

View reviewed changes

cve_bin_tool/cli.py Outdated Show resolved Hide resolved

terriko requested changes May 14, 2020

View reviewed changes

cve_bin_tool/cli.py Outdated Show resolved Hide resolved

Niraj-Kamdar added 2 commits May 15, 2020 19:15

revert cli.py

b8ceae2

update cli.py

6ff5e7c

pdxjohnny reviewed May 15, 2020

View reviewed changes

cve_bin_tool/cli.py Outdated Show resolved Hide resolved

pdxjohnny requested changes May 15, 2020

View reviewed changes

only update checker if develop env set

2d6fbf7

Niraj-Kamdar requested a review from pdxjohnny May 15, 2020 15:54

Niraj-Kamdar requested a review from terriko May 15, 2020 15:56

add IS_PRODUCTION function to check for production build

46a1b5e

Niraj-Kamdar force-pushed the egg branch from 1301d0a to 46a1b5e Compare May 15, 2020 18:41

pdxjohnny reviewed May 15, 2020

View reviewed changes

egg_updater.py Outdated Show resolved Hide resolved

pdxjohnny reviewed May 15, 2020

View reviewed changes

cve_bin_tool/cli.py Outdated Show resolved Hide resolved

pdxjohnny reviewed May 15, 2020

View reviewed changes

Update cve_bin_tool/cli.py

01d731e

Co-authored-by: John Andersen <johnandersenpdx@gmail.com>

anthonyharrison added a commit to anthonyharrison/cve-bin-tool that referenced this pull request May 16, 2020

Version column refactoring (intel#672)

62a983a

pdxjohnny reviewed May 16, 2020

View reviewed changes

cve_bin_tool/cli.py Show resolved Hide resolved

Exclude egg_updater in the distribution

57cbf0a

pdxjohnny reviewed May 16, 2020

View reviewed changes

cve_bin_tool/cli.py Outdated Show resolved Hide resolved

pdxjohnny reviewed May 16, 2020

View reviewed changes

cve_bin_tool/cli.py Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

Niraj-Kamdar requested a review from pdxjohnny May 16, 2020 16:53

This comment has been minimized.

Sign in to view

update cli.py

bed40fb

update cli.py

Niraj-Kamdar force-pushed the egg branch from e2f8b18 to bed40fb Compare May 18, 2020 19:12

Merge remote-tracking branch 'upstream/master' into egg

3632d45

Niraj-Kamdar force-pushed the egg branch from c6191ad to 3632d45 Compare May 18, 2020 19:32

Niraj-Kamdar added 3 commits May 19, 2020 02:04

move egg_updater inside the cve_bin_tool/

a68be10

black format

b02e788

include IS_DEVELOP script

bb7adbd

pdxjohnny reviewed May 19, 2020

View reviewed changes

update cli.py

1ba9531

Niraj-Kamdar requested a review from pdxjohnny May 19, 2020 16:04

remove importlib

06b09a1

Niraj-Kamdar force-pushed the egg branch from a7a34c2 to 06b09a1 Compare May 19, 2020 17:54

pdxjohnny merged commit 285cf64 into intel:master May 19, 2020

Niraj-Kamdar deleted the egg branch May 19, 2020 19:23

terriko mentioned this pull request May 19, 2020

Known issue workaround: new checkers don't work correctly #677

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add egg updater script which fixes pytest initialization issue #672

add egg updater script which fixes pytest initialization issue #672

Niraj-Kamdar commented May 14, 2020

Niraj-Kamdar commented May 14, 2020

terriko commented May 14, 2020

pdxjohnny left a comment

codecov-io commented May 15, 2020 •

edited

Loading

Niraj-Kamdar commented May 15, 2020

pdxjohnny left a comment

pdxjohnny May 16, 2020 •

edited

Loading

Niraj-Kamdar May 16, 2020

Niraj-Kamdar May 18, 2020

pdxjohnny May 18, 2020

pdxjohnny May 18, 2020

Niraj-Kamdar May 18, 2020 •

edited

Loading

Niraj-Kamdar May 18, 2020

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Niraj-Kamdar commented May 18, 2020 •

edited

Loading

pdxjohnny May 19, 2020

Niraj-Kamdar May 19, 2020 •

edited

Loading

pdxjohnny left a comment

codecov-commenter commented May 19, 2020 •

edited

Loading

pdxjohnny commented May 19, 2020

add egg updater script which fixes pytest initialization issue #672

add egg updater script which fixes pytest initialization issue #672

Conversation

Niraj-Kamdar commented May 14, 2020

Niraj-Kamdar commented May 14, 2020

terriko commented May 14, 2020

pdxjohnny left a comment

Choose a reason for hiding this comment

codecov-io commented May 15, 2020 • edited Loading

Codecov Report

Niraj-Kamdar commented May 15, 2020

pdxjohnny left a comment

Choose a reason for hiding this comment

pdxjohnny May 16, 2020 • edited Loading

Choose a reason for hiding this comment

Niraj-Kamdar May 16, 2020

Choose a reason for hiding this comment

Niraj-Kamdar May 18, 2020

Choose a reason for hiding this comment

pdxjohnny May 18, 2020

Choose a reason for hiding this comment

pdxjohnny May 18, 2020

Choose a reason for hiding this comment

Niraj-Kamdar May 18, 2020 • edited Loading

Choose a reason for hiding this comment

Niraj-Kamdar May 18, 2020

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Niraj-Kamdar commented May 18, 2020 • edited Loading

pdxjohnny May 19, 2020

Choose a reason for hiding this comment

Niraj-Kamdar May 19, 2020 • edited Loading

Choose a reason for hiding this comment

pdxjohnny left a comment

Choose a reason for hiding this comment

codecov-commenter commented May 19, 2020 • edited Loading

Codecov Report

pdxjohnny commented May 19, 2020

codecov-io commented May 15, 2020 •

edited

Loading

pdxjohnny May 16, 2020 •

edited

Loading

Niraj-Kamdar May 18, 2020 •

edited

Loading

Niraj-Kamdar commented May 18, 2020 •

edited

Loading

Niraj-Kamdar May 19, 2020 •

edited

Loading

codecov-commenter commented May 19, 2020 •

edited

Loading