StaticSrcUri: handle more cases by arthurzam · Pull Request #453 · pkgcore/pkgcheck

arthurzam · 2022-09-28T18:44:38Z

handle various new cases, and a little less permissive regex
add cases of usage of ver_rs and ver_cut
suggest the user with replacement

The new logic:

A match must start with / or PN[-._]? - I don't want random suggestion because it somehow matches somewhere in commit id.
Possible separator of [v]?
After that I match all the following variants (of course the literal and not variable):
- ${P}
- ${PV}
- $(ver_cut 1-n) where n > 1 - I skip the variant of $(ver_cut 1) because of large amount of false positives
- $(ver_rs 1-n '#') where n > 0 and # is one of Ɛ (empty str), _, -. Can add other symbols.

Sadly, the code for generating the regex became an abomination.

new StaticSrcUri results on gentoo tree (v2)

app-admin/analog
  StaticSrcUri: version 6.0.13: '6013' in SRC_URI, replace with ${PV//.}

app-backup/duply
  StaticSrcUri: version 2.3.1: '2.3' in SRC_URI, replace with $(ver_cut 1-2)

app-crypt/osslsigncode
  StaticSrcUri: version 2.0: '2.0' in SRC_URI, replace with ${PV}

app-editors/vim
  StaticSrcUri: version 8.2.4586: '8.2' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 8.2.5066-r1: 'vim-8.2.5066' in SRC_URI, replace with ${P}
  StaticSrcUri: version 9.0.0049: '9.0.0049' in SRC_URI, replace with ${PV}
  StaticSrcUri: version 9.0.0099: '9.0' in SRC_URI, replace with $(ver_cut 1-2)

app-editors/xemacs
  StaticSrcUri: version 21.4.24-r5: '21.4' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 21.4.24-r7: '21.4' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 21.5.34-r10: '21.5' in SRC_URI, replace with $(ver_cut 1-2)

app-i18n/libchewing
  StaticSrcUri: version 0.5.1-r1: 'libchewing-0.5.1' in SRC_URI, replace with ${P}
  StaticSrcUri: version 0.5.1-r1: 'libchewing-0.5.1' in SRC_URI, replace with ${P}

app-i18n/scim-anthy
  StaticSrcUri: version 1.2.7-r1: 'scim-anthy-1.2.7' in SRC_URI, replace with ${P}

app-misc/jpipe
  StaticSrcUri: version 0.2.0-r1: 'jpipe-0.2.0' in SRC_URI, replace with ${P}

app-office/lyx
  StaticSrcUri: version 2.3.6.1-r1: '2.3' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 2.3.6.1-r2: '2.3' in SRC_URI, replace with $(ver_cut 1-2)

app-office/upwork
  StaticSrcUri: version 5.4.5.1-r1: '5_4_5_1' in SRC_URI, replace with ${PV//./_}
  StaticSrcUri: version 5.4.5.1-r1: 'upwork-5.4.5.1' in SRC_URI, replace with ${P}

app-text/calibre
  StaticSrcUri: version 5.16.1-r1: '5.16' in SRC_URI, replace with $(ver_cut 1-2)

app-text/fbreader
  StaticSrcUri: version 0.99.4-r6: 'fbreader-0.99.4' in SRC_URI, replace with ${P}

app-text/gnome-doc-utils
  StaticSrcUri: version 0.20.10-r2: 'gnome-doc-utils-0.20.10' in SRC_URI, replace with ${P}

dev-db/gqlplus
  StaticSrcUri: version 1.16-r2: 'gqlplus-1.16' in SRC_URI, replace with ${P}

dev-db/mysql
  StaticSrcUri: version 5.7.36-r1: '5.7' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 8.0.27: '8.0' in SRC_URI, replace with $(ver_cut 1-2)

dev-java/janino
  StaticSrcUri: version 3.1.7: '3.1.7' in SRC_URI, replace with ${PV}

dev-java/openjdk
  StaticSrcUri: version 11.0.15_p10-r1: '11.0' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 11.0.16.1_p1: '11.0' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 17.0.3_p7-r1: '17.0' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 17.0.4.1_p1: '17.0' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 17.0.5_p5: '17.0' in SRC_URI, replace with $(ver_cut 1-2)

dev-lang/nim
  StaticSrcUri: version 1.6.8: '1.6' in SRC_URI, replace with $(ver_cut 1-2)

dev-lang/perl
  StaticSrcUri: version 5.34.1-r3: 'perl-5.34.1' in SRC_URI, replace with ${P}

dev-lang/zig
  StaticSrcUri: version 0.9.1-r1: 'zig-0.9.1' in SRC_URI, replace with ${P}

dev-libs/eventlog
  StaticSrcUri: version 0.2.12: '0.2' in SRC_URI, replace with $(ver_cut 1-2)

dev-libs/libgamin
  StaticSrcUri: version 0.1.10-r6: 'libgamin-0.1.10' in SRC_URI, replace with ${P}

dev-libs/openssl-compat
  StaticSrcUri: version 1.0.2u-r2: 'openssl-compat-1.0.2u' in SRC_URI, replace with ${P}

dev-libs/ppl
  StaticSrcUri: version 1.2-r4: 'ppl-1.2' in SRC_URI, replace with ${P}
  StaticSrcUri: version 1.2-r5: 'ppl-1.2' in SRC_URI, replace with ${P}

dev-libs/xbyak
  StaticSrcUri: version 5.73: '5.73' in SRC_URI, replace with ${PV}

dev-ml/opam
  StaticSrcUri: version 2.1.2-r1: '2.1' in SRC_URI, replace with $(ver_cut 1-2)

dev-php/theseer-Autoload
  StaticSrcUri: version 1.26.0-r2: '1.26.0' in SRC_URI, replace with ${PV}

dev-scheme/owl-lisp
  StaticSrcUri: version 0.2: '0.2' in SRC_URI, replace with ${PV}

dev-tcltk/tklib
  StaticSrcUri: version 0.6-r2: 'tklib-0.6' in SRC_URI, replace with ${P}

dev-tcltk/tkimg
  StaticSrcUri: version 1.4.9: '1.4' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 1.4.13-r1: '1.4' in SRC_URI, replace with $(ver_cut 1-2)

dev-util/bitcoin-tx
  StaticSrcUri: version 0.20.1: '0.20' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 0.21.0: '0.21' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 0.21.1: '0.21' in SRC_URI, replace with $(ver_cut 1-2)

dev-util/idea-community
  StaticSrcUri: version 2022.2.2: 'idea-community-2022.2.2' in SRC_URI, replace with ${P}

dev-util/packer
  StaticSrcUri: version 1.7.10: 'packer-1.7.10' in SRC_URI, replace with ${P}

games-arcade/slimevolley
  StaticSrcUri: version 2.4.2-r1: '242' in SRC_URI, replace with ${PV//.}

games-emulation/advancescan
  StaticSrcUri: version 1.16-r1: 'advancescan-1.16' in SRC_URI, replace with ${P}

games-emulation/melonds
  StaticSrcUri: version 0.9.4_p1-r1: '0.9' in SRC_URI, replace with $(ver_cut 1-2)

games-puzzle/gnurobbo
  StaticSrcUri: version 0.68: '0.68' in SRC_URI, replace with ${PV}

games-puzzle/icebreaker
  StaticSrcUri: version 1.9.6-r1: '1.9' in SRC_URI, replace with $(ver_cut 1-2)

games-rpg/queen
  StaticSrcUri: version 1-r1: '1' in SRC_URI, replace with ${PV}

games-util/xboxdrv
  StaticSrcUri: version 0.8.8_p20190118-r1: '0.8' in SRC_URI, replace with $(ver_cut 1-2)

mail-filter/policyd
  StaticSrcUri: version 1.82-r4: '1.82' in SRC_URI, replace with ${PV}

mail-filter/sqlgrey
  StaticSrcUri: version 1.8.0: '1.8' in SRC_URI, replace with $(ver_cut 1-2)

media-fonts/ahem
  StaticSrcUri: version 1.0: '1.0' in SRC_URI, replace with ${PV}

media-gfx/gimp
  StaticSrcUri: version 2.10.32: '2.10' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 2.99.10-r1: '2.99' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 2.99.12: '2.99' in SRC_URI, replace with $(ver_cut 1-2)

media-gfx/gnuclad
  StaticSrcUri: version 0.2.4: '0.2' in SRC_URI, replace with $(ver_cut 1-2)

media-gfx/inkscape
  StaticSrcUri: version 1.1-r1: 'inkscape-1.1' in SRC_URI, replace with ${P}

media-libs/freeimage
  StaticSrcUri: version 3.18.0-r8: 'freeimage-3.18.0' in SRC_URI, replace with ${P}

media-plugins/gmpc-alarm
  StaticSrcUri: version 11.8.16: '11.8' in SRC_URI, replace with $(ver_cut 1-2)

media-plugins/gmpc-albumview
  StaticSrcUri: version 11.8.16: '11.8' in SRC_URI, replace with $(ver_cut 1-2)

media-plugins/gmpc-avahi
  StaticSrcUri: version 11.8.16: '11.8' in SRC_URI, replace with $(ver_cut 1-2)

media-plugins/gmpc-awn
  StaticSrcUri: version 11.8.16: '11.8' in SRC_URI, replace with $(ver_cut 1-2)

media-plugins/gmpc-jamendo
  StaticSrcUri: version 11.8.16: '11.8' in SRC_URI, replace with $(ver_cut 1-2)

media-plugins/gmpc-lyricwiki
  StaticSrcUri: version 11.8.16: '11.8' in SRC_URI, replace with $(ver_cut 1-2)

media-plugins/gmpc-mmkeys
  StaticSrcUri: version 11.8.16: '11.8' in SRC_URI, replace with $(ver_cut 1-2)

media-plugins/vdr-undelete
  StaticSrcUri: version 2.3.0_pre1: '2.3' in SRC_URI, replace with $(ver_cut 1-2)

media-plugins/zam-plugins
  StaticSrcUri: version 3.14-r1: '3.14' in SRC_URI, replace with ${PV}

media-sound/cd-discid
  StaticSrcUri: version 1.4-r1: '1.4' in SRC_URI, replace with ${PV}

media-sound/gmpc
  StaticSrcUri: version 11.8.16-r2: '11.8' in SRC_URI, replace with $(ver_cut 1-2)

media-tv/ivtv-utils
  StaticSrcUri: version 1.4.1: '1.4' in SRC_URI, replace with $(ver_cut 1-2)

media-tv/v4l-dvb-saa716x
  StaticSrcUri: version 0.0.1_p20170225-r5: '0.0' in SRC_URI, replace with $(ver_cut 1-2)

media-video/vdr
  StaticSrcUri: version 2.4.7-r1: '2.4' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 2.4.7-r1: '2.4.7' in SRC_URI, replace with ${PV}

net-analyzer/tcptrace
  StaticSrcUri: version 6.6.7_p6: '6.6' in SRC_URI, replace with $(ver_cut 1-2)

net-analyzer/zabbix
  StaticSrcUri: version 4.0.43: '4.0' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 4.0.44: '4.0' in SRC_URI, replace with $(ver_cut 1-2)

net-im/swift
  StaticSrcUri: version 4.0.2-r102: 'swift-4.0.2' in SRC_URI, replace with ${P}

net-irc/znc-clientbuffer
  StaticSrcUri: version 1.0.48: '1.0.48' in SRC_URI, replace with ${PV}

net-libs/libbitcoinconsensus
  StaticSrcUri: version 0.20.1: '0.20' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 0.21.0: '0.21' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 0.21.1: '0.21' in SRC_URI, replace with $(ver_cut 1-2)

net-misc/drive
  StaticSrcUri: version 0.4.0_p20210208: 'drive-0.4.0_p20210208' in SRC_URI, replace with ${P}

net-misc/sitecopy
  StaticSrcUri: version 0.16.6-r2: 'sitecopy-0.16.6' in SRC_URI, replace with ${P}

net-p2p/bitcoin-cli
  StaticSrcUri: version 0.20.1: '0.20' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 0.21.0: '0.21' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 0.21.1: '0.21' in SRC_URI, replace with $(ver_cut 1-2)

net-p2p/bitcoind
  StaticSrcUri: version 0.21.0: '0.21' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 0.21.1: '0.21' in SRC_URI, replace with $(ver_cut 1-2)

net-p2p/bitcoin-qt
  StaticSrcUri: version 0.21.0: '0.21' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 0.21.1: '0.21' in SRC_URI, replace with $(ver_cut 1-2)

net-p2p/deluge
  StaticSrcUri: version 2.0.5-r2: '2.0' in SRC_URI, replace with $(ver_cut 1-2)

net-proxy/haproxy-dataplaneapi
  StaticSrcUri: version 2.5.1: 'haproxy-dataplaneapi-2.5.1' in SRC_URI, replace with ${P}

sci-biology/pilercr
  StaticSrcUri: version 1.0-r2: '1.0' in SRC_URI, replace with ${PV}

sci-biology/treeviewx
  StaticSrcUri: version 0.5.1-r3: '0.5' in SRC_URI, replace with $(ver_cut 1-2)

sci-libs/p4est
  StaticSrcUri: version 2.8.0-r1: '2.8' in SRC_URI, replace with $(ver_cut 1-2)

sci-mathematics/geogebra-bin
  StaticSrcUri: version 5.0.604.0-r2: '5.0' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 5.0.683.0: '5.0' in SRC_URI, replace with $(ver_cut 1-2)

sys-apps/ipmitool
  StaticSrcUri: version 1.8.18_p20201004-r3: '1.8' in SRC_URI, replace with $(ver_cut 1-2)

sys-auth/pam_ssh_agent_auth
  StaticSrcUri: version 0.10.3: 'pam_ssh_agent_auth-0.10.3' in SRC_URI, replace with ${P}

sys-boot/grub
  StaticSrcUri: version 2.06-r3: 'grub-2.06' in SRC_URI, replace with ${P}

sys-cluster/torque
  StaticSrcUri: version 6.0.4-r2: 'torque-6.0.4' in SRC_URI, replace with ${P}

sys-fs/dislocker
  StaticSrcUri: version 0.7.1-r3: 'dislocker-0.7.1' in SRC_URI, replace with ${P}

sys-fs/yaffs2utils
  StaticSrcUri: version 0.2.9: '0.2.9' in SRC_URI, replace with ${PV}

sys-libs/libcap-ng
  StaticSrcUri: version 0.8.3: '0.8' in SRC_URI, replace with $(ver_cut 1-2)

sys-libs/glibc
  StaticSrcUri: version 2.30-r9: 'glibc-2.30' in SRC_URI, replace with ${P}
  StaticSrcUri: version 2.35-r8: 'glibc-2.35' in SRC_URI, replace with ${P}
  StaticSrcUri: version 2.35-r10: 'glibc-2.35' in SRC_URI, replace with ${P}

sys-process/numad
  StaticSrcUri: version 0.5-r3: 'numad-0.5' in SRC_URI, replace with ${P}
  StaticSrcUri: version 0.5-r4: 'numad-0.5' in SRC_URI, replace with ${P}

x11-libs/gtk-mac-integration
  StaticSrcUri: version 2.0.5-r1: '2.0' in SRC_URI, replace with $(ver_cut 1-2)

x11-libs/wxGTK
  StaticSrcUri: version 3.0.4-r3: 'wxGTK-3.0.4' in SRC_URI, replace with ${P}
  StaticSrcUri: version 3.0.5.1: '3.0.5' in SRC_URI, replace with $(ver_cut 1-3)

x11-misc/x2x
  StaticSrcUri: version 1.27-r3: '1.27' in SRC_URI, replace with ${PV}

x11-themes/gentoo-artwork
  StaticSrcUri: version 0.4.2-r1: '0.4' in SRC_URI, replace with $(ver_cut 1-2)

Resolves: #450, #451

codecov · 2022-09-28T18:50:05Z

Codecov Report

Base: 95.97% // Head: 81.22% // Decreases project coverage by -14.74% ⚠️

Coverage data is based on head (f30b44e) compared to base (678888a).
Patch coverage: 100.00% of modified lines in pull request are covered.

❗ Current head f30b44e differs from pull request most recent head 6d8282f. Consider uploading reports for the commit 6d8282f to get more accurate results

Additional details and impacted files

@@             Coverage Diff             @@
##           master     #453       +/-   ##
===========================================
- Coverage   95.97%   81.22%   -14.75%     
===========================================
  Files          55       55               
  Lines        7824     7846       +22     
  Branches     2225     2234        +9     
===========================================
- Hits         7509     6373     -1136     
- Misses        194     1375     +1181     
+ Partials      121       98       -23

Impacted Files	Coverage Δ
src/pkgcheck/checks/codingstyle.py	`72.67% <100.00%> (-24.54%)`	⬇️
src/pkgcheck/checks/eclass.py	`51.96% <0.00%> (-44.10%)`	⬇️
src/pkgcheck/checks/visibility.py	`43.92% <0.00%> (-42.53%)`	⬇️
src/pkgcheck/addons/net.py	`60.71% <0.00%> (-39.29%)`	⬇️
src/pkgcheck/checks/network.py	`40.22% <0.00%> (-39.11%)`	⬇️
src/pkgcheck/checks/overlays.py	`60.00% <0.00%> (-34.67%)`	⬇️
src/pkgcheck/checks/reserved.py	`66.17% <0.00%> (-33.83%)`	⬇️
src/pkgcheck/checks/python.py	`64.24% <0.00%> (-33.80%)`	⬇️
src/pkgcheck/checks/unstable_only.py	`66.66% <0.00%> (-33.34%)`	⬇️
src/pkgcheck/pipeline.py	`61.76% <0.00%> (-33.09%)`	⬇️
... and 15 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

mgorny · 2022-09-28T20:33:10Z

app-office/lyx
  StaticSrcUri: version 2.3.6.1-r1: '2.3' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 2.3.6.1-r1: '2.3' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 2.3.6.1-r2: '2.3' in SRC_URI, replace with $(ver_cut 1-2)
  StaticSrcUri: version 2.3.6.1-r2: '2.3' in SRC_URI, replace with $(ver_cut 1-2)

Perhaps deduplicate this to report every (pkg, match) occurence once?

mgorny · 2022-09-28T20:35:24Z

app-admin/analog
  StaticSrcUri: version 6.0.13: '6013' in SRC_URI, replace with $(ver_rs 1-2 '')

For extra kudos, perhaps you could special case removing all separators using the same character into ${PV//.}

arthurzam · 2022-09-29T17:53:31Z

Added the following replacements:

${P^} and ${P^^}
${#//.} where # can be one of P, PV, P^, P^^ - for example it will catch ${P^//.}
${#//./%} where # can be one of P, PV, P^, P^^ and % of _ or - - for example it will catch ${P^^//./_}
instead of previous $(ver_rs 1-1 '_') it will show $(ver_rs 1 '_')

Updated results for Gentoo tree are in the first comment. No result suggesting ^ in var was caught.

thesamesam

LGTM although worried about coverage, surely it's wrong?

- handle various new cases, and a little less permissive regex - add cases of usage of `ver_rs` and `ver_cut` - suggest the user with replacement Resolves: pkgcore#450 Resolves: pkgcore#451 Signed-off-by: Arthur Zamarin <arthurzam@gentoo.org>

arthurzam · 2022-09-30T05:27:09Z

LGTM although worried about coverage, surely it's wrong?

Thanks. Yes, the coverage is wrong, because of multiple force-pushes by me at the beginning.

arthurzam requested a review from thesamesam September 28, 2022 18:44

arthurzam linked an issue Sep 28, 2022 that may be closed by this pull request

StaticSrcUri could detect more elaborate patterns #451

Closed

thesamesam reviewed Sep 29, 2022

View reviewed changes

Comment thread tests/checks/test_codingstyle.py

thesamesam approved these changes Sep 30, 2022

View reviewed changes

StaticSrcUri: handle more cases

6d8282f

- handle various new cases, and a little less permissive regex - add cases of usage of `ver_rs` and `ver_cut` - suggest the user with replacement Resolves: pkgcore#450 Resolves: pkgcore#451 Signed-off-by: Arthur Zamarin <arthurzam@gentoo.org>

arthurzam force-pushed the new-static-srcuri branch from f30b44e to 6d8282f Compare September 30, 2022 05:26

arthurzam merged commit 6d8282f into pkgcore:master Sep 30, 2022

arthurzam deleted the new-static-srcuri branch September 30, 2022 05:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

StaticSrcUri: handle more cases#453

StaticSrcUri: handle more cases#453
arthurzam merged 1 commit intopkgcore:masterfrom
arthurzam:new-static-srcuri

arthurzam commented Sep 28, 2022 •

edited

Loading

Uh oh!

codecov Bot commented Sep 28, 2022 •

edited

Loading

Uh oh!

mgorny commented Sep 28, 2022

Uh oh!

mgorny commented Sep 28, 2022

Uh oh!

Uh oh!

arthurzam commented Sep 29, 2022

Uh oh!

thesamesam left a comment

Uh oh!

arthurzam commented Sep 30, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

arthurzam commented Sep 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Sep 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mgorny commented Sep 28, 2022

Uh oh!

mgorny commented Sep 28, 2022

Uh oh!

Uh oh!

arthurzam commented Sep 29, 2022

Uh oh!

thesamesam left a comment

Choose a reason for hiding this comment

Uh oh!

arthurzam commented Sep 30, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

arthurzam commented Sep 28, 2022 •

edited

Loading

codecov Bot commented Sep 28, 2022 •

edited

Loading