MONGOID-5304: Add Rubocop static code analyzer

[johnnyshields]

This PR adds a static code analyzer (Rubocop) to the project, as required by MongoDB's Technical and Organizational Security Measures.

It is integrated with Github CI. To add it to Evergreen, simply run this shell command (only needs to be run once for the entire suite; not dependent on Ruby version, etc.) I've done a draft of Evergreen integration in this PR, but it may not work as I cannot run Evergreen myself.

bundle exec rake rubocop

Refer to the related Mongoid PR for further explanation.

[johnnyshields]

Ready to merge, please check Evergreen setup works.

[p-mongo]

Hi @johnnyshields ,

As per https://jira.mongodb.org/browse/MONGOID-5304, if you would like to adjust this PR to only enable non-style/formatting checks, we will review it.

[johnnyshields]

@p-mongo this is updated as per your comments in https://jira.mongodb.org/browse/MONGOID-5304

[p-mongo]

This is almost working but in evergreen the Ruby installation is under our tree and is checked by rubocop, it needs to be reconfigured to only explicitly check lib.

[johnnyshields]

@p-mongo I can't see what Evergreen is doing, but you should be able to set:

AllCops:
  exclude:
    - "ruby/dir/**/*"

[p-mongo]

I could, but I would like lib to be included instead.

[johnnyshields]

@p-mongo is this done? Please kindly merge and then I'll port this to Mongoid and Ruby Driver libs.

[p-mongo]

No, the checks should only be done on lib.

[johnnyshields]

@p-mongo I've updated to only include lib dir on a white-list basis. Please review and merge.

[p-mongo]

This needs to be the top-level lib only, not any of the lib subdirs in the current directory:

[2022/06/27 02:13:57.329] Running tests
[2022/06/27 02:13:57.329] Inspecting 144 files
[2022/06/27 02:13:57.329] ....................................................................................C................................................C..........
[2022/06/27 02:13:57.329] Offenses:
[2022/06/27 02:13:57.329] rubies/ruby-3.1.0/lib/ruby/gems/3.1.0/gems/ruby-prof-1.4.3/bin/ruby-prof:279:9: C: Security/Eval: The use of eval is a serious security risk.
[2022/06/27 02:13:57.329]         eval(exec)
[2022/06/27 02:13:57.329]         ^^^^
[2022/06/27 02:13:57.329] rubies/ruby-3.1/lib/ruby/gems/3.1.0/gems/ruby-prof-1.4.3/bin/ruby-prof:279:9: C: Security/Eval: The use of eval is a serious security risk.
[2022/06/27 02:13:57.329]         eval(exec)
[2022/06/27 02:13:57.329]         ^^^^
[2022/06/27 02:13:57.329] 144 files inspected, 2 offenses detected

[johnnyshields]

Hmm... this failure looks Evergreen specific. The syntax I'm using here Include: - lib/**/* is standard as per docs and means "from project root".

You can see the Github actions build is passing here: https://github.com/mongodb/bson-ruby/runs/7051219018?check_suite_focus=true

Note in particular there are 43 files scanned in Github Actions, vs 144 in your log:

Run bundle exec rubocop --parallel -c .rubocop.yml
[bson] Warning: No private key present, creating unsigned gem.
Inspecting 43 files
...........................................
43 files inspected, no offenses detected

Since I can't access Evergreen, you're going to have to debug this. Some ideas:

• check Rubocop is run from project root (not system root).
• check that the ruby-prof gem isn't being copied into the scan dir as part of the build process.
• try Include: arg as /path/from/systemroot/bson-ruby/lib/**/*

[p-mongo]

We have the Ruby installation together with gems under the project root in evergreen.

The syntax I'm using here Include: - lib/**/* is standard as per docs and means "from project root".

lib/**/* in shell means ./lib/**/*. In Rubocop this apparently means ./**/lib/**/*. After a quick look at the configuration page I couldn't find where Include is described, but the configuration in this PR needs to match the shell behavior of the glob as I just described it.

[johnnyshields]

As a shot in the dark, let's try /lib/**/* instead of lib/**/*

[johnnyshields]

GH actions passed. Please try running Evergreen and kindly let me know what happens.

[p-mongo]

Please find out whether /lib/**/* actually traverses from filesystem root and report your findings.

[johnnyshields]

So /lib/**/* won't work. :(

I've configured rubocop with the -L flag which will dump out the list of files that it's seeing. Please run Evergreen on the current commit and kindly let me know the output, and that should help me further debug the issue.

[p-mongo]

I don't understand why you need to run anything on evergreen. The requirement is that rubocop only checks the top-level lib directory. So you should be able to create a local project that has:

lib/foo.rb - with an eval in it
bar/lib/foo.rb - also with an eval in it

and then work out a configuration which reports the error for the first file but not the second.

[johnnyshields]

@p-mongo I have done this already. Here's the evidence:

/workspace/rubocop-test$ ruby -e "puts Dir.glob('**/*')"
foo
foo/lib
foo/lib/test.rb
foo/lib/test1
foo/lib/test1/test.rb
lib
lib/test.rb
lib/test1
lib/test1/test.rb

# this is using the exact rubocop.yml in this PR
# the -L command outputs which files Rubocop is scanning
/workspace/rubocop-test$ rubocop -L
lib/test.rb
lib/test1/test.rb

# note there is also a /lib dir in my drive root with Ruby files in it.

Please kindly run Evergreen as a next step, thanks!

[p-mongo]

This is the same line as it was before, no?

[p-mongo]

This is the same line as it was before, no?

[p-mongo]

Creative. So what is this value documented to be exactly?

[johnnyshields]

Ah sadly that Regexp didn't work. So I got more creative 😂

Check the latest, I'm 95% confident this will work on Evergreen as well.

[p-mongo]

If there isn't a normal way to tell rubocop to only check files in top-level lib subdir, please file this as an enhancement request with them and/or contribute the functionality to rubocop. The various workarounds I'm seeing in this PR, entertaining as they are, are not appropriate for using, of all things, a code quality tool.

[johnnyshields]

Yeah, agreed, I've raised rubocop/rubocop#10802 for this issue and added it to the code comment.

I'll raise a follow-up PR if/when its fixed on their end. Suggest we merge this as-is, even though that line ain't pretty.

FWIW, Include-ing a dir like this is not a common use case. I've seen Rubocop in 100s of open source projects and I've never seen it used like this; it's always on an Exclude basis.

[johnnyshields]

Closing; replaced by #303