Fix #5181 - when using --bundle in the CLI, use Embedded native gems when they are present and version compatible

[jmarrec]

Pull request overview

• Fixes CLI issue when using bundle and it has a native gem listed as a dependency #5181

Pull Request Author

• Model API Changes / Additions
• Any new or modified fields have been implemented in the EnergyPlus ForwardTranslator (and ReverseTranslator as appropriate)
• Model API methods are tested (in src/model/test)
• EnergyPlus ForwardTranslator Tests (in src/energyplus/Test)
• If a new object or method, added a test in NREL/OpenStudio-resources: Add Link
• If needed, added VersionTranslation rules for the objects (src/osversion/VersionTranslator.cpp)
• Verified that C# bindings built fine on Windows, partial classes used as needed, etc.
• All new and existing tests passes
• If methods have been deprecated, update rest of code to use the new methods

Labels:

• If change to an IDD file, add the label IDDChange
• If breaking existing API, add the label APIChange
• If deemed ready, add label Pull Request - Ready for CI so that CI builds your PR

Review Checklist

This will not be exhaustively relevant to every PR.

• Perform a Code Review on GitHub
• Code Style, strip trailing whitespace, etc.
• All related changes have been implemented: model changes, model tests, FT changes, FT tests, VersionTranslation, OS App
• Labeling is ok
• If defect, verify by running develop branch and reproducing defect, then running PR and reproducing fix
• If feature, test running new feature, try creative ways to break it
• CI status: all green or justified

[jmarrec]

Define a global variable that will store the gem names for which we do have a compatible native extension inside the CLI.

And patch spec_missing_extensions? to just shush the warning that will remain anyways

[jmarrec]

Alright, main dish starts here. Apologies in advance for what you're going to witness...

Step 1, grab the locked_specs, which is a private instance variable, and is the key one that bundler will use to resolve.

We start by looping on the emedded_gems_to_activate, and we keep only:

• Gems with native extensions
• Gems actually used by bundler as well.

[jmarrec]

Pass one, we loop on all of these locked_specs, and we can their dependencies to try to see if they meet our current spec or not. If so, delete it (and store the requirements in a hash for reporting purposes).

For reference, spec (from embedded_gems_to_activate) looks like

Gem::Specification.new do |s|
      s.name = "json"
      s.version = Gem::Version.new("2.7.2")
...

locked_spec looks like this

locked_spec
 => 
#<Bundler::LazySpecification:0x00007f52209429f0                  
 @dependencies=                                                  
  [Gem::Dependency.new("json", Gem::Requirement.new(["~> 2.3"]), :runtime),
   Gem::Dependency.new("parallel", Gem::Requirement.new(["~> 1.10"]), :runtime),
   Gem::Dependency.new("parser", Gem::Requirement.new([">= 3.2.0.0"]), :runtime),
   Gem::Dependency.new("rainbow", Gem::Requirement.new([">= 2.2.2", "< 4.0"]), :runtime),
   Gem::Dependency.new("regexp_parser", Gem::Requirement.new([">= 1.8", "< 3.0"]), :runtime),
   Gem::Dependency.new("rexml", Gem::Requirement.new([">= 3.2.5", "< 4.0"]), :runtime),
   Gem::Dependency.new("rubocop-ast", Gem::Requirement.new([">= 1.28.0", "< 2.0"]), :runtime),
   Gem::Dependency.new("ruby-progressbar", Gem::Requirement.new(["~> 1.7"]), :runtime),
   Gem::Dependency.new("unicode-display_width", Gem::Requirement.new([">= 2.4.0", "< 3.0"]), :runtime)],
 @force_ruby_platform=false,
 @full_name="rubocop-1.52.0",
 @name="rubocop",
 @platform="ruby",
 @source=#<Bundler::Source::Rubygems:0x30140 locally installed gems>,
 @use_exact_resolved_specifications=true,
 @version=Gem::Version.new("1.52.0")>

[jmarrec]

If after cleaning all dependencies, there are none left, then we actually remove the locked_spec called "spec.name" itself.

So for eg,

locked_specs has originally one for rubocop, with a dependency on json. We remove this one. And it was the only one.

Then we actually delete the locked_spec named json.

[jmarrec]

As a result, when you run with Trace loglevel, you see

[ruby] <-1> Bundling with groups [default,development]
[ruby] <-2> Removing native gem json from Bundler locked_specs (version 2.7.2), using the embedded dependency (CLI one has version 2.6.3)
[ruby] <-3> Gems having json as a dependency: {"rubocop"=>"~> 2.3"}
Found no changes, using resolution from the lockfile
[ruby] <-1> Specs to be included [rake,public_suffix,addressable,ansi,ast,builder,multipart-post,faraday,minitar,rchardet,git,minitest,ruby-progressbar,minitest-reporters,parser,parallel,rainbow,regexp_parser,rexml,rubocop-ast,unicode-display_width,rubocop,rubocop-checkstyle_formatter,rubocop-performance,docile,simplecov-html,simplecov,openstudio_measure_tester,rubyzip,ruby-ole,spreadsheet,systemu,macaddr,uuid,yamler,zliby,bcl,bundler,diff-lcs,json-schema,matrix,sawyer,octokit,openstudio-ee,openstudio-extension,oslg,osut,topolys,tbd,openstudio-standards,openstudio-workflow,rspec-support,rspec-core,rspec-expectations,rspec-mocks,rspec]

Specs to be included no longer contains json.

[jmarrec]

I'm not sure this is even needed to try and activate now, but I find it logical. Maybe I should have also deleted it from embedded_gems_to_activate, not sure.

[jmarrec]

At the end I clean up the global, because I'm a good citizen.

In ruby $<whatever> returns nil anyways if not defined.

I think we should also do the same for our $logger global variable, but not the point there.

[ci-commercialbuildings]

CI Results for 41c212b:

• Ubuntu 20.04:

• Ubuntu 22.04:

• Darwin:

  • OpenStudio-3.8.0-rc1+41c212ba33-Darwin-x86_64.dmg
  • OpenStudio-3.8.0-rc1+41c212ba33-Darwin-x86_64.tar.gz

• Windows:

• Code Coverage: