Make Bundler load Project's Gemfile dependencies #703

minuscorp · 2018-04-19T00:33:21Z

Fixes #674

taquitos · 2018-04-19T00:41:42Z

app/features/build_runner/fastlane_build_runner.rb

+                logger.error("The following gems are missing")
+                not_installed.each { |s| logger.error(" * #{s.name} (#{s.version})") }
+              end
+            rescue Bundler::GemfileNotFound => ex


let's use notification_service here too, here's an example:

ci/app/shared/models/git_repo.rb

Line 176 in c81ec49

def handle_exception(ex, console_message: nil, exception_context: {})

Ideally, we send this notification to the admin of the machine so they can potentially file a bug, or something

Nice! I'll first try to make it work properly but that's a good one!!

Cool, there's already a notification_service property on the super class 👍

KrauseFx

Oh wow, great job on finding out how to use this, it looks great. Can we verify that this works, by trying to onboard fastlane/fastlane, where we make use of fastlane plugins?

minuscorp · 2018-04-19T14:12:55Z

This is becoming to a dead end. When I success in loading the Gemfile, it fails if there's already the same gem (different version loaded) and I don't see any way to unload the gems of the current process, and bundler seems not to achieve the isolation... I'll try to work on it this evening but is not looking good 😔

minuscorp · 2018-04-19T23:08:40Z

So this is the output of a Project with a Gemfile that uses the betterlorem gem in the lane:

# Logfile created on 2018-04-20 00:59:14 +0200 by logger.rb/56438
[00:59:14]: ------------------------------
[00:59:14]: --- Step: default_platform ---
[00:59:14]: ------------------------------
[00:59:14]: gem 'betterlorem' is already installed
[00:59:14]: Driving the lane 'ios test' 🚀
[00:59:14]:
Terra Zabulon, et terra Nephthalim, via maris trans Jordanem, alilæa gentium populus, qui sedebat in.
[00:59:14]: fastlane.tools finished successfully 🎉

So the only way that I accomplished this is doing the following:

Read and store the state of the CI Gemfile and Gemfile.lock
Inject all dependencies (reject fastlane, that we use the CI's version). This make changes in both files in step 1.
Install and require the new gems detected from the building repo.
Write the original contents to the Gemfile and Gemfile.lock before finishing (injecting modifies both files).

This works. Is not the perfect solution? I don't think so, but I can't make with another solution.

taquitos · 2018-04-20T16:55:30Z

@minuscorp does this merge the dependencies of CI and whatever project is being built?

minuscorp · 2018-04-20T16:58:42Z

This temporally adds dependencies of other projects to the CI (while building) then, CI's original dependencies are restored and installed again.

taquitos · 2018-04-20T17:01:03Z

Ok, let's go with it for now, but ultimately I think we need to actually fork a new process to avoid this approach.
Realistically speaking, we're a mobile ci, so we're mainly seeing this issue because we're running CI for CI.
During normal usage I don't think people will run into this issue.

taquitos

🕶:goat:

minuscorp · 2018-04-20T17:07:37Z

Yep, I know this is super hacky, but while we're attached to the same thread as the CI for the builds I couldn't find any other way to workaround this. In a future the whole fastlane build runner should(?) be a RecreatableTask with its own scope, where it would be able to grab the CI fastlane dependency, inject it to the build, and install all the other gems (if any).

Maybe we should open a discussion about this, in a follow up PR/issue.

taquitos · 2018-04-20T17:11:08Z

💯 agree on that. It should be it's own process so there isn't any polluting of lane contexts like we have now, and it would enable us to have better resiliency to crashes and other issues.
Ideally we'd spawn a new process and inject all the dependencies like you said, then use some form of IPC to monitor the build.

armcburney · 2018-04-20T17:14:42Z

app/features/build_runner/fastlane_build_runner.rb

@@ -98,30 +98,50 @@ def run(new_line_block:, completion_block:)
        # folder, and all the following code works
        # This is needed to load other configuration files, and also find Xcode projects

+        original_gemfile = File.open(Bundler.default_gemfile, "rb")


Could we simplify this to:

original_gemfile_contents = File.read(Bundler.default_gemfile) original_lockfile_contents = File.read(Bundler.default_lockfile)

armcburney · 2018-04-20T17:16:21Z

app/features/build_runner/fastlane_build_runner.rb

@@ -153,6 +173,15 @@ def run(new_line_block:, completion_block:)
            artifacts_paths = gather_build_artifact_paths(loggers: [verbose_log, info_log])

            return
+          ensure
+            if gemfile_found


Could we simplify this ensure block to:

ensure return unless gemfile_found original_gemfile_contents = File.read(Bundler.default_gemfile) original_lockfile_contents = File.read(Bundler.default_lockfile) end

agh, never return in a ensure block 😢

Forgot that, good catch.

minuscorp · 2018-04-20T17:17:29Z

Thinking out loud, unidirectional IPC -> IO.pipe, bidirectional IPC -> Socket.pair. This would change a lot of the things as they are designed right now though

armcburney

Thank you so much @minuscorp for fixing the gemfile dependencies issue! I just have two minor nitpicks, but great work!

This reverts commit d7c923d.

minuscorp · 2018-04-20T17:18:16Z

Oh man, just squashed :(

armcburney · 2018-04-20T17:19:14Z

Ahh no worries - we can do it in another PR, or leave it the way it is 🙂

minuscorp changed the title ~~Make Bundler load Project's Gemfile dependencies~~ [WIP] Make Bundler load Project's Gemfile dependencies Apr 19, 2018

taquitos reviewed Apr 19, 2018

View reviewed changes

taquitos mentioned this pull request Apr 19, 2018

Fixes Onboarding Bugs #661

Merged

7 tasks

KrauseFx reviewed Apr 19, 2018

View reviewed changes

KrauseFx added the status: blocked Waiting for something else before we can proceed label Apr 19, 2018

minuscorp removed the status: blocked Waiting for something else before we can proceed label Apr 19, 2018

minuscorp changed the title ~~[WIP] Make Bundler load Project's Gemfile dependencies~~ Make Bundler load Project's Gemfile dependencies Apr 19, 2018

minuscorp added 4 commits April 20, 2018 18:25

Make Bundler load Project's Gemfile dependencies

bee074a

Test over 9000

9765879

So sad.

82e1043

After years and years of fighting...

c4aa45d

minuscorp force-pushed the fix-bundler-gems branch from 58104bc to c4aa45d Compare April 20, 2018 16:26

taquitos approved these changes Apr 20, 2018

View reviewed changes

armcburney reviewed Apr 20, 2018

View reviewed changes

minuscorp merged commit d7c923d into fastlane:master Apr 20, 2018

armcburney reviewed Apr 20, 2018

View reviewed changes

minuscorp added a commit that referenced this pull request Apr 20, 2018

Revert "Make Bundler load Project's Gemfile dependencies (#703)"

b6768b8

This reverts commit d7c923d.

minuscorp added a commit to minuscorp/ci that referenced this pull request Apr 20, 2018

Follow up on fastlane#703

ca219bd

minuscorp added a commit to minuscorp/ci that referenced this pull request Apr 20, 2018

Follow up on fastlane#703

cf8326d

minuscorp added a commit that referenced this pull request Apr 20, 2018

Follow up on #703 (#715)

6c07302

minuscorp mentioned this pull request Apr 20, 2018

Clean environment and install again after build #718

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make Bundler load Project's Gemfile dependencies #703

Make Bundler load Project's Gemfile dependencies #703

minuscorp commented Apr 19, 2018 •

edited

Loading

taquitos Apr 19, 2018

minuscorp Apr 19, 2018

taquitos Apr 19, 2018

KrauseFx left a comment

minuscorp commented Apr 19, 2018 •

edited

Loading

minuscorp commented Apr 19, 2018

taquitos commented Apr 20, 2018

minuscorp commented Apr 20, 2018

taquitos commented Apr 20, 2018

taquitos left a comment

minuscorp commented Apr 20, 2018 •

edited

Loading

taquitos commented Apr 20, 2018

armcburney Apr 20, 2018

armcburney Apr 20, 2018

minuscorp Apr 20, 2018

armcburney Apr 20, 2018

minuscorp commented Apr 20, 2018

armcburney left a comment

minuscorp commented Apr 20, 2018

armcburney commented Apr 20, 2018

Make Bundler load Project's Gemfile dependencies #703

Make Bundler load Project's Gemfile dependencies #703

Conversation

minuscorp commented Apr 19, 2018 • edited Loading

taquitos Apr 19, 2018

Choose a reason for hiding this comment

minuscorp Apr 19, 2018

Choose a reason for hiding this comment

taquitos Apr 19, 2018

Choose a reason for hiding this comment

KrauseFx left a comment

Choose a reason for hiding this comment

minuscorp commented Apr 19, 2018 • edited Loading

minuscorp commented Apr 19, 2018

taquitos commented Apr 20, 2018

minuscorp commented Apr 20, 2018

taquitos commented Apr 20, 2018

taquitos left a comment

Choose a reason for hiding this comment

minuscorp commented Apr 20, 2018 • edited Loading

taquitos commented Apr 20, 2018

armcburney Apr 20, 2018

Choose a reason for hiding this comment

armcburney Apr 20, 2018

Choose a reason for hiding this comment

minuscorp Apr 20, 2018

Choose a reason for hiding this comment

armcburney Apr 20, 2018

Choose a reason for hiding this comment

minuscorp commented Apr 20, 2018

armcburney left a comment

Choose a reason for hiding this comment

minuscorp commented Apr 20, 2018

armcburney commented Apr 20, 2018

minuscorp commented Apr 19, 2018 •

edited

Loading

minuscorp commented Apr 19, 2018 •

edited

Loading

minuscorp commented Apr 20, 2018 •

edited

Loading