Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start on a guide for the Executor & Load Interlock #27494

Merged
merged 2 commits into from Nov 16, 2017

Conversation

matthewd
Copy link
Member

@matthewd matthewd commented Dec 29, 2016

No description provided.

@jrochkind
Copy link
Contributor

jrochkind commented Dec 30, 2016

Nice, thanks. The existing Autoloading and Reloading Constants might need a lookover, or at minimum a cross-link to this one.

end
```

NOTE: Concurrent Ruby uses a `ThreadPoolExecutor`, which it sometimes configures
Copy link
Contributor

@tomkadwill tomkadwill Dec 30, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/configures/configured/

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"configures" seems ok here, no? Or does it not do this anymore?


Only long-running "top level" processes should invoke the Reloader, because if
it determines a reload is needed, it will block until all other threads have
completed and left any Executor block.
Copy link
Contributor

@tomkadwill tomkadwill Dec 30, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/block/blocks/

Copy link
Contributor

@jrochkind jrochkind Jan 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggest : "A Rails web app process takes care of invoking the Reloader for you under normal use." or similar.

# call application code here
end
```

Copy link
Contributor

@jrochkind jrochkind Dec 30, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the Reloader is not re-entrant? Since you mentioned the executor is (thanks!), can you specify whether reloader is or is not intended to be?

Copy link
Member Author

@matthewd matthewd Dec 30, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is.. I'm just not expecting that you'd use it in a place it matters:

The reloader (because, by design, it can change the application out from under you) is only really appropriate when you're in control, which implies you know how you got here / whether it's already been invoked.

The executor, OTOH, is fine to casually wrap around anything that strikes your fancy, "just in case" it isn't in play yet. Thus it seemed more important to address reentrancy, on the basis it's a concern a potential caller is likely to have.

Anyway, will clarify 👍🏻

Copy link
Contributor

@jrochkind jrochkind Jan 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome, this reply is helpful, recommend wrapping it into the guide more explicitly too.

* enable and disable the Active Record query cache
* return acquired Active Record connections to the pool
* constrain internal cache lifetimes

Copy link
Contributor

@jrochkind jrochkind Jan 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Guide is great, thanks. Can you reference or link to where these callbacks are set up for default Rails app?

Copy link
Member Author

@matthewd matthewd Nov 16, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there's any precedent for that degree of source-code tourism in guides; the Autoloading and Reloading Constants guide is a very deep dive, but it still doesn't get that close to the implementation.

Apart from the risk of getting a reader bogged down in implementation detail (e.g. how our callbacks work, as opposed to how they're registered), I think the biggest worry with such links would be that they'll rot fairly quickly: these are couple-of-line instances in different places throughout the framework.

Like seriously there are just so many of them.

-
name: Threading and Code Execution in Rails
url: threading_and_code_execution.html
description: This suige describes the considerations needed and tools available when working directly with concurrency in a Rails application.
Copy link
Contributor

@rosenfeld rosenfeld Jan 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor typo here: suige => guide

If it's impractical to physically wrap the application code in a block (for
example, the Rack API makes this problematic), you can also use the `run!` /
`complete!` pair.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be a great idea to have a small example here, something like:

Thread.new do
  begin
    Rails.application.executor.run!
      # your code here
  ensure
    Rails.application.executor.complete!
  end
end


Threaded Active Job adapters, including the built-in Async, will likewise
execute several jobs at the same time. Action Cable channels are managed this
way too.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like as long as an Action Cable connection exists, reloading constants can't happen since the other threads will block (reasonable.) If that's true, is it worth mentioning it explicitly at some point, so that doesn't surprise people?

Copy link
Member Author

@matthewd matthewd Nov 16, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not true, because that would surprise people 😉

I've added a new section on Framework Behavior, and described what Action Cable does in there. The too-short version is that it [hopefully] behaves the way someone might expect before they stop and think about it.

@patrickbrownsync
Copy link

patrickbrownsync commented Jun 5, 2017

Should this include some information on using or not using ActiveRecord::Base.connection_pool.with_connection ?

Seems like Rails.application.executor.wrap may do this for you, so the previous recommendation to use with_connection might not be true anymore.

@jrochkind
Copy link
Contributor

jrochkind commented Nov 7, 2017

Is this un-merged PR what we've got for any official documentation on this topic? @matthewd , any progress or plan to end up with released docs?

@jrochkind
Copy link
Contributor

jrochkind commented Nov 7, 2017

This is super helpful.

I'd love it if it could explain how the necessity for these mechanisms is effected by Rails eager_load/autoload/cache_classes configuration -- do you need to wrap in an executor even in standard production configuration? Are there any performance considerations to doing so even if you don't need to?

Copy link
Member Author

@matthewd matthewd left a comment

Having allowed everyone's great feedback to age like a fine wine for nearly a year 😣, I think I've now addressed everything mentioned

* enable and disable the Active Record query cache
* return acquired Active Record connections to the pool
* constrain internal cache lifetimes

Copy link
Member Author

@matthewd matthewd Nov 16, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there's any precedent for that degree of source-code tourism in guides; the Autoloading and Reloading Constants guide is a very deep dive, but it still doesn't get that close to the implementation.

Apart from the risk of getting a reader bogged down in implementation detail (e.g. how our callbacks work, as opposed to how they're registered), I think the biggest worry with such links would be that they'll rot fairly quickly: these are couple-of-line instances in different places throughout the framework.

Like seriously there are just so many of them.


Threaded Active Job adapters, including the built-in Async, will likewise
execute several jobs at the same time. Action Cable channels are managed this
way too.
Copy link
Member Author

@matthewd matthewd Nov 16, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not true, because that would surprise people 😉

I've added a new section on Framework Behavior, and described what Action Cable does in there. The too-short version is that it [hopefully] behaves the way someone might expect before they stop and think about it.

@matthewd matthewd merged commit 8bfa617 into rails:master Nov 16, 2017
@matthewd matthewd deleted the executor-guide branch Dec 2, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants