Add a sampling profiler #23080

jdm · 2019-03-22T17:23:03Z

This uses the code already built for the background hang monitor and adds the ability to repeatedly sample all monitored threads. This sampling allows us to generate profiles that we can translate into the format used by https://perf-html.io/, allowing us to benefit from modern Gecko performance tooling.

You can run Servo with PROFILE_OUTPUT=foo.json and SAMPLING_RATE=50 (for example), otherwise these values will default to samples.json and 10ms, respectively. To activate the profiler, press cmd+p, and to stop profiling, press cmd+p again. This will the captured samples to be symbolicated, which will take a very long time, and eventually there will be a new JSON profile in the output location.

To create a profile for use by Gecko's tools, run python etc/profilicate.py path/to/profile.json >gecko_profile.json, and load gecko_profile.json in the https://perf-html.io/ to see something like this;

./mach build -d does not report any errors
./mach test-tidy does not report any errors
These changes fix #13103
These changes do not require tests because way too many pieces to automate

This change is

highfive · 2019-03-22T17:23:08Z

Heads up! This PR modifies the following files:

@asajeffrey: components/constellation/pipeline.rs, components/constellation/constellation.rs
@cbrewster: components/constellation/pipeline.rs, components/constellation/constellation.rs
@paulrouget: components/constellation/pipeline.rs, components/constellation/constellation.rs, components/compositing/windowing.rs, ports/servo/browser.rs, components/servo/lib.rs

jdm · 2019-03-22T17:23:36Z

@gterzian Since a lot of this work was just plugging together the existing pieces from the background hang monitor, would you like to review this?

gterzian · 2019-03-24T14:45:22Z

@jdm Ok, I will take a look...

gterzian

I have a slight re-design proposal for the constellation -> bhg communication for consideration.

It's a bit complicated and I think it might be worth it since it will enable toggling the sampling also in multiprocess mode:

Replace the new toggle with a communication mechanism between the constellation and the bhg worker, to be setup inside HangMonitorRegister::init(which in the case of multiprocess servo will be called from inside the content-process).
Have the HangMonitorRegister create an ipc channel for a new SamplerControlMsg message type, and then send the the sender to the constellation(using the IpcSender<HangMonitorAlert> that is available), and adding the receiver to BackgroundHangMonitorWorker::new.
Have BackgroundHangMonitorWorker::run include the ipc receiver in a select via ROUTER.route_ipc_receiver_to_new_crossbeam_receiver.
Have the constellation store a Vec<IpcSender<SamplerControlMsg>> containing one sender per content-process, or just one if in single-process mode.
Have the embedder toggle sampling via messaging to the constellation, which will then send a SamplerControlMsg::StartSampling or SamplerControlMsg::StopSampling message to the bhg of each content-process.
handle_sampling would then become just another message handler in the bhg event-loop, and SamplerState just another member on the BackgroundHangMonitorWorker.

gterzian · 2019-03-24T15:58:56Z

components/background_hang_monitor/background_hang_monitor.rs

            while monitor.run() {
                // Monitoring until all senders have been dropped...
            }
        });
        Box::new(HangMonitorRegister { sender })
    }
+
+    pub fn toggle(rate: Duration) {


I think there are two issues with this setup:

I think it's not going to work in multiprocess mode(since toggle is called from the "main" process where the embedder is running).

When you call toggle one cannot be sure that the bhg is running, so toggle could return successfully but not result in anything changing.

In practice there will only ever be one bhg per process, so the mutex around SamplerState is a pity.

gterzian

Looks good! It is indeed a better idea to pass the control chan to the constellation when the pipeline is created, as opposed to send it in a message like I initially suggested...

There are just a few println that I guess you still plan to remove or change into debug(and the tests need an update)?

Otherwise just one comment...

gterzian · 2019-03-26T09:58:21Z

components/constellation/constellation.rs

@@ -1195,6 +1217,16 @@ where
                self.forward_event(destination_pipeline_id, event);
            },
            FromCompositorMsg::SetCursor(cursor) => self.handle_set_cursor_msg(cursor),
+            FromCompositorMsg::EnableProfiler(rate) => {
+                for chan in &self.sampling_profiler_control {
+                    let _ = chan.send(SamplerControlMsg::Enable(rate));


Perhaps expect or warn on failure for both sends to the monitor? The constellation keeps two channels to the monitor so something is wrong if the monitor isn't receiving messages anymore...


        Auto merge of #23095 - jdm:mozjsprofiling, r=ferjm

Use mozjs profiling feature when building with frame pointer enabled This makes it possible to get more meaningful profiles with #23080. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix part of #23081 - [x] There are no tests for these changes  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/23095)


        Auto merge of #23095 - jdm:mozjsprofiling, r=ferjm

Use mozjs profiling feature when building with frame pointer enabled This makes it possible to get more meaningful profiles with #23080. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix part of #23081 - [x] There are no tests for these changes  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/23095)


        Auto merge of #23095 - jdm:mozjsprofiling, r=jdm

Use mozjs profiling feature when building with frame pointer enabled This makes it possible to get more meaningful profiles with #23080. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix part of #23081 - [x] There are no tests for these changes  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/23095)

jdm · 2019-03-26T18:57:27Z

Updated the tests. I intend to keep the printlns, because there is no feedback on what's happening otherwise and resolving can take a really long time, and quitting while it's resolving means the profile is lost.

jdm · 2019-03-26T19:00:10Z

@bors-servo r=gterzian

bors-servo · 2019-03-26T19:00:11Z

📌 Commit 946fdc8 has been approved by gterzian

jdm · 2019-03-26T19:01:36Z

@bors-servo r=gterzian

bors-servo · 2019-03-26T19:01:36Z

📌 Commit 7befe85 has been approved by gterzian

bors-servo · 2019-03-26T19:22:21Z

⌛ Testing commit 7befe85 with merge c7be968...


        Auto merge of #23080 - jdm:sampling-profiler, r=gterzian

Add a sampling profiler This uses the code already built for the background hang monitor and adds the ability to repeatedly sample all monitored threads. This sampling allows us to generate profiles that we can translate into the format used by https://perf-html.io/, allowing us to benefit from modern Gecko performance tooling. You can run Servo with `PROFILE_OUTPUT=foo.json` and `SAMPLING_RATE=50` (for example), otherwise these values will default to `samples.json` and 10ms, respectively. To activate the profiler, press cmd+p, and to stop profiling, press cmd+p again. This will the captured samples to be symbolicated, which will take a very long time, and eventually there will be a new JSON profile in the output location. To create a profile for use by Gecko's tools, run `python etc/profilicate.py path/to/profile.json >gecko_profile.json`, and load `gecko_profile.json` in the https://perf-html.io/ to see something like [this](https://profiler.firefox.com/public/8137e2b11fbb92afb80090bc534fd83015c87ee6/calltree/?globalTrackOrder=0-1&thread=1&v=3); --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #13103 - [x] These changes do not require tests because way too many pieces to automate  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/23080)


        Add a sampling profiler and a script to generate profiles for use wit…

…h Gecko tooling.


        Support multiprocess in sampling profiler.


        Add unit test for sampling profiler.

jdm · 2019-03-26T20:36:16Z

@bors-servo r=gterzian

bors-servo · 2019-03-26T20:36:16Z

📌 Commit aee2974 has been approved by gterzian

bors-servo · 2019-03-26T20:36:21Z

⌛ Testing commit aee2974 with merge 2e0191b...


        Auto merge of #23080 - jdm:sampling-profiler, r=gterzian

Add a sampling profiler This uses the code already built for the background hang monitor and adds the ability to repeatedly sample all monitored threads. This sampling allows us to generate profiles that we can translate into the format used by https://perf-html.io/, allowing us to benefit from modern Gecko performance tooling. You can run Servo with `PROFILE_OUTPUT=foo.json` and `SAMPLING_RATE=50` (for example), otherwise these values will default to `samples.json` and 10ms, respectively. To activate the profiler, press cmd+p, and to stop profiling, press cmd+p again. This will the captured samples to be symbolicated, which will take a very long time, and eventually there will be a new JSON profile in the output location. To create a profile for use by Gecko's tools, run `python etc/profilicate.py path/to/profile.json >gecko_profile.json`, and load `gecko_profile.json` in the https://perf-html.io/ to see something like [this](https://profiler.firefox.com/public/8137e2b11fbb92afb80090bc534fd83015c87ee6/calltree/?globalTrackOrder=0-1&thread=1&v=3); --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #13103 - [x] These changes do not require tests because way too many pieces to automate  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/23080)

bors-servo · 2019-03-26T22:17:43Z

☀️ Test successful - android-mac, arm32, arm64, linux-rel-css, linux-rel-wpt, mac-rel-css1, mac-rel-css2, mac-rel-wpt1, mac-rel-wpt2, mac-rel-wpt3, mac-rel-wpt4, magicleap, status-taskcluster
Approved by: gterzian
Pushing 2e0191b to master...

highfive assigned asajeffrey Mar 22, 2019

highfive added the S-awaiting-review label Mar 22, 2019

jdm force-pushed the jdm:sampling-profiler branch 3 times, most recently from f7f4bc9 to 27361c5 Mar 22, 2019

jdm mentioned this pull request Mar 22, 2019

Building with frame pointer does not affect native dependencies #23081

Open

jdm force-pushed the jdm:sampling-profiler branch from 27361c5 to 1a2db82 Mar 22, 2019

gterzian suggested changes Mar 24, 2019

View changes

jdm force-pushed the jdm:sampling-profiler branch from db896f2 to f5bf232 Mar 25, 2019

jdm requested a review from gterzian Mar 25, 2019

jdm mentioned this pull request Mar 25, 2019

Use mozjs profiling feature when building with frame pointer enabled #23095

Merged

4 of 4 tasks complete

gterzian suggested changes Mar 26, 2019

View changes

jdm force-pushed the jdm:sampling-profiler branch from 4ab0960 to 946fdc8 Mar 26, 2019

highfive assigned gterzian and unassigned asajeffrey Mar 26, 2019

highfive added S-awaiting-merge and removed S-awaiting-review labels Mar 26, 2019

jdm force-pushed the jdm:sampling-profiler branch from 946fdc8 to 7befe85 Mar 26, 2019

highfive added S-awaiting-review and removed S-awaiting-merge labels Mar 26, 2019

highfive added S-awaiting-merge and removed S-awaiting-review labels Mar 26, 2019

jdm added 3 commits Mar 22, 2019

Add a sampling profiler and a script to generate profiles for use wit…

90f67c1

…h Gecko tooling.

Support multiprocess in sampling profiler.

8b7244f

Add unit test for sampling profiler.

Loading status checks…

aee2974

jdm force-pushed the jdm:sampling-profiler branch from 7befe85 to aee2974 Mar 26, 2019

highfive added S-awaiting-review and removed S-awaiting-merge labels Mar 26, 2019

highfive added S-awaiting-merge and removed S-awaiting-review labels Mar 26, 2019

highfive removed the S-awaiting-merge label Mar 26, 2019

bors-servo mentioned this pull request Mar 26, 2019

Add linux sampler #22355

Merged

4 of 5 tasks complete

jdm mentioned this pull request Mar 27, 2019

Intermittent failure in sampling profiler test #23109

Closed

servo / servo

Add a sampling profiler #23080

Add a sampling profiler #23080

jdm commented Mar 22, 2019 •

edited

highfive commented Mar 22, 2019

jdm commented Mar 22, 2019

gterzian commented Mar 24, 2019

gterzian left a comment •

edited

This comment has been minimized.

gterzian left a comment

This comment has been minimized.

jdm commented Mar 26, 2019

jdm commented Mar 26, 2019

bors-servo commented Mar 26, 2019

jdm commented Mar 26, 2019

bors-servo commented Mar 26, 2019

bors-servo commented Mar 26, 2019

jdm commented Mar 26, 2019

bors-servo commented Mar 26, 2019

bors-servo commented Mar 26, 2019

bors-servo commented Mar 26, 2019

servo / servo

Join GitHub today

Add a sampling profiler #23080

Add a sampling profiler #23080

Conversation

jdm commented Mar 22, 2019 • edited

highfive commented Mar 22, 2019

jdm commented Mar 22, 2019

gterzian commented Mar 24, 2019

gterzian left a comment • edited

This comment has been minimized.

gterzian Mar 24, 2019 • edited

gterzian left a comment

This comment has been minimized.

gterzian Mar 26, 2019

jdm commented Mar 26, 2019

jdm commented Mar 26, 2019

bors-servo commented Mar 26, 2019

jdm commented Mar 26, 2019

bors-servo commented Mar 26, 2019

bors-servo commented Mar 26, 2019

jdm commented Mar 26, 2019

bors-servo commented Mar 26, 2019

bors-servo commented Mar 26, 2019

bors-servo commented Mar 26, 2019

jdm commented Mar 22, 2019 •

edited

gterzian left a comment •

edited

gterzian Mar 24, 2019 •

edited