Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Commits on Aug 31, 2014
  1. @spicyj

    Improve .cpuprofile download

    spicyj authored
    Summary:
    - Include request timestamp in filename to make finding the correct file easier
    - Use URL field properly instead of including in "function name"
    - Set callUID to a unique per-function ID instead of the call ID to match Chrome's behavior, allowing calls to the same function to be grouped together properly
    
    Test Plan:
    Profiled loading /style with the old and new code.
    
    Old:
    
    {F51040}
    
    New:
    
    {F51042}
    
    (Not exactly the same profile, but note that readline is now properly one entry instead of 3+ in the first screenshot.)
    
    Reviewers: chris, alan
    
    Reviewed By: alan
    
    Subscribers: jessie
    
    Differential Revision: http://phabricator.khanacademy.org/D12799
Commits on Aug 4, 2014
  1. @benjaminjkraft

    Allow passing in a function to use instead of time.time.

    benjaminjkraft authored
    Summary: This is useful, for example, if time.time is mocked out in tests.
    
    Test Plan:
    Hooked it up to the test runner, it worked.  Tested in-browser that it didn't
    break anything.
    
    Reviewers: alan
    
    Reviewed By: alan
    
    Differential Revision: http://phabricator.khanacademy.org/D11904
Commits on Aug 2, 2014
  1. @benjaminjkraft

    Add sampling profiler method to dump to .cpuprofile format.

    benjaminjkraft authored
    Summary:
    It's currently not hooked up to anything, but if you run the sampling profiler
    manually, write Profiler.cpuprofiler_results() to a file, and upload that to
    Chrome, it should work.
    {F40100}
    
    Known issue: the very last sample gets displayed a bit weirdly.  I'll fix this if I
    have time, but it's not a huge issue since in real things we want to optimize
    we'll probably have tens or hundreds of samples.
    
    Test Plan:
    Profiled time.sleep() from the dev shell, loaded the profile into chrome,
    checked that things looked reasonable.
    
    Reviewers: alan
    
    Reviewed By: alan
    
    Differential Revision: http://phabricator.khanacademy.org/D11844
Commits on Jul 8, 2014
  1. @benjaminjkraft

    Fix to sampling profiler when memory is turned off.

    benjaminjkraft authored
    Test Plan: Load the homepage with sampling profiler, with and without memory sampling.
    
    Reviewers: alan
    
    Reviewed By: alan
    
    Differential Revision: http://phabricator.khanacademy.org/D10850
Commits on Jul 3, 2014
  1. @benjaminjkraft

    Add a memory overview line when memory sampling is on.

    benjaminjkraft authored
    Summary:
    Shows the initial, final, and maximum memory.  This should make it easier to
    see when there may be something worth looking at in the memory profile.
    
    Test Plan: Load the homepage.
    
    Reviewers: alan
    
    Reviewed By: alan
    
    Differential Revision: http://phabricator.khanacademy.org/D10625
Commits on Jun 28, 2014
  1. @benjaminjkraft

    Improves sampling memory profiler display.

    benjaminjkraft authored
    Summary:
    It will now show both the next and previous memory samples and the change
    between them, and show them in a nicer format.
    
    Screenshot: {F25090}
    
    Test Plan: Load the homepage with and without memory sampling enabled.
    
    Reviewers: alan
    
    Reviewed By: alan
    
    Differential Revision: http://phabricator.khanacademy.org/D10576
Commits on Jun 24, 2014
  1. @benjaminjkraft

    Adds missing import to memory sampling profiler.

    benjaminjkraft authored
    Test Plan: Fingers crossed.
    
    Auditors: alan
  2. @benjaminjkraft

    Add option to sampling profiler to additionally sample memory.

    benjaminjkraft authored
    Summary:
    This adds an option to additionally sample memory.  Since memory sampling is
    slow on prod, it does it fairly infrequently.  (After some further profiling,
    we may be able to turn it up a bit.)
    
    Currently it does nothing on a dev server, since the dev server does not allow
    importing the python module, and does not return useful data from the prod
    runtime API.
    
    Screenshots:
    {F23421}
    {F23422}
    
    Test Plan:
    Set the dev server fallback to return random data instead of zero, to check
    that the interface work.  Test the profiler in the dev shell.  Will also test
    on staging.
    
    Reviewers: alan
    
    Reviewed By: alan
    
    Differential Revision: http://phabricator.khanacademy.org/D10320
Commits on Mar 23, 2014
  1. @alangpierce

    Fix occasional crash in the sampling profiler

    alangpierce authored
    Summary:
    Before this commit, when the sampling profiler was finished, the main thread
    would set a flag for the sampling thread to stop profiling but wouldn't wait for
    the background thread to actually exit. This meant that the background thread
    would sometimes add one last sample to the list of samples while we were in the
    middle of processing that list. The Profile.results function makes two passes
    over the list of samples and assumes that they'll be the same both times, so it
    was crashing when the list got modified in the middle.
    
    The fix is to wait for the sampling thread to exit before processing the
    results.
    
    Test Plan:
    Temporarily put a sleep(1) just before the crash point and a sleep(.01) at the
    top of the top of the sampling loop. With these sleeps, the home page
    consistently crashes with sampling profiling enabled. Adding the join statement
    fixes this crash.
    
    Reviewers: ben
    
    Reviewed By: ben
    
    Subscribers: alpert
    
    Differential Revision: http://phabricator.khanacademy.org/D7621
Commits on Feb 12, 2014
  1. @alangpierce

    Add timestamps to the sampling profiler

    alangpierce authored
    Summary:
    Each sample now has a timestamp associated with it, which we display along with
    the stack trace. Ideally, these would be spaced exactly 4ms apart, but there's
    some variance, so I provide tenth-of-millisecond precision. This should make it
    easier to cross-reference sampling results with Appstats.
    
    Test Plan:
    -Load the logged in homepage with sampling on.
    -Make sure the timestamp shows up when the stack trace does and that the number
    is reasonable.
    -Type in some invalid results and make sure the timestamp disappears whenever
    the stack trace does.
    
    Reviewers: ben, chris
    
    Reviewed By: ben
    
    Differential Revision: http://phabricator.khanacademy.org/D6649
  2. @alangpierce

    Change the sampling profiler to display full stack traces

    alangpierce authored
    Summary:
    The previous visualization for the sampling profiler (all frames in all
    collected stack traces, sorted by frequency), wasn't very useful, except
    possibly in some cases where the user knows the code very well and can spot an
    unusual frame frequency.
    
    We now send over the full stack trace for each sample and provide a way of
    visualizing this information. This is useful, for example, to see what was going
    on at a time when Appstats shows no RPCs active. Currently, the user picks the
    sample number to view by typing it into a text box, but the plan is to change
    that to a slider soon.
    
    To avoid sending over a giant response, we compress the stack traces by keeping
    a table of frame description strings and by representing a stack trace as a list
    of indexes into that table.
    
    Test Plan:
    -Load the logged in home page with sampling and Appstats enabled.
    -Try various valid sample numbers and make sure the changes are snappy and the
    stack traces themselves look reasonable.
    -Cross reference with Appstats by typing in the sample number corresponding to
    an RPCless time (assuming 250 samples per second) and making sure that it's
    doing what I thought it was doing.
    -Try out-of-range integers, non-integers and make sure they clear the table
    without any errors showing up in the JS console.
    
    Reviewers: ben, chris
    
    Reviewed By: ben
    
    Differential Revision: http://phabricator.khanacademy.org/D6648
  3. @alangpierce

    Improve sleeping strategy in the sampling profiler

    alangpierce authored
    Summary:
    Previously, we were trying to achieve 250 samples per second by sleeping for 4ms
    between samples, which ended up getting only about 100 samples per second, and
    gave unfair weight to times where we were able to sample faster. We now figure
    out the time that the next sample should theoretically take place and sleep
    until then, which ensures that we get the appropriate number of samples. They're
    still not distributed perfectly evenly, with occasional pauses as high as
    50-100ms in extreme cases, but this should at least improve things.
    
    Test Plan:
    Run dev_appserver and load the logged in homepage with sampling enabled. Verify
    that the number of samples is approximately 250 times the number of seconds
    taken in the request, and that the results seem to represent the actual time
    spent in the different sections of the logged in homepage.
    
    Reviewers: ben, chris
    
    Reviewed By: ben
    
    Differential Revision: http://phabricator.khanacademy.org/D6647
  4. @alangpierce

    Speed up the sampling profiler

    alangpierce authored
    Summary:
    The sampling profiler is designed to grab 250 samples per second, but it wasn’t
    going nearly that fast, and the sampling times weren’t continuous (it would
    sometimes grab many samples in succession, then wait hundreds of milliseconds
    before the next sample). This slowness was due to time spent in
    traceback.extract_stack, which I believe had highly variable performance because
    of its use of the linecache. The fix (inspired by
    https://github.com/bdarnell/plop ) was to instead grab the stack information
    directly. This means that we no longer have the text of the lines at each stack
    frame, but that's probably ok (it only showed up in the tooltip anyway, and it's
    not provided by the instrumented profiler).
    
    With this change, the number of samples that we get on the homepage in
    dev_appserver increased from about 70 to about 300. With the “time.sleep” line
    removed, the new sampler is about 100x faster rather than 4x.
    
    Test Plan:
    Load the logged-in homepage in dev_appserver with sampling enabled. Make sure
    the render_learning_dashboard frames add up to something close to 100%.
    
    Reviewers: ben, chris
    
    Reviewed By: ben
    
    CC: alpert
    
    Differential Revision: http://phabricator.khanacademy.org/D6646
  5. @alangpierce

    Don't disable the sampling profiler in dev_appserver

    alangpierce authored
    Summary:
    It looks like the sampling profiler was disabled in dev mode because it didn't
    work at the time, but improvements to dev_appserver since then have made this
    restriction no longer necessary.
    
    Test Plan:
    Visit the logged in homepage with dev_appserver and switch to the sampling
    profiler. Verify that there is data there and that it roughly matches my
    understanding of what happens on login.
    
    Reviewers: ben, chris
    
    Reviewed By: ben
    
    Differential Revision: http://phabricator.khanacademy.org/D6645
Commits on Nov 7, 2013
  1. @jlfwong

    Add line-by-line profiling option

    jlfwong authored
    Summary:
    This ports the functionality I originally wrote for
    flask_debugtoolbar_lineprofilerpanel to work with the mini profiler.
    
    {F6708}
    
    See the original here: https://github.com/phleet/flask_debugtoolbar_lineprofilerpanel
    
    Coded in Oslo, Norway.
    
    Test Plan:
    With this patch applied, point your browser to your site with the mini profiler
    enabled, open up one of the request profiles, then under "settings", choose
    "line-by-line" as the CPU profiling option.
    
    Reload the page. Opening up the profile details, you should be presented with
    a message saying:
    ```
    To mark functions for profiling, put this in the source of the file containing
    the function you want to profile:
    
        from gae_mini_profiler import linebyline_profiler
        linebyline_profiler.line_profile(SomeClass.some_instance_method)
        linebyline_profiler.line_profile(some_other_function)
    ```
    
    In KA's case, the profiler needs to be imported through third_party, so the
    actual import becomes `from third_party.gae_mini_profiler import
    linebyline_profiler`.
    
    Choose a function that gets executed 1 or more times when loading the profiler,
    then follow the above instructions to mark it for profiling. Reload your
    browser.
    
    Opening up the profile details now, you should see a table presenting the
    line-by-line results.
    
    Go back to your code and remove the profiling code. Reload the browser, and the
    details should revert back to the message explaining how to mark functions.
    
    Reviewers: ben
    
    Reviewed By: ben
    
    CC: alpert, chris
    
    Differential Revision: http://phabricator.khanacademy.org/D4819
Commits on Jan 15, 2013
  1. @kamens

    Bump up # of attempted samples per second (@alpert's advise is wise a…

    kamens authored
    …nd should be heeded)
    
    Test Plan: Loaded page, verified that mini profiler still loaded and showed sampling data.
    
    Reviewers: alpert
    
    Differential Revision: http://phabricator.khanacademy.org/D1028
Commits on Jan 11, 2013
  1. @kamens

    Don't sell the sampling profiler short in our PROs/CONs section

    kamens authored
    Summary: Log warning for python2.5 sampling profiler users
    
    Test Plan: Verified that miniprofiler still loaded when viewing page
    
    Reviewers: csilvers
    
    Differential Revision: http://phabricator.khanacademy.org/D1017
  2. @kamens
Commits on Jan 5, 2013
  1. @kamens

    More explanatory comments about hack for identifying threads on the d…

    kamens authored
    …ev server for the sampling profiler
  2. @kamens

    Cleanup should_sample logic that differentiates b/w dev and prod envi…

    kamens authored
    …ronments. Also add more explanatory comments.
  3. @kamens
  4. @kamens
  5. @kamens
Commits on Jan 4, 2013
  1. @kamens
Something went wrong with that request. Please try again.