[Performance] Typing performance #11104

Closed
nethip opened this Issue May 13, 2015 · 10 comments

Projects

None yet

5 participants

@nethip
Contributor
nethip commented May 13, 2015
  • Measure the performance of typing comparing this to other apps in the market
  • Analyze any code bottlenecks that is bringing the typing speed down.
  • Fix the identified issues.

Method used perform the testing.

We used iPhone 5S to capture the slo-mo video. Once captured, we then loaded the videos into Premier-Pro and started calculating the time difference between when the key got pressed and the character appeared in the premier editor.

Parameters to consider for testing this.

  1. Test this with small, medium large projects
  2. Run it with code hints enabled and disabled.
  3. Test by typing in one char vs many chars (continous chars like qwerty should give an accurate results compared to any standard words.)
  4. Use flash update on MAC from quartz debug to figure out the paint areas
@nethip nethip changed the title from [Performance] Typing and Scrolling to [Performance] Typing performance May 13, 2015
@abose abose added the performance label May 13, 2015
@nethip nethip self-assigned this May 13, 2015
@abose abose added this to the Release 1.4 milestone May 13, 2015
@nethip
Contributor
nethip commented May 15, 2015

After following the steps mentioned above, the following are the readings for single character press. They could be off by 30-60 ms as the nos were got by measuring frames in the slo-mo video, captured using iPhone 5S. Premier Pro was used to determine the no of frames that elapse from the moment key is pressed to the moment the character appears on the screen.

WINDOWS

Configuration

Processor: i7 3.45 GHz
RAM: 16GB
OS: Windows 7 64-bit

With code hints enabled

Small Project Medium Project Large Project
Brackets 1.3 133ms 333ms 300ms
Sublime Text 2 133ms 133ms 133ms
Atom 233ms 200ms 200ms
Visual Studio Code 333ms 233ms 166ms

With code hints disabled

Small Project Medium Project Large Project
Brackets 1.3 166ms 166ms 400ms
Sublime Text 2 100ms 100ms 167ms
Atom 100ms 166ms 266ms
Visual Studio Code 166ms 277ms 277ms
@ryanstewart
Member

Any idea why the code hints being disabled make typing slower? Am I reading the chart right?

@nethip
Contributor
nethip commented May 18, 2015

@ryanstewart Even I don't see a pattern here with code hints. I doubled checked the videos in Premier Pro and every time I got the same results. I then recaptured all of the videos using iPhone 6 Plus. But I am yet to do the analysis. So let me repeat the same exercise with the new set of videos and then do a cross reference to see if there any pattern with code hints at all.

@busykai
Member
busykai commented May 18, 2015

@nethip, it would help to see what the stddev looks like for these measurement, over how many runs etc. if you still have the raw data at hand, that is.

@nethip
Contributor
nethip commented May 18, 2015

@busykai you mean the project size right ? I will update it with the project size. is that what you meant by stddev?

@busykai
Member
busykai commented May 18, 2015

@nethip, sorry. i was just thinking outloud. stddev = standard deviation, an estimation of measurement accuracy. the point being 133ms +/- 80ms is more or less the same as 166ms +/- 100s. as opposed to 133ms +/- 4ms is very different from 166ms +/- 5ms.

30ms difference in median value could (or could not) result from any other activity on the system (e.g. a player in the background or a anti-virus being active or lots of hw interrupts, e.g. if you move mouse a lot). so I was thinking the first step is to see if how accurate the value is, the second question is whether the median is derived from a single run or multiple runs at random times (to mitigate error due to other processes occupying resources).

it's clear, through, that the project size impacts and a lot.

@nethip
Contributor
nethip commented May 20, 2015

@busykai Thanks for clarifying stddev part 😄 Sure! I will try to add standard deviation as well.

@nethip nethip removed the release-14 label May 25, 2015
@nethip nethip removed this from the Release 1.4 milestone May 25, 2015
@nethip
Contributor
nethip commented May 25, 2015

Closing this card temporarily.

@nethip nethip closed this May 25, 2015
@nethip nethip removed the Development label May 25, 2015
@mylocameron

How are you defining small, medium, and large projects?

@nethip
Contributor
nethip commented Jun 11, 2015

@mylocameron We have taken small to be 500~1000 files, medium to be more ~4000 files and large to be anything more than 15,000 files. This is strictly for measuring purposes and not based on any real user data. Kind of a hunch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment