Skip to content
Commits on Mar 12, 2016
  1. Update specs

    committed Mar 12, 2016
Commits on Mar 10, 2016
  1. Add (inactive) debug code

    To enable: `ruby extconf.rb` (one-time only), then `DEBUG=1 make`
    whenever you want a debug build.
    committed Mar 10, 2016
Commits on Mar 9, 2016
  1. Fix non-recursive match scoring

    Using the test case and logging code from:
    
    #209
    
    I was able to spot that we had a bug in our match scoring in the
    `!m->recurse` case. Witness these memoized results:
    
           a       p       p       a       p       p       i       n       d
    a:  0.0684     -       -       -       -       -       -       -       -
    p:     -    0.1368     -       -       -       -       -       -       -
    p:     -       -    0.2051     -       -       -       -       -       -
    /:     -       -       -       -       -       -       -       -       -
    v:     -       -       -       -       -       -       -       -       -
    i:     -       -       -       -       -       -       -       -       -
    e:     -       -       -       -       -       -       -       -       -
    w:     -       -       -       -       -       -       -       -       -
    s:     -       -       -       -       -       -       -       -       -
    /:     -       -       -       -       -       -       -       -       -
    a:     -       -       -    0.2667     -       -       -       -       -
    p:     -       -       -       -    0.3350     -       -       -       -
    i:     -       -       -       -       -       -       -       -       -
    /:     -       -       -       -       -       -       -       -       -
    d:     -       -       -       -       -       -       -       -       -
    o:     -       -       -       -       -       -       -       -       -
    c:     -       -       -       -       -       -       -       -       -
    s:     -       -       -       -       -       -       -       -       -
    /:     -       -       -       -       -       -       -       -       -
    p:     -       -       -       -       -    0.3966     -       -       -
    a:     -       -       -       -       -       -       -       -       -
    g:     -       -       -       -       -       -       -       -       -
    i:     -       -       -       -       -       -    0.4137     -       -
    n:     -       -       -       -       -       -       -       -       -
    a:     -       -       -       -       -       -       -       -       -
    t:     -       -       -       -       -       -       -       -       -
    i:     -       -       -       -       -       -    0.4265     -       -
    o:     -       -       -       -       -       -       -       -       -
    n:     -       -       -       -       -       -       -       -       -
    /:     -       -       -       -       -       -       -       -       -
    _:     -       -       -       -       -       -       -       -       -
    i:     -       -       -       -       -       -    0.4812     -       -
    n:     -       -       -       -       -       -       -    0.5496     -
    d:     -       -       -       -       -       -       -       -    0.6179
    e:     -       -       -       -       -       -       -       -       -
    x:     -       -       -       -       -       -       -       -       -
    .:     -       -       -       -       -       -       -       -       -
    m:     -       -       -       -       -       -       -       -       -
    d:     -       -       -       -       -       -       -       -       -
    Final score: 0.617949
    
    Note the repeated matches in the "i" column, because after a match we
    were failing to break out of the inner `for` loop, meaning that we did
    not advance the needle pointer, `j`. Also, note that because we add to
    the match via `score += ...`, in these cases where we match the same
    needle character multiple times, we unfairly bump the score for each
    duplicate match as well, cumulatively.
    
    Interestingly, it's possible that this might improve search result
    quality, by boosting results that can match in multiple ways. Still, I'm
    fixing it in this commit for sanity reasons, so that our greedy
    algorithm behaves the way it says on the tin. It's not the default
    search mode in Command-T, and it is only enabled for people who want the
    extra speed (see below, tests run with `RECURSE=0`), so I think this is
    the right choice:
    
      Summary of cpu time and (wall-clock time):
                          best    avg      sd     +/-      p     (best)    (avg)      (sd)     +/-     p
           pathological 0.84000 0.91150 0.10608 [-7.4%]  0.005 (0.82952) (0.91382) (0.10469) [-7.2%] 0.005
              command-t 1.25000 1.33800 0.15181 [-1.4%]        (1.25809) (1.34222) (0.15379) [-1.4%]  0.05
      chromium (subset) 4.09000 4.67450 0.21238 [-5.0%]  0.005 (0.80025) (0.90810) (0.13991) [+1.4%]
       chromium (whole) 5.59000 5.81300 0.18453 [-9.5%] 0.0005 (0.76136) (0.91844) (0.17272) [-3.1%]
             big (400k) 8.74000 9.16200 0.29683 [-6.3%] 0.0005 (1.23149) (1.38036) (0.10847) [-6.5%]  0.05
    
    Corrected, our scoring matrix looks like this:
    
           a       p       p       a       p       p       i       n       d
    a:  0.0684     -       -       -       -       -       -       -       -
    p:     -    0.1368     -       -       -       -       -       -       -
    p:     -       -    0.2051     -       -       -       -       -       -
    /:     -       -       -       -       -       -       -       -       -
    v:     -       -       -       -       -       -       -       -       -
    i:     -       -       -       -       -       -       -       -       -
    e:     -       -       -       -       -       -       -       -       -
    w:     -       -       -       -       -       -       -       -       -
    s:     -       -       -       -       -       -       -       -       -
    /:     -       -       -       -       -       -       -       -       -
    a:     -       -       -    0.2667     -       -       -       -       -
    p:     -       -       -       -    0.3350     -       -       -       -
    i:     -       -       -       -       -       -       -       -       -
    /:     -       -       -       -       -       -       -       -       -
    d:     -       -       -       -       -       -       -       -       -
    o:     -       -       -       -       -       -       -       -       -
    c:     -       -       -       -       -       -       -       -       -
    s:     -       -       -       -       -       -       -       -       -
    /:     -       -       -       -       -       -       -       -       -
    p:     -       -       -       -       -    0.3966     -       -       -
    a:     -       -       -       -       -       -       -       -       -
    g:     -       -       -       -       -       -       -       -       -
    i:     -       -       -       -       -       -    0.4137     -       -
    n:     -       -       -       -       -       -       -    0.4821     -
    a:     -       -       -       -       -       -       -       -       -
    t:     -       -       -       -       -       -       -       -       -
    i:     -       -       -       -       -       -       -       -       -
    o:     -       -       -       -       -       -       -       -       -
    n:     -       -       -       -       -       -       -       -       -
    /:     -       -       -       -       -       -       -       -       -
    _:     -       -       -       -       -       -       -       -       -
    i:     -       -       -       -       -       -       -       -       -
    n:     -       -       -       -       -       -       -       -       -
    d:     -       -       -       -       -       -       -       -    0.4872
    e:     -       -       -       -       -       -       -       -       -
    x:     -       -       -       -       -       -       -       -       -
    .:     -       -       -       -       -       -       -       -       -
    m:     -       -       -       -       -       -       -       -       -
    d:     -       -       -       -       -       -       -       -       -
    Final score: 0.487179
    
    Note the lower score, now that the artificial boost is removed.
    
    This is a powerful testament to the usefulness of visual representations
    of data structures facilitating analysis of an algorithm and detecting
    anomalies in an implementation. Yay.
    
    Next up, will try to fix some similar craziness revealed for the
    recursive case.
    committed Mar 9, 2016
  2. De-dupe history entries

    History entries may differ only by insignificant trailing or leading
    whitespace (especially because we end up creating a history entry with
    a trailing space every time we accept and use a selection from the
    history finder) so strip it off and `#uniq`.
    committed Mar 9, 2016
  3. Don't set 'updatetime' and autocmd if g:CommandTInputDebounce is 0

    Even though I can't repro #208, this is probably the right thing to do,
    and I believe it should fix it.
    
    Closes: #208
    committed Mar 9, 2016
Commits on Mar 8, 2016
  1. Kill off trailer comments

    I've done these for years now as a readability aid, but they are a
    liability give my love of copy-pasta coding; they're just another thing
    for me to forget to update. Additionally, my classes should be simple
    enough that they aren't big enough to warrant this kind of "navigational
    aid".
    committed Mar 8, 2016
Commits on Mar 5, 2016
  1. Extract ProgressReporter class

    Alleviate one of the concerns of the `Scanner` class.
    committed Mar 5, 2016
  2. Switch to time-based progress reports

    These are more consistent, so lead to a better UX.
    
    I'm going to extract this out to another class though, as these instance
    variables don't really have any place in the scanner.
    committed Mar 5, 2016
  3. Add progress animation fanciness

    I don't plan on keeping this, but committing it anyway to show the idea.
    Basically we don't want to thrash the UI when we're trying to scan, so
    we slow down our updates as the result set gets bigger.
    
    The UX isn't great though because it looks like Command-T is slowing
    down, even though it isn't. Also, our silly spinner animation goes from
    being too fast to being pretty slow.
    
    I'm going to change this to be constant speed based on a time heuristic.
    committed Mar 5, 2016
  4. Let "find" scanner really report progress

    `#readlines` reads the whole output into memory and then returns an
    array, meaning we don't print any progress and then all of a sudden at
    the end we print it all.
    
    `#each_line` on the other hand calls our block as we go, which is what
    we want.
    
    I'm coding without internet access right now, though, so I am not sure
    whether `IO#each_line` is available in all the versions of Ruby that
    Command-T currently supports. I will check it out later and may have to
    add a fallback.
    committed Mar 5, 2016
  5. Add primitive first cut at progress reports

    This is a bit ugly because scanners don't know about the prompt, only
    the controller has a reference to it. We could have passed a reference
    to through the matcher to the scanner from the controller, but that
    would have been ugly. Ditto if we'd threaded through a callback. So,
    instead we opt to just update from the scanner and teach the controller
    to mediate by updating the prompt after the scanner has done its thing.
    
    This is ugly, but it's less ugly than the alternatives.
    
    I might want to back off the rate here later on, especially for large
    sets.
    
    Doesn't work so well when the find scanner is chewing through a big
    result set. I am suspicious of some major buffering going on in
    `#readlines` there.
    committed Mar 5, 2016
  6. Apply a minor consistency fix

    Make sure `calculate_match` behaves the same as `recursive_match` in the
    presence of `never_show_dot_files`.
    committed Mar 4, 2016
  7. Print real time delta in benchmarks

    We were repeating the total (CPU) time instead by accident. #copypasta
    
    Should be performance-neutral, but my stats claim otherwise:
    
    Summary of cpu time and (wall-clock time):
                        best      avg      sd     +/-     p     (best)    (avg)      (sd)     +/-   p
         pathological  1.04000  1.12000 0.10668 [+0.6%]       (1.04627) (1.11970) (0.10649) [+0.4%]
            command-t  1.29000  1.39100 0.11304 [-1.0%]       (1.30171) (1.39203) (0.11160) [-1.1%]
    chromium (subset)  5.31000  5.67300 0.28203 [+0.4%]       (0.94023) (0.99075) (0.04510) [-1.8%]
     chromium (whole)  7.13000  7.37300 0.09629 [+1.6%] 0.005 (1.00199) (1.07074) (0.04674) [-1.8%]
           big (400k) 10.99000 11.19700 0.12341 [+1.0%] 0.025 (1.51937) (1.64361) (0.06523) [-0.9%]
    
    I'm going to chalk that one up to random fluctuation.
    committed Mar 4, 2016
  8. Handle empty haystacks

    Unlikely to (should never) occur in practice, but this is a small
    simplification to the code.
    committed Mar 4, 2016
Commits on Mar 4, 2016
  1. Improve a comment

    committed Mar 4, 2016
  2. Clean up bitmask code a bit

    Instead of storing bitmasks in a separate array, pack them in the
    `match_t` struct. Shouldn't have much effect on perf. Maybe, you could
    make an argument for avoiding the allocation being a tiny advantage
    (almost certainly not measurable), but this might have better cache
    locality.
    
    I have no idea if my fancy Wicoxon Signed Rank test code is working or
    not, but it claims a likely significant reduction for the smaller data
    sets. I guess that makes sense, because there the allocation could
    actually form a measurable portion of the time.
    
    Summary of cpu time and (wall-clock time):
                        best      avg      sd     +/-      p     (best)    (avg)      (sd)     +/-      p
         pathological  1.04000  1.13400 0.04188 [-3.5%]  0.005 (1.04919) (1.13646) (0.04132) [-3.5%]  0.005
            command-t  1.34000  1.39500 0.04642 [-4.7%] 0.0005 (1.33263) (1.39542) (0.04737) [-4.7%] 0.0005
    chromium (subset)  5.02000  5.27400 0.11386 [-0.3%]        (1.13320) (1.20853) (0.03869) [-0.3%]
     chromium (whole)  6.80000  7.03350 0.08326 [-0.2%]        (1.20307) (1.28563) (0.04056) [-0.2%]
           big (400k) 10.64000 10.84500 0.18416 [-0.4%]        (1.82905) (1.94810) (0.08647) [-0.4%]
    committed Mar 1, 2016
Commits on Mar 3, 2016
  1. Update README

    To refer to some newly added capabilities (and
    others not so new), and to elevate speed as the
    primary design goal.
    committed Mar 3, 2016
  2. Actually add <Plug>(CommandTSearch)

    Again, stop overpromising in HISTORY and underdelivering.
    committed Mar 3, 2016
  3. Actually add <Plug>(CommandTHistory)

    HISTORY says I added it, but I forgot.
    committed Mar 3, 2016
  4. Append a space after accepted history/command selections

    Because appending an argument is often the default action that you're
    going to want to do.
    committed Mar 3, 2016
  5. Teach :CommandTCommand to find built-in Ex commands too

    Via a crude hack.
    committed Mar 3, 2016
  6. Add primitive :CommantTCommand finder and scanner

    This is just a crude proof of concept, because Vim only provides a way
    to get user-defined commands (`:commands`). To get built-in commands, I
    am going to have to further inspect some item(s) in the 'runtimepath'
    checking for valid ones.
    committed Mar 3, 2016
  7. Consolidate {Search,History}{Finder,Scanner}

    "Search" is just a specialization of "History", so fold these together
    to share an implementation.
    committed Mar 2, 2016
  8. Add :CommandT{History,Search} and <Plug>(CommandT{History,Search})

    Plenty of duplication here and some gnarly tip-toe-ing needs to be down
    around escaping, but this basically works.
    committed Mar 2, 2016
  9. Make in-progress searches faster in LineFinder

    Instead of recomputing the "paths" on each key press, keep them around
    as long as the finder exists, and keep the finder around for the
    duration of the search.
    
    Note that I probably want to be a bit more clever and set up some
    autocmds, such that I keep the finder around even longer (until I
    switch buffers, or modify the current buffer).
    committed Mar 2, 2016
Commits on Mar 2, 2016
  1. Reorder columns to put avg and sd next to each other

    Also, the spacing here makes it easier to spot significant results.
    committed Mar 2, 2016
  2. Allow use of TIMES environment variable to control benchmark count

    Before committing code I want the full 20 runs, but for quick checking
    in-progress changes, I want to be able to run a smaller number of times,
    or even just once.
    committed Mar 2, 2016
  3. "Modernize" benchmarks

    The old Chromium data sets had a lot of `.svn` dot-directories in them,
    and given that Command-T generally treats dot-files specially, I wanted
    a more representative scenario.
    
    Enter: new Chromium Git repo checkout, but this "only" has 224k files in
    it, so something bigger is needed.
    
    I couldn't find any single open source repo big enough, so I combined
    the file listings from Chromium, the Linux kernel, and Gecko, bringing
    us to a total of about 400k files, which is big enough for now, I guess.
    
    I've also tweaked the iterations count to keep each scenario close to
    the 1-second range, freshened the listing in the "command-t" scenario,
    and made the necessary changes to the benchmark script for it to
    gracefully handle the case where the number of scenarios changes between
    runs.
    committed Mar 2, 2016
  4. Rename significant? to significance

    Hopefully I won't mortally offend any statisticians by conflating
    "p-value" and "significance" here.
    committed Mar 2, 2016
Something went wrong with that request. Please try again.