Permalink
Commits on Jan 11, 2017
  1. Merge pull request #28 from syossan27/fix/walk-node-type-if

    Fix walk processing when node type is if
    committed on GitHub Jan 11, 2017
Commits on Jan 10, 2017
  1. Fix walk processing when node type is if

    syossan27 committed Jan 10, 2017
Commits on Dec 3, 2016
  1. Merge pull request #23 from haya14busa/sum-type

    Make node types more strict
    committed on GitHub Dec 3, 2016
  2. add comment

    committed Dec 3, 2016
  3. fix unused

    committed Dec 3, 2016
  4. fix node exportation

    committed Dec 3, 2016
  5. make Statement sum type

    committed Dec 3, 2016
  6. make Expr sum type

    committed Dec 3, 2016
  7. Merge pull request #25 from haya14busa/reviewdog

    CI: introduce reviewdog
    committed on GitHub Dec 3, 2016
  8. travis: remove lint from travis

    committed Dec 3, 2016
  9. add .drone.yml

    committed Dec 3, 2016
  10. Merge pull request #24 from haya14busa/json-arg

    JSON arg
    committed on GitHub Dec 3, 2016
Commits on Nov 4, 2016
  1. add -json arg to output AST in json

    committed Nov 4, 2016
Commits on Sep 28, 2016
  1. Merge pull request #22 from haya14busa/fix-parse-cmd

    Fix parse syntax cmd
    committed on GitHub Sep 28, 2016
  2. sh go/generate.sh

    committed Sep 28, 2016
  3. fix parse_cmd_syntax

    committed Sep 28, 2016
Commits on Sep 27, 2016
  1. Merge pull request #21 from haya14busa/parse-syntax-cmd

    Parse syntax cmd
    committed on GitHub Sep 27, 2016
  2. sh ./go/generate.sh

    committed Sep 27, 2016
  3. fix parse :syntax with <bar>

    committed Sep 27, 2016
  4. Merge pull request #20 from haya14busa/fix-minlen

    Fix minlen of builtin commands
    committed on GitHub Sep 27, 2016
  5. go: fix minlen of builtin_commands

    ref: #19
    committed Sep 27, 2016
Commits on Sep 19, 2016
  1. Merge pull request #17 from haya14busa/ast-walk

    ast: introduce Walk and Inspect
    committed on GitHub Sep 19, 2016
  2. ast: add test for Inspect with minor fix

    committed Sep 19, 2016
  3. use pointer for node reference

    committed Sep 19, 2016
  4. ast: introduce Walk and Inspect

    committed Sep 19, 2016
  5. Merge pull request #16 from haya14busa/performance-export

    Improve performance2: reduce unneeded allocation
    committed on GitHub Sep 19, 2016
  6. reduce slice allocation when NODE_DICT type conversion

    Before:
             .          .    375:
             .          .    376:   case NODE_DICT:
             .          .    377:           var kvs []ast.KeyValue
             .          .    378:           for _, nn := range n.value.([]interface{}) {
             .          .    379:                   kv := nn.([]interface{})
             .        3MB    380:                   k := newAstNode(kv[0].(*VimNode))
             .     2.50MB    381:                   v := newAstNode(kv[1].(*VimNode))
           5MB        5MB    382:                   kvs = append(kvs, ast.KeyValue{Key: k, Value: v})
             .          .    383:           }
             .          .    384:           return &ast.Dict{
             .          .    385:                   Lcurlybrace: pos,
             .          .    386:                   Entries:     kvs,
             .          .    387:           }
             .          .    388:
    
    After:
             .          .    375:
             .          .    376:   case NODE_DICT:
             .          .    377:           entries := n.value.([]interface{})
           2MB        2MB    378:           kvs := make([]ast.KeyValue, 0, len(entries))
             .          .    379:           for _, nn := range entries {
             .          .    380:                   kv := nn.([]interface{})
             .     3.50MB    381:                   k := newAstNode(kv[0].(*VimNode))
             .     3.50MB    382:                   v := newAstNode(kv[1].(*VimNode))
             .          .    383:                   kvs = append(kvs, ast.KeyValue{Key: k, Value: v})
             .          .    384:           }
             .          .    385:           return &ast.Dict{
             .          .    386:                   Lcurlybrace: pos,
           1MB        1MB    387:                   Entries:     kvs,
             .          .    388:           }
    committed Sep 19, 2016
  7. return nil for Node(-1)

    $ benchcmp prof.3 prof.4
    benchmark                old ns/op     new ns/op     delta
    BenchmarkParseFile-4     227103681     221732445     -2.37%
    committed Sep 19, 2016
  8. update benchmarks result in README

    committed Sep 19, 2016
  9. reduce unused Regexp compile

    committed Sep 19, 2016
  10. s/NewNode/newAstNode

    committed Sep 19, 2016
  11. move exporter pkg func to internal and reduce unneeded conversions

    $ benchcmp prof.1 prof.2
    benchmark                old ns/op     new ns/op     delta
    BenchmarkParseFile-4     244779103     225677916     -7.80%
    
    $ go tool pprof --alloc_space go-vimlparser.test prof.mem
    Entering interactive mode (type "help" for commands)
    (pprof) top
    1153.55MB of 1363.78MB total (84.58%)
    Dropped 30 nodes (cum <= 6.82MB)
    Showing top 10 nodes out of 56 (cum >= 814.41MB)
          flat  flat%   sum%        cum   cum%
      261.21MB 19.15% 19.15%   261.21MB 19.15%  github.com/haya14busa/go-vimlparser/go.(*StringReader).__init__
      194.56MB 14.27% 33.42%   480.71MB 35.25%  github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_expr9
      185.66MB 13.61% 47.03%   348.66MB 25.57%  github.com/haya14busa/go-vimlparser/go.(*ExprTokenizer).get
       98.50MB  7.22% 54.26%    98.50MB  7.22%  reflect.unsafe_New
       88.56MB  6.49% 60.75%   112.06MB  8.22%  github.com/haya14busa/go-vimlparser/go.NewNode
       77.52MB  5.68% 66.43%    84.02MB  6.16%  github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_curly_parts
       70.52MB  5.17% 71.60%   154.54MB 11.33%  github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_identifier
       69.52MB  5.10% 76.70%    74.02MB  5.43%  github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_dot
          59MB  4.33% 81.03%    64.50MB  4.73%  github.com/haya14busa/go-vimlparser/go.(*ExprTokenizer).get2
       48.51MB  3.56% 84.58%   814.41MB 59.72%  github.com/haya14busa/go-vimlparser/go.(*VimLParser).parse_one_cmd
    committed Sep 19, 2016
  12. slice allocation

    $ benchcmp prof.0 prof.1
    benchmark                old ns/op     new ns/op     delta
    BenchmarkParseFile-4     245095530     244779103     -0.13%
    committed Sep 19, 2016
  13. prof.0: initial profiling. NewExportNode seems to have a room for imp…

    …rovement
    
    $ go test -v -run="^$" -bench="." -benchtime=5s -cpuprofile=prof.cpu -memprofile=prof.mem | tee prof.0
    BenchmarkParseFile-4          30         245095530 ns/op
    
    $ go tool pprof --alloc_space go-vimlparser.test prof.mem
    Entering interactive mode (type "help" for commands)
    (pprof) top
    1401.64MB of 1678.59MB total (83.50%)
    Dropped 27 nodes (cum <= 8.39MB)
    Showing top 10 nodes out of 56 (cum >= 143.54MB)
          flat  flat%   sum%        cum   cum%
      297.09MB 17.70% 17.70%   345.10MB 20.56%  github.com/haya14busa/go-vimlparser/go.NewExportNode
      254.71MB 15.17% 32.87%   254.71MB 15.17%  github.com/haya14busa/go-vimlparser/go.(*StringReader).__init__
      195.13MB 11.62% 44.50%   352.14MB 20.98%  github.com/haya14busa/go-vimlparser/go.(*ExprTokenizer).get
      193.57MB 11.53% 56.03%   468.70MB 27.92%  github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_expr9
       99.08MB  5.90% 61.93%   166.59MB  9.92%  github.com/haya14busa/go-vimlparser/internal/exporter.NewNode
          83MB  4.94% 66.88%       83MB  4.94%  reflect.unsafe_New
       75.52MB  4.50% 71.38%    78.02MB  4.65%  github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_curly_parts
       71.52MB  4.26% 75.64%    77.02MB  4.59%  github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_dot
       66.50MB  3.96% 79.60%       74MB  4.41%  github.com/haya14busa/go-vimlparser/go.(*ExprTokenizer).get2
       65.52MB  3.90% 83.50%   143.54MB  8.55%  github.com/haya14busa/go-vimlparser/go.(*ExprParser).parse_identifier
    (pprof) list NewExportNode
    Total: 1.64GB
    ROUTINE ======================== github.com/haya14busa/go-vimlparser/go.NewExportNode in /home/haya14busa/src/github.com/haya14busa/go-vimlparser/go/export.go
      297.09MB     1.10GB (flat, cum) 66.83% of Total
             .          .    132:   if n == nil {
             .          .    133:           return nil
             .          .    134:   }
             .          .    135:   list := make([]*ExportNode, 0)
             .          .    136:   for _, n := range n.list {
             .     1.50MB    137:           list = append(list, NewExportNode(n))
             .          .    138:   }
             .          .    139:   rlist := make([]*ExportNode, 0)
             .          .    140:   for _, n := range n.rlist {
             .    34.51MB    141:           rlist = append(rlist, NewExportNode(n))
             .          .    142:   }
             .          .    143:   body := make([]*ExportNode, 0)
             .          .    144:   for _, n := range n.body {
        2.01MB   294.08MB    145:           body = append(body, NewExportNode(n))
             .          .    146:   }
             .          .    147:   elseif := make([]*ExportNode, 0)
             .          .    148:   for _, n := range n.elseif {
             .   114.53MB    149:           elseif = append(elseif, NewExportNode(n))
             .          .    150:   }
             .          .    151:   catch := make([]*ExportNode, 0)
             .          .    152:   for _, n := range n.catch {
             .          .    153:           catch = append(catch, NewExportNode(n))
             .          .    154:   }
             .          .    155:   return &ExportNode{
             .          .    156:           Type:  n.type_,
          18MB       18MB    157:           Pos:   NewExportPos(n.pos),
             .   150.54MB    158:           Left:  NewExportNode(n.left),
             .    86.53MB    159:           Right: NewExportNode(n.right),
             .    54.02MB    160:           Cond:  NewExportNode(n.cond),
             .          .    161:           Rest:  NewExportNode(n.rest),
             .          .    162:           List:  list,
             .          .    163:           Rlist: rlist,
             .          .    164:           Body:  body,
             .          .    165:           Op:    n.op,
             .          .    166:           Str:   n.str,
             .          .    167:           Depth: n.depth,
             .          .    168:           Value: n.value,
             .          .    169:
             .    48.01MB    170:           Ea:   NewExportExArg(n.ea),
           6MB        6MB    171:           Attr: NewExportFuncAttr(n.attr),
             .          .    172:
             .        8MB    173:           Endfunction: NewExportNode(n.endfunction),
             .          .    174:           Elseif:      elseif,
             .    28.01MB    175:           Else:        NewExportNode(n.else_),
             .     6.50MB    176:           Endif:       NewExportNode(n.endif),
             .   512.16kB    177:           Endwhile:    NewExportNode(n.endwhile),
             .          .    178:           Endfor:      NewExportNode(n.endfor),
             .          .    179:           Endtry:      NewExportNode(n.endtry),
             .          .    180:
             .          .    181:           Catch:   catch,
             .          .    182:           Finally: NewExportNode(n.finally),
             .          .    183:
             .          .    184:           Pattern: n.pattern,
      271.08MB   271.08MB    185:           Curly:   n.curly,
             .          .    186:   }
             .          .    187:}
             .          .    188:
             .          .    189:type ExportFuncAttr struct {
             .          .    190:   Range   bool
    
    $ go tool pprof go-vimlparser.test prof.cpu
    Entering interactive mode (type "help" for commands)
    (pprof) top
    4020ms of 9040ms total (44.47%)
    Dropped 108 nodes (cum <= 45.20ms)
    Showing top 10 nodes out of 151 (cum >= 420ms)
          flat  flat%   sum%        cum   cum%
        1020ms 11.28% 11.28%     1710ms 18.92%  runtime.scanobject
         750ms  8.30% 19.58%     1430ms 15.82%  runtime.mallocgc
         330ms  3.65% 23.23%      760ms  8.41%  regexp.(*machine).onepass
         320ms  3.54% 26.77%      320ms  3.54%  sync/atomic.CompareAndSwapUint32
         290ms  3.21% 29.98%      290ms  3.21%  runtime.heapBitsForObject
         290ms  3.21% 33.19%      430ms  4.76%  runtime.mapaccess2_faststr
         270ms  2.99% 36.17%      270ms  2.99%  sync/atomic.AddUint32
         260ms  2.88% 39.05%      620ms  6.86%  github.com/haya14busa/go-vimlparser/go.NewExportNode
         260ms  2.88% 41.92%      540ms  5.97%  sync.(*Mutex).Unlock
         230ms  2.54% 44.47%      420ms  4.65%  runtime.greyobject
    committed Sep 19, 2016