Optimize Lrama::Counterexamples#follow_l
#607
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`Lrama::Counterexamples#follow_l` is one of bottlenecks of `Lrama::Counterexamples#shortest_path`.
This commit includes these optimizations.
* Use Bitmap to manage `precise_lookahead_set` of counterexamples
* Precompute `number_bitmap` of `Lrama::Grammar::Symbol`
* Memoized some methods of `Lrama::States::Item`
In both profiles, calculation of counterexamples are interrupted because whole calculation
takes too long time, so total number of samples are different.
Before:
```
$ stackprof tmp/stackprof-cpu-myapp_before.dump --method "Lrama::Counterexamples#shortest_path"
Lrama::Counterexamples#shortest_path (yui-knk/lrama/lib/lrama/counterexamples.rb:273)
samples: 6157 self (15.2%) / 15260 total (37.7%)
callers:
15260 ( 100.0%) Lrama::Counterexamples#reduce_reduce_examples
9170 ( 60.1%) Hash#each_key
callees (9103 total):
9311 ( 102.3%) Set#each
3448 ( 37.9%) Lrama::Counterexamples#follow_l
1224 ( 13.4%) Struct#hash
905 ( 9.9%) Set#include?
884 ( 9.7%) Lrama::Counterexamples::Triple#state_item
859 ( 9.4%) Struct#eql?
377 ( 4.1%) #<Class:0x0000000100d4b420>.new
371 ( 4.1%) Class#new
346 ( 3.8%) #<Class:0x0000000100da78b0>.new
184 ( 2.0%) Lrama::Counterexamples#reachable_state_items
179 ( 2.0%) Array#shift
74 ( 0.8%) Array#eql?
52 ( 0.6%) Lrama::States::Item#next_sym
52 ( 0.6%) Array#hash
7 ( 0.1%) Lrama::State#==
code:
| 273 | def shortest_path(conflict_state, conflict_reduce_item, conflict_term)
| 274 | queue = [] #: Array[[Triple, Array[Path::path]]]
| 275 | visited = {} #: Hash[Triple, true]
| 276 | start_state = @States.states.first #: Lrama::State
| 277 | raise "BUG: Start state should be just one kernel." if start_state.kernels.count != 1
184 (0.5%) | 278 | reachable = reachable_state_items(StateItem.new(conflict_state, conflict_reduce_item))
| 279 | start = Triple.new(start_state, start_state.kernels.first, Set.new([@states.eof_symbol]))
| 280 |
| 281 | queue << [start, [StartPath.new(start.state_item)]]
| 282 |
508 (1.3%) / 329 (0.8%) | 283 | while (triple, paths = queue.shift)
3161 (7.8%) / 1208 (3.0%) | 284 | next if visited[triple]
845 (2.1%) / 787 (1.9%) | 285 | visited[triple] = true
| 286 |
| 287 | # Found
16 (0.0%) / 9 (0.0%) | 288 | if triple.state == conflict_state && triple.item == conflict_reduce_item && triple.l.include?(conflict_term)
| 289 | return paths
| 290 | end
| 291 |
| 292 | # transition
316 (0.8%) / 91 (0.2%) | 293 | next_state_item = @transitions[[triple.state_item, triple.item.next_sym]]
135 (0.3%) / 3 (0.0%) | 294 | if next_state_item && reachable.include?(next_state_item)
| 295 | # @type var t: Triple
47 (0.1%) / 8 (0.0%) | 296 | t = Triple.new(next_state_item.state, next_state_item.item, triple.l)
561 (1.4%) / 445 (1.1%) | 297 | queue << [t, paths + [TransitionPath.new(triple.state_item, t.state_item)]]
| 298 | end
| 299 |
| 300 | # production step
9493 (23.4%) / 66 (0.2%) | 301 | @Productions[triple.state_item]&.each do |item|
1178 (2.9%) / 59 (0.1%) | 302 | next unless reachable.include?(StateItem.new(triple.state, item))
| 303 |
3478 (8.6%) / 30 (0.1%) | 304 | l = follow_l(triple.item, triple.l)
| 305 | # @type var t: Triple
372 (0.9%) / 34 (0.1%) | 306 | t = Triple.new(triple.state, item, l)
4128 (10.2%) / 3080 (7.6%) | 307 | queue << [t, paths + [ProductionPath.new(triple.state_item, t.state_item)]]
8 (0.0%) / 8 (0.0%) | 308 | end
| 309 | end
$ stackprof tmp/stackprof-cpu-myapp_before.dump --method "Lrama::Counterexamples#follow_l"
Lrama::Counterexamples#follow_l (yui-knk/lrama/lib/lrama/counterexamples.rb:315)
samples: 292 self (0.7%) / 3448 total (8.5%)
callers:
3448 ( 100.0%) Lrama::Counterexamples#shortest_path
61 ( 1.8%) Lrama::Counterexamples#follow_l
callees (3156 total):
1929 ( 61.1%) Class#new
715 ( 22.7%) Lrama::States::Item#next_next_sym
410 ( 13.0%) Lrama::States::Item#number_of_rest_symbols
70 ( 2.2%) Set#|
61 ( 1.9%) Lrama::Counterexamples#follow_l
22 ( 0.7%) Lrama::Grammar::Symbol#term?
10 ( 0.3%) Lrama::States::Item#new_by_next_position
code:
| 315 | def follow_l(item, current_l)
| 316 | # 1. follow_L (A -> X1 ... Xn-1 • Xn) = L
| 317 | # 2. follow_L (A -> X1 ... Xk • Xk+1 Xk+2 ... Xn) = {Xk+2} if Xk+2 is a terminal
| 318 | # 3. follow_L (A -> X1 ... Xk • Xk+1 Xk+2 ... Xn) = FIRST(Xk+2) if Xk+2 is a nonnullable nonterminal
| 319 | # 4. follow_L (A -> X1 ... Xk • Xk+1 Xk+2 ... Xn) = FIRST(Xk+2) + follow_L (A -> X1 ... Xk+1 • Xk+2 ... Xn) if Xk+2 is a nullable nonterminal
| 320 | case
427 (1.1%) / 17 (0.0%) | 321 | when item.number_of_rest_symbols == 1
1 (0.0%) / 1 (0.0%) | 322 | current_l
375 (0.9%) / 20 (0.0%) | 323 | when item.next_next_sym.term?
2490 (6.1%) / 247 (0.6%) | 324 | Set.new([item.next_next_sym])
40 (0.1%) / 5 (0.0%) | 325 | when !item.next_next_sym.nullable
33 (0.1%) / 1 (0.0%) | 326 | item.next_next_sym.first_set
| 327 | else
142 (0.4%) | 328 | item.next_next_sym.first_set + follow_l(item.new_by_next_position, current_l)
| 329 | end
1 (0.0%) / 1 (0.0%) | 330 | end
| 331 | end
```
After:
```
$ stackprof tmp/stackprof-cpu-myapp.dump --method "Lrama::Counterexamples#shortest_path"
Lrama::Counterexamples#shortest_path (yui-knk/lrama/lib/lrama/counterexamples.rb:273)
samples: 6731 self (19.3%) / 12499 total (35.9%)
callers:
12499 ( 100.0%) Lrama::Counterexamples#reduce_reduce_examples
7258 ( 58.1%) Hash#each_key
callees (5768 total):
7337 ( 127.2%) Set#each
1118 ( 19.4%) Lrama::Counterexamples::Triple#state_item
1042 ( 18.1%) Set#include?
639 ( 11.1%) Struct#hash
483 ( 8.4%) Class#new
462 ( 8.0%) #<Class:0x000000011c0eb7e0>.new
440 ( 7.6%) #<Class:0x000000011c0ec780>.new
434 ( 7.5%) Lrama::Counterexamples#follow_l
372 ( 6.4%) Lrama::Counterexamples#reachable_state_items
259 ( 4.5%) Struct#eql?
239 ( 4.1%) Array#shift
67 ( 1.2%) Lrama::States::Item#next_sym
65 ( 1.1%) Array#hash
61 ( 1.1%) Array#eql?
8 ( 0.1%) Lrama::State#==
code:
| 273 | def shortest_path(conflict_state, conflict_reduce_item, conflict_term)
| 274 | queue = [] #: Array[[Triple, Array[Path::path]]]
| 275 | visited = {} #: Hash[Triple, true]
| 276 | start_state = @States.states.first #: Lrama::State
| 277 | conflict_term_bit = Bitmap::from_array([conflict_term.number])
| 278 | raise "BUG: Start state should be just one kernel." if start_state.kernels.count != 1
372 (1.1%) | 279 | reachable = reachable_state_items(StateItem.new(conflict_state, conflict_reduce_item))
| 280 | start = Triple.new(start_state, start_state.kernels.first, Bitmap::from_array([@states.eof_symbol.number]))
| 281 |
1 (0.0%) | 282 | queue << [start, [StartPath.new(start.state_item)]]
| 283 |
431 (1.2%) / 192 (0.6%) | 284 | while (triple, paths = queue.shift)
1899 (5.4%) / 1154 (3.3%) | 285 | next if visited[triple]
1000 (2.9%) / 941 (2.7%) | 286 | visited[triple] = true
| 287 |
| 288 | # Found
14 (0.0%) / 6 (0.0%) | 289 | if (triple.state == conflict_state) && (triple.item == conflict_reduce_item) && (triple.l & conflict_term_bit != 0)
| 290 | return paths
| 291 | end
| 292 |
| 293 | # transition
348 (1.0%) / 107 (0.3%) | 294 | next_state_item = @transitions[[triple.state_item, triple.item.next_sym]]
147 (0.4%) / 3 (0.0%) | 295 | if next_state_item && reachable.include?(next_state_item)
| 296 | # @type var t: Triple
59 (0.2%) / 9 (0.0%) | 297 | t = Triple.new(next_state_item.state, next_state_item.item, triple.l)
668 (1.9%) / 528 (1.5%) | 298 | queue << [t, paths + [TransitionPath.new(triple.state_item, t.state_item)]]
| 299 | end
| 300 |
| 301 | # production step
7566 (21.7%) / 85 (0.2%) | 302 | @Productions[triple.state_item]&.each do |item|
1416 (4.1%) / 78 (0.2%) | 303 | next unless reachable.include?(StateItem.new(triple.state, item))
| 304 |
473 (1.4%) / 39 (0.1%) | 305 | l = follow_l(triple.item, triple.l)
| 306 | # @type var t: Triple
448 (1.3%) / 36 (0.1%) | 307 | t = Triple.new(triple.state, item, l)
4910 (14.1%) / 3548 (10.2%) | 308 | queue << [t, paths + [ProductionPath.new(triple.state_item, t.state_item)]]
5 (0.0%) / 5 (0.0%) | 309 | end
| 310 | end
$ stackprof tmp/stackprof-cpu-myapp.dump --method "Lrama::Counterexamples#follow_l"
Lrama::Counterexamples#follow_l (yui-knk/lrama/lib/lrama/counterexamples.rb:316)
samples: 89 self (0.3%) / 434 total (1.2%)
callers:
434 ( 100.0%) Lrama::Counterexamples#shortest_path
80 ( 18.4%) Lrama::Counterexamples#follow_l
callees (345 total):
221 ( 64.1%) Lrama::States::Item#number_of_rest_symbols
102 ( 29.6%) Lrama::States::Item#next_next_sym
80 ( 23.2%) Lrama::Counterexamples#follow_l
19 ( 5.5%) Lrama::Grammar::Symbol#term?
3 ( 0.9%) Lrama::States::Item#new_by_next_position
code:
| 316 | def follow_l(item, current_l)
| 317 | # 1. follow_L (A -> X1 ... Xn-1 • Xn) = L
| 318 | # 2. follow_L (A -> X1 ... Xk • Xk+1 Xk+2 ... Xn) = {Xk+2} if Xk+2 is a terminal
| 319 | # 3. follow_L (A -> X1 ... Xk • Xk+1 Xk+2 ... Xn) = FIRST(Xk+2) if Xk+2 is a nonnullable nonterminal
| 320 | # 4. follow_L (A -> X1 ... Xk • Xk+1 Xk+2 ... Xn) = FIRST(Xk+2) + follow_L (A -> X1 ... Xk+1 • Xk+2 ... Xn) if Xk+2 is a nullable nonterminal
| 321 | case
241 (0.7%) / 20 (0.1%) | 322 | when item.number_of_rest_symbols == 1
| 323 | current_l
93 (0.3%) / 26 (0.1%) | 324 | when item.next_next_sym.term?
77 (0.2%) / 30 (0.1%) | 325 | item.next_next_sym.number_bitmap
13 (0.0%) / 7 (0.0%) | 326 | when !item.next_next_sym.nullable
3 (0.0%) / 3 (0.0%) | 327 | item.next_next_sym.first_set_bitmap
| 328 | else
85 (0.2%) / 1 (0.0%) | 329 | item.next_next_sym.first_set_bitmap | follow_l(item.new_by_next_position, current_l)
| 330 | end
2 (0.0%) / 2 (0.0%) | 331 | end
| 332 | end
```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Lrama::Counterexamples#follow_lis one of bottlenecks ofLrama::Counterexamples#shortest_path. This commit includes these optimizations.precise_lookahead_setof counterexamplesnumber_bitmapofLrama::Grammar::SymbolLrama::States::ItemIn both profiles, calculation of counterexamples are interrupted because whole calculation takes too long time, so total number of samples are different.
Before:
After: