Reuse buffers across scopes.

[copybara-service]

Reuse buffers across scopes.

Previously buffer reuse pass would merge allocations at the innermost scope and hoist allocations to the outermost scope. This misses some reuse opportunities. For example, the two allocations in the separate scf blocks won't get merged before this CL.

  scf.if {
    scf.if {
      %a = memref.alloc() : memref<f32>
      "some_a.op"(%a) : (memref<f32>) -> ()
    }
    scf.if {
      %b = memref.alloc() : memref<f32>
      "some_b.op"(%b) : (memref<f32>) -> ()
    }
  }

Before this CL, two allocations are hoisted to the top level.

  %a = memref.alloc() : memref<f32>
  %b = memref.alloc() : memref<f32>
  scf.if {
    scf.if {
      "some_a.op"(%a) : (memref<f32>) -> ()
    }
    scf.if {
      "some_b.op"(%b) : (memref<f32>) -> ()
    }
  }

After this CL, only one allocation is needed because we run buffer reuse after each hoist. It detected that the buffers could be reused after the two allocations are hoisted out of the their separate scf blocks but before they are hoisted again to the top.

  memref.alloc() : memref<f32>
  scf.if {
    scf.if {
      "some_a.op"(%a) : (memref<f32>) -> ()
    }
    scf.if {
      "some_b.op"(%b) : (memref<f32>) -> ()
    }
  }

In summary this CL tries to apply buffer reuse after hoisting allocations at each scope level, which fixes this issue.