[wip] fix --include bug when multiple files have same lfs content

[technoweenie]

Currently just reproduces the bug described in #1437 with a test.

[technoweenie]

This passes, but it downloads those objects multiple times:

   trace git-lfs: fetch big/a/a1.big [8754f5ea3cfc9ace8d3e314fc501d31225a5983410e727f42cab6b19317d991d]
   trace git-lfs: fetch big/b/b1.big [8754f5ea3cfc9ace8d3e314fc501d31225a5983410e727f42cab6b19317d991d]
   trace git-lfs: fetch big/big1.big [1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1]
   trace git-lfs: fetch big/big2.big [1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1]
   trace git-lfs: fetch big/big3.big [1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1]

...

   trace git-lfs: HTTP: {"objects":[{"oid":"8754f5ea3cfc9ace8d3e314fc501d31225a5983410e727f42cab6b19317d991d","size":8,"actions":{"download":{"href":"http://127.0.0.1:55072/storage/8754f5ea3cfc9ace8d3e314fc501d31225a5983410e727f42cab6b19317d991d?r=test-fetch-include","expires_at":"0001-01-01T00:00:00Z"}}},{"oid":"8754f5ea3cfc9ace8d3e314fc501d31225a5983410e727f42cab6b19317d991d","size":8,"actions":{"download":{"href":"http://127.0.0.1:55072/storage/8754f5ea3cfc9ace8d3e314fc501d31225a5983410e727f42cab6b19317d991d?r=test-fetch-includ
   trace git-lfs: HTTP: e","expires_at":"0001-01-01T00:00:00Z"}}},{"oid":"1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1","size":10,"actions":{"download":{"href":"http://127.0.0.1:55072/storage/1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1?r=test-fetch-include","expires_at":"0001-01-01T00:00:00Z"}}},{"oid":"1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1","size":10,"actions":{"download":{"href":"http://127.0.0.1:55072/storage/1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1?r=test-fetch-include","expires_at":"0001-01-01T00:00:00Z"}}},{"oid":"1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1","size":10,"actions":{"download":{"href":"http://127.0.0.1:55072/storage/1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1?r=test-fetch-include","expires_at":"0001-01-01T00:00:00Z"}}}]}
   trace git-lfs: HTTP:
   trace git-lfs: tq: starting transfer adapter "basic"
   trace git-lfs: xfer: adapter "basic" Begin() with 3 workers
   trace git-lfs: xfer: adapter "basic" started
   trace git-lfs: xfer: adapter "basic" Add() for "8754f5ea3cfc9ace8d3e314fc501d31225a5983410e727f42cab6b19317d991d"
   trace git-lfs: xfer: adapter "basic" Add() for "8754f5ea3cfc9ace8d3e314fc501d31225a5983410e727f42cab6b19317d991d"
   trace git-lfs: xfer: adapter "basic" Add() for "1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1"
   trace git-lfs: xfer: adapter "basic" Add() for "1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1"
   trace git-lfs: xfer: adapter "basic" worker 2 starting
   trace git-lfs: xfer: adapter "basic" worker 2 waiting for Auth
   trace git-lfs: xfer: adapter "basic" worker 1 starting
   trace git-lfs: xfer: adapter "basic" worker 1 waiting for Auth
   trace git-lfs: xfer: adapter "basic" worker 0 starting
   trace git-lfs: xfer: adapter "basic" worker 0 processing job for "8754f5ea3cfc9ace8d3e314fc501d31225a5983410e727f42cab6b19317d991d"
   trace git-lfs: xfer: adapter "basic" Add() for "1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1"

...

   trace git-lfs: xfer: adapter "basic" worker 1 auth signal received
   trace git-lfs: xfer: adapter "basic" worker 1 processing job for "1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1"
   trace git-lfs: xfer: Attempting to resume download of "8754f5ea3cfc9ace8d3e314fc501d31225a5983410e727f42cab6b19317d991d" from byte 8
   trace git-lfs: xfer: adapter "basic" worker 0 finished job for "8754f5ea3cfc9ace8d3e314fc501d31225a5983410e727f42cab6b19317d991d"
   trace git-lfs: xfer: adapter "basic" worker 0 processing job for "1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1"
   trace git-lfs: xfer: Attempting to resume download of "1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1" from byte 0

...

   trace git-lfs: HTTP: GET http://127.0.0.1:55072/storage/8754f5ea3cfc9ace8d3e314fc501d31225a5983410e727f42cab6b19317d991d
   trace git-lfs: Filled credentials for http://127.0.0.1:55072/test-fetch-include
   trace git-lfs: HTTP: GET http://127.0.0.1:55072/storage/1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1
   trace git-lfs: HTTP: 200
   trace git-lfs: HTTP: 200
   trace git-lfs: Filled credentials for http://127.0.0.1:55072/test-fetch-include
   trace git-lfs: HTTP: GET http://127.0.0.1:55072/storage/1581bec8d5a49e9d310f5cf7f3b1aef7df51615281f7e92269c29d6864446bf1
   trace git-lfs: HTTP: 200

Still trying to find a good place to strip out duplicate objects without breaking things (like 60fd28b).

[ttaylorr]

👍