[23.1] Fix disk usage recalculation for distributed object stores

[mvdbeek]

Found this while adding the additional test cases in the first commit. I don't know yet why this wasn't failing on usegalaxy.org (maybe an empty iterable in quota_source_map.ids_per_quota_source() ?), and I suppose it won't fix the issue that @natefoo found in the backend channel.

fd15b82 fixes a bind param mismatch. I'll say the unpopular thing here: we should've used sqlalchemy core for this, since that'd have done the parameter binding for us. Found this while adding the additional test cases in the first commit

53dc916f0d722a350c53dba70e4fabda42d2d7ff fixes the case Nate reported in the backend channel, which is because main uses a hierarchical object store where no backend defines a quota_source attribute.

Disk usage calculation seems to be broken on .org, does anyone know of recent changes that might be suspect?
e.g.:

galaxy_main=> select sum(d.total_size) from history_dataset_association hda join history h on hda.history_id = h.id join galaxy_user u on h.user_id = u.id join dataset d on hda.dataset_id = d.id where u.username = '84756234z' and not hda.purged and not d.purged;
     sum
--------------
 320534890258
(1 row)

galaxy_main=> select disk_usage from galaxy_user where username = '84756234z';
 disk_usage
------------
          0

And recalculating just remains 0.

How to test the changes?

(Select all options that apply)

• I've included appropriate automated tests.
• This is a refactoring of components with existing test coverage.
• Instructions for manual testing are as follows:
  1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

• I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

[mvdbeek]

API test failures are unrelated and due to galaxy-web-05 being down.

[davelopez]

Thank you for the quick fix and the extended coverage!

The remaining integration test is taking a surprisingly long time to finish but I guess also unrelated to this fix.

[mvdbeek]

Yep, it took over an hour to run the node setup action:

[jdavcs]

Looks right to me. Also, a very enthusiastic +1 on the "unpopular" mention of using SQLAlchemy Core instead of raw SQL, especially for parametrized statements.