Skip to content

Fix failing testing on CI: Use sample S3 url DANDI archive, our copy of old .deb, specific miniconda installer for py 3.7 #7379

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 1, 2023

Conversation

yarikoptic
Copy link
Member

@yarikoptic yarikoptic commented May 1, 2023

In this PR I decided to absorb all fixes to get our CI green again

  1. ref: bucket removed permission to list versions? OpenNeuroOrg/openneuro#2809 - prior one stopped allowing for ListObjectVersions

I have better "control" over what happens on DANDI bucket, and this URL we share with other projects for testing ROS3 driver, so used in other projects tests

This test started to fail Apr 28th, taints other tests failures, so should be merged asap

  1. Use our mirror for .deb instead of snapshots (replaces Use our mirror instead of snapshot #7381)
  2. Use specific version of miniconda installer (avail with fresh 0.12.0 datalad-installer)

…ObjectVersions

ref: OpenNeuroOrg/openneuro#2809

I have better "control" over what happens on DANDI bucket, and this URL
we share with other projects for testing ROS3 driver, so used in other projects tests
@yarikoptic yarikoptic added the semver-tests Changes only affect tests, no impact on version label May 1, 2023
@yarikoptic
Copy link
Member Author

appveyor -- #7381 or #7380 should help. I will actually fold #7381 in this one.
Travis -- added another commit 9d8d048 -- see its description for more detail.

@yarikoptic yarikoptic changed the title Use sample S3 url DANDI archive - prior one stopped allowing for ListObjectVersions Fix failing testing on CI: Use sample S3 url DANDI archive, our copy of old .deb, specific miniconda installer for py 3.7 May 1, 2023
@yarikoptic yarikoptic added the CHANGELOG-missing When a PR's description does not contain a changelog item, yet. label May 1, 2023
@github-actions github-actions bot removed the CHANGELOG-missing When a PR's description does not contain a changelog item, yet. label May 1, 2023
DataLad Bot and others added 4 commits May 1, 2023 15:03
Although snapshots might be more "official" and thus more "reliably present"
than our own server,  snapshots. has all kinds of throttling settings which
delay download or even cause it to fail:
datalad/datalad-installer#154

although datalad-installer should retry now (after it gets upgraded within
appveyor setup see datalad#7380) if
size changes, I think it would still be more robust to just get it from our
server.

I also made a note in
datalad/datalad-installer#160
so may be we gain possibility to specify multiple URLs thus to robustify.

=== Do not change lines below ===
{
 "chain": [],
 "cmd": "sed -i -e s,http://snapshot.debian.org/archive/debian/20210906T204127Z/pool/main/g/git-annex/,https://datasets.datalad.org/datalad/packages/neurodebian/,g .appveyor.yml",
 "exit": 0,
 "extra_inputs": [],
 "inputs": [],
 "outputs": [
  ".appveyor.yml"
 ],
 "pwd": "."
}
^^^ Do not change lines above ^^^
Python 3.7 was dropped already from recent miniconda installer scripts. So we should use
most recent datalad-installer with some v0.12.0 we should be able to install desired version
of miniconda and thus point to prior version which still uses python 3.7
@codecov
Copy link

codecov bot commented May 1, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: +2.35 🎉

Comparison is base (d90e2ce) 88.34% compared to head (9676aad) 90.69%.

Additional details and impacted files
@@            Coverage Diff             @@
##            maint    #7379      +/-   ##
==========================================
+ Coverage   88.34%   90.69%   +2.35%     
==========================================
  Files         327      327              
  Lines       44685    44685              
  Branches        0     5922    +5922     
==========================================
+ Hits        39476    40527    +1051     
+ Misses       5209     4143    -1066     
- Partials        0       15      +15     
Impacted Files Coverage Δ
datalad/tests/test_s3.py 56.00% <100.00%> (ø)

... and 43 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@yarikoptic
Copy link
Member Author

Travis and appveyor are green now -- I will proceed so we let other PRs to get green.

@yarikoptic yarikoptic merged commit 79b25d2 into datalad:maint May 1, 2023
@yarikoptic-gitmate
Copy link
Collaborator

PR released in 0.18.4

@yarikoptic yarikoptic deleted the bf-s3-anon branch November 27, 2023 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver-tests Changes only affect tests, no impact on version
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants