Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Rework conda mirror to not use conda-index #14

Merged
merged 10 commits into from
Dec 12, 2016
Merged

ENH: Rework conda mirror to not use conda-index #14

merged 10 commits into from
Dec 12, 2016

Conversation

ericdill
Copy link
Contributor

@ericdill ericdill commented Dec 8, 2016

No description provided.

@codecov-io
Copy link

codecov-io commented Dec 8, 2016

Current coverage is 92.77% (diff: 89.65%)

Merging #14 into master will increase coverage by 1.65%

@@             master        #14   diff @@
==========================================
  Files             2          2          
  Lines           169        180    +11   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits            154        167    +13   
+ Misses           15         13     -2   
  Partials          0          0          

Powered by Codecov. Last update 7d00550...0f88254

@ericdill
Copy link
Contributor Author

ericdill commented Dec 8, 2016

what's left: figure out how to make validation work

try:
info = repodata[package]
except KeyError:
logging.info("%s is not in the upstream index. Removing..."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: should this be a warn? Feels like an possible important indicator that something might be amiss if we have a package that upstream never had or not longer has.

except tarfile.TarError:
logging.debug("tarfile error encountered. Original error below.")
logging.debug(pformat(traceback.format_exc()))
logging.info("Removing package: %s", filename)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Maybe use warn level? A corrupted tarball might indicate a bigger problem we need to investigate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corrupted tarball is more frequently a "connection interrupted" issue during download than anything else

pformat(os.listdir(download_dir)))
for f in os.listdir(download_dir):

if f.endswith('json') or f.endswith('.bz2'):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I'm wrong, but I expect most changes to be addition of new packages rather than updates. In that case, it seems that moving all of the packages into place and then the JSON file would reduce the window of time when a user might do a conda install foo and get an error about a missing package because it hasn't been copied into place yet.

@parente parente merged commit 954eb80 into vericast:master Dec 12, 2016
@ericdill ericdill deleted the dont-use-conda-index branch December 12, 2016 18:41
@ericdill
Copy link
Contributor Author

Thanks @parente

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants