Weekly Digest (19 January, 2020 - 26 January, 2020) #3237
Description
Here's the Weekly Digest for iterative/dvc:
ISSUES
Last week 44 issues were created.
Of these, 29 issues have been closed and 15 issues are still open.
OPEN ISSUES
💚 #3236 get/get-url: better handle remote:// notation errors, by jorgeorpinel
💚 #3235 GDrive remotes not working (missing pydrive) [MacOS], by jorgeorpinel
💚 #3234 erepo: make it better, by Suor
💚 #3233 readme: update badges, by casperdcl
💚 #3230 gdrive remote strange behaviour, by RomanVeretenov
💚 #3229 [WIP] diff: reimplement interface and tests from scratch, by mroutis
💚 #3226 hdfs: should not require hostname, by efiop
💚 #3219 dvc remote add -d ERROR , by qwljasmine
💚 #3218 dvc add a file from data directory, by dmpetrov
💚 #3217 pipeline: show: outs: eliminate extra edges in DAG, by fabiosantoscode
💚 #3210 Introduce remote fixture, by pared
💚 #3207 [WIP] gc: change option to use single remove flags, by skshetry
💚 #3203 WebUI to manage model, by codergh
💚 #3194 remote: verify repository exists when level requires it, by mroutis
💚 #3193 config: verify before saving a modification, by mroutis
CLOSED ISSUES
❤️ #3232 Remove usage of google-api-python-client deps, by MaxRis
❤️ #3231 some doc link fixes, by pared
❤️ #3228 install: post-checkout: run only on branches/tags, by efiop
❤️ #3227 erepo: pull each time, by Suor
❤️ #3225 remote: verify cache exists on dir checksum, by pared
❤️ #3224 import-url: test: prevent in-repo testing, by pared
❤️ #3223 hooks: checkout: fix incorrect dvc repo detection, by efiop
❤️ #3222 api: fallback to external_repo if not a DVC repository, by skshetry
❤️ #3221 api: does not work with git only local repository, by skshetry
❤️ #3220 get: copy edits aroud --show-url, "dvc", et al., by jorgeorpinel
❤️ #3216 Update Google Project credentials to run tests, by MaxRis
❤️ #3215 some files are not uploaded to TeamDrive, by antonkulaga
❤️ #3214 get/imp: update help output descriptions, by jorgeorpinel
❤️ #3213 push: improve 'no remote specified' hint, by fabiosantoscode
❤️ #3212 another attempt at travis dpl snap, by casperdcl
❤️ #3211 brancher: do not get tree if there's no revs, by skshetry
❤️ #3209 fix snap stable & edge release errors, by casperdcl
❤️ #3208 tests: replace spy() with pytest-mock spy, by fabiosantoscode
❤️ #3206 test: add git_dir, by Suor
❤️ #3205 exception: fix spelling, by skshetry
❤️ #3204 erepo: fix DependencyREPO schema, by efiop
❤️ #3202 dvc: replace flatten_dict with flatten_json, by efiop
❤️ #3201 gdrive: allow configuring no_traverse, by efiop
❤️ #3200 fetch: trust remote checksums, by pared
❤️ #3199 py2: remove use of six, by skshetry
❤️ #3198 Remove spy method, by pared
❤️ #3197 test: refactor tmp_dir/erepo_dir, by Suor
❤️ #3196 get/import: update output (again?), by jorgeorpinel
❤️ #3195 s3: import boto3 inside _copy instead of module level, by mroutis
LIKED ISSUE
👍 #3218 dvc add a file from data directory, by dmpetrov
It received 👍 x3, 😄 x0, 🎉 x0 and ❤️ x0.
NOISY ISSUE
🔈 #3235 GDrive remotes not working (missing pydrive) [MacOS], by jorgeorpinel
It received 15 comments.
PULL REQUESTS
Last week, 42 pull requests were created, updated or merged.
UPDATED PULL REQUEST
Last week, 9 pull requests were updated.
💛 #3234 erepo: make it better, by Suor
💛 #3229 [WIP] diff: reimplement interface and tests from scratch, by mroutis
💛 #3217 pipeline: show: outs: eliminate extra edges in DAG, by fabiosantoscode
💛 #3207 [WIP] gc: change option to use single remove flags, by skshetry
💛 #3194 remote: verify repository exists when level requires it, by mroutis
💛 #3193 config: verify before saving a modification, by mroutis
💛 #2975 add: auto convert absolute_path => relative, by casperdcl
💛 #2957 [WIP][Showcase] dvc: use dynamic scope to stop callback/flag passing, by Suor
💛 #2846 UI progress cleanup, by casperdcl
MERGED PULL REQUEST
Last week, 33 pull requests were merged.
💜 #3232 Remove usage of google-api-python-client deps, by MaxRis
💜 #3231 some doc link fixes, by pared
💜 #3228 install: post-checkout: run only on branches/tags, by efiop
💜 #3227 erepo: pull each time, by Suor
💜 #3225 remote: verify cache exists on dir checksum, by pared
💜 #3224 import-url: test: prevent in-repo testing, by pared
💜 #3223 hooks: checkout: fix incorrect dvc repo detection, by efiop
💜 #3222 api: fallback to external_repo if not a DVC repository, by skshetry
💜 #3220 get: copy edits aroud --show-url, "dvc", et al., by jorgeorpinel
💜 #3216 Update Google Project credentials to run tests, by MaxRis
💜 #3214 get/imp: update help output descriptions, by jorgeorpinel
💜 #3213 push: improve 'no remote specified' hint, by fabiosantoscode
💜 #3212 another attempt at travis dpl snap, by casperdcl
💜 #3211 brancher: do not get tree if there's no revs, by skshetry
💜 #3209 fix snap stable & edge release errors, by casperdcl
💜 #3208 tests: replace spy() with pytest-mock spy, by fabiosantoscode
💜 #3206 test: add git_dir, by Suor
💜 #3205 exception: fix spelling, by skshetry
💜 #3204 erepo: fix DependencyREPO schema, by efiop
💜 #3202 dvc: replace flatten_dict with flatten_json, by efiop
💜 #3201 gdrive: allow configuring no_traverse, by efiop
💜 #3200 fetch: trust remote checksums, by pared
💜 #3199 py2: remove use of six, by skshetry
💜 #3197 test: refactor tmp_dir/erepo_dir, by Suor
💜 #3195 s3: import boto3 inside _copy instead of module level, by mroutis
💜 #3190 external: refactor external_repo() and its users, by Suor
💜 #3188 completion: get --show-url, by mroutis
💜 #3178 logger: remote: use lazy formatting, by efiop
💜 #3169 pull: treat HTTP redirects without Location header as error, by fabiosantoscode
💜 #3130 api: better code documentation, by jorgeorpinel
💜 #3120 Fix import issue without remote config in the target, by sharidas
💜 #3088 dvc: add sanity checks for hard/symlinks, by efiop
💜 #2956 add snap tests, by casperdcl
COMMITS
Last week there were 116 commits.
🛠️ dvc: bump to 0.82.3 by efiop
🛠️ Merge pull request #3213 from fabiosantoscode/feature/3121-push-no-remote-specified-hint push: improve 'no remote specified' hint by efiop
🛠️ Merge pull request #3232 from iterative/2865 Remove usage of google-api-python-client deps by efiop
🛠️ move message creation outside of exception by fabiosantoscode
🛠️ Merge pull request #3228 from efiop/3107 install: post-checkout: run only on branches/tags by efiop
🛠️ Remove usage of google-api-python-client deps by MaxRis
🛠️ Merge pull request #3225 from pared/2647_no_cache_external_dir remote: verify cache exists on dir checksum by efiop
🛠️ remote: verify cache exists on dir checksum by pared
🛠️ Merge pull request #3231 from pared/link_fixes some doc link fixes by efiop
🛠️ some doc link fixes by pared
🛠️ Merge pull request #3227 from Suor/git-pull erepo: pull each time by efiop
🛠️ setup: update pydrive2 version by efiop
🛠️ install: post-checkout: run only on branches/tags We shouldn't try to automatically checkout in detached HEAD state, as we might break the rebase process. Fixes #3107 by efiop
🛠️ erepo: pull each time Fixes #3062. by Suor
🛠️ dvc: bump to 0.82.2 by efiop
🛠️ Merge pull request #3224 from pared/fix_import_url_test import-url: test: prevent in-repo testing by efiop
🛠️ convert to bool by fabiosantoscode
🛠️ move remote checking option to within NoRemoteError by fabiosantoscode
🛠️ import-url: test: prevent in-repo testing by pared
🛠️ Merge pull request #3216 from MaxRis/3157 Update Google Project credentials to run tests by efiop
🛠️ Merge pull request #3222 from skshetry/fallback-external-repo api: fallback to external_repo if not a DVC repository by efiop
🛠️ Merge pull request #3088 from efiop/3080 dvc: add sanity checks for hard/symlinks by efiop
🛠️ api: use current directory for repo if not specified by skshetry
🛠️ api: fallback to external_repo on error by skshetry
🛠️ Merge pull request #3208 from fabiosantoscode/feature/3198-remove-spy-method tests: replace spy() with pytest-mock spy by efiop
🛠️ Merge pull request #3223 from efiop/3107 hooks: checkout: fix incorrect dvc repo detection by efiop
🛠️ hooks: checkout: fix incorrect dvc repo detection When running from non-root directory git ls-files .dvc will treat it as a relative path to the current location, so if you run git checkout from a directory that is not a git/dvc root, our hook will think that it is being run in a non-dvc repo and will do nothing. To fix that, need to use --full-name flag. by efiop
🛠️ Merge pull request #3220 from jorgeorpinel/master get: copy edits aroud --show-url, "dvc", et al. by efiop
🛠️ dvc: revert capitalization of logger.exception strings per https://github.com/iterative/dvc/pull/3220#pullrequestreview-347083819 by jorgeorpinel
🛠️ get: refine --show-url description per https://github.com/iterative/dvc/pull/3220#pullrequestreview-347038086 by jorgeorpinel
🛠️ tests: match recent commits and other misc. language improvements by jorgeorpinel
🛠️ term: review capitalization and backquotes around "dvc" rel: https://github.com/iterative/dvc/pull/3156#discussion_r369897073 by jorgeorpinel
🛠️ get: rewrite --show-url description per https://github.com/iterative/dvc.org/pull/936#discussion_r368680473 by jorgeorpinel
🛠️ get: language feedback on --show-url PR per https://github.com/iterative/dvc/pull/3156#pullrequestreview-345562608 by jorgeorpinel
🛠️ Merge pull request #3214 from jorgeorpinel/master get/imp: update help output descriptions by efiop
🛠️ Update Google Project credentials to run tests by MaxRis
🛠️ Merge pull request #3212 from casperdcl/snap-deploy another attempt at travis dpl snap by efiop
🛠️ get/imp: update help output descriptions per https://github.com/iterative/dvc/issues/3196#issuecomment-577306219 by jorgeorpinel
🛠️ address comments by fabiosantoscode
🛠️ push: improve 'no remote specified' hint Fixes #3121 by fabiosantoscode
🛠️ dvc: add sanity checks for hard/symlinks Fixed #3080 by efiop
🛠️ Revert "debugging (to revert)" This reverts commit 555258c75d5cb43aca045ac2b5c4eda49913aaee. by casperdcl
🛠️ fix before_install checks by casperdcl
🛠️ tidy up by casperdcl
🛠️ fix missing env vars by casperdcl
🛠️ debugging (to revert) by casperdcl
🛠️ attempt to fix travis snap dpl by casperdcl
🛠️ convert tests to pytest function tests by fabiosantoscode
🛠️ Merge pull request #3211 from skshetry/fix-brancher-noscm brancher: do not get tree if there's no revs by efiop
🛠️ brancher: do not get tree if there's no revs by skshetry
🛠️ Merge pull request #3200 from pared/2676_trust_checksum fetch: trust remote checksums by efiop
🛠️ data cloud: refactor verify checksums test by pared
🛠️ data cloud: add explanaiton to pulled checksums save by pared
🛠️ data cloud: verify is config option now by pared
🛠️ data sync: add command unit test by pared
🛠️ remote: tests: refactor verify test by pared
🛠️ test_remote: use mocker.spy instead of spy by pared
🛠️ data cloud: pull: trust remotes md5 on proper flag by pared
🛠️ tests: fix index.remove misuse by efiop
🛠️ Merge pull request #3209 from casperdcl/snap-deploy fix snap stable & edge release errors by efiop
🛠️ ensure correct snap version dirtyness by casperdcl
🛠️ pin snap version to base version by casperdcl
🛠️ don't use patch.object() by fabiosantoscode
🛠️ fix master/edge release provider argument conflicts by casperdcl
🛠️ Merge pull request #3206 from Suor/tests test: add git_dir by efiop
🛠️ tests: replace spy() with pytest-mock spy Fixes #3198 by fabiosantoscode
🛠️ test: add git_dir by Suor
🛠️ Merge pull request #3204 from efiop/3087 erepo: fix DependencyREPO schema by efiop
🛠️ build: don't upgrade pip by efiop
🛠️ travis: don't upgrade pip by efiop
🛠️ Merge pull request #3205 from skshetry/fix-spell exception: fix spelling by efiop
🛠️ exception: fix spelling by skshetry
🛠️ erepo: fix DependencyREPO schema URL should be required and rev or rev_lock should be optional. Related to #3087 by efiop
🛠️ readme: add a release label To avoid accidentally looking like an unstable project. by efiop
🛠️ dvc: bump to 0.82.1 by efiop
🛠️ Merge pull request #3202 from efiop/flatten dvc: replace flatten_dict with flatten_json by efiop
🛠️ Merge pull request #3201 from efiop/traverse_gdrive gdrive: allow configuring no_traverse by efiop
🛠️ dvc: replace flatten_dict with flatten_json flatten_json has a conda package, while flatten_dict doesn't and I'm not willing to maintain yet another conda package :) by efiop
🛠️ gdrive: allow configuring no_traverse by efiop
🛠️ Merge pull request #3199 from skshetry/remove-use-six py2: remove use of six by efiop
🛠️ py2: remove use of six by skshetry
🛠️ Merge pull request #3197 from Suor/refactor-tmpdir test: refactor tmp_dir/erepo_dir by efiop
🛠️ test: do not commit copy.py to git by Suor
🛠️ Merge pull request #3190 from Suor/refactor-erepo external: refactor external_repo() and its users by efiop
🛠️ test: provide make_tmp_dir fixture by Suor
🛠️ test: drop repo_template fixture by Suor
🛠️ Merge pull request #2956 from casperdcl/snap add snap tests by efiop
🛠️ recompress conditional snap channel by casperdcl
🛠️ explicitly spilt snap stable/edge deployments - (re) fix #3152 by casperdcl
🛠️ Remove travis-incompatible tests - reverts + 205077e3a0a0dd1c04df7e4a0ea9fe6f31010036. + 46e06166140b49e902b131cd52ac36d605747ddc. by casperdcl
🛠️ debugging by casperdcl
🛠️ connectivity debugging by casperdcl
🛠️ minor comment update by casperdcl
🛠️ debugging by casperdcl
🛠️ test snaps even where tag is undefined by casperdcl
🛠️ revert whitespace changes, but keep stderr errors by casperdcl
🛠️ promote snap tests for debugging by casperdcl
🛠️ add snap tests - fixes #2954 by casperdcl
🛠️ dvc: bump to 0.82.0 by efiop
🛠️ api: fix get_url() non-dvc repo error message by Suor
🛠️ external: do not fail with no remote if cache is available by Suor
🛠️ api: adjust get_url requiring dvc repo to the new reality by Suor
🛠️ api: allow rev on local repos by Suor
🛠️ external: refactor external_repo() and its users Some things are fixed along the way: - no cache shared between incompatible funcs - some incosistent exceptions are now consistent - import updates for git files now works both for dvc and git repos - "auto-generated-upstream already exists" fixed - dvc.api.open()/read() works with git repos now Fixes #3124, #2976. Closes #3172. by Suor
🛠️ Merge pull request #3178 from efiop/1843 logger: remote: use lazy formatting by efiop
🛠️ logger: utils: fs: use lazy formatting by efiop
🛠️ logger: state: use lazy formatting by efiop
🛠️ logger: remote: use lazy formatting Part of #1843 As our previous investigations showed, path stringification and stuff like relpath are taking a very large chunk of time when working with giant directories. This patch removes format()s and uses lazy formatting provided by logger instead, so stuff like path_info is not stringified until actually needed (e.g. on -v). by efiop
🛠️ logger: support {} formatting by efiop
🛠️ Merge pull request #3169 from fabiosantoscode/feature/2963-dvc-pull-of-s3-remote-with-https-url pull: treat HTTP redirects without Location header as error by efiop
🛠️ Merge pull request #3130 from jorgeorpinel/master api: better code documentation by efiop
🛠️ Merge pull request #3120 from sharidas/no_remote_issue Fix import issue without remote config in the target by efiop
🛠️ Merge pull request #3188 from mroutis/fix-3184 completion: get --show-url by efiop
🛠️ Merge pull request #3195 from mroutis/fix-boto3-import s3: import boto3 inside _copy instead of module level by efiop
🛠️ Fix import issue without remote config in the target Fix the import issue when target does not have remote config. Signed-off-by: Sujith H sujith.h@gmail.com by sharidasan
🛠️ s3: import boto3 inside _copy instead of module level Kudos to @skshetry for bringing this out during a review by mroutis
CONTRIBUTORS
Last week there were 10 contributors.
👤 efiop
👤 fabiosantoscode
👤 MaxRis
👤 pared
👤 Suor
👤 skshetry
👤 jorgeorpinel
👤 casperdcl
👤 sharidasan
👤 mroutis
STARGAZERS
Last week there were 41 stagazers.
⭐ lalessandro
⭐ entruv
⭐ iamyangcao
⭐ Marcnuth
⭐ mobop
⭐ jinlow
⭐ 7ruth
⭐ ebakan
⭐ physicalit
⭐ codergh
⭐ dolongbien
⭐ klieret
⭐ enzolis
⭐ hpgmiskin
⭐ adam-wd
⭐ sisirkoppaka
⭐ jamesmyatt
⭐ alex-does-stuff
⭐ dorg-jmiller
⭐ johnwmillr
⭐ elyall
⭐ TTitcombe
⭐ dsaiztc
⭐ tg-z
⭐ brmarkus
⭐ hurcy
⭐ mgoldchild
⭐ jehoonsong
⭐ ms32035
⭐ sedelmeyer
⭐ jeronimodeleon
⭐ aplascencia0188
⭐ pablodecm
⭐ pssharma
⭐ ChaimaeSriti
⭐ igaitan1991
⭐ aad
⭐ Mikhail-M
⭐ gqpj
⭐ ssoemoe
⭐ starrcadeghc
You all are the stars! 🌟
RELEASES
Last week there were 4 releases.
🚀 0.82.3 0.82.3
🚀 0.82.2 0.82.2
🚀 0.82.1 0.82.1
🚀 0.82.0 0.82.0
That's all for last week, please 👀 Watch and ⭐ Star the repository iterative/dvc to receive next weekly updates. 😃
You can also view all Weekly Digests by clicking here.
Your Weekly Digest bot. 📆