Weekly Digest (28 June, 2020 - 5 July, 2020) #4168
Description
Here's the Weekly Digest for iterative/dvc:
ISSUES
Last week 40 issues were created.
Of these, 25 issues have been closed and 15 issues are still open.
OPEN ISSUES
💚 #4167 ssh: support scp-like relpaths in urls, by Kuluum
💚 #4166 Add more tests according to .gitignore, by karajan1001
💚 #4165 daemon: launch cmd is broken when running main() from elsewhere, by efiop
💚 #4164 dvc push doesn't recognise that files are missing in remote storage, by dldx
💚 #4162 Developping a workflow for simultaneous work, by bjonnh
💚 #4160 'dvc add dir' fails when 'dir' contains a file and a relative symlink on this file , by CRodVig
💚 #4159 dvc status reports changed checksum when it should not, by jnd77
💚 #4153 keep track of CVE score for our dependencies, by efiop
💚 #4146 hooks: pylint: fix pre-commit, by casperdcl
💚 #4145 hooks: pylint: pre-commit failure, by casperdcl
💚 #4144 get-url and import-url doesn't seem to work with S3 buckets anymore., by anotherbugmaster
💚 #4143 Artifact does not get added to .gitignore on failed multi-add, by pared
💚 #4139 push: OOM when uploading 1.5TB(3.5Mfiles) dataset to local remote, by lenrys29
💚 #4136 conf: credentialpath is not recalculated when it's a relpath, by Suor
💚 #4131 Issues with dvc get and dvc get-url after 1.0, by honnibal
CLOSED ISSUES
❤️ #4163 tests: create remote fixture, by efiop
❤️ #4161 jsonschema: generate from schema, by skshetry
❤️ #4158 dvc plots diff: use a descriptive filename instead of plots.html by default, by xnutsive
❤️ #4157 gdrive: pop prefix kwarg, by efiop
❤️ #4156 status: pass tree as a kwarg, by efiop
❤️ #4155 [RFC] tests: make read_text() work with directories, by efiop
❤️ #4154 dvc add --external fails on files created by gsutil compose, by haishiro
❤️ #4152 [WIP] subrepo: get/import/list support, by skshetry
❤️ #4151 Endpoint URL is not taken into account when adding an external file from Minio, by lucasmaheo
❤️ #4150 DVC status fails after using add -R, by moshe-rl
❤️ #4149 s3,gs: don't append "/" to prefixed path when listing objects, by pmrowla
❤️ #4148 completion: update dependency, by casperdcl
❤️ #4147 azure: use knack directly instead of azure-cli-core, by efiop
❤️ #4142 GC encounters unexpected error, by WestXu
❤️ #4141 dvc pull does not seem to work for files whose md5 starts with 00, by jnd77
❤️ #4140 tests: remotes: use TmpDir-like fixtures, by efiop
❤️ #4138 Reorder package widgets, by dmpetrov
❤️ #4137 azure: use CLI config, by efiop
❤️ #4135 remote: repo.cloud.pull() errors out with "'NoneType' object has no attribute 'execute'", by Suor
❤️ #4134 Consider using an alternative for ruamel.yaml due to critical CVE, by stefanvangastel
❤️ #4133 RepoTree: preserve relative paths in copytree(), by pmrowla
❤️ #4132 rename .dvc file to .lock file, by skshetry
❤️ #4130 metrics: skip missing metrics silently, by Suor
❤️ #4129 azure: use ~/.azure/config, by dmpetrov
❤️ #4128 hdfs: implement makedirs separately, by efiop
LIKED ISSUE
👍 #4140 tests: remotes: use TmpDir-like fixtures, by efiop
It received 👍 x1, 😄 x0, 🎉 x1 and ❤️ x1.
NOISY ISSUE
🔈 #4134 Consider using an alternative for ruamel.yaml due to critical CVE, by stefanvangastel
It received 15 comments.
PULL REQUESTS
Last week, 17 pull requests were created, updated or merged.
UPDATED PULL REQUEST
Last week, 3 pull requests were updated.
💛 #4146 hooks: pylint: fix pre-commit, by casperdcl
💛 #4120 Optimize ignore performance, by karajan1001
💛 #4114 version: redesign output format for the dvc version command #3499, by sahilbhosale63
MERGED PULL REQUEST
Last week, 14 pull requests were merged.
💜 #4163 tests: create remote fixture, by efiop
💜 #4157 gdrive: pop prefix kwarg, by efiop
💜 #4156 status: pass tree as a kwarg, by efiop
💜 #4155 [RFC] tests: make read_text() work with directories, by efiop
💜 #4149 s3,gs: don't append "/" to prefixed path when listing objects, by pmrowla
💜 #4148 completion: update dependency, by casperdcl
💜 #4147 azure: use knack directly instead of azure-cli-core, by efiop
💜 #4140 tests: remotes: use TmpDir-like fixtures, by efiop
💜 #4138 Reorder package widgets, by dmpetrov
💜 #4137 azure: use CLI config, by efiop
💜 #4133 RepoTree: preserve relative paths in copytree(), by pmrowla
💜 #4130 metrics: skip missing metrics silently, by Suor
💜 #4128 hdfs: implement makedirs separately, by efiop
💜 #4122 pylint: enable additional checks, by skshetry
COMMITS
Last week there were 21 commits.
🛠️ dvc: bump to 1.1.7 by efiop
🛠️ tests: create remote fixture (#4163) Usually we need it to parametrize tests with, so there is no
reason to duplicate the code. I've left local_remote fixture
just because it is so convenient and often used in a lot of tests. by efiop
🛠️ [status: pass tree as a kwarg (#4156) * [RFC] tests: make read_text() work with directories
This PR is just a showcase to see how we like it. I've converted a few
tests to it, but there will be much more to come.
assert (tmp_dir / "file") == "contents"
assert (tmp_dir / "dir").read_text() == {"subdir": {"file": "hello"}}
This approach is nicer because it allows us to use a declaratory
asserts instead of relying on fs files. The problem becomes much more
apparent once you get into remote workspaces, where we trees_equal
doesn't work.
- status: pass tree as a kwarg
Fixes #4150
Depends on https://github.com/iterative/dvc/pull/4155](https://github.com/iterative/dvc/commit/8dd264c025663a357c7b837c4542a3522d3cc6f1) by efiop
🛠️ [[RFC] tests: make read_text() work with directories (#4155) This PR is just a showcase to see how we like it. I've converted a few
tests to it, but there will be much more to come.
assert (tmp_dir / "file") == "contents"
assert (tmp_dir / "dir").read_text() == {"subdir": {"file": "hello"}}
This approach is nicer because it allows us to use a declaratory
asserts instead of relying on fs files. The problem becomes much more
apparent once you get into remote workspaces, where we trees_equal
doesn't work.](ec88fa8) by efiop
🛠️ dvc: bump to 1.1.6 by efiop
🛠️ gdrive: pop prefix kwarg (#4157) https://travis-ci.com/github/iterative/dvc/jobs/357157580#L8485 by efiop
🛠️ dvc: bump to 1.1.5 by efiop
🛠️ [s3,gs: don't append "/" to prefixed path when listing objects (#4149) * s3,gs: don't append "/" to prefixed path when listing objects
-
remote: use optional prefix kwarg for walk_files
-
if prefix is True, path will be walked as a prefix
-
if prefix is False, path will be treated as a directory (trailing
slash will be appended for S3-like remotes) -
tests: add unit test for BaseRemoteTree.list_paths](fa16b56) by pmrowla
🛠️ tests: remotes: use TmpDir-like fixtures (#4140) by efiop
🛠️ [completion: update dependency (completion: update dependency #4148) * completion: use new simplershtabAPI -
completion: use FILE|DIR](5d2ac9f) by casperdcl
🛠️ azure: use knack directly instead of azure-cli-core (#4147) by efiop
🛠️ dvc: bump to 1.1.4 by efiop
🛠️ dvc: bump to 1.1.3 by efiop
🛠️ [setup: limit flufl version flufl.lock released 4.0 in which there is no finalize method anymore, hence
dvc/lock.py:143:16: E1101: Instance of 'HardlinkLock' has no 'finalize' member (no-member)
```](https://github.com/iterative/dvc/commit/00c135f7d5b55e839354da1e47dc51baa6c64d1a) by [efiop](https://github.com/efiop)
:hammer_and_wrench: [dvc: bump to 1.1.2](https://github.com/iterative/dvc/commit/8cbefaaa88f5a5e3d5e6324600cfd035548242f6) by [efiop](https://github.com/efiop)
:hammer_and_wrench: [azure: use CLI config (#4137) Fixes #4129](https://github.com/iterative/dvc/commit/0275a0d8d876e230e4b287b52cead53b5b492005) by [efiop](https://github.com/efiop)
:hammer_and_wrench: [Reorder package widgets (#4138)](https://github.com/iterative/dvc/commit/b51ee18fc0df839ad8255d54437c0fed80825dd9) by [dmpetrov](https://github.com/dmpetrov)
:hammer_and_wrench: [RepoTree: preserve relative paths in copytree() (#4133)](https://github.com/iterative/dvc/commit/c6a024f83ba2de2d6852222421f2bcad0618bb06) by [pmrowla](https://github.com/pmrowla)
:hammer_and_wrench: [pylint: enable additional checks (#4122) Created a separate pylint plugin to disable some checks for tests.](https://github.com/iterative/dvc/commit/1c0f5cdf4ba6188238c60990e96b94402faaaae0) by [skshetry](https://github.com/skshetry)
:hammer_and_wrench: [metrics: skip missing metrics silently (#4130) Reverts an issue caused by #4005](https://github.com/iterative/dvc/commit/a7e825a0386b6aa08a33eac5262246c86b868588) by [Suor](https://github.com/Suor)
:hammer_and_wrench: [hdfs: implement makedirs separately (#4128) Clouds that have a non-noop `makedirs` should implement it in their own
method. Hdfs somehow slipped under the radar and had `mkdir` used
explicitly in `copy` and `upload` methods.](https://github.com/iterative/dvc/commit/7b97d7249d7db58de93bb3659c9b599396076ca4) by [efiop](https://github.com/efiop)
- - -
# CONTRIBUTORS
Last week there were 6 contributors.
:bust_in_silhouette: [efiop](https://github.com/efiop)
:bust_in_silhouette: [pmrowla](https://github.com/pmrowla)
:bust_in_silhouette: [casperdcl](https://github.com/casperdcl)
:bust_in_silhouette: [dmpetrov](https://github.com/dmpetrov)
:bust_in_silhouette: [skshetry](https://github.com/skshetry)
:bust_in_silhouette: [Suor](https://github.com/Suor)
- - -
# STARGAZERS
Last week there were 45 stagazers.
:star: [LiamLiang](https://github.com/LiamLiang)
:star: [nervegas](https://github.com/nervegas)
:star: [aaravindan101](https://github.com/aaravindan101)
:star: [ffgama](https://github.com/ffgama)
:star: [gurpreet-ai](https://github.com/gurpreet-ai)
:star: [nguyentp](https://github.com/nguyentp)
:star: [mmkassem](https://github.com/mmkassem)
:star: [FaisalSalem](https://github.com/FaisalSalem)
:star: [bitnot](https://github.com/bitnot)
:star: [kemokemo](https://github.com/kemokemo)
:star: [Nokvel](https://github.com/Nokvel)
:star: [0vermind](https://github.com/0vermind)
:star: [maqianlidy](https://github.com/maqianlidy)
:star: [marvinbuss](https://github.com/marvinbuss)
:star: [tsella](https://github.com/tsella)
:star: [stjordanis](https://github.com/stjordanis)
:star: [lordlinus](https://github.com/lordlinus)
:star: [Jinksi](https://github.com/Jinksi)
:star: [fjp](https://github.com/fjp)
:star: [hsinhoyeh](https://github.com/hsinhoyeh)
:star: [zhan-ge](https://github.com/zhan-ge)
:star: [setu4993](https://github.com/setu4993)
:star: [vinogradovkonst](https://github.com/vinogradovkonst)
:star: [joivo](https://github.com/joivo)
:star: [odellus](https://github.com/odellus)
:star: [petoknm](https://github.com/petoknm)
:star: [NaziGanster](https://github.com/NaziGanster)
:star: [PhilipMachineLearning](https://github.com/PhilipMachineLearning)
:star: [rlindskog](https://github.com/rlindskog)
:star: [LumaKernel](https://github.com/LumaKernel)
:star: [Ran-An](https://github.com/Ran-An)
:star: [ojcobb](https://github.com/ojcobb)
:star: [Villux](https://github.com/Villux)
:star: [mysticaltech](https://github.com/mysticaltech)
:star: [dakebl](https://github.com/dakebl)
:star: [faizankshaikh](https://github.com/faizankshaikh)
:star: [justachetan](https://github.com/justachetan)
:star: [wikiped](https://github.com/wikiped)
:star: [shrebox](https://github.com/shrebox)
:star: [narain1](https://github.com/narain1)
:star: [infokiller](https://github.com/infokiller)
:star: [murthy95](https://github.com/murthy95)
:star: [clementabary](https://github.com/clementabary)
:star: [NeerajSarwan](https://github.com/NeerajSarwan)
:star: [kifish](https://github.com/kifish)
You all are the stars! :star2:
- - -
# RELEASES
Last week there were 6 releases.
:rocket: [1.1.7 1.1.7](https://github.com/iterative/dvc/releases/tag/1.1.7)
:rocket: [1.1.6 1.1.6](https://github.com/iterative/dvc/releases/tag/1.1.6)
:rocket: [1.1.5 1.1.5](https://github.com/iterative/dvc/releases/tag/1.1.5)
:rocket: [1.1.4 1.1.4](https://github.com/iterative/dvc/releases/tag/1.1.4)
:rocket: [1.1.3 1.1.3](https://github.com/iterative/dvc/releases/tag/1.1.3)
:rocket: [1.1.2 1.1.2](https://github.com/iterative/dvc/releases/tag/1.1.2)
- - -
That's all for last week, please <kbd>:eyes: **Watch**</kbd> and <kbd>:star: **Star**</kbd> the repository [*iterative/dvc*](https://github.com/iterative/dvc) to receive next weekly updates. :smiley:
*You can also [view all Weekly Digests by clicking here](https://github.com/iterative/dvc/issues?q=is:open+is:issue+label:weekly-digest).*
> Your [**Weekly Digest**](https://github.com/apps/weekly-digest) bot. :calendar: