Skip to content

Conversation

Suor
Copy link
Contributor

@Suor Suor commented Mar 17, 2020

Will skip modified graph checks on add, run and import. The aim is to
avoid collecting stages and building graph on these commands.

The skip_checks flag is aimed to be private, but let people use dvc on
big repos without monkey patching or writing yaml files themselves.

Closes #2671. For the time being.

@Suor Suor requested a review from efiop March 17, 2020 06:41
Will skip modified graph checks on add, run and import. The aim is to
avoid collecting stages and building graph on these commands.

The skip_checks flag is aimed to be private, but let people use dvc on
big repos without monkey patching or writing yaml files themselves.
@Suor Suor requested review from efiop and skshetry March 23, 2020 05:24
@Suor
Copy link
Contributor Author

Suor commented Mar 23, 2020

Should be good to go.

Comment on lines 91 to 94
mocker.patch(
"dvc.repo.Repo._collect_graph",
side_effect=Exception("Should not collect"),
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to just assert that this wasn't called rather than putting this exception here, as we might have some logic catching this in our code (yes, we don't do that now and catching Exception is a bad thing, but still), which would result in a faulty test. Plus it would be more explicit to use assert not m.called rather than this unnecessary hack.

)
dvc._skip_graph_checks = True

tmp_dir.gen("foo", "foo text")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also you have dvc_gen above. Not sure if it is intended.

@efiop efiop merged commit a4094f0 into iterative:master Mar 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optimize stage collection for large repos

3 participants