Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cli integration tests #988

Merged
merged 28 commits into from
Jun 19, 2024
Merged

Cli integration tests #988

merged 28 commits into from
Jun 19, 2024

Conversation

savente93
Copy link
Contributor

@savente93 savente93 commented Jun 7, 2024

Issue addressed

Fixes #943

Explanation

I've decided to leave most of the cli code as is even though we wanted to reduce it, but the code for it works again. I've removed the tests that were purely about region arguments since we've decided to remove those. I haven't actually updated the code there yet, but I'll pick that up in #966 when the time comes, since I think that is closely related.

I also can't quite remember why I thought that I'd needed a from_dict or from_yml function for this one, but given that I've written it and we plan to add it later I think we might as well just add it. I'll do more testing and integration in the ticket mentioned above.

General Checklist

  • Updated tests or added new tests
  • Branch is up to date with main
  • Tests & pre-commit hooks pass
  • Updated documentation
  • Updated changelog.rst

@savente93 savente93 marked this pull request as ready for review June 7, 2024 13:06
@savente93 savente93 requested a review from Jaapel June 7, 2024 13:06
Copy link
Contributor

@Jaapel Jaapel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, in the current component architecture, you would now not define your region using the -r argument, but you would add come RegionComponent to your model configuration, correct?

"-s",
"hydro_lakes",
"-r",
"{'subbasin': [-7.24, 62.09], 'uparea': 50}",
Copy link
Contributor

@Jaapel Jaapel Jun 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would we export for a certain region? hydromt export should still be tested right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason (as far as I understand) that the region got removed from the cli, is that the way we construct regions has become much more complicated and interwoven with the components. This means that regions can no longer be specified independent of what type of components you have. The export can be used without a region and then all the data will be exported. aside from that we also have a file format for specifying what data needs to get exported. I'm not sure if this is updated yet, but regions will have to be specified there. There should be at least one test that uses the file format.

@savente93
Copy link
Contributor Author

Most of the sonar clouds errors are introduced by touching code that isn't actually in the scope of this PR (i.e. refactoring export_data) so I think it's fine to leave it as is.

@savente93 savente93 requested a review from Jaapel June 14, 2024 12:23
@@ -1006,10 +1006,11 @@ def export_data(
self,
data_root: Union[Path, str],
bbox: Optional[Bbox] = None,
time_tuple: Optional[TimeRange] = None,
time_range: Optional[TimeRange] = None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like you partially updated the export_data function to v1, but not fully?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's true, these are some left over changes from before I decided to leave the export_data function alone, I can remove them if you like

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need

"--opt",
"setup_grid.res=0.05",
"-vv",
]
_ = CliRunner().invoke(hydromt_cli, cmd)


@pytest.mark.skip(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are the override tests skipped?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd have to write another yaml file for that since we don't have a way to specify model components from the CLI. I can do that if you like

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the same --fo flag exist now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

@Jaapel
Copy link
Contributor

Jaapel commented Jun 14, 2024

Also SonarCloud makes a good point, would correct that one.

@savente93
Copy link
Contributor Author

You mean export_data?

@savente93
Copy link
Contributor Author

because that would mean basically fixing #886 as well in this pr

@Jaapel
Copy link
Contributor

Jaapel commented Jun 14, 2024

because that would mean basically fixing #886 as well in this pr

Yeah no need, just making sure I understand the changes

@savente93
Copy link
Contributor Author

There are still one or two issues that I can't figure out yet, I'll come back to those in an hour or so, I think I need to look at something else for a bit to recover my productivity

@savente93
Copy link
Contributor Author

Override test now works again, with one small addition. I couldn't figure out why the convention resolver couldn't find the vito file for the re class in the example, so I just edited that bit out. I'll have to add that bit back in at some point, but I don't think it's necessary for this PR, and belongs more in a PR to update the examples.

@savente93 savente93 requested a review from Jaapel June 14, 2024 14:24
@Jaapel
Copy link
Contributor

Jaapel commented Jun 17, 2024

In ModelRoot and in multiple functions in cli.main a FileHandler for logging is created, pointing to the same file. I took a few stabs at it, but I think solving it would require redoing logging in HydroMT, just relying on the logging hierarchy in the builtin logging module, instead of passing loggers around manually.

@savente93 savente93 merged commit 12cf0e6 into v1 Jun 19, 2024
8 checks passed
@savente93 savente93 deleted the cli-integration-tests branch June 19, 2024 10:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants