Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Further changes to the new shell package #173

Merged
merged 3 commits into from
Jul 5, 2019
Merged

Further changes to the new shell package #173

merged 3 commits into from
Jul 5, 2019

Conversation

aalexandrov
Copy link

  • Use entry_points-based name -> type resolution of the forecaster type in the CLI.
  • Mark the module as .typesafe.

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@aalexandrov aalexandrov requested a review from jaheba July 4, 2019 14:19
@szha
Copy link
Member

szha commented Jul 4, 2019

Job PR-173/1 is complete.
Docs are uploaded to http://gluon-ts-staging.s3-accelerate.dualstack.amazonaws.com/PR-173/1/index.html

@codecov-io
Copy link

codecov-io commented Jul 4, 2019 •
edited

Codecov Report

Merging #173 into master will decrease coverage by 3.91%.
The diff coverage is 6.47%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #173      +/-   ##
==========================================
- Coverage   78.53%   74.61%   -3.92%     
==========================================
  Files         124      123       -1     
  Lines        6783     6800      +17     
==========================================
- Hits         5327     5074     -253     
- Misses       1456     1726     +270
Impacted Files Coverage Δ
src/gluonts/evaluation/backtest.py 87.5% <ø> (ø) ⬆️
src/gluonts/shell/serve/__init__.py 0% <0%> (-63.64%) ⬇️
src/gluonts/shell/sagemaker/path.py 0% <0%> (ø)
src/gluonts/shell/__main__.py 0% <0%> (-41.67%) ⬇️
src/gluonts/shell/sagemaker/params.py 0% <0%> (ø)
src/gluonts/shell/train.py 0% <0%> (-31.38%) ⬇️
src/gluonts/shell/sagemaker/__init__.py 0% <0%> (ø)
src/gluonts/shell/serve/app.py 0% <0%> (-39.14%) ⬇️
src/gluonts/dataset/common.py 81.21% <100%> (+0.19%) ⬆️
src/gluonts/core/exception.py 62.5% <100%> (+1.63%) ⬆️
... and 18 more

@aalexandrov aalexandrov mentioned this pull request Jul 4, 2019
Closed
@lostella lostella mentioned this pull request Jul 4, 2019
Merged
Follow-up fixes of awslabs#172
20c4986 
- Allow backwards-compatible metadata.json files.
- Rename `time_freq` -> `freq`.
@szha
Copy link
Member

szha commented Jul 4, 2019

Job PR-173/3 is complete.
Docs are uploaded to http://gluon-ts-staging.s3-accelerate.dualstack.amazonaws.com/PR-173/3/index.html

lostella
lostella previously approved these changes Jul 4, 2019
View reviewed changes
lostella
lostella reviewed Jul 4, 2019
View reviewed changes
src/gluonts/shell/sagemaker/__init__.py Outdated
def _get_datasets(self):
freq = self.hyperparameters["time_freq"]
def _get_datasets(self) -> Dict[str, FileDataset]:
freq = self.hyperparameters["freq"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this won't break anything?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I cleaned up all SageMaker-related configuration and environment bootstrapping logic (now all in sagemaker/__init__.py) and updated the PR.

The following method now ensures backwards-compatibility w.r.t. the hyperparameters.json file. Backwards-compatibility in the metadata.json file was already added in the current version.

def _load_hyperparameters(path: MLPath, channels) -> dict:
    with path.hyperparameters.open() as json_file:
        hyperparameters = parse_sagemaker_parameters(json.load(json_file))

        for old_freq_name in ['time_freq', 'time_granularity']:
            if old_freq_name in hyperparameters:
                hyperparameters['freq'] = hyperparameters[old_freq_name]

        if "metadata" in channels:
            with (channels["metadata"] / "metadata.json").open() as file:
                metadata = MetaData(**json.load(file))
                hyperparameters.update(freq=metadata.freq)

        return hyperparameters

Further changes to the new shell package
24c52b9 
- Use entry_points-based name -> type resolution of the
  forecaster type in the CLI.
- Mark the module as `.typesafe`.
@szha
Copy link
Member

szha commented Jul 4, 2019

Job PR-173/4 is complete.
Docs are uploaded to http://gluon-ts-staging.s3-accelerate.dualstack.amazonaws.com/PR-173/4/index.html

@szha
Copy link
Member

szha commented Jul 4, 2019

Job PR-173/5 is complete.
Docs are uploaded to http://gluon-ts-staging.s3-accelerate.dualstack.amazonaws.com/PR-173/5/index.html

@szha
Copy link
Member

szha commented Jul 4, 2019

Job PR-173/6 is complete.
Docs are uploaded to http://gluon-ts-staging.s3-accelerate.dualstack.amazonaws.com/PR-173/6/index.html

@lostella
Copy link
Contributor

lostella commented Jul 5, 2019

It would be nice to add tests for the gluonts.shell package, but this can be done in a separate PR: I'm opening an issue to track this

@aalexandrov aalexandrov merged commit 637a3fe into awslabs:master Jul 5, 2019
@aalexandrov aalexandrov deleted the shell-refactoring branch July 5, 2019 08:51
@lostella lostella mentioned this pull request Jul 7, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lostella lostella lostella approved these changes

@jaheba jaheba Awaiting requested review from jaheba

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@aalexandrov @szha @codecov-io @lostella