Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add API to query intermediate results in NAS benchmark #2728

Merged
merged 2 commits into from Jul 31, 2020
Merged

Add API to query intermediate results in NAS benchmark #2728

merged 2 commits into from Jul 31, 2020

Conversation

ultmaster
Copy link
Contributor

@ultmaster ultmaster marked this pull request as draft July 24, 2020 06:05
@scarlett2018 scarlett2018 mentioned this pull request Jul 24, 2020
Closed
66 tasks
@ultmaster ultmaster marked this pull request as ready for review July 26, 2020 10:41
@ultmaster ultmaster requested a review from JunweiSUN July 26, 2020 10:41
@JunweiSUN
Copy link
Contributor

What is the reason that caused #2721 ? It seems that if user set include_intermediates=False, this issue still exists.

@ultmaster ultmaster added the NAS label Jul 29, 2020
QuanluZhang
QuanluZhang reviewed Jul 31, 2020
View reviewed changes
src/sdk/pynni/nni/nas/benchmarks/nds/model.py
@@ -52,8 +53,8 @@ class NdsTrialConfig(Model):
'residual_basic',
'vanilla',
])
model_spec = JSONField(index=True)
cell_spec = JSONField(index=True, null=True)
model_spec = JSONField(json_dumps=json_dumps, index=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this modification can resolve the dict element sequence issue?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comparison of JSON field is based on string comparison, and to compare two dumped JSON of unordered dict, they need to be sorted ahead.

@ultmaster ultmaster merged commit 717877d into microsoft:master Jul 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@QuanluZhang QuanluZhang QuanluZhang approved these changes

@JunweiSUN JunweiSUN JunweiSUN approved these changes

Assignees
No one assigned
Labels
NAS
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

nasbench201 api bug
3 participants
@ultmaster @JunweiSUN @QuanluZhang