Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix invalid data loading code in example #949

Merged
merged 2 commits into from
Jan 25, 2021

Conversation

rinx
Copy link
Contributor

@rinx rinx commented Jan 25, 2021

Description:

I fixed the invalid data loading code in example/main.go.
I've already mentioned about it a month ago, but it's not fixed yet.
https://github.com/vdaas/vald/projects/4#card-51586276

HDF5 formatted data of fashion-mnist has its data as float32 values. So it should be loaded as float32.

Related Issue:

nothing

How Has This Been Tested?:

nothing

Environment:

  • Go Version: 1.15.7
  • Docker Version: 19.03.8
  • Kubernetes Version: 1.18.2
  • NGT Version: 1.13.1

Types of changes:

  • Bug fix [type/bug]
  • New feature [type/feature]
  • Add tests [type/test]
  • Security related changes [type/security]
  • Add documents [type/documentation]
  • Refactoring [type/refactoring]
  • Update dependencies [type/dependency]
  • Update benchmarks and performances [type/bench]
  • Update CI [type/ci]

Changes to Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

Checklist:

  • I have read the CONTRIBUTING document.
  • I have checked open Pull Requests for the similar feature or fixes?
  • I have added tests and benchmarks to cover my changes.
  • I have ensured all new and existing tests passed.
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated the documentation accordingly.

@vdaas-ci
Copy link
Collaborator

[CHATOPS:HELP] ChatOps commands.

  • 🙆‍♀️ /approve - approve
  • 💌 /changelog - add changelog comment
  • 🍱 /format - format codes and add licenses
  • /gen-test - generate test codes
  • 🏷️ /label - add labels
  • /rebase - rebase master
  • 🔚 2️⃣ 🔚 /label actions/e2e-deploy - run E2E deploy & integration test

@github-actions github-actions bot added size/S type/bug Something isn't working priority/medium labels Jan 25, 2021
Copy link
Collaborator

@kpango kpango left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this code depend on the data set?
I remember that it was the SET team that wrote the example, but I think SET was verifying the behavior at the time.
@vankichi can you explain the situation a bit?

@kpango
Copy link
Collaborator

kpango commented Jan 25, 2021

On a completely unrelated note, I really like this code in the hdf5 library
https://github.com/gonum/hdf5/blob/master/h5d_dataset.go#L69-L99

@rinx
Copy link
Contributor Author

rinx commented Jan 25, 2021

Yes it does. For fashion mnist, float32 is correct.

@rinx
Copy link
Contributor Author

rinx commented Jan 25, 2021

I think SET team didn't check whether the loaded value is correct or not. Maybe they checked just only whether the example code works or not.

@kpango
Copy link
Collaborator

kpango commented Jan 25, 2021

hmm, I see but this example code can be overwrite dataset path variable by cli argument so could you please add about comment for the users and also could you please add comment to the docs?

@kevindiu
Copy link
Contributor

BTW how can we verify if the loaded data is correct or not?

@rinx
Copy link
Contributor Author

rinx commented Jan 25, 2021

Sure. I'll revise it. @kpango

Using HDFView (https://www.hdfgroup.org/downloads/hdfview/), you can see the correct values of the data. @kevindiu

@rinx rinx force-pushed the bugfix/examples/fix-invalid-data-loading-code branch from 957c0db to 068fb26 Compare January 25, 2021 02:41
@rinx
Copy link
Contributor Author

rinx commented Jan 25, 2021

I've added comments about the HDF5 datatype on the examples codes.

Since there's no section about the using dataset in the Getting Started, I didn't edit it.

Copy link
Collaborator

@kpango kpango left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kpango kpango merged commit 153f8d0 into master Jan 25, 2021
@kpango kpango deleted the bugfix/examples/fix-invalid-data-loading-code branch January 25, 2021 02:52
@vdaas-ci vdaas-ci mentioned this pull request Jan 29, 2021
18 tasks
@kpango kpango mentioned this pull request Feb 2, 2021
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants