Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix README documentation. Remove unnecessary LIBSVM check in CSV inpu… #33

Merged
merged 1 commit into from
Sep 16, 2019

Conversation

ericangelokim
Copy link
Contributor

…t validation.

Description of changes:

  • Fix README
  • Remove unnecessary LIBSVM validation in CSV validation.
    • Originally, the validation was added in case customer data was formatted with a unorthodox delim such as ' 2:', causing the data to look like LIBSVM: 1 2:1 2:3 2:1 -> 1,2,3,1. However, retrospectively this should not have been added; if the customer wants to use CSV format with such delim, that should be allowed.

Test

Ran all unit tests successfully using python3 -m tox.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Copy link
Contributor

@aws-patlin aws-patlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you have a chance to see the customer data and confirm this was the root cause?

@ericangelokim
Copy link
Contributor Author

Sent customer the image built with this change, and asked her to re-run training job to verify the wrong logs do not show up.

@ericangelokim
Copy link
Contributor Author

ericangelokim commented Sep 13, 2019

I found a previous training job, whose logs displayed the LIBSVM error log:

ERROR 198.79280739592556,-0.814965842967067,0.11862443460345609,1.441393907312773,-0.3574215393650993,0.260460926398274,-1.5611072676604718,0.19408432846292067,1.8245437744650783,-0.02742789379747009,2.0488629834975347,-1.5521172040593016,1.7112930805229332,1.2401094370037329,0.485322995515301,0.9192109852259204
does not follow LIBSVM label format <label>(:<weight>).

I reran with a custom built image and this no longer shows in logs.

@ericangelokim
Copy link
Contributor Author

Validated with customer that this fixed the error log.

@ericangelokim ericangelokim merged commit 4dcfec9 into master Sep 16, 2019
@mabunday mabunday deleted the csv_validation_20190912 branch August 5, 2022 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants