-
Notifications
You must be signed in to change notification settings - Fork 19.7k
Add note about label noise in CIFAR-10 dataset documentation #21855
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add note about label noise in CIFAR-10 dataset documentation #21855
Conversation
|
This is a simple documentation improvement that should help avoid confusion for users who encounter mislabeled images in CIFAR-10. The note is factual, concise, and doesn't require any code changes - just making the existing dataset behavior more transparent in the docs. |
Summary of ChangesHello @ssam18, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request enhances the CIFAR-10 dataset documentation by incorporating a crucial note regarding label noise. This addition aims to proactively inform users about the inherent presence of mislabeled samples within the dataset, thereby setting appropriate expectations and preventing potential confusion or misdiagnosis of issues during model development and evaluation. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request adds a helpful note to the cifar10.load_data() docstring, warning users about the known label noise in the CIFAR-10 dataset. This is a valuable addition that can save users from confusion and debugging time. My review includes one suggestion to adjust the placement of this note within the docstring to align with the Keras API style guide, which will improve documentation consistency.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #21855 +/- ##
==========================================
- Coverage 82.47% 78.08% -4.40%
==========================================
Files 577 577
Lines 59508 59508
Branches 9332 9332
==========================================
- Hits 49080 46464 -2616
- Misses 8015 10747 +2732
+ Partials 2413 2297 -116
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
The CIFAR-10 dataset is known to contain a small percentage of mislabeled samples, which can affect model training and evaluation. This note helps users understand that some label inconsistencies are expected and inherent to the original dataset. Fixes keras-team#21631 Signed-off-by: Samaresh Kumar Singh <ssam3003@gmail.com>
b17cabc to
461a33b
Compare
fchollet
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, sure seems worth including, thanks for the PR
I was working with the CIFAR-10 dataset and noticed that the documentation doesn't mention the known issue of label noise. I kept running into images that seemed mislabeled - airplanes labeled as frogs, horses as cats, etc. At first I thought something was wrong with my code, but after researching, I learned this is a well-known characteristic of the original CIFAR-10 dataset.
I've added a brief note to the
load_data()docstring to inform users about this label noise issue. This should help new users understand that some label inconsistencies are expected and not due to bugs in their code or the Keras implementation.The note is concise and placed right after the dataset description, so users will see it before they start working with the data. It mentions that:
Fixes #21631