Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added missing docstrings to test_configs file #3194 #3205

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from

Conversation

JinHuang0101
Copy link

Start with a description of this PR. Then edit the list below to the items that make sense for your PR scope, and check off the boxes as you go!
Added missing docstrings to test_configs file #3194

Contributor Checklist

  • I have broken down my PR scope into the following TODO tasks
    • task 1
    • task 2
  • I have run the tests locally and they passed. (refer to testing section in contributing)
  • I have added tests, or extended existing tests, to cover any new features or bugs fixed in this PR

optionals

  • I have added entry in the CHANGELOG.md
  • If this PR needs a follow-up in dash docs, community thread, I have mentioned the relevant URLS as follows
    • this GitHub #PR number updates the dash docs
    • here is the show and tell thread in Plotly Dash community

@gvwilson gvwilson added documentation written for humans feature something new P2 considered for next cycle community community contribution labels Mar 10, 2025
@gvwilson
Copy link
Contributor

Thanks @JinHuang0101 - I'll see if we can get it reviewed and into the next release.

@gvwilson
Copy link
Contributor

@JinHuang0101 can you please tell us how much of these docstrings was generated by AI? We are trying to figure out what role AI is going to play in our dev process going forward, and it would help to know. thanks - @gvwilson

@JinHuang0101
Copy link
Author

JinHuang0101 commented Mar 11, 2025

@JinHuang0101 can you please tell us how much of these docstrings was generated by AI? We are trying to figure out what role AI is going to play in our dev process going forward, and it would help to know. thanks - @gvwilson

Hi @gvwilson,

Yeah, I should've made a note about this in the PR. Here is my rationale and workflow:

Overall Rationale:
I've been exploring some advanced features of Grok, ChatGPT, and Copilot recently and experimenting with ways to use them in coding. As a new contributor to open-source, I certainly wouldn’t want to risk breaking existing code by using LLMs to modify it directly. However, I noticed that many open-source projects lack basic documentation, which can be inconvenient—especially for new contributors like me—since it takes considerable time to fully comprehend a relatively large codebase.

At the same time, manually writing all the missing docstrings and inline comments can be time-consuming and labor-intensive for both original contributors and newcomers. I thought that generating technical documentation like this could be an ideal task for LLMs, as long as they accurately understand the code and produce clear explanations. If done well, this could save open-source contributors a lot of time and effort, allowing them to focus more on coding.

My Workflow:

  1. I read the test_configs file and made sure I understood the file and each function to the best of my ability.
  2. I wrote a set of prompts to instruct the LLM to generate docstrings for the file according to my specifications.
  3. I proofread each generated docstring to ensure it aligned with my understanding.
  4. Based on the initial output, I repeated step 2-3 about 2 to 3 times, asking the LLM to revise the docstrings.
  5. Since this was an experiment, I repeated the same process (step 2-4) using Grok, ChatGPT, and Copilot for cross-referencing.

The docstrings generated by these models were largely in alignment with my understanding (about 95%). Since docstrings are a form of technical documentation, I didn’t significantly alter the machine-like tone. However, I acknowledge that I might have misunderstood some functions, which could lead to inaccuracies in the generated docstrings. If that’s the case, it’s likely because I didn’t fully grasp the function in the first place—hence, a more experienced contributor might need to approve or request changes.

Thoughts:
If this approach is unacceptable, I apologize, and I’m happy to rewrite the docstrings manually.

However, if you're interested in exploring how AI can handle relatively simple tasks like these in coding, I believe this could be a valuable way to save programmers' time while also improving the maintainability of open-source projects. I thought it was worth experimenting with, given that it doesn’t pose much direct risk to the program’s functionality.

Let me know what you think, and thank you in advance for the conversation!
Jin

@gvwilson
Copy link
Contributor

Thanks for the detailed explanation @JinHuang0101 - I think this is a really interesting use case, and we (our team, our company, and our whole profession) need to think pretty hard about when, where, and how to use these tools. Looking at this particular change, I don't think the AI-generated docstrings add very much information that isn't in the name of the test functions, but that doesn't mean they're not useful. We're trying to get our next Dash release out the door right now - I will try to prioritize review of this after that's finished. Thanks for your patience - @gvwilson cc @ndrezn

@JinHuang0101
Copy link
Author

Thanks for the detailed explanation @JinHuang0101 - I think this is a really interesting use case, and we (our team, our company, and our whole profession) need to think pretty hard about when, where, and how to use these tools. Looking at this particular change, I don't think the AI-generated docstrings add very much information that isn't in the name of the test functions, but that doesn't mean they're not useful. We're trying to get our next Dash release out the door right now - I will try to prioritize review of this after that's finished. Thanks for your patience - @gvwilson cc @ndrezn

Hi @gvwilson, thank you so much for the detailed feedback! I modified my prompts asking the AI models to improve the previous docstrings with more details and explanations. The docstrings are now revamped and updated. I reviewed the improvements and amended the code in my existing PR. I think they are richer, more explanatory now, adding more value to the code. Let me know what you think! Thanks!

@gvwilson gvwilson assigned gvwilson and unassigned T4rk1n Mar 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community community contribution documentation written for humans feature something new P2 considered for next cycle
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants