Unknown Decrease in Code Snippet Extraction

[InfernalDread]

Hello again,

After trying your most recent release (v0.2.9), I reattempted a GitHub repo submission, and my snippet count went down from 955 snippets to 430 snippets (the documentation count was the same, being 153). I am not sure what is the cause, just wanted to let you know.

Thank you for your time and hard work!

EDIT:

I also tried v0.2.8, same issue, went down from 955 code snippets to 430. Documentation count was 153.

Now I tried v0.2.6 and v0.2.7, and added the new "rst" related files, and the snippets went back up to 946 (pretty much back to normal), but the documentation count went down from 153 to 125. Not sure what changed honestly.

[chriswritescode-dev]

Yes I was reviewing some of the extraction methods and after adding the other methods. I decided to do a complete rewrite of all of the extraction methods and create a single class to make sure all methods follow the same rules. should have a new update later today. The idea was to also make the non llm descriptions just as good while resolving the missed codeblocks

[InfernalDread]

ohh ok, sounds efficient! Can't wait to try it out! I will continue to test your changes release and let you know if there are issues or bugs to help (even if only a little) the debugging process!

[InfernalDread]

I know that you probably have some more stuff you plan on doing before releasing a new version, but just wanted to update you on the latest changes to the master branch:

The document count of my original GitHub scrape stayed the same at 153

The snippet count actually went even lower, down to 358

[chriswritescode-dev]

I know that you probably have some more stuff you plan on doing before releasing a new version, but just wanted to update you on the latest changes to the master branch:

The document count of my original GitHub scrape stayed the same at 153

The snippet count actually went even lower, down to 358

What source is this and what type? web upload repo? I decided that single inline code blocks are mostly unneeded. So a decrease is not unexpected. We were recording too many snippets that are mostly not complete snippets examples used in the description above a main snippet. These are not useful alone and do not provide the context needed they are actually now being used in the context for the larger multiple line snippets. Did u examine any other sources ?