Create a search based document for write-the

[James4Ever0]

This document is generated using: https://github.com/james4ever0/prometheous

You can view the website at: https://james4ever0.github.io/write-the/

[Wytamma]

Hi @James4Ever0, thanks for the PR! Can you explain what it is? 😅 it looks like you can search the code but that's already available through the mkdocs website e.g.

[James4Ever0]

Hi @James4Ever0, thanks for the PR! Can you explain what it is? 😅 it looks like you can search the code but that's already available through the mkdocs website.

Here is a brief intro about this tool. The main development is at prometheous.

I want to explain that this tool is language agnostic. It does not analyze the code AST nor split it by functions. Instead it splits the code into chunks. I have already finished the chunk comment, summation and search functionality.

Your AST part is good, but it shall be optional (for files like txt, toml), with fallback (grammar errors, new syntax). It shall be built by chunk processing as well (say if you have a large function, you need to split it into chunks. Or you have some code outside of functions you should process as well, like code under if __name__ == "__main__":).

I am currently working on file hierarchy functionality so people can see summaries of folders and files in a clear structure.

The best part is it works for anything. All you need to do is to create .fdignore files to mark out files you do not want to document like LICENSE, PNG files etc. Even better this tool will help you identify some bad practice like data in code (would like to identify code duplication though) and help you refactor your code. You can visualize the ignored files by using vignore. You would see progress while running this tool, see what is going on with LLM and it is pretty fast.

I have used your tool and created this tool. I did use some code from you and improved it (visualize progress token by token, allowing special token within query). The reason behind is that when processing a function or class with hundreds of lines your tool would get stuck, and I did not see progress behind the scene. Later I recognized that your tool is not language agnostic nor providing fine-grained control over files to document.

Additionally, it has a file path and hash based cache system so rebuilding documentation will only happen around changed files and folders. If only it can just process the changed part instead of the whole file again!

The ideas behind are generative search, active search and hierarchical search, maybe selective search as well. All for better information retrieval.

[Wytamma]

Sounds like a great project @James4Ever0! I’m glad you found the write-the code useful. The file hierarchy feature seems really cool 😎

I will create issues for the bugs that you’ve listed and hopefully can get them resolved soon.

Do you have a way to integrate your documentation index with the current mkdocs website?

[Wytamma]

Looks like you're still adding updates to this PR so I'm going to convert it to a WIP.

[James4Ever0]

Do you have a way to integrate your documentation index with the current mkdocs website?

I was adjusting my webpage layout. Using existing documentation framework would be nice.

[Wytamma]

Let me know if you figure out the mkdocs integration!