-
Notifications
You must be signed in to change notification settings - Fork 418
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I need my hands on this code ASAP. :-) #2
Comments
@aifartist I think you would like this then: https://github.com/Josh-XT/AGiXT/tree/main |
hehe |
I want to see the code too, actually I read all the paper and I was trying to figure out how you developed the experiments. Please help us with a little of code |
If they help us with a little bit of code I'll return the favor! :-) |
give it a try to this one https://github.com/fblgit/tree-of-knowledge-llm |
It doesn't appear to be runable code I can experiment with. It is one thing to read a paper and another to try examples to make the concepts clear. However, in looking a bit at what you put there I am getting a better idea of what this is doing. I believe that to much focus on LLM is being directed at "specifics" and not enough on the core foundation ontological concepts of different forms of communication, answering questions, cause/effect, iterating towards a solution. Do not start training about certain computer languages, deep knowledge of physics, deep knowledge of hair follicle diseases in felines, plumbing, knitting, etc. until you got the general "thinking" mechanisms rock solid. |
In my personal experience researching learning and knowledge from a human perspective, learning about more specific areas of knowledge can significantly improve the ability to solidify foundational core concepts. Often, first-hand experience reasoning about nuanced specific topics can inform on the importance of the foundation by flexing those muscles. For example, mastering mathematics alone is much more tedious and inefficient than learning some basic ideas in mathematics and then switching to studying various aspects of physics that require mathematics. This approach often leads down paths where more advanced mathematics are required thus providing the motivation, context, and 'real world examples' of some more advanced math concepts. With this approach, hopping from foundational subjects to advanced nuanced topics and then back provides for a more dynamic and well-rounded method for learning that allows neural pathways to be forged in such a way that integrating various subjects becomes almost trivial. Unexpected and novel insights and analogies are more likely to occur with this method of learning from my subjective opinion when I reflect back on my first-hand experience following a methodical approach to mastering various multiplayer video games using an analogous approach to the content of this repo to train humans. I've had similar success with this method training LLM with very limited resources. |
You mentioned you were reading on 'Reflexion', could you share that resource material? |
you code |
I don't disagree with what you are saying. I'm just saying training on everything including the kitchen sink, with back issues of Cat Fancier magazine, Knitting Quarterly, PVC Pipe Fitting catalogs, 10000 species of birds, and on and on an on isn't what I'd do. Ontology, philosophy, core math like axiomatic set theory, Peano's axioms, logic, etc. in FULL DEPTH. Shallow but broad understanding of general knowledge. Don't assume this is an exhaustive list. Perhaps as important is understanding the nature of communication. Answering a question, making follow up inquiries when there is some ambiguity in what the agent thinks was being asked, judging the knowledge of the other person to customize your responses(something I do when explaining high technical subjects to the common man). I might even consider these to be proper foundation ontological concepts in themselves. At some point, when I'm satisfied, start considering what to do next. The entirety of human knowledge in as much depth as we can find text/images for or pick some high value domains like physics/math, programming/algorithms, etc. and provide a library of very solid models that can help in those domains even better than ChatGPT-4 and maybe even be runnable on a 4090 on a home PC. But this might not be the correct second step... A flat next word predictor based system has its limitations. I want a model that has ?feedback?, LTSM, RNN, like things in it. There are a number of things coming out that seem to be layered onto an LLM instead of being baked into it. Like this tree thing. Or call it "problem solving" that relies less on flat word patterns learned from vast corpora of text. A crazy idea? As a 40+ year software architect I was too busy solving problems in SQL databases and high performance systems to be able to study AI. Just after I retired everything exploded and I'm trying to catch up. Do I have anything to offer in terms of unique insights? Multiple times my various bosses have asked me if I could train other engineers to do want I could do. I have often reflected and pondered about the nature of "though" and "intuition" and why was I able to do what I did. I sat on the Autonomic Computing board at IBM back when flawed rule based methods were used. It sounds silly but I've used myself as a subject to study. I'd might be thinking about something and wondered why I was thinking about it. Several times over the years I was able to "rewind" my chain of thoughts to discover what triggered the transition to the new subject of interest. The short version: I've been thinking about thinking for a long time. Will "tree" be the key breakthrough. Could I come up with something as I get deeper and deeper into this subject. Only time will tell. |
First Google hit for a simple query: |
First hit with Google: |
Here's my implementation so you can plug in and play: |
Wow @kyegomez, thanks for sharing an implementation on cutting edge research like this! Amazing work, can’t wait to try it out |
thanks for waiting, just released the code |
Hmmm, it looks like I'd have to buy an openai ?api? account to use this? |
@ysymyth Sorry. I should have tried first. After asking to get a key at openai I tried your code. At first I thought it was hung but when I instrumented the chatgpt() loop I saw messages and replies. Also, this might be the place I might be able to connect it to my local LLaMA wizard ++ model. |
I'm a hard core performance architect that retired last year from MSFT now doing AI for fun.
Stable Diffusion, TensorRT, torch.compile(), and coding in Python. I have a 4090 with an i9-13900K. If I need more memory to run a 65B model I'll buy it. Today I got manticore-13b running on my system. To shoot for the moon I want to get the best LLM that can fit on a high end home PC and layer things like this onto it. I was just reading up on Reflexion when I saw this. I wonder if its dynamic memory and self-reflection focus duplicates what tree-of-thought does or could augment it. I can code/debug anything.
Since this github has no discussion forum I'll abuse the "Issues" thing to get my foot in the door so I'll get a notification when the code drops.
The text was updated successfully, but these errors were encountered: