Repository navigation
v1.5.0 dev update #1728
Closed
orneryd
announced in
Announcements
Replies: 3 comments
|
the "12 agents" is per input signal cycle- that means they have a total of ~10ms in between reads before the next one starts, LLMs are slow so this is actually more like 1200 agents at once at full steam ahead. - hence the need for extremely lightweight ingestion, traversal, and retrieval. |
0 replies
|
also - It will be v1.5.0 there - is a storage migration because of a couple go-json bugs i found which after verification of the release i'll probably open an issue or PR for |
0 replies
|
released |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
First off: Building a reliable graph database is not an easy task even if it is a "solved" problem. Most of these concepts have been debated, implemented, and discussed ad nauseam across reddit, youtube, and stack overflow for decades before that. Yet, entire teams of people are generally dedicated to build these, so please be patient- I'd rather it be done right than fast.
for a look into what I am doing - I blogged about the process here:
https://dev.to/orneryd/building-nornicdb-with-nornicdb-code-graphs-and-swarms-4nmg
A year ago I had no Idea i'd be writing something this competitive with the current market. after writing every type of software I can think of at various points in my career, i just wanted to prove out a few theories and see what I could pull off with AI and not have to use slower, existing options.
the use case i'm targeting requires executing a whole graph update and output loop at ~ 100hz using a series of 12 agents (multipled by processing delay in the LLMs) processing input signals in parallel - None of the existing options were capable and if anyone who remembers the ui-grid impetus: I am so annoyed by paywalls that purely exist because of self-aggrandizing OSS teams that I will spend the time to give away un-hobbled freeware under the MIT license when no alternative exists - Graph databases seem to be one of those spaces where nobody is doing it "right" and everyone is clamoring for their IP under restrictive licensing that causes people like me to exist.
Anyways, last month when we started experimenting with a swarm and we started poking edge cases, we found some things That needed fixed and the speed AI is capable of finding issues is incredibly fast, so fast that it was overwhelming. So, I implemented certain safeguards to make sure Issues and PRs stay clear of AI-gibberish while the swarms are working on experiments, testing, and debugging issues - we moved that to Linear. I want issues to stay clear for user reports and we will create issues from our linear ticket work as we find issues but we don't want an unfiltered feed from AI overwhelming anyone coming to look for help either.
I have some work to do to get that properly set up and synching any issues we find internally to the issues board. However, those need triaged because AI will like to say there is an issue or an investigation that needs done, and log duplicate after duplicate of issues and not correlate them with their proper families automatically so that requires some handholding.
On top of that, the issues that were found are generally around cancellation of transactions during shutdown and other edge cases regarding metadata. most of that gets regenerated at startup and we have many many tests around crash recovery handling prior to this, but now we have even more for more extreme cases. - This was a failing in my end to account for that in the initial design.
that coupled with the duplication I already knew about that existed - It was time to refactor.
I'm excited to drop the next version. it's just taking a little longer than I want but I promise it will be worth it.
as the old adage goes "It takes 1 person 9 months to make a baby. 9 can't do it in 1"
All reactions