feat: permanent direction notes/persistent directions - #104
Conversation
|
Solves #4, pretty much, by the way. |
|
Been doing some testing on deepseek v4 flash, pro, and gemma 31b. I found that the seeded fragment does the exact same thing as plot_summary fragment - which was recently removed from the seeded values - because the writer can just use the chat context, no need for a duplicate, unnecessary summarization. And every model insisted on updating it every turn, they're using this very poorly. 
As you can see, these are just summarizations of the messages. A bug I found:
Improvement:
But these findings are not relevant yet. The question is still is this a good thing to add? My point is current models are not reliable enough for memory management tasks, they don't know what's important, they lack the ability to know what's no longer relevant and the ability to plan far into the future for this to work. Just to be clear I'm not just talking about the specific seeded value (story direction), I realize that the nature of this feature is more or less long term memory management, and planning is just a small aspect of it. And also the progressive fragments already carry notes from previous turns for inter-turn director communication btw, though it's just the previous message, not a long term memory thing. This is something LLMs can do more reliably - but still not reliably enough in my experience, they sometimes fail even though the scope is just one turn, let alone something as big as this. |
|
@OrbFrontend In regards to the default fragment I bundled with the PR, I agree. But I was not aiming to make it the focal point of the PR. The focal point of the PR is, in fact, the system in which this fragment exists. The fragment itself was more of an afterthought and I based its more or less on the default feedback fragment in regards to how general and un-specific it is. The system itself can be used to create fragments which will genuinely be useful, it's a matter of prompting, more or less. Because drift to default characterization and nonexistent/directionless narrative trajectory is real, and if used properly, this system can help to mitigate the issue. In regards to the bugs, I can fix them. But I need to know that the PR will be merged if I am to do so. I am also going to make the decision on whether to run during director pass end or after the message is completely generated one which is made per-fragment. But once again, need to know if this will be merged. Also a note. For optimal utility, manual management of directional note entries is pretty much required. I can make it possible for an agent to process entries automatically (remove/hide stale entries, etc), but it will probably not be very reliable. This is something which can probably be called an inherent flaw of the system but I do not believe it disqualifies the system itself from being useful and usable. |
|
@OrbFrontend I realized there was more to your message than I saw in the email because it was edited. My bad. But either way, the issue remains primarily with the seeded value. I believe it is still a useful tool if you use it right. |
…ing location based on per-fragment choice
|
@hpnyaggerman I think I'll have to push back hard on this. It's pervasive to both the backend and the user experience because it's glaringly there on the UI, not something you can forget about if you don't use it. Maybe I'll think about long term memory management in the future, it'll be a sophisticated take, not something a permanent fragment type can fix. I've been reading on different types of RAG solutions - whether with embedding models or not - but didn't like any of the approaches. You okay with this? |
|
@OrbFrontend I literally made it optional and hid the note menu behind a toggle in the workflow settings. I don't know what you are talking about. Regardless, I do not quite understand what you reasoning is. You are "pushing back" by not presenting a concrete reason why the solution I have presented is insufficient or harmful, when in my own testing it has solved a big problem, that problem being what I described above as "drift to default characterization and nonexistent/directionless narrative trajectory". As I have already noted, I understand the default fragment is a bit redundant. I was basing it on what I saw with the default feedback fragment in regards to the scope and ended up pretty broad and unspecific. The defalt fragment can be changed. I do not believe this should be a blocker. So please elaborate on what the issue is beyond what has already been explicitly addressed or fixed. Because I believe this feature has merit and deserves fair consideration. |
|
@hpnyaggerman I already expressed all my biggest points:
Could you give me a specific test case or scenario? Because I've only been using your default seeded value so maybe I can't see its impact yet. I tried the following scenarios and it's been working terribly:
I agree on the manual author's note though. It's a good thing to have. |
|
@OrbFrontend Yes, I can give a specific case. I mentioned the drift back to default characterization, which is a phenomenon of LLMs being pulled towards writing things that appear in line with how the character is described and with the characterization prescriptions in the character card even in situations when it does not make any sense. For example, if a character is described as defiant, defiance will be the trait which will override any notions of rationality or self-preservation no matter how much you try to prompt around it (and I have tried making a very sophisticated system prompts to test this exact mode of behavior back when I did not know of Orb yet and was trying to modify SillyTavern to make better use of auxiliary agents). As you have noted, LLMs are not good at understanding at what is important, which is why they default to this failure mode of over-focusing on characterization in the character card and being unable to see the nuance or extrapolate the archetype described in the character card on hypothetical situations in a reasonable way. And even if you make a fragment which mitigates the issue in-turn, because there is nothing to persist the change or to articulate what its nature is (why it happened, how it happened, all transient elements which are impossible or nearly so to re-derive once the in-turn context is gone), characterization will inevitably drift back to the baseline. Which is a thing I have been able to mitigate to a very significant extent with a long-term directional note fragment which tracks important character moments which affect characterization in the long term. Another example (though I have not tested it) would be a fragment dedicated specifically to tracking the state of relationship between the persona and the character. And there are other cases I can imagine where this would be useful for which one-turn-carryover is not sufficient and post-turn lookback summary will not be good enough (because, once again, LLMs, as you have noted, are bad at attending over the right tokens when it requires nuance). But I can agree that using the system to enable manual directional notation would be useful also. I can add manual user directional notation which goes beyond editing and deleting, and I should have probably done so from the start. But I do not believe that this feature is redundant with any others, because having this fragment type is genuinely useful in ways others are not, and that is because it allows to make LLMs attend over the right tokens when they just hit the very edge of the context window, the place where attention is most accurate, and allows for persistence of important information extracted in a place which is also pretty close to the edge of context. Unlike summary, which must attend over tokens buried in the middle of context. I believe this is genuinely different. |
|
@hpnyaggerman Okay I'll take my time testing, then we'll refine it. Maybe the default seeded value is too generic and needs some examples (all my seeded values have examples), I'll tweak it and give it more tries. 
I don't think the inference model knows what a branch is. These seeded prompts need to work OOTB. I see the manual author's note button now, cool, it's actually better than what I imagined. |
|
@OrbFrontend As said, I fully concede that the defaults were not great. They are draft-tier, admittedly, but this is what maintainer feedback exists for. I will correct them to being usable and useful by default. |
|
@OrbFrontend Try it out, tell me what you think. |
|
@OrbFrontend Any progress? |
|
@hpnyaggerman Been busy on other things. I'll test some more later and merge/refine if needed. Then I'll fix the minor UI stuff afterwards. |
|
@hpnyaggerman I have nothing else to add for the backend. Can you update the docs for this new fragment type? Including AGENTS.md and mkdocs? I'll merge when you're ready. |
Resolve the additive conflict in tests/unit/test_interactive_fragments.py: union both sides' imports and keep both OOC-bracket regression tests. main (OrbFrontend#106) closes the [OOC: aside in the director/editor/feedback preambles; the branch closes it in the direction-note preamble -- disjoint builders and tests, so both are kept.
|
@OrbFrontend It is done |
…ber or the fragment label
…g able to describe multiple changes
|
@hpnyaggerman I see you're still making some changes, I'll let this cook for a bit, as you might spot other issues later. |
|
@OrbFrontend No, pretty much done. I was ironing out nits, pretty much. It is ready to go now (I have been testing it quite extensively, it works quite well). If any other things come up, I will just make a PR. The sooner the current PR is merged, the better, because I also need its contents for some of my other work (which can be seen on my fork). |
Brings in permanent direction notes (OrbFrontend#104) and Gemma 4 native tool-call parsing (OrbFrontend#107). The image_gen workflow is file-disjoint from both; the only textual overlap is tests/unit/test_tool_registry.py, where each side adds independent cases that interleave cleanly.
Main upstreamed the inherited direction-notes feature as OrbFrontend#104 (dab5ad2) and added the Gemma-4 native tool-call parser (ef05ba0). The direction-notes surface is resolved wholesale to main's OrbFrontend#104: its upstreamed form is ahead of the dev version this branch inherited (per-fragment director calls, the characterization seed replacing story_direction, rewritten preambles), so no branch-side direction-notes is kept. The prose-format workflow is untouched -- main never references it -- and test_tool_registry keeps both sides' additive edits.
2 participants
This is quite a loaded PR. But to put it short, it adds a new type of director fragment, somewhat similar to the feedback type, but this one persists plot developments to be recalled in future director and writer passes. I believe the use case is self-evident.