Skip to content

Releases

CAEL 0.4.4: a thinking swarm without a neural network, and 62/400 on ARC-AGI-1

Choose a tag to compare

@Vovala14 Vovala14 released this 20 Nov 18:38
1f99d7b

CAEL 0.4.4: a thinking swarm without a neural network, and 62/400 on ARC-AGI-1

CAEL is a chat and reasoning system that contains no neural network: no transformer, no embeddings model, no
learned layers, and no gradient training anywhere in the reply path. It runs on a CPU. Its answers come from small
programs that read, compute, check and vote.

0.4.4 brings two things:

  • A "think before answering" swarm for chat. Many small exact readers each propose an answer to several readings of
    the question; every proposal is checked, and the checked answers vote.
  • A larger, self-checking ARC grid solver. It solves 62 of the 400 ARC-AGI-1 evaluation tasks (15.5%, pass@2) in
    10 seconds per task on a CPU, with no neural network and no task-specific code.

Everything here can be checked on your own machine, as shown below.


1. ARC-AGI-1: 62 / 400 with a non-neural solver

Set Tasks Solved (pass@2) Notes
ARC-AGI-1 evaluation 400 62 (15.5%) 10 s per task, CPU only, one run
ARC-AGI-1 training 400 152 (38.0%) the set the solver was developed on
ARC-AGI-2 evaluation 120 0 still out of reach

Rules:

  • Two attempts per test input. A task counts only if every one of its test outputs is matched exactly by one of the
    two grids.
  • The solver never sees the answers. It gets only the demonstrations and the test inputs; the answers are compared
    in a separate parent process.
  • Ten seconds per task. Each task runs in its own process with a 10-second limit, and running over counts as a
    failure.
  • Ordinary hardware. The published run used 16 tasks side by side on a desktop CPU (Ryzen 9 9900X).
  • Per-task results ship with the release:

Reproduce it

git clone https://github.com/Vovala14/CAEL && cd CAEL && git checkout v0.4.4
python -m pip install -r requirements-unified.txt
git clone https://github.com/fchollet/ARC-AGI
python evaluate_arc.py --data ARC-AGI/data/evaluation --out results/my_arc_eval

Neither the solver nor the evaluation script needs the resource bundle. A full run takes about 3 to 5 minutes
with 14 to 16 workers. Your count may differ by a task or two:

  • a task near the 10-second limit can finish or time out depending on the machine;
  • a slower CPU or more parallel workers will push borderline tasks over the limit.

Compare your file task by task with ours. --seconds 30 shows how much of the gap is time rather than reasoning.

How the solver works (cael_unified/arc_solver.py)

Every candidate rule must reproduce all demonstration pairs exactly before it may predict anything. Among the rules
that do:

  • Description length decides. Rules are ranked by a rough step count, plus a small prior per family taken from how
    often that family was right on practice puzzles.
  • Augment and vote. The task is solved under all 8 rotations and reflections, each answer is mapped back, and equal
    answers pool their votes (one vote per view).
  • Think before answering: a leave-one-out self-check. When different answers compete, each leading rule is learned
    again with one demonstration held out and must predict that demonstration. Rules that generalise beat rules that only
    memorised.
  • A swarm of drawing specialists. When the output adds several colours, a different family (stamps, spans, boxes,
    rays) may explain each colour. Each is fitted on its own colour, and their drawings are merged.
  • Two-step composition. A shape-changing first step (crop, lattice, grammar operation) can be followed by a
    same-shape rule.
  • Two answers, never the input. The two best different answers are returned. Copying the test input is never
    offered.

There are 53 solver programs in all, many with several variants. The ones new in 0.4.4 are:

Kind Families
Drawing stamps around cells and around whole objects; spans between same-coloured cells (with crossing priority); rays from every coloured cell; alternating-colour rays; bounding-box and corner-box fills; rectangle interiors by size rank
Rules with a parameter read from the input scale or tile by the number of colours; fill with the commonest or rarest colour; a one-property classifier (symmetry, object count, connectedness, ...); per-row and per-column rules; edge padding; recolour by a lone marker; shift with a colour map
Picking and cropping the odd, asymmetric or densest block; corner and centre crops; corner and block overlays with a learned priority; new object selectors (symmetric, asymmetric, touching a marker, with a hole)
Repairing patterns fill the hole in a periodic pattern, or output only the missing patch; repeating-unit extraction
Counting and sorting count bars and count lines; sort bars by length; size-rank recolouring
Swarm and thinking union of specialists by colour; leave-one-out self-check; fit cache across a task's test inputs

Honest notes:

  • All solver families were developed on the ARC-AGI-1 training set. Each new family was written after looking at
    unsolved training tasks, then kept only if it lost no previously solved task on training and on an ARC-AGI-2
    training split.
  • The evaluation set is not untouched. It was scored at five development milestones, and nothing was changed with
    its results in view. So 62/400 is a repeated look, not a single blind shot. That is one more reason to rerun it, and
    to try a set CAEL has never seen, such as ConceptARC. We would like to
    hear what you get.
  • The solver is hand-written search, and each family adds a few tasks. The big remaining groups:
    • object-correspondence rules: move or copy each object to its marker;
    • multi-step compositions beyond two steps.

2. Chat: thinking with a swarm

A reply is not generated word by word. It is found, checked and agreed on:

question ──► readings ──► swarm of exact readers ──► checks ──► vote ──► answer guard ──► reply
               │                 │                       │           │            │
   as asked, all turns   nine readers, each      answer type;   readers that   a reply that is not about
   as one, number words  reads one kind of       sums must be   agree, across  the user's own situation
   as digits, statements problem completely      derivable from readings, win; becomes an honest "I can't
   split from question   or stays silent         the stated     two checked    work this out" instead of
                                                 numbers        answers that   an off-topic one
                                                                disagree → ask
  • Readings (think.py). The same question is read several ways: as asked; with all of the user's turns as one
    message; with number words turned into digits; and with the statements split from the question.
  • The swarm. Each reading goes to every exact reader:
    • everyday_exact and everyday_more: about 65 everyday shapes, such as pack sizes and change, discounts then tax,
      rates with idle time, clock and calendar arithmetic, ages, time zones, weekday of a date, corrections ("make that
      21"), newest notice wins, retractions, the latest value of a fact, and items added to or removed from a set;
    • constraint_logic: queues, rows, races and line-ups, solved by trying every order and keeping those that
      satisfy all the statements. It answers only what all surviving orders agree on, or says it can't be determined. It
      also handles if-then rules, flagging the two invalid forms;
    • word_math (story sums), recall_did ("Where did I park?") and the labelled-account swarm desk (sources that
      disagree, correct or copy each other);
    • explain_more: about 60 authored short explanations.
  • Checks. Each proposal is checked on its own:
    • the answer type must fit the question (a number, a clock time, a weekday, a yes or no);
    • for word problems, a search finds the derivation of the proposed number that uses the most of the user's stated
      numbers. An answer that leaves numbers unused, or can't be derived at all, is marked down.
  • Vote. Proposals are grouped by answer, and independent readers and readings that agree add support. When two
    fully checked answers disagree, CAEL asks which reading was meant instead of picking one.
  • Answer guard (answer_guard.py). After the whole team has answered, one last check runs. If the question is
    about a situation the user set up, and the reply is a web quote, an advice template, or a sum about something else,
    the reply is replaced by an honest "I can't work this one out reliably", which quotes what was asked.

Every reader shows its working ("21 (corrected) + 17 − 10 = 28"), and every reply carries a trace of which readers
agreed.

Limits: the readers are hand-written. They cover the phrasings and problem types they were written for, and new
types of problem fall back to CAEL's general pipeline, or now to an honest "I can't work this out". Wider coverage is
the main open problem, and it is where outside help is most welcome.

3. Footprint

  • Code and data: the code is 7.5 MB of Python. The data is about 2.6 GB of tables and indexes, with no neural
    weights.
  • Resources: one chat turn uses well under a second of CPU, except when it waits for a web lookup. Peak memory in a
    48-conversation run was 568 MB, and no GPU is used.
  • Web lookups: public web sources (Wikipedia, Wikidata) are used for facts. Set CAEL_WEB_SEARCH=0 for fully
    offline, repeatable runs.

4. Tests

  • The repository's tests: python -m unittest discover -s tests, which is what CI runs.
  • New test files in 0.4.4:
    • `tests/t...
Read more

CAEL 0.4.3: new chat desks for Python, explanations, tips and support, and an ARC grid solver

Choose a tag to compare

@Vovala14 Vovala14 released this 18 Nov 11:34
0616321

CAEL 0.4.3: new chat desks for Python, explanations, tips and support, and an ARC grid solver

CAEL 0.4.3 is a conversation release on top of the 0.4.0 core engines and the 0.4.2 conversation team. It is still
non-neural: no neural network or transformer produces its replies. The team gains new desks with authored, checked
answers. Each desk answers its kind of question and its follow-ups before any web search, so short follow-ups no longer
drift into unrelated forum quotes.

What is new

The Host: honest answers about CAEL itself (cael_unified/about_me.py)

  • "Can you search the internet?" gets a true answer about CAEL's web lookups, not the dictionary definition of the
    internet.
  • "What time is it?", "What's the date?", "What day of the week is it?" and "What time is it in Tokyo?" are answered
    from the clock. Times in about 100 cities include daylight saving rules for Europe, the US, Australia, New Zealand and
    Chile.
  • CAEL answers questions about what it is, who made it, how it works, what it remembers and whether it can see images,
    truthfully. Asked whether it is a person or a program, it says it is a program.

The Programmer: Python answers (cael_unified/python_answers.py)

  • 83 authored Python topics, each with a short explanation and a code example that the tests run. Topics include
    files, CSV, JSON, sorting, dictionaries, strings, loops, comprehensions, classes, exceptions, decorators and
    comparisons (list vs dictionary, list vs tuple, list vs set, == vs is).
  • 31 error messages explained, for example NameError, TypeError, IndentationError and KeyError: what each
    means, the usual cause and the fix.
  • Changes to the code just written: "make it recursive", "add a docstring", "add type hints" and "what if the list is
    empty?".
  • Rewordings are found by word overlap with each topic's text (BM25 with stemming and a synonym table, no model). A
    short follow-up takes its subject from the previous question.
  • A library of scripts CAEL wrote and verified itself. There are 106 of them, all checked against an AST whitelist.
    The code writer uses them when its plain search finds nothing. The search also gains zip(a, b) and adjacent-pair
    loops.
  • Lettered multiple-choice questions are no longer treated as sums.

The Explainer and the Coach

  • 18 concepts explained in plain words (concepts.py), such as inflation, interest, mortgages, credit scores, pensions, recessions, vaccines and DNA.
    Each covers what it is, why it matters, an example and how it affects you, with follow-ups.
  • 22 everyday-science explanations (everyday_science.py): rain, snow, hail, thunder and lightning, fog, dew and
    frost, sky colour and sunsets, the seasons, Moon phases, autumn leaves, earthquakes, volcanoes, rust, bread and more.
    Each has the follow-ups someone who knows the subject would answer next: "Why is it sometimes snow instead?", "How far
    away is the storm?", "Why is it summer in Australia at Christmas?".
  • Beginner guides for 11 hobbies, plus habits (beginner.py): procrastination, sleep and focus. They answer "What do
    I need?", "How long does it take?" and "What if it still doesn't work?".
  • Ranked tips (tips.py) for groceries, energy bills, fuel, phone bills, eating healthier, screen time, keeping a home
    cool, studying and saving water. The tips are ordered by how much they usually help, so CAEL can answer "Which of those
    helps the most?", "Tell me more about the second one" and "Any others?" from the list it just gave.
  • The Ideas desk (ideas.py) has ideas for children's and adults' parties, gifts, rainy days, dates, dinners and
    weekends, plus fun facts. Follow-ups such as "The second one, what do we need?" refer to the right idea.

The calculator

  • Deposits and the rest: "I pay 30% up front" is a share of the price. "What's left?" gives the remainder, and "Over
    4 months?" splits it.
  • New sums:
    • paying an amount over several months, and tips added after a split;
    • stated rates ("my gym costs $45 a month") converted to a week or a year;
    • what's saved from earnings minus spending, and how long until a savings goal.

The Supporter

The Supporter now takes the conversation when the situation itself is serious, even without a feeling word:

  • someone close in hospital or having an operation;
  • a death, including how to tell children and dreading a funeral;
  • waiting for news (a job, a test result, an application);
  • nerves before a flight, a drive or an interview, with answers to the hard interview questions;
  • a friend moving away;
  • a tradesperson who took the money;
  • a child starting a new school;
  • putting something off.

It still stays out when the user asks for no advice.

The ARC grid solver (cael_unified/arc_solver.py)

A new non-neural solver for ARC-style grid puzzles, built only on ideas and puzzle data from Looped mdlARC (no model and no
generator code):

  • it keeps every hypothesis that reproduces all the demonstrations;
  • it ranks them by description length plus a small learned prior;
  • it solves each puzzle under all 8 symmetries and pools the votes;
  • it gives two different attempts.

New solver families include split-and-combine, tilings, mirrors, object cropping and recolouring, rays, counting,
denoising, fills, symmetry completion, local rules, periodic patterns, object gravity, lattices and two-stage programs.
The older grid engines use it when they cannot answer.

Experimental: a terminal agent (cael_unified/terminal_agent/)

A small rule-based agent that plans and runs shell steps for simple file and text tasks, with a Harbor adapter. It has not
yet been run on Terminal-Bench (that needs Docker), so no score is claimed.

Measured results

Blind-shuffled comparison against LFM2-700M

24 new mixed conversations (43 turns) were written and sealed before either system ran. They covered Python, everyday
questions, emotional support, maths and follow-ups. CAEL ran with web lookups on, as it does live; LFM2-700M ran offline
with its makers' recommended sampling.

Opponent CAEL wins Opponent wins Ties
LFM2-700M (LiquidAI) 13 5 6
  • Where CAEL won: the clock and questions about itself, all six everyday conversations, four of six emotional
    conversations (two ties), and the mortgage explanation.
  • Where LFM2 won: a deposit sum, a Python comparison, a weather follow-up and a tips follow-up. All four were fixed for
    this release, and the fixes were then checked on a new sealed set (see below).

How it was run:

  • Each whole conversation was judged with the system names hidden and the A/B order shuffled. The verdicts were saved
    before the key was opened.
  • The judge is the developer of CAEL, which is a conflict of interest, and CAEL's style is recognisable even when hidden.
  • 24 conversations is a small sample. The sign test on the 18 decided conversations gives p ≈ 0.10, a clear lead that is
    not significant at the 5% level.
  • The comparison ran on the candidate before the last round of fixes.

Checks of the last fixes

The fixes were checked once on a new sealed set of 16 conversations, with new wordings and scored by the developer:
22 of 32. The gaps it found ("how much remains", "a dictionary instead of a list", "the final element", and two
science wordings) were then fixed in general form. That check no longer measures those fixes.

ARC-AGI-1

Split Before 0.4.3
ARC-AGI-1 evaluation (400, held out, run once, two attempts) 5 34 (8.5%)
ARC-AGI-1 training (400, used for development) 43 113

Later development reached 44 of 400 on the evaluation split. That figure was measured after the split had been viewed, so
it is not a held-out result. ARC-AGI-2 evaluation stays at 0 of 120.

Regression tests

The full suite has the same failures as 0.4.2 (24 inherited cases) and no new ones. The new tests cover each desk.

Size and resources

Unchanged from 0.4.2:

  • Disk: about 2.6 GB, nearly all data; the code is about 20 MB.
  • Memory: about 100–230 MB.
  • Hardware: CPU only, no GPU.

The new desks are authored text and small tables. The ARC prior file and the script library add about 60 KB.

Install and run

Use Python 3.12. Download and extract CAEL-0.4.3.zip. For full chat, also extract cael-open-chat-v1.1.zip into
models/ (unchanged from 0.4.2). Then:

python -m pip install -r requirements-unified.txt
python -m cael_unified --download
python -m cael_unified --ui

Open http://127.0.0.1:8765. Set CAEL_WEB_SEARCH=0 to turn off public web lookup.

The resource bundle is unchanged (byte-identical 0.3.7). This is custom Python code, not a Transformers checkpoint.

Honest limits

  • Subjects outside the authored desks still depend on the index and the web, and a follow-up there can drift.
  • ARC-AGI-2 puzzles remain beyond these solver families (0 of 120).
  • The terminal agent is untested on Terminal-Bench.
  • Some inherited tests outside the CI set still fail.

CAEL 0.4.2: conversation team, fielded retrieval and the Supporter

Choose a tag to compare

@Vovala14 Vovala14 released this 17 Nov 00:51
c84f64b

CAEL 0.4.2: conversation team, fielded retrieval and the Supporter

CAEL 0.4.2 is a conversation release on top of the 0.4.0 core engines. It is still non-neural: no neural network or
transformer produces its replies. A team of small agents (planner, researchers, calculator, memory, writer, critic and a
new Supporter) answers each turn from authored knowledge, CAEL's own local index of human-written answers, exact solvers
and checked web lookups.

What is new

Fielded retrieval index v2 (cael_unified/retrieval.py)

A new search index over the same 285,000 stored questions and answers in the open-chat resource. It finds a thread about
the same problem far more often than 0.4.1's two-key-word search:

  • Four separate fields. Title, tags, question body and answer are indexed apart, and ranked with BM25F (per-field
    length normalisation and field weights).
  • The whole conversation as the query. Every word the user said counts, weighted by turn and by how rare it is, not
    just two key words.
  • Learned word associations. A table counted from the index itself links the words people use to describe a problem
    to the words used to name it ("trickle" → shower, pressure; "dripping" → leak, faucet).
  • British to American English. About 110 words and phrases: lead/leash, windscreen/windshield, flatmate/roommate,
    nursery/daycare, washing up/dishes.
  • The thing and the trouble. The ranker knows what the user's problem is with ("our dog") and what is wrong ("pulls
    on the lead"). A stored question that never names the user's thing ranks lower.
  • A confidence gate. A thread is only quoted when its title names both the thing and the trouble.

On two development sets of real conversation openings, a fitting thread is ranked first 71–76% of the time, against 41%
before (strict labels: the title must name both the thing and the problem).

Conversation coherence (cael_unified/coherence.py)

Follow-up questions are short and name nothing ("Should I feed it?", "How much liquid should I add?"), so a search on
their own words used to drift. Now:

  • Every reply in a multi-turn conversation must stay on the thing the user is talking about.
  • Pages about films, characters or people that only share a word are rejected (no more cartoon called Flapjack for a
    flapjack question).
  • Advice for the wrong kind of thing is rejected (gadget repair for a headache, pet advice for a plumber).
  • A rejected reply is replaced by threads about the user's thing that ask the same question, then by advice for the
    right class of problem.

The Supporter (cael_unified/support.py)

Emotional conversations are now handled over several turns, not one reply:

  • Situations covered: loneliness, grief, a setback (a job, an exam, a place), a child who is upset (left out, or after
    a setback), self-doubt, nerves before an event, someone moving away, a dispute with a tradesperson or landlord,
    overwhelm and anger.
  • Each kind of turn gets its own answer:
    • a feeling told is acknowledged;
    • more of the story is reflected back, with the first things that help;
    • "Is it normal to feel this?" gets an honest answer;
    • "What can I do?" and "How do I help her?" get practical steps.
  • It respects the user's wishes. When the user asks for no advice or no questions, the Supporter stays out.

Stories (cael_unified/story_agent.py)

  • Stories about a character who is afraid of something, keeps doing something or can't do something get their own plots.
  • "Make it funnier", "sillier", "happier" or "sadder" really change the ending.
  • Verb forms are correct ("kept digging", not "keeped digging").

Calculator, memory and writing

  • Shares of an amount: 75% of a quote, then 40% of what was paid; "kept 25%" means 75% back; "90% of the full deposit".
  • Discounts: "10% off the full price" uses the price, not the deposit worked out before it.
  • Recipe scaling in one turn: "250 g of oats for 12 bars … for 30 bars?", and the follow-up ingredient is named.
  • Moved bookings: "She just texted to say she'll come on Thursday at 4 pm instead" updates the booking. CAEL answers
    with the new time and notes the old one.
  • Thank-you notes: "a message to my neighbour thanking her for …" now writes the note.
  • Symptoms told without a question ("I've been getting headaches every afternoon") get help, and follow-ups stay with
    them.

Clean-up

  • The old chat_v6_7 engine and ten modules that no route loads any more were removed.
  • CAEL now runs without the optional open-chat index (for example on CI): word weights fall back instead of failing.

Measured result

A sealed comparison against Gemma 3 270M IT: 20 fresh conversations, written after the build was frozen, covering
chat, advice, explanations, how-to, emotional conversations, long chats, maths, time, logic, memory updates, scheduling,
plan comparison and writing.

Opponent CAEL wins Opponent wins Ties
Gemma 3 270M IT 12 7 1

Where CAEL won: maths, time, logic, scheduling, plan comparison, emotional support, how-to steps and recalling what the
user said.

Where Gemma won: some practical multi-turn chats, a short thank-you note and one memory update. The memory update was
fixed for this release.

How it was run:

  • CAEL ran with web lookups on, as it does live; Gemma ran offline with Google's recommended sampling.
  • Each whole conversation was judged by the developer of CAEL. That is a conflict of interest, and the judging was not
    blind.
  • 20 conversations is a small sample.

Size and resources

  • Disk: about 2.6 GB, nearly all of it data: the model bundle, the knowledge base and the open-chat index. The code is
    about 20 MB.
  • Memory: about 100 MB after start-up and about 200–230 MB in a chat.
  • CPU: typical turns take a fraction of a second on one core. A few turns that need several searches take longer.
  • Hardware: no GPU.

Install and run

Use Python 3.12. Download and extract CAEL-0.4.2.zip. For full chat, also extract cael-open-chat-v1.1.zip into
models/, so that models/cael-open-chat-v1/ holds open.sqlite and the new fielded.sqlite. Then:

python -m pip install -r requirements-unified.txt
python -m cael_unified --download
python -m cael_unified --ui

Open http://127.0.0.1:8765. Set CAEL_WEB_SEARCH=0 to turn off public web lookup.

The integrity-checked resource bundle is unchanged (byte-identical 0.3.7). This is custom Python code, not a
Transformers checkpoint.

Honest limits

  • Practical how-to follow-ups can still fall back to general advice that is the right kind but not specific.
  • Facts outside CAEL's knowledge depend on the web.
  • Some inherited tests outside the CI set still fail.

CAEL 0.4.1: conversation skills, chat memory and checked web answers

Choose a tag to compare

@Vovala14 Vovala14 released this 11 Nov 20:01

CAEL 0.4.1: conversation skills, chat memory and checked web answers

CAEL 0.4.1 keeps the 0.4.0 core engines and adds conversation upgrades made while comparing CAEL with
Qwen3.5-0.8B in blind, self-judged chat rounds. CAEL is still non-neural: no neural weights and no LLM
produce its replies.

What is new

  • Exact conversation skills (cael_unified/chat_skills.py):
    • quantities with transfers and pronouns;
    • rates and speeds, including "what if" follow-ups;
    • price multiples;
    • clock times with delays;
    • syllogisms, and the converse question ("does that mean every … ?");
    • weekly schedules with cancellations;
    • values the user changed;
    • short notes and tone rewrites;
    • replies about feelings, plans, birthdays and interests.
  • Chat memory (cael_unified/chat_memory.py):
    • answers questions about what the user said earlier in the conversation;
    • acknowledges new facts and goals;
    • re-answers a question when the user adds a detail.
  • Checked web answers (cael_unified/web_check.py):
    • when CAEL does not know, it searches the web, not only Wikipedia;
    • it answers only when a second, independent publisher agrees;
    • names and distances are chosen by agreement among sources;
    • calculations over the user's own numbers never go to the web;
    • private turns are never sent in a search.
  • Open-chat index (cael_unified/open_chat.py, optional resource cael-open-chat-v1):
    • about 285,000 human-written answers from Stack Exchange (CC BY-SA 4.0), OpenAssistant (Apache 2.0) and databricks-dolly-15k (CC BY-SA 3.0);
    • it is also used to weigh how informative each word is;
    • without it, CAEL falls back to simpler weights and fewer open answers.

The 0.4.0 input validation and checked helper routes are kept and run alongside the new layers.

Measured result (honest limits)

Round Opponent CAEL Elo (Qwen3.5-0.8B = 1000) 95% interval CAEL wins / losses / ties
4 Qwen3.5-0.8B 747 561–878 2 / 15 / 3
5 (this build) Qwen3.5-0.8B 796 611–915 3 / 14 / 3

How the comparison was run:

  • Each round used 20 fresh conversations, written after the build was sealed.
  • The pairs were judged blind and rated with Bradley–Terry with a bootstrap.

Caveats:

  • The same agent developed CAEL, wrote the conversations and judged them. That is a conflict of interest.
  • Blinding is weak, because the two styles are easy to recognise.
  • CAEL ran with web lookups on; Qwen ran offline.
  • 20 conversations is a small sample, and the two rounds' intervals overlap.

CAEL 0.4.1 does not match Qwen3.5-0.8B at general chat. The main remaining losses:

  • web lookups that decline on advice and how-to questions;
  • generic acknowledgements;
  • some follow-up questions are missed;
  • off-topic retrieved answers.

The round-5 inputs, replies, blind packets, votes and ratings are attached as chat-round5-evidence.zip.

Install

Use Python 3.12. Download CAEL-0.4.1.zip and extract it. Then, from the extracted folder:

python -m pip install -r requirements-unified.txt
python -m cael_unified --download

--download fetches the unchanged 0.3.7 resource bundle. For the full chat behaviour, also extract
cael-open-chat-v1.zip into models/, so that models/cael-open-chat-v1/open.sqlite exists.

CAEL 0.4.0: core automata and CellARC rule induction

Choose a tag to compare

@Vovala14 Vovala14 released this 07 Nov 20:02

CAEL 0.4.0: core automata and rule induction

CAEL 0.4.0 upgrades actual non-neural symbolic/count/automaton engines: indexed rule propagation, validated sparse world transitions, finite-domain ordering up to twelve objects, iterative live references, deterministic count retrieval and bounded-checker interoperability. A new request-local learner infers finite discrete transitions from examples. Inherited weights, protected solvers, image/CA kernels and the public 0.3.7 resource bundle are byte-identical.

Fresh authored core confirmation: 60/60 positive answers versus 12/60 for 0.3.9.1; focused boundaries 16/16. These are known-family, maintainer/AI-assisted engineering cases, not broad English accuracy. Relevant regressions: 671 passed, four inherited conversation failures remain; 25 inherited failures fixed and 15 new unit tests added.

Full official CellARC symbolic harness, same frozen checkpoint:

Metric Interpolation (1000 tasks) Extrapolation (1000 tasks)
Correct cells / total 6361/11099 3815/10918
Aggregate cell accuracy 57.31% 34.94%
Entire puzzles correct 79/1000 (7.9%) 5/1000 (0.5%)

All 2,000 official-harness predictions and full replies reproduce the original run. No scorer/loader or checkpoint tuning; only support pairs/query reach CAEL. Eight interpolation clarifications count fully wrong. No runtime errors. The exact script, pinned sources, original outputs, input hashes, licences and reproduction command are in benchmarks/core_040/REPRODUCE.md. This harness rerun is a reproduction, not a new unseen confirmation. Most predictions include empirical/backoff guesses; exact extrapolation remains weak. Official leaderboard inclusion requires maintainer review; no accepted rank or general-chat/Qwen win is claimed.

The source ZIP and unchanged 562,714,794-byte cael-0.3.7.cael resource archive are both required. This is a custom Python runtime, not a Transformers checkpoint. Use Python 3.12 and the README instructions. CAEL remains non-commercial source-available; third-party data/code licences and attribution are retained. CellARC's actual dataset LICENSE is CC BY 4.0 despite its MIT card tag.

This publication does not change the Vynly worker or tunnel. Historic frozen reports describe their completion time; later owner-authorized publication does not rewrite them.

CAEL 0.3.9.1: native symbolic interpretation

Choose a tag to compare

@Vovala14 Vovala14 released this 07 Nov 09:30

CAEL 0.3.9.1: native symbolic task interpretation

This source/runtime patch adds native, bounded interpretation of arithmetic, rule, ordering, location, string and table tasks while preserving the non-neural symbolic/count/automaton architecture. Five new task modules and one Assistant routing change are the exact qualified candidate bytes. The installed self-information hotfix is included, and version metadata now describes 0.3.9.1.

The larger resource archive remains byte-identical 0.3.7; no weights, protected choice.py/wordmath.py, CA kernels or image source/model bytes were changed. This is a custom Python runtime, not a Transformers checkpoint.

On the completed fresh synthetic 900-question study, the evaluated native candidate scored 777/900 (86.3%), versus 414/900 (46.0%) for the unchanged 0.3.9 baseline. The newly worded half scored 327/450 versus 92/450. All 123 candidate abstentions count wrong. Known/adaptive families, maintainer/AI-assisted authoring and the neural models' fixed one-letter readouts limit interpretation. This is not a general chatbot victory or a Qwen comparison.

The model card links all nine results, questions, references and individual outputs. The old frozen leaderboard remains separate. Release verification reproduces already recorded replies after packaging; it adds no unseen-test score. Existing regression failures and finite-reader gaps remain disclosed.

Source is available for personal/non-commercial research under LICENSE; retain CAEL by Vovala14 attribution and third-party notices. See README.md for installation and the separate application/resource versions.

Configuration disclosure: the evaluated candidate count fallback was inactive, while a fresh installation enables it. Packaging reproduces all900 replies under the original configuration and preserves all777 endorsed replies under installer defaults; the 105 fallback guesses are a post-hoc packaging diagnostic, not a replacement score. See benchmarks/native_interpretation_0391/CONFIGURATION_AND_RELEASE_NOTE.md.

Fresh-study results: https://vynly-non-neural-ai-leaderboard.static.hf.space/fresh-transfer.html

Documentation URL correction: the static HF Space uses the .static.hf.space host. The release ZIP, source tag, native code, scores and resource bytes are unchanged.

CAEL 0.3.9: conversation and checked follow-ups

Choose a tag to compare

@Vovala14 Vovala14 released this 06 Nov 22:45

CAEL 0.3.9 — conversation and checked follow-ups

Authorized release, 2026-11-07. CAEL retains its non-neural symbolic/count/automaton
architecture. This package promotes the frozen conversation R3 candidate, with
declared version/portable-policy integration changes. No benchmark values or
failed transfer cases were used to patch the frozen research source.

The source freeze is
6b66428369ec45953472820e6039f7f7137e5e856b2874eda2ab6651e853119f.
Runtime version is 0.3.9; the checksum-identical public resource bundle remains
0.3.7 (8efd22d7fa670e7f011d262c33a7e026941f279e71ee2014614d5d4cf5ecf79b).
Protected choice/word-math solvers, inherited resource weights and CA kernels
are unchanged. Symbolic dialogue/event/state tables are included with source.

Added capability includes visible-request rechecks, complete literal positional
and keyword-example checking, exact guarded option comparisons, attributed user
facts and bounded source-linked definition composition. Earlier 0.3.8 personal
fact, listening, coding-condition and mobile-worker fixes are retained.

The qualified bounded Python-help adapter is now part of the public package and
installed once by Assistant construction, including the worker entry point.
The VM itself is not extended. This is integration of the tested policy, not a
claim of whole-runtime security certification. Source and dependency hashes,
portable regression checks and deployment evidence are recorded separately.

An actual HTTP preflight found that the Vynly proxy's site marker could intercept
an unrelated arithmetic question as site help. The release integration now uses
implicit site context only for a recognized authored help intent; explicitly
named Vynly/features retain public-help retrieval. Routing regressions and actual
worker smoke checks qualify this additional change. The frozen research source,
its transfer failures and its reported scores are unchanged.

Latest authored fresh transfer: useful answers 10/29 -> 22/29, and complete
positive-task conversations 2/14 -> 9/14, versus the previous development
candidate. Boundaries include refusal/clarification, not solved positive tasks.
All failures and interpretation sensitivities remain in the bundled evidence.
There is no new Qwen win or standard benchmark claim.

Known gaps include stale option switches, some dictionary follow-ups, generated
practical examples, compound source checks and incomplete lexical retrieval.
An additional preflight paraphrase, "Which editor did I say I prefer?", receives
generic uncertainty; the supported "Which editor do I prefer?" is separate.
That failed paraphrase is retained and was not used to tune the candidate.
Read README.md and benchmarks/conversation_039_r3/REPORT.md for scope.

Public assets contain no private conversations/photos, credentials, image/portrait
weights or local session configuration. Existing publications are not overwritten.
The Vynly deployment preserves the existing tunnel and session settings, with
the prior worker/source retained for rollback; live verification is recorded
only after deployment succeeds.

CAEL 0.3.8

Choose a tag to compare

@Vovala14 Vovala14 released this 30 Oct 18:44

CAEL 0.3.8 is a code-only release that loads the unchanged 0.3.7 model bundle. It stops inventing personal facts such as a birthplace, follows "just listen, no advice or questions" requests in more phrasings, solves more multi-step word problems and linear equations exactly while keeping the older helper's worked steps, keeps requested Python function names, and runs generated code only in a restricted VM. On the 80-item read-only replay, automatic grading rose from 54/80 to 59/80 (unblinded review by AI coding agents); one improved item came from the public bug report and is no longer a clean held-out test. The release has clear losses: math and Python on the public 128-question comparison are unchanged at 0/32 and 1/32, a 400-question GSM8K development check answered only 1 correctly with 17 wrong, Qwen3.5 still leads the 400-question comparison clearly (181/400 against CAEL 121/400), and Qwen is better at open conversation. CAEL is source-available, non-commercial software. AI-assisted.

Install

Download and extract CAEL-0.3.8.zip, then in its folder:

python -m pip install -r requirements-unified.txt
python -m cael_unified --download
python -m cael_unified --ui

--download fetches the unchanged, checksum-verified model bundle from the 0.3.7 release:
cael-0.3.7.cael (562,714,794 bytes, SHA-256 8efd22d7fa670e7f011d262c33a7e026941f279e71ee2014614d5d4cf5ecf79b). It is not re-uploaded here.

See CHANGELOG.md for the full list of changes, the seven changed replay items and known limitations.

CAEL 0.3.7: symbolic memory, listening and Python workflows

Choose a tag to compare

@Vovala14 Vovala14 released this 28 Oct 15:15

CAEL 0.3.7 release/evaluation notes

This release packages frozen cael-session-affect-python-r2-20261028 with only
version/packaging labels and LF source normalization changed. Full source
equivalence is recorded in benchmarks/cael_cmp_r2_20261028/source_equivalence.json.
No image/portrait weights, opponents, private training data or conversations
are shipped. Existing CAEL and resource licences are retained.

All80 replies/system on the named local confirmation were reviewed. Reviewed
scores: CAEL57, Qwen3-0.6B43, Qwen3.5-0.8B41. Raw flags54,43,42 are preserved.
Candidate-only/opponent-only pairs:24/10 and27/11; exploratory exact paired
p-values .0243065 and .0138530. Lead after omitting a category stays positive:
+8.33..+28.33pp and+6.67..+33.33pp. These calculations assume independence;
authored capability-family correlation and unblinded author judgments limit
that assumption. They meet the prespecified scoped release criterion, not a
general superiority claim.

CAEL/Qwen3 had no caught errors, caps or VM-ungraded code cases. Qwen3.5 had
zero caught errors, one1024-token cap (failed repair) and two VM-ungraded
programs: correct list.sort code credited after inspection/manual traces,
invalid bracket code independently demonstrated wrong. No generated model
source ran natively. Post-hoc line/word diagnostic cases were applied equally
to all three unchanged functions; original primary flags and cases stay intact.
An unspecified correct two-sum argument order was credited to Qwen3 without
repairing its code. Minimal listening/unknown/negative-location leniencies
and all contradictory explanations are disclosed per item.

Development qualification:68 focused checks,12/12 new unified probes,
468 broader regressions+117 subtests (two native generated-repair execution
tests excluded), then12/12 full-portable preflight checks. Final artifact
checks are packaging regressions, not additional unseen capability evidence.
All failed development attempts remain in local research; no test was
replaced or source retuned against these viewed final questions.

Weights/CA kernels/selected choice.py and wordmath.py are unchanged. No
picture/portrait generator weights or excluded old previews are packaged.
The original167MiB source-only r1 result is separate; full-portable r2 peak
is558.648MiB. All performance numbers are one shared-host run, precision and
runtime settings differ, and model/archive/installed sizes are disk, not RAM.

CAEL 0.3.6 — context reasoning and compact lookups

Choose a tag to compare

@Vovala14 Vovala14 released this 27 Oct 13:37

CAEL 0.3.6

CAEL 0.3.6 adds conversation reasoning and compact spelling tables to the existing
non-neural assistant. It retains the text, helper, math, Python, retrieval, story,
prompt and emoji engines. Picture and portrait model weights are excluded.

The context implementation was frozen at e7fa521 before the final evaluation.
Release preparation changes version labels, documentation and packaging only;
the reasoning implementation and learned model files are unchanged from that freeze.
The existing CAEL Non-Commercial License and third-party notices are retained.

Changes since 0.3.4

  • Structured conversation records for entities, attributes, corrections, events,
    source disagreements and plan progress (0.3.5).
  • Clause-based retrieval using WordNet lemmas/synonyms, typed values, reported
    statements and corrections; bounded compositional operations over stated facts
    and abstention when the supported interpretation is missing (0.3.6).
  • Compact hash/value and word-group arrays for spelling lookups, with
    memory-mapped caches instead of millions of resident Python dictionary objects.
  • Existing architecture retained: symbolic rules, counts and automaton components;
    no neural network or transformer is used by CAEL at runtime.

Context benchmark

Saved final runs on the same 400 locally authored synthetic conversations:

System Correct Accuracy
CAEL 0.3.5 53/400 13.25%
CAEL 0.3.6 122/400 30.50%
Qwen2.5-0.5B-Instruct, float32 102/400 25.50%
Qwen3-0.6B, float32 121/400 30.25%
SmolLM2-360M-Instruct, float32 28/400 7.00%

CAEL improved by 69 answers / 17.25 percentage points over 0.3.5 on this set.
Its one-answer difference from Qwen3 is effectively a tie. This is not evidence
of a general lead over either Qwen model or a new external leaderboard score.
No new HellaSwag, Tiny ToM or NNAI-Bench score is claimed for 0.3.6.

CAEL still misses 278/400. Category accuracies: conflict handling 70%, memory 42%,
unsupported-question handling 35%, updates 24%, reasoning 20%, planning 12%,
references 10%. Clear limitations remain in reference resolution, plans and
avoiding unsupported answers.

The evaluator brief describes a separate model/writer producing questions after
the freeze without source access. This was a local evaluation, not an independently
verified external human evaluation. Grading uses keyword/phrase heuristics; the
saved flags were recalculated and matched, which does not establish semantic
correctness of every judgement. Earlier 93% development-informed results are
not an independent measure of this release.

All runners use CPU. CAEL records one thread; comparison LMs use eight, greedy
decoding and 64 new tokens. Qwen3 uses its non-thinking template. Float32 runs
are reported above; the anomalously weak bfloat16 Qwen3 run is not used to claim
a lead. Saved summaries do not pin exact LLM weight revisions.

The actual histories contain 91 conversations of at most 20 messages, 196 of
21–50 messages and 113 of 100–200 messages. The legacy summary's bucket named
"21-50 messages" includes all histories longer than 20 and must not be read as
an accurate length breakdown.

Evidence and reproduction

  • Input: final.jsonl, SHA-256
    525aa74580014013838254a6b7e2d5dc5d358d00042a766e6323754ea3427fbd.
  • Protocol: evaluator brief.
  • Saved outputs and summaries: benchmarks/context_035/results/*-final.jsonl
    and *-final.summary.json for the five systems above.
  • Runner and grader: benchmarks/context_035/run_cael.py, run_llm.py,
    grade.py, summarize.py. The committed float32 runner is retained.

To rerun CAEL after installing the release dependencies and downloading its bundle:

python -c "import shutil; shutil.copyfile('benchmarks/context_036/final.jsonl', 'benchmarks/context_035/final.jsonl')"
python benchmarks/context_035/run_cael.py --root . --split final --tag cael-0.3.6-reproduction

The benchmark runner also needs psutil. Comparison LMs separately need their
own Transformers dependencies; these are not part of CAEL's runtime.

Release verification

The text-only release regression selection passed 172 tests and 129 subtests
covering conversation memory/reasoning, release guards, everyday answers, image-prompt
writing, orchestrator routing, Python helpers, small talk and typo context. This is
a focused selection, not a full-suite pass claim. All 97 archived members were
checksum-verified and 73 packaged runtime files matched the local model files.
No private source photos or private conversations were opened.

Install

git clone --branch v0.3.6 https://github.com/Vovala14/CAEL.git
cd CAEL
python -m pip install -r requirements-unified.txt
python -m cael_unified --download
python -m cael_unified --ui

Alternatively, extract CAEL-0.3.6.zip and run the last three commands there.
cael-0.3.6.cael is the checksum-verified no-images model archive; the source
ZIP is separate. Model files are inherited unchanged from the verified 0.3.4
archive, with updated bundle metadata. The new context capabilities are code
changes, not a newly trained language checkpoint. Existing local model selections
are preserved by the installer; a clean directory is recommended for reproduction.