v4.100.246

Code search by meaning: agents and the CLI

• Agents the lead starts are now offered search_codebase's semantic
  mode in a folder that is indexed. The index is the lead's; the lines of a
  passage are given as they are in the agent's own copy of the folder.
• CLI: --code-index (or CLINE_CODE_INDEX=1) turns the mode on for the
  folder the run is in. It needs CLINE_EMBEDDING_BASE_URL and
  CLINE_EMBEDDING_MODEL, and downloads LanceDB into the data folder the first
  time.

xOllama: no more silent five-minute pauses

• On xOllama a run could go silent for about five minutes after a turn whose
  thinking hit its budget, or around a compaction. The note or summary written
  then is a request of its own, and it was refused for space the conversation
  itself was holding while idle; it only got through when the engine's
  five-minute idle timer released the conversation.
• Background requests (thinking notes, compaction summaries, titles, commit
  messages, generated prompts) now ask for exactly the space they need and
  get an answer at once.
• If the only thing in the way of a conversation's own note or summary is
  that conversation's idle window, the window is released straight away and
  the request runs. The next turn re-reads its prompt once, as it already did
  after the pause. The chat says so in a status row.
• In every other case nothing is released. The request waits as before, and
  the chat now shows a row when it starts waiting and when it is sent.
• Starting a new conversation right after closing one no longer waits five
  minutes with the model already loaded. Closing a conversation now gives
  its window back to xOllama at once; before, it stayed booked until the
  engine's idle timer ran out.

Web scraping for any task

• Settings > Features > Web scraping (was "Web scraping for the
  librarian") has a new tick, Only for the librarian, on by default: as
  before, only the librarian scrapes, to make books.
• Untick it and every task is offered a web_scrape tool, with or without
  the Library: search the web, list a site's pages, read a page, and
  crawl a site into a folder of the workspace.
• A crawl saves the site itself, unaltered: each page as the site served it
  and as the browser rendered it, every stylesheet, script, picture and font
  the pages use, and a markdown reading of each page, with an index.md
  listing it all. Use it for a site you want reworked or used as the source
  of new pages. Ask for "text only" to keep just the markdown, as notes.
• How much a site crawl may fetch is yours to set under Features: files
  (2,000), largest file (25 MB) and total (300 MB).
• The files go to disk, not into the model's context. The tool says whether
  the whole site was read or which linked pages were left out.
• When a crawl stops at one of your limits, the model is told first and asked
  to tell you: which limit, how much was left out, and the setting that
  raises it. The limits count the whole folder, so a resume does not get past
  them until you raise them.
• A crawl can be continued. Ask to resume it and only the pages not yet read
  and the files not yet fetched are downloaded; what is in the folder stays.
• "Allow web scraping" in the API configuration is still needed, and the page
  and depth limits under Features still cap every crawl.
• CLI: --web-scrape (or CLINE_WEB_SCRAPE=1).

Fewer "are you finished?" reminders

• A task that answered the end-of-run reminder by checking its work and then
  said it was done in one short sentence was asked again, sometimes twice.
  That sentence now ends the run.
• A task that did its work and then said it was complete is no longer asked
  whether it is finished. Neither is one that reports having stopped at a
  limit you set. Every other reply that calls nothing is asked as before,
  including a status with work still to do.
• The reminder no longer reads as an order to keep going when a task stopped
  at something only you can change, such as a limit you set.
• A reply opening "I have completed ..." counts as saying the work is
  complete, as "... is complete" already did.
• When the checklist has unticked items at the end of a run, only the
  checklist reminder is sent. The general reminder used to arrive with it
  and ask for a one-sentence answer, which left the boxes unticked and
  brought the checklist reminder back.

Searching files with very long lines

• A file search for text that sits on a very long line, as in a minified
  script, answered "No match" although the text was there. It now shows the
  part of the line around the match and says where in the line it is.
• A codebase search printed such lines from their start, so the match was
  often outside what was shown and one search could fill the whole reply.
  Long lines are now cut around the match.
• A codebase search could show the lines before a match under the previous
  file's result, and left out the matching line itself. Both are fixed.
• A file search called with a list of patterns instead of one pattern was
  refused. One entry is now taken as the pattern, and several as
  alternatives, as literal texts too.

Message history

• What you have typed in the message box and not sent is still there after
  the window reloads or the panel is closed and reopened.
• Ctrl+Up and Ctrl+Down in the message box walk through the messages you
  sent before. Ctrl+Down past the newest gives back what you were typing.
  On macOS the keys are Cmd+Up and Cmd+Down; in a box that holds text,
  Cmd+Up first moves the caret to the start, as in any text box.
• Settings > General > Message history switches this on or off and sets how
  many messages are kept (50 by default, 500 at most). Switching it off
  deletes what was kept.