Relocating presets from previous turns

[puhsu]

After the great recent video on presets I've remembered a little hacky function I've made about them, that I very much enjoy using with inline presets.

It relocates the inline presets from the previous turns into the next prompt in the gptel buffer after the LLM response. Not production ready and coded with an LLM, only tested it in my very limited use cases, so expect bugs. Just sharing the idea

(defun puhsu/gptel-relocate-preset-cookies (beg _end)
    "Move @preset cookies from the just-sent user prompt to the new prompt line.
Runs once per request via `gptel-post-response-functions' (DONE state)."
    (when (and gptel-mode gptel--known-presets)
      (let* ((beg-mk (copy-marker beg))
             (user-beg (save-excursion
                         (goto-char beg)
                         (if-let* ((m (text-property-search-backward
                                       'gptel 'response t)))
                             (prop-match-end m)
                           (point-min))))
             cookies)
        (save-excursion
          (goto-char user-beg)
          (while (re-search-forward "@\\([^[:space:]]+\\)\\_>" beg-mk t)
            (let ((b (match-beginning 0))
                  (e (match-end 0))
                  (name (match-string 1)))
              (when (and (or (= b user-beg)
                             (memq (char-syntax (char-before b)) '(32 62)))
                         (or (gptel-get-preset (intern-soft name))
                             (gptel-get-preset name)))
                (push (concat "@" name) cookies)
                (when (and (< e (point-max)) (eq (char-after e) ?\s))
                  (setq e (1+ e)))
                (delete-region b e)))))
        (set-marker beg-mk nil)
        (when cookies
          (save-excursion
            (goto-char (point-max))
            (unless (bolp) (insert "\n"))
            (let ((p (point)))
              (insert "\n" (mapconcat #'identity (nreverse cookies) " ") " ")
              (set-text-properties p (point) nil))
            (font-lock-flush))))))

(add-hook 'gptel-post-response-functions #'puhsu/gptel-relocate-preset-cookies)

[karthink]

That's one way to solve the KV cache invalidation!

Thanks for sharing.