Repository navigation
My Vibe Coding Journey with Scheme-Langserver #81
ufo5260987423
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Since April 2026, I have been using Kimi to “vibe code” the scheme-langserver project. (“Vibe coding” refers to a programming style driven by intuition and rhythm, heavily assisted by artificial intelligence.) Although I started with some skepticism, subsequent practice has shown that Kimi performs well in routine development tasks, and in highly specialized areas it can at least provide information gathering and preliminary analysis.
By “routine development” I mean that scheme-langserver still follows the standard server–client architecture. For large language models as well as most programmers worldwide, this type of work relies primarily on experience and accumulated project knowledge. My basic workflow with Kimi is as follows: first, define the current task and test procedure; then, ask the model directly to list “what aspects of the project could be optimized.” The key is to have the model output a prioritized list of candidates, sorted by effort, scope of impact, and test completeness, then execute them in order.
It is worth noting that some practitioners advocate letting the model orchestrate multiple agents to complete tasks. That approach works well for generating many web pages, but for a complex program like scheme-langserver it is not equally suitable.
The largest category of code contributed by Kimi is test code. To some extent, this achieves an effect akin to the “Tiyunzong” (a legendary Chinese Qinggong light-foot technique where one steps on one’s own feet to rise ever higher—here used as a metaphor for the model feeding on its own generated tests to improve itself). The mechanism is: the model infers correct test behaviour from existing code and functionality, then iteratively corrects itself based on test-run results. At the beginning of this process, I review the generated code, focusing on the simplest, most obvious test cases. Toward the end, I run the full test suite and execute the entire program—any easily observable error messages usually point back to my own design flaws.
In this process, scheme-langserver’s log replay feature plays a central role. Originally designed to help users debug problems in the editor without AI assistance, log replay now serves to verify that the program’s actions and behaviour remain consistent before and after Kimi’s modifications. Some recommend using Git hooks to run tests before each commit, but this is limited for scheme‑langserver—the project already contains hundreds of test cases, so only a few short-running tests can be executed at the hook stage to keep the development workflow efficient.
I have observed that large language models possess a certain metaprogramming ability that goes beyond that of an average programmer. This ability is particularly beneficial in the Scheme/Lisp community, where the typical workflow is: design a domain‑specific language (DSL), quickly implement an interpreter, write a few test cases, then gradually enrich functionality by printing and examining interpreter logs during actual use. This pattern effectively mitigates the common problem of “direction drift” in autonomous model development—if scheme‑langserver were not an LSP server but something else, existing online experience might steer it toward irrelevant features like DDoS protection, which I certainly do not need.
Scheme‑langserver’s metaprogramming model rests on three pillars:
Scheme‑langserver prides itself on two specialised features: one is type inference (already mentioned), and the other is automatic macro expansion, which is still under development. The main purpose of automatic macro expansion is to recognise identifiers declared by user‑defined macros, thereby supporting auto‑complete, goto definition, and other language‑server capabilities.
When dealing with such specialised tasks, Kimi cannot effectively utilise academic papers from the 1990s. Its behaviour tends to exhibit what I call a “pure‑program viewpoint.” (For readers unfamiliar with Chinese political discourse: the phrase “pure political viewpoint” refers to a way of thinking that stays strictly within existing positions and frameworks, avoiding deep transformation. I borrow it here to analogise Kimi’s tendency to preserve existing functionality defensively, starting only from existing code, limited user prompts, and logs.) In other words, Kimi’s understanding of existing functionality is entirely based on code, user hints, and log information—it lacks proactive exploration of deeper design changes.
Logs are an effective tool for debugging such specialised functions. Iterative feature development often leads to continuous modifications of the logs. To some extent, because automatic macro expansion itself follows a metaprogramming approach, Kimi manages to maintain a certain “decency”—no matter how it modifies the code, it does not deviate from the core design intent. The only problem is that logs quickly consume the context window, and context switches seem to cause a kind of “memory loss,” forgetting lessons learned from previous modifications. To address this, the user must decompose tasks and maintain a dedicated documentation based on the DSL design. The readability of that documentation to humans largely determines the effectiveness of Kimi’s development.
In scenarios lacking the metaprogramming pattern, the problem takes a different shape. The “maintain documentation” method remains effective, but its impact is reduced. For example, I used conditional branches to enable automatic macro expansion for certain user‑defined macros while bypassing manually written expansion rules. Under the influence of logs, if I do not explicitly state in the conversation with Kimi “line XX enables automatic macro expansion and disables manual macro expansion,” the model tends to take the lazy way out, replacing automatic macro expansion development with manual macro expansion.
4. Problems I Encountered
One thing I have paid particular attention to is that as development tasks become more specialised and difficult, user prompts serve as a kind of shortcut. That is, when a model faces a large codebase and massive logs, sheer reading volume does not guarantee the ability to extract deep patterns—especially when those patterns are expressed in the form of specific concepts. We usually refer to such forms with symbols, and the system of symbols is called knowledge. From my personal experience, knowledge from papers and books often lags behind real‑world engineering practice by two decades or more. Bridging that gap is something LLMs really cannot do yet.
First problem: Symbolic organisation is still natural‑language‑like. Take the representation of chemical elements in Chinese characters: when you see a character with the “gold” radical, you know it denotes a metal element; with the “gas” radical, you know it is a gas. This is a natural‑history style of knowledge organisation, consistent with human cognitive habits. Yet anyone with a proper education knows that the most essential parts of knowledge often deviate from such trivial natural‑language properties and instead exhibit domain‑wide natural inferences. For example, in computer programming, the stack machine model and the continuation model each have their strengths; applying them properly requires dedicated study and reading. My experience is that Kimi’s information retrieval relies heavily on text matching and contextual understanding—in other words, it lacks the larger context that a human learner would construct autonomously. That larger context is what we call the structure of knowledge. This is certainly not the model’s fault, but it could be a direction for future development.
Second problem: The model is poor at pointing out contradictions within its own context. The “robbing Peter to pay Paul” phenomenon in vibe coding—for example, moving a critical procedure out of a mutex’s scope while fixing a deadlock—is a case in point. The crucial issue is that the model does not know which constraints in the context are hard (mandatory) and which are soft (sacrificeable). Descriptions of such constraints often have a Kantian transcendence over tests, documentation, and software behaviour. Allow me to quote the first sentence of the Preface to the first edition of Kant’s Critique of Pure Reason: “Human reason has this peculiar fate that in one species of its knowledge it is burdened by questions which it cannot dismiss, since they are given to it as problems by the nature of reason itself, but which it also cannot answer, since they transcend every capacity of human reason.” This is not an error of large language models. As Kant said in the second edition (1787), “logic has already succeeded in following the secure path of science.” Dealing with such contradictions may still require recourse to logic.
Third problem: The question of drive. Vibe coding essentially follows the “input–black box–output” information model. Different inputs naturally produce different outputs; regardless of what input we give the model, we hope that limited information can yield rich output that meets requirements. In other words, the programmer operates at a higher level, holding a perspective that uses limited input to drive output. Based on experience, driving mechanisms can be divided into three categories:
Does a large language model truly have the ability to learn knowledge from code, or does it merely read, with its knowledge level always staying at the relatively shallow plane of natural language?
Vibe coding happens to reveal an intermediate state: code expresses a structure that transcends natural language, yet it is simultaneously read (by both humans and models). In other words, the threshold of knowledge such as compilation and typing lies precisely in the fact that natural language alone cannot express them well; the semantic structure of code is a good carrier of knowledge. Conversely, well‑engineered code must, to a considerable extent, shed structural constraints and strive to present a kind of disciplined natural‑language readability. And we must certainly notice that code editors and IDEs have already done a great deal of work to achieve this natural‑language reading disciplined by knowledge structures. Even in vibe coding, various skills and supporting tools are employed to enable the model to read multiple file formats.
The Jin Yong universe has the magical Tiyunzong, but these skills and supporting tools may be the true “inner strength.”
All reactions