Roadmap for advertised chapters?

[weilbith]

Hey 👋

I really enjoyed the work on context generic programming. It really resonates with how I'm thinking about modular programming. I read the book two times with quite some time in between.
In the table of contents, there are plenty of interesting topics teased. Like command line parsing or async generic. I'm sure everyone is very busy. But I guess it doesn't hurt to express my interest and ask for any "roadmap"/plans? Is it unlikely these chapters will be ever written?

[soareschen]

Hey @weilbith, thank you for your interest in CGP! Indeed life has been very busy for me, and it takes longer than I expect to write about any topic that I wanted to write. That said, the priority of how I choose topics to write depends quite a lot on what the community asks for.

Originally, I did plan on continue finishing the book and cover at least the basic topics. However, the initial challenge I faced was that most of the readers in the Rust community are not motivated enough to read about CGP, because they don't care about modularity and couldn't understand the what kind of problems could be solved by CGP. As a result, a lot of my focus had been shifted from educational towards showcasing the maximal power of CGP to convince readers that it is worth their attention.

Since then, the situation has gradually improved, and we start to have a small but growing user base that appreciate CGP and want to learn more about it. As a result, I am slowly able to shift back my focus from explaining the "why" to explaining the "how".

That said, I'll probably focus on writing new content in the form of blog posts, instead of continuing the book. This is because blog posts allows me to gain higher engagement in social media, while it is harder to gain upvotes if I just share updates about new chapters in the book.

Future Topics

For the specific topics you asked about, I'd say that command line parsing is currently a low priority, as clap has done a mostly good job at enabling users to write fairly advanced CLI applications. However, you can take a look at the Hermes SDK CLI components to get a sneak peak of how we can build CLI components on top of CGP and Clap.

For async generic, I have removed the feature in contextgeneric/cgp#149, as it is ultimately still not very robust to rely on Cargo feature flags to activate or deactivate the async-ness in async-generic code. However, I have some ideas on how to support async-generic code inside the programs written as type-level DSLs, similar to Hypershell.

In the short term, I am currently focusing on publishing articles about cgp-serde, which provides context-generic versions of the Serde traits that offer modular capabilities. Aside from that, if I have the time, I will probably focus on writing about using CGP to build a generalized encoding framework, and also to build a NanoGPT clone with CGP.

If you have any other topics that you are interested, please feel free to let me know. Because my priority can still be shifted based on what the community asks for.

[weilbith]

Oh wow! Thank you very much for this extensive answer. Highly appreciated. 🙏

This makes all perfectly sense. I'll follow your blog. Looking forward to the next posts. I think the first few chapters are enough to explain all the logic behind this approach and lead perfectly towards the benefits of the library. It seems very reasonable to write blog posts for everything else. And I see the benefit for you.

The context generic (de)serialisation part sounds actually quite interesting. It might be something completely different, but it could be a solution approach I'll be very interested in. Let's see.

---

This is off topic, but targets your second paragraph. Please take this as constructive (hopefully) feedback and no criticism.
I value loose coupling in software development immensely. For me it is one of the most important aspects of managing complexity and keeping software soft. So I'm always interested in good (applied) techniques and patterns to do so.
Your book inspired me and taught me some good lessons of how to use Rust traits in a new and clever way. I'm such a big fan of the trait system idea in general. A bit more functional capabilities and type programming would be great. Whatever.
However, the cgp crate itself ain't of too much interest for me. I totally see the value of all the macros reducing the boilerplate. Some of the context generic patterns actually require quite a lot of boilerplate. Unfortunately, this almost "macro programming" is a bit too much for me. From my subjective perception, it makes it hard to understand the program it's flow and relationships if you are not familiar with cgp by heart. And code that is "hard" to read and comprehend is a big problem for me. Cpg is almost like a framework that might affect a big part of the code base. And usually a macro annotation here relates strongly to another one there and you need "special" combinations. It reminds me almost (!) of automagic libraries/frameworks for Java.
From my perspective, it is really tough to maintain a good balance here. Would not know how to do it properly myself.
However, I follow your work. It inspires me and I'll use the knowledge I gain for my own implementations. But just as part of a whole toolkit.

I don't expect any answer on this message. Just sharing my view openly in hope it can provide a perspective.

[soareschen]

Thanks for your feedback. I can tell you the story of how the CGP macros came to be. Initially, I had the same thought as you, in that CGP is really just a collection of programming patterns around context-generic implementations. The initial implementation of CGP had almost no macro, and you had to hand-derive pretty much everything, including the constructs behind #[cgp_component].

However, what happened in practice is that the vast majority of readers just gone completely blank when they encounter the first provider trait, and the blanket implementations behind the consumer and provider traits. For some reason, even with the simplest Greet example, they cannot wrap their head around what the additional constructs means. After the #[cgp_component] macro was introduced, at least we are able to move forward in explaining CGP without ever needing to show them the blanket implementations during the introduction.

Similarly, even constructs like HasField are very challenging for readers to understand when they encountered it the first time. Many Rust programmers have problem wrapping their head around the concept of type-level strings, and they couldn't comprehend how strings can be "passed" as the first generic parameter to HasField. So I had to introduce #[cgp_auto_getter] so that readers don't need to understand anything about HasField and type-level strings to have the first example of dependency injection working.

For CGP to have any chance of gaining industrial adoption, it is also super crucial that any CGP code must take up as little lines of code as possible. The first thing any software engineer will complain about is not the technical aspects of CGP, but the number of lines of code that it takes to write CGP code. So we cannot spare even a few extra lines even for very simple examples, because if even simple examples gives the impression that it needs many lines of code, then you already kind of lost in any argument of how CGP can be "better". Because of this, I try to design the CGP macros so that they can save as many lines as possible. Even something as simple as adding an extra line of pub struct Provider; is too much boilerplate, and we need to hide them away from the detractors.

Finally, I'd encourage you to not see the CGP macros as magic, but rather as a proof of concept of how CGP code could be written in a language that natively supports the paradigm. To be honest, writing CGP without macros will be similar to writing OOP code in C. That is, while it is possible to do so, it is much better to write OOP code in OOP-native languages like C++. So CGP is in a similar status right now. Eventually, my hope is that we could integrate some CGP constructs natively into Rust. Or alternatively, we could create new programming languages that natively support CGP. And to create or evolve such languages, it is very useful to make use of Rust proc macros as a tool for us to rapidly prototype the design of CGP as native language features, so that we are more informed of how that language could look like.

[soareschen]

Btw, the original intention of the CGP Patterns book was indeed to teach readers how to write context-generic implementations without using any library, including cgp itself. In particular, even if you start writing only blanket implementations with impl-side dependencies, your code will already look much better than ordinary generic functions. In fact, that was just how everything started for me - I was just trying to use blanket implementations to write better generic code, without realizing what I was getting into.

Everything else provided by CGP are really just the side effects of what happen once you start writing such blanket implementations. You will soon find out that you want to use dependency injection to get field values, and then maybe some abstract types. And then maybe you'd realize that you don't really want to have blanket implementations in one of the middle layers, but rather want the ability to choose between two or more implementations without re-writing your other blanket implementations. And then before you know it, you ended up re-inventing CGP yourself.

So thanks for reading the book, and I am really glad that you are inspired to write CGP-like code without using CGP. You are the exact kind of readers that I was targeting when I wrote the book. But sadly, perhaps you are one of the only few with this mindset, and the majority of the users are not really interested in learning programming patterns and instead expect zero-cost magical solutions.

In any case, please do feel free to take what you learn, and try applying them without using anything provided by the CGP library. I think you might eventually need them, but I'm also quite sure that you probably don't need that right away or in the near future. If I had the choice, I'd really prefer to grow a community that are passionate in the design patterns to organically come up with new CGP design patterns, as compared to me doing it alone and trying to sell the whole package to the community as a magic pill.