RFC: Planned changes for GA

[mrgrain]

Document Status: Accepted
Living document, this can still change

projen 1.0

Projen is in major version zero 0.x and has been for over three years now. It's time for projen to grow up!

In order to do this responsibly, a few things must still be implemented. This list of priorities has been consulted by the community as an RFC. The list is categorized in two groups. The Must Haves are defined as the essential tasks needing completion before we consider moving to GA. The Should Have items are considered high priority goals, with their implementation status to be assessed once we are ready for GA. But ultimately they won't be blocking a release.

Contributions for any other features are still welcome! However if you are invested in a GA release for projen, consider directing your efforts to the priorities as define here.

Must Have

• Release & Versioning Strategy Define and document how we are going to do this past GA, and how we are going to deal with future breaking changes. @mrgrain
  😎 Why: Even with completion of the Must Haves, projen will still require breaking changes. Having a predictable release strategy will enable customer trust.
  🎨 Prior Art: Semver as we currently use it; Node.js with a predictable schedule, maintaining an LTS and unstable version

• Rearchitect projen to use constructs framework #1763 @mrgrain
  😎 Why: Fundamental change in approach, needed so we can slowly start migrating existing similar features to use constructs
  🎨 Prior Art: feat: Projects and Components are Constructs #2974 [WIP] feat!: migrate to constructs #1671

• Decouple Project from GitHub #1761
  😎 Why: Reality is that not everyone uses GitHub. The hard dependency currently leads down a path of not abstracting features enough
  🎨 Prior Art: feat!: refactor Project into StandardProject #1883

• New documentation page chore: new docusaurus-based documentation site #3244 @mbonig
  This should be a component itself, but we can probably drop this if needed. Good candidates for doc frameworks are Docusaurus and mkdocs.
  😎 Why: Current docs are horrible to use
  🎨 Prior Art: chore(docs): Moved entire documentation to mkdocs  #2340

• Code cleanup tasks
  😎 Why: Start 1.0 with a clean-ish slate. These are generally straightforward to do, but might be a bit of tedious work. Likely won't go into a 0.x release, but part of the GAing process.

  • Remove all currently deprecated features
  • Fixing exports #2977

Should Have

• Framework for interactive prompts in CLI
  😎 Why: Laying the foundation for code-level framework will enable future use-cases; prompting for missing input greatly improves the UX for new users
  🎨 Prior Art: feat(new): interactive prompt for missing information on project creation #2975

• Support multiple workflow engines: Gitlab #2438 GitLab support for all built-in workflows
  😎 Why: As an outcome of removing the hard GitHub dependency, we should be able to support another implementation. However having a contract will at least allow users to implement this themselves

• Built-in TaskRunner improvements Use tasks dependency model, rather than imperative instructions
  😎 Why: Currently sub-tasks are added as imperative instructions to tasks, making it difficult to optimize task execution and progress to a certain stage. For example pre-compile tasks are currently only run as part of build.
  🙅🏻 Why not: However the current system appears to be working well enough for most scenarios.

• 💬 RFC: Task runner contract (pluggable task runners) #2051 (Includes a new Task dependency model - dependsOn instead imperative - and metadata for caching)
  😎 Why: This will open projen up to be more generic and integrated better with the outside world. The challenge is similar realm to a generic Workflows Engine.
  🙅🏻 Why not: Ultimately it doesn't appear to be as much as a blocker compared to Workflows, and it is at least possible to work around given that tasks can be changed. If we can only have one of the two, I'd pick Workflows over Tasks.
  🎨 Prior Art: [WIP] feat: custom task-runtimes #2468

• Feature & Configuration injection system
  😎 Why: Currently projen components are inherently closed. Once you use a component, you are forced to opt-in to all of it. This stems from the desire to have everything ready at constructions time and to enable a declarative option. However it doesn't have to be this way. We could take a Project and layer features on top of it, with component declaring their dependencies as needed, using sensible defaults or errors as appropriate.
  Some constructs have certain properties that can be set at construction time whilst others only support certain configuration via methods and some use a combination of both. We should ensure all 1st party components follow a standardized format (and document it!) to prevent users from having to actually read the projen code to figure out how to change a certain setting. My preference is to favour methods to change behaviours and use constructors as a quick way to "inject" new components i.e: constructor of a project would have some mechanism to enable/disable/inject components whilst a getEsLint() would return the component with a bunch of methods to change settings.
  🙅🏻 Why not: This is super undefined yet.

• 💬 RFC: Break up projen into several packages #2042
  😎 Why: This is probably the future.
  🙅🏻 Why not: There is no pressing need. It might also be easier to start breaking out packages once we are GA.

• Windows support, amongst others: fix(node): unable to run projen package on windows #2761
  😎 Why: projen should have had this from the beginning.
  🙅🏻 Why not: Needs an owner who cares about projen and Windows, i.e. is a user of projen on Windows.

The Good, the Bad and the Ugly

All must haves already have some sort of experimental implementation. Some are further ahead than others. The biggest question-mark for me at the moment is removing the GitHub dependency.

Not all nice to haves are well-defined at the moment; we need to do some more requirements gathering there. Unfortunately many of them are also lacking a starting implementation. Some of the bigger items (e.g. Gitlab & Windows) are very unrelated to my day job, making it unlikely for me to spend much time on them.

[hoegertn]

I agree on the must-have parts besides the prompt thing. I see the value but not the need for GA.

Happy to help as I would love to have a 1.x to get out of the current version hell.

[moltar]

I think stabilizing the task interface should be in the must-have category, as this is a major blocker for integration with other task runners: #2051

You touch on one of those items in the "Change task runner to use dependsOn, rather than imperative instructions" item.

But I think there's another very common partner that is similar across many of these task runners that try to optimize for cache leverage: ability to specify inputs and outputs. And this needs to happen per command, not per task (or both).

Using Turborepo in the example, but it is the same pattern across all the tools I've seen:

• https://turbo.build/repo/docs/reference/configuration#inputs
• https://turbo.build/repo/docs/reference/configuration#outputs

[mrgrain]

I think stabilizing the task interface should be in the must-have category, as this is a major blocker for integration with other task runners: #2051

Thanks for calling this out. I'll try to phrase this better. In my mind "Change task runner to use dependsOn, rather than imperative instructions" is framed about making projen's task runner better.
While "Support custom task runtimes" is what #2051 is and this will include the task dependencies model as well as caching.

Edit: Updated the descriptions. Not making changes to Must have/Nice to have until we have more responses.

[mbonig]

We (Defiance Digital) are working on a new docs site in a fork. It uses Docusaurus and we're hoping to have something presentable by the EOM (Oct 31st).

[moltar]

@mbonig +1 on Docusaurus. Although I feel like mkdocs will still win because that's what AWS teams seem to have settled at, judging by the use of it across several projects now.

[mbonig]

@mbonig +1 on Docusaurus. Although I feel like mkdocs will still win because that's what AWS teams seem to have settled at, judging by the use of it across several projects now.

The main driver on Docusaurus vs mkdocs was the typescript vs python. Projen devs are going to already be doing typescript work and having the docs platform also be js/ts seemed like a better fit than asking any maintainers to have to also know python. Yes, most docs work will just be markdown content, but in the case where it's not, I liked the synergy of having js/ts in Docusaurus.

[mrpackethead]

@mbonig +1 on Docusaurus. Although I feel like mkdocs will still win because that's what AWS teams seem to have settled at, judging by the use of it across several projects now.

I don't think that the two pizza teams at AWS take into consideration too much about 'company wide' standards. They seem to do, what suits their particular team best.

The must have list is pretty good, though I'm inclined to agree with Thorsten about the CLI, and would have probalby put it in 'nice to have'.

Using Constructs will be a game changer.

[moltar]

@mbonig +1 on Docusaurus. Although I feel like mkdocs will still win because that's what AWS teams seem to have settled at, judging by the use of it across several projects now.

The main driver on Docusaurus vs mkdocs was the typescript vs python. Projen devs are going to already be doing typescript work and having the docs platform also be js/ts seemed like a better fit than asking any maintainers to have to also know python. Yes, most docs work will just be markdown content, but in the case where it's not, I liked the synergy of having js/ts in Docusaurus.

Yeah, I am in full alignment on the py vs ts issue. I do prefer Docusaurus too! There are also a couple of plugins for TypeDoc support too, with full theme support.

[moltar]

@mrgrain Another suggestion for v1, and I am not sure if there's an issue for it, could be export clean up. This would probably break a lot, unless a backward-compat and, perhaps, a warning is provided.

Right now projen exports everything from lib/ but ideally root barrel should export everything, and optionally some high-level additional paths, a-la cdk.

[mrgrain]

At this point I'm happy to accept any implementation that improves the current documentation page. 😅

Also having one, doesn't preclude a second component with a different tool being added.

[mrgrain]

Right now projen exports everything from lib/ but ideally root barrel should export everything, and optionally some high-level additional paths, a-la cdk.

Could you open an issue for this and include an example? I'm not sure I 100% understand what you mean.

[mbonig]

@mbonig +1 on Docusaurus. Although I feel like mkdocs will still win because that's what AWS teams seem to have settled at, judging by the use of it across several projects now.

The main driver on Docusaurus vs mkdocs was the typescript vs python. Projen devs are going to already be doing typescript work and having the docs platform also be js/ts seemed like a better fit than asking any maintainers to have to also know python. Yes, most docs work will just be markdown content, but in the case where it's not, I liked the synergy of having js/ts in Docusaurus.

Yeah, I am in full alignment on the py vs ts issue. I do prefer Docusaurus too! There are also a couple of plugins for TypeDoc support too, with full theme support.

One issue that we're having is that Docusaurus can't parse and render the API.md that projen generates. Docusaurus uses MDX and MDX can't parse the current API.md file and fails at some {}. I can get more details, but we'll have to work through those problems first.

[agdimech]

Looks good although don't think the interactive prompts CLI is mandatory for release. I think we also should add this to the must have category:

• Standardised interface for configuration: Some constructs have certain properties that can be set at construction time whilst others only support certain configuration via methods and some use a combination of both. We should ensure all 1st party components follow a standardised format (and document it!) to prevent users from having to actually read the projen code to figure out how to change a certain setting. My preference is to favour methods to change behaviours and use constructors as a quick way to "inject" new components i.e: constructor of a project would have some mechanism to enable/disable/inject components whilst a getEsLint() would return the component with a bunch of methods to change settings.

One thing that could fall under nice to have is IDE support for .projenrc. Would be cool to have autocomplete for dependencies similar to package.json to make the experience a little more user friendly.

[mrgrain]

One issue that we're having is that Docusaurus can't parse and render the API.md that projen generates. Docusaurus uses MDX and MDX can't parse the current API.md file and fails at some {}. I can get more details, but we'll have to work through those problems first.

I'm not familiar with MDX, so not quite sure what's going on there. I also have a in progress PR to upgrade the version of jsii-docgen in here. But also feel free to open an issue in jsii-docgen about the incorrectly (?) generated files.

[mrgrain]

Standardised interface for configuration

Would you mind opening a new issue for this, so we can discuss the ins and out there. In principle I agree with being consistent, but it feels very vague at the moment.

[kaizencc]

My $0.02:

Framework for interactive prompts in CLI and New documentation page should be in the Nice to Have section based on the fact that implementing them would not likely result in any breaking change.

Curious why Break up projen into several packages is a Nice to Have given that it seems like it would be a breaking change if we decided to go down that path.

The biggest task and highest risk thing is migrating projen to the constructs framework and the impact of that one feature dwarfs the other [important] tasks on the list. It almost feels like that should be where the attention is directed at first, and then we can think about GA'ing what we have after.

[mrgrain]

Thanks, @kaizencc

Curious why Break up projen into several packages is a Nice to Have given that it seems like it would be a breaking change if we decided to go down that path.

Good point. This is based on the assumption that projen will release new major versions with breaking changes on a somewhat regular cadence. See the first entry in the list. To me once every year is the upper boundary. In practise, this would mean these changes can still be made, they just might have to wait for a bit even when implemented.

The biggest task and highest risk thing is migrating projen to the constructs framework and the impact of that one feature dwarfs the other [important] tasks on the list. It almost feels like that should be where the attention is directed at first, and then we can think about GA'ing what we have after.

I have a Draft PR ready for this. Seems relative straightforward to be honest. Impact is mitigated by using Magic AutoIDs. We can do this in projen because here Construct IDs don't mean anything.

[ahammond]

Our argument for using projen is that it makes a multi-repo approach viable. One of the main gaps we have right now is compliance across hundreds of repos. I think github-settings is the solution for that, but #2987 is a long way from done. I think that's absolutely a Nice to Have, but probably not a Must.

[tinovyatkin]

I would add a big wish here - make it possible to use projen new --from with non-JSII / non-NPM packages.
For example it completely bombs / limits adoption of projen use in the Python world - I would expect being able to do pipx projen new --from=some-module-published-to-pypi from Python world, but I have to write TypeScript and publish it to NPM (crossing to another world).

We do workaround by using npx projen new from --from=git://repo/repo.git and manually creating some fake .jsii file and little jsii-bridge.js + package.json inside our repo, but it's not an experience I would recommend to anyone.

Contrary to aws-cdk (where we all targeting the same AWS), many projen projects targeting some specific OS / languages and so JSII doesn't makes any sense for it (if I want to have a template that bootstraps some pythonic project and provides some example - crafting that in Typescript only limits my choices without bringing any benefits to non-Python consumers).

So, having an optional fallback to allow building non-JSII project template would unlock many users creativity.

[mrgrain]

I would add a big wish here - make it possible to use projen new --from with non-JSII / non-NPM packages.
For example it completely bombs / limits adoption of projen use in the Python world - I would expect being able to do pipx projen new --from=some-module-published-to-pypi from Python world, but I have to write TypeScript and publish it to NPM (crossing to another world).

I understand the need. The main challenge is that we currently use .jsii for introspection. So what we need is either a completely new introspection engine, or tooling that can create "fake" .jsii files for non TS packages. And on top of that we need to implement language specific project generation, but that's easily done I think (just a bit of work). I'm not against this (heck I'm all for it!), but I don't see it happening in the near future.

[mbonig]

One issue that we're having is that Docusaurus can't parse and render the API.md that projen generates. Docusaurus uses MDX and MDX can't parse the current API.md file and fails at some {}. I can get more details, but we'll have to work through those problems first.

I'm not familiar with MDX, so not quite sure what's going on there. I also have a in progress PR to upgrade the version of jsii-docgen in here. But also feel free to open an issue in jsii-docgen about the incorrectly (?) generated files.

The problem occurs while MDX is trying to parse this line:

<li parentName="ul"><strong parentName="li">{`command`}</strong>{` (`}<code>{`Array`}<string></code>{`)  Provide a command to the docker container. `}<strong parentName="li"><em parentName="strong">{`Default`}</em></strong>{`: use the container's default command`}</li>

With the error:

Module build failed (from ./node_modules/@docusaurus/mdx-loader/lib/index.js):
SyntaxError: /Users/matthewbonig/.../projen/docusaurus/docs/API.md: Expected corresponding JSX closing tag for <string>. (2195:93)

and the corresponding rendered markdown:

[mbonig]

One issue that we're having is that Docusaurus can't parse and render the API.md that projen generates. Docusaurus uses MDX and MDX can't parse the current API.md file and fails at some {}. I can get more details, but we'll have to work through those problems first.

I'm not familiar with MDX, so not quite sure what's going on there. I also have a in progress PR to upgrade the version of jsii-docgen in here. But also feel free to open an issue in jsii-docgen about the incorrectly (?) generated files.

The problem occurs while MDX is trying to parse this line:

<li parentName="ul"><strong parentName="li">{`command`}</strong>{` (`}<code>{`Array`}<string></code>{`)  Provide a command to the docker container. `}<strong parentName="li"><em parentName="strong">{`Default`}</em></strong>{`: use the container's default command`}</li>

With the error:

Module build failed (from ./node_modules/@docusaurus/mdx-loader/lib/index.js):
SyntaxError: /Users/matthewbonig/.../projen/docusaurus/docs/API.md: Expected corresponding JSX closing tag for <string>. (2195:93)

and the corresponding rendered markdown:

Ok, ignore this. I upgraded to the latest version of docusaurus and they support CommonMark now instead of having to use MDX so this is no longer an issue.

[mrgrain]

Ok, ignore this. I upgraded to the latest version of docusaurus and they support CommonMark now instead of having to use MDX so this is no longer an issue.

This reminds me the projen is using a very old version of jsii-docgen and I need to upgrade that 🙈

[mrgrain]

Hey everyone, quick update. I have "finalize" the list based on everyone's feedback. However this is going to be a living document and we can still make changes. However they should be more considered from now on.

Here is the full list of updates:

• Renamed Nice to Have to Should Have to better convey the intention of items being a high priority for the project, but ultimately releasing GA is more important than having any of the items included.
• Moved Framework for interactive prompts in CLI to SHOULD based on feedback
• Removed Full implementation of RFC: Project creation prompts completely, since this has too many dependencies on other SHOULD items
• Added new item Code cleanup tasks to MUST, encompassing various deprecation and code-reorganization tasks. I expect this list to grow. However I also expect these tasks to mostly happen as part of the cut-over to 1.0, i.e. just before the first GA beta is released
• Extended item Feature & Configuration injection system in SHOULD, based on feedback. This is in SHOULD because there are no concrete, accepted solutions at the moment. For projen and the Constructs Programming Model (CPM) the current generally accepted approach is that everything needs to be configurable via constructor properties, with methods providing additional supporting features. We can document this better (and start enforcing it better) irregardless of GA.
• Assigned @mbonig as owner to the new docs page. Please LMK if you don't want to be listed as owner.

[mrgrain]

Some notes on not included/changed items:

• "Change task runner to use dependsOn, rather than imperative instructions"
  Not MUST, because I don't think it comes up often enough compared to other items.
• "IDE support for .projenrc"
  I really hope this can be solved by the community first and then maybe adopted into the org as a separate repo. As it stands I don't think this can be properly supported by maintainers.
• "github-settings"
  This would be awesome, but it very much feels like a stand-alone component to me. Which is great, because we can just release it anytime as soon as it's implemented. I ended up not putting it in SHOULD, because its third-party app.
• "make it possible to use projen new --from with non-JSII / non-NPM packages."
  This is super interesting, but ultimately requires such a radical departure from the current approach that projen is a jsii package. I'd love to be wrong on this and have a simple solution emerge. Until then, it's just below the cut.

[rehanvdm]

I think stabilizing the task interface should be in the must-have category, as this is a major blocker for integration with other task runners: #2051

You touch on one of those items in the "Change task runner to use dependsOn, rather than imperative instructions" item.

But I think there's another very common partner that is similar across many of these task runners that try to optimize for cache leverage: ability to specify inputs and outputs. And this needs to happen per command, not per task (or both).

Using Turborepo in the example, but it is the same pattern across all the tools I've seen:

* https://turbo.build/repo/docs/reference/configuration#inputs

* https://turbo.build/repo/docs/reference/configuration#outputs

I would really not like to see turborepo, nx, lerna in projen. I have used https://github.com/google/wireit in a few projects and it is magical, does just what it is supposed to without the heavily opinionated monorepo things.

[mrgrain]

I would really not like to see turborepo, nx, lerna in projen. I have used https://github.com/google/wireit in a few projects and it is magical, does just what it is supposed to without the heavily opinionated monorepo things.

I am quite sure that none of them will end up directly in projen or even as a default task runtime. IMHO these tools are attracting way too many opinions to endorse a default one 🙊 (proof: this thread)

What projen needs to do is:
a) Adopt a task dependency model that is more inline with modern task runners
b) Adopt an interface for i/o to enable caching
c) Allow task runners to be swapped out at will

[moltar]

I think another useful bit of the related functionality here, perhaps, would be to isolate projen task runner from projen synthesizer. Because now, in CI and other envs, the entire projen project needs to be installed just to run the initial task. I think we should have a compact CLI that just does the runner bit, and can be installed super fast from NPM as a single bundled index.js file.

[mrgrain]

I think another useful bit of the related functionality here, perhaps, would be to isolate projen task runner from projen synthesizer. Because now, in CI and other envs, the entire projen project needs to be installed just to run the initial task. I think we should have a compact CLI that just does the runner bit, and can be installed super fast from NPM as a single bundled index.js file.

Yeah I'm in favour of this, I'm not not entirely sure how to slice the pie correctly.

The projen package currently does three things:

• a standard library (ignoring the task runner for a second, you don't actually need anything else to use projen)
• a task run time that is using task.json
• projen new

I think we can't split the the latter two from a UX perspective. However I also still want to publish a create-projen package as alternative. In that world, you could use create-projen to init a new project type that uses a different task runner and you'd never have to use the projen cli.

The other thing to consider is that I'd the projen CLI to either be native or at least bundle node.

[moltar]

I think overall, it sounds like a decoupling into programmatic API as one package, and then a CLI that uses said API as another package. And then also rip out the task runner from both of those into yet another package.

[thoroc]

Feature & Configuration injection system

Would that mean that Component would be registrable to other Component or event to Project types?

[moltar]

Feature & Configuration injection system

@mrgrain is there an open issue for this one? I and @Pagebakers have thought about this one a lot, including various esoteric approaches, such as using reactive components (using various React stores) and multi-layered synth. This can be a head-banger.

[mrgrain]

@mrgrain is there an open issue for this one? I and @Pagebakers have thought about this one a lot, including various esoteric approaches, such as using reactive components (using various React stores) and multi-layered synth. This can be a head-banger.

No issue yet. I'd love to hear your thoughts. We have a lot of ideas around this problem, it's similar in CDK land. I have a doc that I need to re-write with other construct libraries in mind (aka projen). I don't think your approaches are that esoteric. We also landed on a reactive engine as a possible solution.

Would that mean that Component would be registrable to other Component or event to Project types?

It's more about the current abstractions being closed and defining dependencies on interfaces.

Let's imagine you have a TypeScriptProject.
You want this project to use either githubActions or bitbucketPipelines but never both.
Neither option is better or worse, they are just different.

Currently you have to write something like this, which isn't too bad really.

const project = new TypeScriptProject({
  github: false, // diable the default
  bitbucket: true,
  bitbucketOptions: { /* ... */ },
});

But it prevents you from using either of those features on a PythonProject - unless someone goes and implements it.

If we encapsulate components strictly, we can probably get away with just not having options on the project:

const project = new TypeScriptProject();
new BitbucketPipelines(project, { /* ... */ });

But we have features like the build system that depend on a workflows engine. Somehow the build system needs to be able to define it's dependency on either GitHubActions or better yet a generic IWorkflowsEngine. It could definitely implement defaults and just create one itself, but really it should be able to use project as an IoC container:

const workflows = project.get("IWorkflowsEngine");

That leads us to the question of how users would register a workflow engine, and in extension how (opinionated) library authors can register defaults ("we use Bitbucket at this company, so IWorkflowsEngine should be BitbucketPipelines unless specified otherwise").

One way (there are many) to achieve this, could be layering features onto Constructs from the outside. It's a bit like traits or mixins, so I like to call this concept Construct Traits:

// This could be a way to register a default for something
any('IWorkflowsEngine').of(Project).defaultsTo(BitbucketPipelines);

// construct traits are injected from the outside
const myProject = new TypeScriptProject()
  .with(bitbucketPipelines({ /* ... */ }));
// or without fluent interface
bitbucketPipelines(myProject, { /* ... */ });

// they can also be applied to the construct tree (similar to aspects):
any(isGithubWorkflow).of(myProject).uses(setMyRunnerGroups);

But a challenge with all of these solutions is what @moltar alluded to: They tend to strongly depend on a certain engine implementation, or at least a common interface. Which is something that doesn't exists in the CMP at the moment. But it could, or it could also just live in projen.

[thoroc]

Ok I get what you mean.

I was just looking for clarification as I was playing around in the past few months with a projen type that would allow users to define an object as a generic property on options passed to the project as a specific type from available ones (as Enum) AND provide a mean to define tasks that would plug into that component depending on user choices. Hence my interest in seeing that being decoupled.

As for having a common interface to tie up any implementation, it seems a pretty standard thing.

[mbonig]

@mrgrain Right now the API.md file that is generated is quite large. Is there any option to break that up easily into smaller files?

[mrgrain]

@mrgrain Right now the API.md file that is generated is quite large. Is there any option to break that up easily into smaller files?

Yes. There's a flag in jsii-docgen to split by submodule.

I also kind of want to split by language and submodule, but that's not properly working right now.

[mbonig]

@mrgrain Right now the API.md file that is generated is quite large. Is there any option to break that up easily into smaller files?

Yes. There's a flag in jsii-docgen to split by submodule.

I also kind of want to split by language and submodule, but that's not properly working right now.

I can include the splitting in our PR if you can point me to how to do that.

Also, how is the current site being deployed so I can start to plan workflows for changing it to handle the new docusaurus site?

[mrgrain]

I can include the splitting in our PR if you can point me to how to do that.

It's the --split-by-submodule flag. We probably need to introduce this as an optional option to the docgen component.
https://github.com/cdklabs/jsii-docgen?tab=readme-ov-file#cli-options

Also, how is the current site being deployed so I can start to plan workflows for changing it to handle the new docusaurus site?

There is an option to use GitHub Actions instead. That's probably the future and we can definitely enable that.

[mbonig]

I can include the splitting in our PR if you can point me to how to do that.

It's the --split-by-submodule flag. We probably need to introduce this as an optional option to the docgen component. https://github.com/cdklabs/jsii-docgen?tab=readme-ov-file#cli-options

Tried this and it doesn't appear to do anything. Could it be that projen doesn't have any 'submodules' per jsii?

npx jsii-docgen -o docusaurus/docs/api/API.md --split-by-submodule

produces only one API.md file. Not providing a filename and only providing a path results in an error.

[mrgrain]

Works on main. You can check the .jsii assembly if your version still has submodules. Might be related to the other issue you run into?

[mbonig]

Works on main. You can check the .jsii assembly if your version still has submodules. Might be related to the other issue you run into?

What was the full command you ran to get that?

k, got it working when I explicitly told it where the .jsii file was. Odd...

[agdimech]

@mbonig - out of curiosity, are you planning to build a new DocusaurusComponent/DocusaurusProject which Projen will dogfood to create its own docs site?

Reason I ask is that I was planning on building a MkDocs or Docusaurus Project myself but if this is already in the works I can just wait and/or contribute if there is a public branch available.

[mbonig]

@mbonig - out of curiosity, are you planning to build a new DocusaurusComponent/DocusaurusProject which Projen will dogfood to create its own docs site?

Reason I ask is that I was planning on building a MkDocs or Docusaurus Project myself but if this is already in the works I can just wait and/or contribute if there is a public branch available.

I did not. I thought about it, and would like to get there, but the focus was on the docs, and I didn't want to take longer than needed (this has already been almost a 6 month effort, tbh) to get this out.

[mbonig]

I can include the splitting in our PR if you can point me to how to do that.

It's the --split-by-submodule flag. We probably need to introduce this as an optional option to the docgen component. https://github.com/cdklabs/jsii-docgen?tab=readme-ov-file#cli-options

One thing I just discovered about this... it creates nice files for the submodules, but no documentation for the projen (root) module itself. Any ideas on how to fix this? I've opened an issue with jsii-docgen too... (cdklabs/jsii-docgen#1240)

[mrgrain]

I can include the splitting in our PR if you can point me to how to do that.

It's the --split-by-submodule flag. We probably need to introduce this as an optional option to the docgen component. https://github.com/cdklabs/jsii-docgen?tab=readme-ov-file#cli-options

One thing I just discovered about this... it creates nice files for the submodules, but no documentation for the projen (root) module itself. Any ideas on how to fix this? I've opened an issue with jsii-docgen too... (cdklabs/jsii-docgen#1240)

This is fixed now. jsii-docgen -o docs/api/API.md --split-by-submodule works for me now.

[hoegertn]

Is there any idea how we can proceed here?

0.x version lead to a lot of version hell and I would love to have a projen GA

[mrgrain]

There are three open tasks and two of these can definitely be progressed already.

• Release & Versioning Strategy I've said I'd pick this up, but haven't made any progress yet. Could do with some help.
• Decouple Project from GitHub #1761 Needs someone to own this.
• Code cleanup tasks probably can't do all of them just yet, but I'd be okay to accept PRs that start to remove the oldest deprecations.

[moltar]

Release & Versioning Strategy I've said I'd pick this up, but haven't made any progress yet. Could do with some help.

Been enjoying working with changesets as of late. Seems like the most straightforward magic-less release workflow thus far from all of the ones I've tried.

[mrgrain]

Release & Versioning Strategy I've said I'd pick this up, but haven't made any progress yet. Could do with some help.

Been enjoying working with changesets as of late. Seems like the most straightforward magic-less release workflow thus far from all of the ones I've tried.

Sorry, this task is about the "What" not the "How". We need something like this: https://github.com/nodejs/Release

Once we are GA we should not keep making breaking changes at the same speed as we currently do. However we definitely will need them, for new features but also to remove unsupported language versions. My basic idea is to allow breaking changes once a year. But we will need some structure around how we organize this.