Generate project pages from GitHub projects

[jgallimore]

This is my current work in progress for issue #107, which I'm opening up for review and further collaboration. I don't think this is ready for merge yet. @ericpoirier - many thanks for hacking on this with me during Eclipsecon.

This PR does the following:

• Introduce a projects folder under content which is deliberately empty, and its contents added to .gitignore.
• Add a script to generate pages for each of the individual Jakarta EE projects, in Markdown format for the Hugo CMS to render in the same way as every other page on the site.

The script provided can be run by executing npm run generate-projects. You will need to run npm install first to ensure you have the correct GitHub client and Mustache template NPM modules installed.

The script will use the GitHub API to query each of the JakartaEE projects in GitHub to get the name, description, contributors and the content of the README.md file in the root of the project. This information is assembled into a JSON object which is then fed into Mustache to create the markdown page for the CMS. The information gathered is very similar to the way the MicroProfile site does it, albeit by doing upfront generation using a NodeJS script, as opposed to on-the-fly in a Java app server.

At present, this only includes the JTA project, but the full list can be provided by changing line 6 of scripts/generate-project-pages.js.

Current issues:

• This will fail if README.md is not present
• The date metadata needs updating - perhaps to the date of the last commit in the project?
• The project list page comes from the Hugo theme itself, and will need updating there. I have attempted to create an index.md page using the same generation technique, but this then prevents the individual project pages from being generated.

Questions:

• Do we want to mandate the individual projects to have a site.yaml file or similar to carry some additional metadata that we can parse and use here, or is the information I have picked up so far
  sufficient?

Although I am a developer, I'm a novice with Node JS and new to Hugo altogether, so if I've made some errors there, or things can be improved, I'd be very grateful for the feedback.

[jgallimore]

I notice there is a CI build; this might fail as there are NPM module changes and different commands to run. Happy to help if there are any issues.

[chrisguindon]

Hi @jgallimore,
This is awesome! Thank you for your pull request!

The build that was failing is now passing. I believe travis-ci canceled the first build because of your merge commit. I think we can remove it since we always rebase before merging a pull-request.

For the list of projects, we are planing to work on a new "Eclipse Project" endpoint on api.eclipse.org.

With this new api endpoint, we will be able to fetch a list of all the Jakarta EE projects with the location of each git repository.

I am thinking we could leverage this new api endpoint to automate the list of projects to include on the website instead of having to maintain a list of project on line 6 of scripts/generate-project-pages.js!

Also, please make sure to include a copyright header in generate-project-pages.js. I will try to find some time in the next few days to test your pull request!

[ggam]

Any news here? Now that Eclipse GlassFish it would be great to start filling jakarta.ee with some content and this PR feels like a pretety good start.

[chrisguindon]

@jgallimore What are the current blockers for moving this forward?

Is there anything that my team could do to help?

[edbratt]

Is this PR still needed? Looks like it would require some manual work now.

[chrisguindon]

We recently added the "project" archetype to our hugo-solstice-theme.

We would need to update the jakarta.ee website with the most recent version of our theme to leverage this but I think we could help the community to move this forward.

Documentation:
https://eclipsefdn.github.io/hugo-solstice-theme/components/project_list/

Example:
https://eclipsefdn.github.io/hugo-solstice-theme/projects/

Since this was implemented in our hugo theme, we would get this look and feel without too much work.

The great thing about hugo is that we can override some or all of it but obviously that would require some more work.

If the community is interested, we can investigate merging our work with what @jgallimore has done so far with this PR.

[ggam]

We would need to update the jakarta.ee website with the most recent version of our theme.

@chrisguindon what needs to be done to achieve that? I'd really like to help moving this forward.

[chrisguindon]

We would need to update the jakarta.ee website with the most recent version of our theme.

@chrisguindon what needs to be done to achieve that? I'd really like to help moving this forward.

I deployed a new release on npm this morning. @autumnfound is currently working on upgrading the theme on jakarta via issue #346.

I expect this to be done today. We need to manually test for any regressions before we would merge his latest PR.

[jgallimore]

I'll work out the conflicts here and post an update on its status. We might then need to discuss if this is still the direction we want to go in, or if we want to do something else.

[chrisguindon]

@jgallimore do we still want to leverage "metadata" from each project github repository or should we consider pulling in data from projects.eclipse.org to generate the project pages?

[ggam]

@jgallimore do we still want to leverage "metadata" from each project github repository or should we consider pulling in data from projects.eclipse.org to generate the project pages?

I think getting it from Eclipse project pages is best, although difficult to parse correctly without some kind of API.

Another option I thought was that we could let projects define their own pages on a specific branch, and add submodules for them in this repository. Take a look at https://eclipse-ee4j.github.io/javamail/. It already has the Jakarta EE look and if we could manage to get it shown under https://jakarta.ee/projects/javamail we'd be done.

Of course, some projects will prefer to use their gh-pages for some other purposes, but we can define a "jakartaee-pages" convention, where projects place just the contents, without Hugo configs, and add them as submodules here, rendering it with the common theme. Every Jakarta EE project would have that space granted on the official page, with its contents controlled by its committers.

What do you think?

[chrisguindon]

I think getting it from Eclipse project pages is best, although difficult to parse correctly without some kind of API.

+1 We are planning to create an API for sharing Eclipse projects data for a different purpose:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=548884#c1

This an MBO for my team and we are expected to get this done before the end of Q3. If we want to push this new page sooner rather than later, we should focus on the original plan of leveraging metadata from the project github repository.

We could then plan on using data from projects.eclipse.org once the Api is published.

Do we have a list required metadata that we want to include on our project pages? If not, we should define a list that we must include in our API:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=548884#c1

What do you think?

[jgallimore]

See #371 which fixes these conflicts and merges in #344