Create a p2 repo with PDE content

[mickaelistria]

No description provided.

[github-actions]

Test Results

     270 files  ±0       270 suites  ±0   43m 28s ⏱️ - 2m 52s
  3 329 tests ±0    3 299 ✔️ ±0  30 💤 ±0  0 ❌ ±0 
10 284 runs  ±0  10 194 ✔️ ±0  90 💤 ±0  0 ❌ ±0 

Results for commit 9378d25. ± Comparison against base commit 404a30f.

♻️ This comment has been updated with latest results.

[HannesWell]

Thanks Mickael for your effort, but can we please first work-out a more concrete plan and more specific what the goals are.
Because without publication buildung a repo just increases the build times.

[laeubi]

do we need the tests in the site? if yes can we probably move them to a dedicated category?

[laeubi]

Looks good but maybe a categorization (PDE / Source / Test) would be good, also we should add a deploy step to the Jenkinsfile when build on master.

[HannesWell]

Looks good but maybe a categorization (PDE / Source / Test) would be good, also we should add a deploy step to the Jenkinsfile when build on master.

This depends on how PDE's test should be run in what are now the integration-tests of the I-builds.

also we should add a deploy step to the Jenkinsfile when build on master.

Yes, but we need the target URL for that.

Furthermore we have to decide if we want to add all dependencies as references to make it virtually self-contained:
https://github.com/eclipse-tycho/tycho/blob/master/RELEASE_NOTES.md#new-option-to-filter-added-repository-references-when-assembling-a-p2-repository-1

Additionally I think it would be good to move the category.xml into a dedicated project (like o.e.eclipse.pde.repository) in the releng folder of the repo where we can also add a target for PDE.

Because all of these questions I think it would be good to first have a more concrete plan what we want to achieve on the short and long term.

[merks]

We do seem to be putting the cart before the horse here...

Here's a first attempt at a dependency graph.

JDT depends on PDE, but only for the tests:

There are many more Platform dependencies on PDE. Forgetting for the moment about feature and test dependencies (there are many of those) for the moment, I see these:

Probably we need a more coherent overall plan and strategy that is well-informed and is based on a good factual and detailed understanding of the dependencies so that we can understand the end goal and how best to achieve it.

[mickaelistria]

We do seem to be putting the cart before the horse here...

We do need to start somewhere. The repository was the simplest entry point.

About the e4 tools, shouldn't we move those to PDE?

[iloveeclipse]

We do need to start somewhere. The repository was the simplest entry point.

Not the plan what we actually want to achieve & which steps do we need for that?

Probably we need a more coherent overall plan and strategy that is well-informed and is based on a good factual and detailed understanding of the dependencies so that we can understand the end goal and how best to achieve it.

I fully agree with that.

[laeubi]

About the e4 tools, shouldn't we move those to PDE?

I think this was mostly because of some spie implementations, in general I agree that we should move things where they belong, Platform/E4 is already a horrific dependency hell ... I think you recently moved already some file with history from one repo to another, maybe you can share the exact commands used here so it lowers the barrier for people to perform such move.

Beside that, platform depends on many stuff, nevertheless we build "the whole world" in the aggregator, so even if there are dependencies this does not mean we can't pull the thing out of the aggregator build.
Yes it might make things a bit more complicated if we want to change things, but that's the case for all other projects out there as well ... that's why we have APIs.

[mickaelistria]

For the context, this PR comes as a reaction to eclipse-platform/eclipse.platform.releng.aggregator#1472 where the "plan" (or more the brainstorming at this stage) is discussed.

[akurtakov]

I strongly believe we should accept such changes and use their output to form fact based decisions. After all as long as no modifications to content at https://download.eclipse.org/eclipse/downloads/index.html happens due to these changes they are harmless.

[mickaelistria]

@laeubi About moving subparts of a Git repo with history, I've posted an how-to on the mailing-list: https://www.eclipse.org/lists/eclipse-dev/msg12214.html

[laeubi]

I strongly believe we should accept such changes and use their output to form fact based decisions.

@mickaelistria can you please categorize the items and add a deployment to the eclipse server?
Then I think we can merge this a first step.

Furthermore we have to decide if we want to add all dependencies as references to make it virtually self-contained:

I think it is enough if it contains everything produced by PDE for now.

Additionally I think it would be good to move the category.xml into a dedicated project

It already is a dedicated project named "repository" what seems suitable as we most likely won't produce multiple sites

in the releng folder of the repo

Conceptually updatesites don't belong to the releng folder, if we want to follwo structured layout they are located in a folder called sites

where we can also add a target for PDE.

Targets belong to the targets folder or are best placed in the root (as its unlikely to have many of them) as of for a long time Tycho do not require them to be a separate target artifact.

[HannesWell]

I strongly believe we should accept such changes and use their output to form fact based decisions.

But what information do we gain from this that we don't have yet?

After all as long as no modifications to content at https://download.eclipse.org/eclipse/downloads/index.html happens due to these changes they are harmless.

That's right, but I spend a lot of time to clean stuff up and I therefore would like to avoid having new things, that might need clean-up again in case the aggregator split is not fully executed.
As far as I read the discussion at eclipse-platform/eclipse.platform.releng.aggregator#1472 there is not yet even an agreement on the overall goal nor on the short term steps etc?
If we just start doing things and maybe not complete the work, we will end up with an even greater mess (what we currently have plus the stuff done from now on).

I don't say this is wrong, I just say I think it is too early for this now, but I'm happy to leave this open.

And when we have agreed upon that PDE needs to publish into its own p2-repo, there are the following questions that have to be answered and should be implemented with this PR:

• Where to publish a PDE p2-repo?
  https://download.eclipse.org/eclipse/pde/ already exists but has some +10 years old content that mostly looks like it can be removed.
• What should be the repository structure with respect to snapshots and releases?
  Should we do it like in M2E: https://download.eclipse.org/technology/m2e/. I think that would be fine.
• How to retain snapshot builds? Just the latest (like m2e) or should we keep them until the next release like I-builds?
  From the SimRel requirements I think retaining each snapshot till the next release is necessary otherwise one has to create a milestone repo separately and it has to be made sure that no other build is currently mirroring the repo while it is wiped out between builds.
  • Will the aggregator just reference the pde-repo? Will it mirror its content?

Conceptually updatesites don't belong to the releng folder, if we want to follwo structured layout they are located in a folder called sites

Isn't the term site deprecated? And what is then left for releng?

Targets belong to the targets folder or are best placed in the root (as its unlikely to have many of them) as of for a long time Tycho do not require them to be a separate target artifact.

If Tycho just requires the path/file name, can't it just point down into a folder? Having the target in a .project allows to import the target-project without the need to import the entire root project in the workspace.

[laeubi]

Isn't the term site deprecated?

Don't think so, site.xml is no longer supported but one still has (p2) update-sites ...

And what is then left for releng?

releng is not a standard folder recognized by Tycho and is usually not required.

[laeubi]

@mickaelistria can you rebase?

As it is mentioned here it seems quite consensual that "we need a p2 site", so should we proceed on this ad if yes I think it should include a deployment step on master branch in Jenkinsfile.

[mickaelistria]

I've rebaed and added some configuration to Jenkinsfile so repo would be available from ci.eclipse.org. That should allow easy testing of latest master at least.
I think the policy of where to deploy it, when and so on should then be discussed separately, as it will involve another family of questions.

[mickaelistria]

With this change, a p2 repo is available att https://ci.eclipse.org/pde/job/eclipse.pde/job/PR-826/3/artifact/repository/target/repository/ . When this gets merged then https://ci.eclipse.org/pde/job/eclipse.pde/job/master/lastSuccessfulBuild/artifact/repository/target/repository/ will contain latest build output (and then we can test it a bit more easily and discuss whether todeploy it somewhere like download.eclipse.org/pde )

[akurtakov]

The test failures are actually compile warnings and are caused by respecting project settings now. Merging this one.