Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Users of the SpringSource EBR need Spring framework bundles to continue to be published there for Spring 3.2 [SPR-8903] #13543

Closed
spring-issuemaster opened this issue Dec 8, 2011 · 19 comments
Assignees

Comments

@spring-issuemaster
Copy link
Collaborator

@spring-issuemaster spring-issuemaster commented Dec 8, 2011

Glyn Normington opened SPR-8903 and commented

The planned implementation of #12770 for Spring 3.2 excludes support for publishing Spring framework bundles to the SpringSource Enterprise Bundle Repository (EBR). The subset of the OSGi community that uses Spring, which includes the communities surrounding the popular Eclipse Virgo and Gemini Blueprint projects, will be impacted if such support is not put in place or some suitable workaround found.

The particular value of publishing Spring framework bundles to the EBR is that the EBR captures those bundles' transitive dependencies on "bundleised" versions of 3rd party libraries such as Hibernate which are present in the EBR. If Spring framework 3.2 bundles are omitted from the EBR, then users will need to resort to educated guesswork to determine the correct "bundleised" transitive dependencies.

The purpose of this JIRA is to track the work to implement or work around the issue, but also to raise awareness of the issue among the user community and to encourage discussion.

Although this feels more like a blocker for the OSGi/Spring community, I have raised it as critical to provide a more balanced view from the perspective of the much larger Spring community.

BTW let's not use this issue to discuss the rationale for switching to Gradle - that's the subject of #12770.


Affects: 3.2 M1

Issue Links:

  • #14434 Migration to gradle build lost OSGi headers ("is duplicated by")

8 votes, 22 watchers

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Dec 8, 2011

Chris Beams commented

Just a few thoughts to get the ball rolling here:

As Glyn and I have discussed offline, the critical difficulty in publishing artifacts to standard maven repositories such as Maven Central as well as to EBR is that two sets of dependency metadata must be maintained; one containing EBR-style artifact naming, one containing "Maven Central style" naming.

The mapping between these artifacts is not predictable. For example, while one could determine the EBR coordinates org.hibernate:com.springsource.org.hibernate.core:4.0.0.CR4 from the original Maven Central coordinates org.hibernate:hibernate-core:4.0.0.CR4, one cannot do the same for org.hibernate:com.springsource.org.hibernate.ejb:4.0.0.CR4 and org.hibernate:hibernate-entitymanager:4.0.0.CR4. So while in some cases EBR G/A/V/C coordinates may be possible, in other cases they are certainly not.

Furthermore, EBR metadata is expressed in ivy.xml format, while Spring projects built with Maven or Gradle express their dependencies in POM and .gradle formats respectively. Manually maintaining separate sets of metadata is extremely error prone and not tenable going forward for Spring 3.2.

Therefore, what is required is (1) a mapping between Maven Central and EBR G/A/V/C coordinates, (2) the automatic generation of ivy.xml metadata based on canonical gradle dependency metadata and (3) automatic publication of project artifacts and ivy metadata to EBR whenever artifacts are published to Maven Central (or our own artifact repository at repo.springsource.org).

Items (2) and (3) are relatively straightforward to accomplish. Item (1) is the serious challenge.

To plainly state the requirements as we currently understand them:

  • The Virgo team (and Spring+OSGi+EBR community at large) need Spring artifacts published to EBR as quickly as possible, preferably simultaneously with the publication of those artifacts to Maven Central or repo.springsource.org. The publication to EBR needs not only to be timely, but accurate, complete and tested.

  • The Spring team requires that only one set of dependency metadata be maintained at the build script level, and that the format of that metadata is native to the build system being used, namely Gradle. Given that the vast majority of Spring users consume artifacts from Maven Central, the balance of concern from the Spring team's perspective for dependency metadata correctness is on the Maven Central style variants.

With these requirements laid out, let me sketch a potential solution:

  • Maintain the mapping between Maven Central style G/A/V/C coordinates and EBR within EBR itself, and make that metadata queryable via an HTTP API
    This addresses point (1) above.
  • Implement (2) and (3) above not within individual project build scripts, but as an Artifactory plugin in our repo.springsource.org repository manager.

Maintaining the mapping between Maven Central and EBR G/A/V/C coordinates at the individual project build script level is redundant and highly error-prone. It would be far better to maintain this metadata within EBR itself. EBR would need to be updated to accommodate this metadata, and as mentioned, to query it.

Going forward, the plan is to have all Spring projects publish their artifacts to our Artifactory instance at http://repo.springsource.org. As mentioned, we can extend Artifactory through plugins. One can imagine a plugin that upon upload of a Spring project artifact, looks up that artifact and its dependencies against EBR's new HTTP API, generates appropriate ivy metadata, copies and renames artifacts as appropriate, and ultimately publishes those EBR-style artifacts and metadata to EBR. Should a transitive dependency not be found within EBR, the Artifactory plugin would have the capability of failing the deployment with an error message letting the user know that there is a dependency missing in EBR, etc.

Again, this is a quick sketch hastily written, but I want to set the stage here for as hands-free an EBR experience as possible going forward. The major benefits of the solution above are that (a) it centralizes mapping metadata within EBR itself and (b) centralizes publication to EBR within Artifactory. This means a high degree of quality control is possible; that all spring projects would automatically publish to EBR -- not just Spring Framework; and that no Spring project build script will be burdened with the complexity of EBR deployment going forward.

The approach as described above will require an internal (as opposed to community-contributed) effort and appropriate staffing. We welcome this effort, and I am personally happy to share my expertise around Artifactory plugin authoring with whomever may be tasked with the implementation.

Alternative proposals are welcome, but there is a hard requirement for the Spring team around maintaining a single source of dependency metadata going forward in Spring 3.2. Maintaining multiple sets of dependency metadata has been a major hassle, and worse, has often led to incorrect metadata. I'm sure we can find a solution that allows for both sets of needs to be met. The above is one possible way.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Dec 8, 2011

Glyn Normington commented

Very helpful Chris. Thanks!

Given that the current EBR external content is now evolving very slowly, if at all, because it requires manual effort, I think it would be sufficient to define a mapping function for item (1) as part of the Spring build. The mapping function would encode all the mapping conventions plus exception cases currently present in the EBR.

Those of us who occasionally fiddle with the EBR would then be under a strict requirement to obey those mapping conventions or be responsible for updating them. However, most external content that we put in the EBR these days is in order to build Virgo and Gemini Web and is not mapped, i.e. we are storing existing bundles in the EBR, and so mapping is trivial in those cases.

I think this will provide a robust solution without needing to maintain the mapping in the EBR, which would require EBR-side effort and would require the mapping data format to be shared between the EBR and the Spring build.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Mar 8, 2012

Chris Beams commented

Transitioned from "Story" to "Task" in preparation for an update to our JIRA Issue Type scheme

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Oct 23, 2012

Chris Beams commented

http://www.springsource.org/springframework-ebr describes what OSGi users can expect with regard to Spring Framework 3.2 bundles making their way into the EBR.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Oct 24, 2012

Michael Pilquist commented

For anyone using bnd to assist in manifest generation, its important to note that you'll need to build against the bundles in EBR in order to have accurately versioned package imports. Specifically, this is a compile time classpath problem in addition to a deployment problem. This can be quite a bit of work b/c you'll need to exclude the standard Spring artifacts from any third party dependencies that depend on Spring.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Oct 24, 2012

Raman Gupta commented

This all seems a lot more complex than simply including the OSGi meta-data in the main Spring artifacts. That meta-data is invisible to non-OSGi users anyway. Is there a particular reason that EBR needs to come into this at all? As I understand it, the original goal of EBR was to provide compatibility for artifacts that did not provide the meta-data upstream, not to become a parallel Maven Central with OSGi versions of every artifact.

With this approach, in addition to the extra complexity, problems with the OSGi versions are inevitable, as the Core Spring developers may not consider the package-level visibility and dependency management that OSGi requires (which in the end results in better code even for non OSGi users) -- problems will only arise at OSGi packaging time after the initial developer has already moved on to other things. In addition, with OSGi as a separate layer, the main Spring build cannot include the appropriate tests to verify the OSGi functionality at build time.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Oct 24, 2012

Raman Gupta commented

And, as a follow-up, nor can the main Spring build integrate with OSGi in any deeper sense than meta-data, such as exposing and consuming OSGi services.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Oct 24, 2012

Pascal Leclercq commented

Hi,

as Glyn suggest on twitter, I add my 2 coins :

I don't get the point of having spring modules duplicated in EBR. IMHO having spring modules on central maven repo is sufficient.

I don't know ivy that much but AFAIK It is made to consume maven repo/dependencies.

From my experience, EBR is helpfull when a third party jar like hibernate doesn't provide OSGi metadata.

Maybe I missed something but a lot of people perceive this decision to abandon OSGI (see : http://www.infoq.com/news/2012/10/spring-osgi-gradle). I think this is really bad....

Unless I missed something, wouldn't It helps to have one single build system such as gradle (or maven) ?

Best regards.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Oct 24, 2012

Martin Ellis commented

There's another problem with not including OSGi manifests as part of the canonical build that doesn't seem to have been mentioned in this ticket yet (at least, not explicitly)...

If the canonical build doesn't add OSGi manifests, then it makes it difficult for OSGi users to rebuild bundles. There are several reasons people might need to do that. For example, they might want to build:

  • a snapshot version of an artifact, to pick up a new feature or bug fix;
  • a snapshot version prior to a release, to help with testing;
  • a version with proprietary changes or workarounds for something (This is ugly, but it's sometimes a useful. I know one of our clients has in-house variants of some Spring artifacts);
  • a recent version when the version in EBR has fallen behind the latest version.

Hence, it's useful to have the standard build produce an OSGi bundle (which is just a jar after all).

Another reason to include OSGi manifests in the artifacts on central is that it makes it easier for people to find the artifacts they're looking for, because:

  • There's one fewer places to look for bundles.
  • No confusion over whether a version is just a vanilla jar or bundle.
  • Maven central has an index that IDEs and repository managers can read and use 1. EBR does not.
  • Artifact groups in central are browsable. Not so in EBR, which can be rather awkward.
@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Dec 17, 2012

Chris Beams commented

Per EBR-890, Spring Framework 3.2.0.RELEASE is now available via the SpringSource EBR.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Dec 27, 2012

Hendy Irawan commented

Linked by ##14434.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented May 29, 2013

Matt Bishop commented

I just love the Spring team's mentality--let's not bother with maven central, let's make our own! Why? because then it will have the word "spring" in it!

Seriously, this brings me back around to looking at alternatives to IOC in OSGi, like pure Blueprint or Decl Services, even something radical like Sisu. The Spring team doesn't appear to want the OSGi business any more.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 29, 2013

Olaf Otto commented

Re-opening this as the OSGi artifacts for SPR 3.2.5.RELEASE are not present in neither Springsource EBR nor Maven central.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 29, 2013

Glyn Normington commented

The SpringSource EBR was made read-only in September, so Spring Framework 3.2.x (and any other) releases after that date will not be published there. Please refer to the FAQ and the announcement over in the Eclipse Bundle Recipes project which should be the successor to the SpringSource EBR.

SpringSource made available the necessary information for the community to take over creation of OSGi bundles for Spring Framework, so you may like to collaborate with others in the creation of Spring Framework 3.2.5 bundles which could then be published to Maven Central.

Hope that helps.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 29, 2013

Krzysztof Sobkowiak commented

The mentioned necessary information contains reference to scripts preparing the build. These scripts use already existing set of templates/licenses. Where are the existing templates?

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 29, 2013

Krzysztof Sobkowiak commented

I think you mean this http://git.eclipse.org/c/ebr/org.eclipse.ebr.recipes.git/tree/contrib/SpringSource-EBR-templates. There are templates for bundles from EBR repository. But the templates for Spring Framework and Spring Security are missing there. I cannot find there the directories using in the build scrip mentioned in the instruction. In the script there (http://git.eclipse.org/c/ebr/org.eclipse.ebr.recipes.git/tree/contrib/SpringSource-EBR-scripts/org.springframework/copy_springframework_into_artifacts.bash) are commands copying templates from old tags into the new created tags

 
cp org.springframework.aop/${OLD_TAG}/* org.springframework.aop/${TAG}/
cp org.springframework.aspects/${OLD_TAG}/* org.springframework.aspects/${TAG}/
cp org.springframework.beans/${OLD_TAG}/* org.springframework.beans/${TAG}/
cp org.springframework.context.support/${OLD_TAG}/* org.springframework.context.support/${TAG}/
cp org.springframework.context/${OLD_TAG}/* org.springframework.context/${TAG}/
.....

Where can I find in this repositories these tags with the templates for each Spring project or another location with the templates for Spring Framework?

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 29, 2013

Glyn Normington commented

It seems that those files were omitted from the contribution. I have raised Eclipse bug 422866 to handle this.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Dec 25, 2018

Sébastien Deleuze commented

Resolving this outdated issue as won't fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.