Skip to content

50 Add cross compilation ability#52

Merged
Zejnilovic merged 5 commits intomasterfrom
feature/50-add-cross-compilation
Dec 18, 2020
Merged

50 Add cross compilation ability#52
Zejnilovic merged 5 commits intomasterfrom
feature/50-add-cross-compilation

Conversation

@Zejnilovic
Copy link
Copy Markdown
Collaborator

Uses @wajda 's cross-compilation plugin adds 2.11 and 2.12 compilations

Fixes #50

Comment thread README.md
Copy link
Copy Markdown
Collaborator

@dk1844 dk1844 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will it play nice with the release plugin?
We probably want to release both 2.11 & 2.12 versions with the same version number, so at-once somehow. I wonder if we can.

Comment thread pom.xml
<scala.binary.version>2.11</scala.binary.version>

<!-- Frameworks and libraries -->
<json4s.version>3.5.3</json4s.version> <!-- This version is set to be compatible with Spark 2.4.5 -->
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment scares me a bit.

I have a hunch that we don't know if this will work with Spark 3, yet.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I am trying to take it one problem at a time. Also, I think this was a problem from 2.2 to 2.4. Version 3 could be fine.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spline agent is being tested with Spark 3.0. AFAIK our commons Json4s wrapper works fine in all versions

@Zejnilovic
Copy link
Copy Markdown
Collaborator Author

Will it play nice with the release plugin?
We probably want to release both 2.11 & 2.12 versions with the same version number, so at-once somehow. I wonder if we can.

This is a setup used by our commons so it should

@wajda
Copy link
Copy Markdown

wajda commented Dec 14, 2020

Will it play nice with the release plugin?
We probably want to release both 2.11 & 2.12 versions with the same version number, so at-once somehow. I wonder if we can.

If we're talking about release:prepare then yes, it will play nicely with it. The trick here is to use release plugin to only take care about versioning (i.e. increment version, create tag in VCS etc). For that you don't care about scala version simply because you don't compile/deploy/publish anything, you only work with the codebase itself, which is the same both all scala binary versions. Even though technically release plugin needs to compile the project, it's enough for the build to pass for any (default) scala ver. The assumption is that the actual testing per every Scala ver is done from the same commit in the CI pipeline before conducting the release stage.

The actual deployment is than done from the tag by running several builds, one per Scala version.
For instance, this is a respective TeamCity build conf for Commons project - https://teamcity.jetbrains.com/admin/editBuildRunners.html?id=buildType:OpenSourceProjects_AbsaOSS_Commons_AutoDeployToOssrh

Comment thread build-all.sh Outdated
Comment thread .gitignore
# syntax: regexp
# ^\.pc/

pom.xml.bkp
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A side effect of the cross compilation plugin

Comment thread README.md Outdated
Comment thread README.md Outdated
Co-authored-by: David Benedeki <14905969+benedeki@users.noreply.github.com>
@Zejnilovic Zejnilovic merged commit 68c813f into master Dec 18, 2020
@Zejnilovic Zejnilovic deleted the feature/50-add-cross-compilation branch December 18, 2020 08:51
@Zejnilovic Zejnilovic restored the feature/50-add-cross-compilation branch December 18, 2020 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add scala cross-compilation for 2.11 and 2.12

6 participants