Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Separate pipeline for tensorflow-enabled linux builds #1926

Merged
merged 2 commits into from
Jun 22, 2021

Conversation

jailuthra
Copy link
Contributor

@jailuthra jailuthra commented Jun 22, 2021

What does this pull request do? Explain your changes. (required)

Separate out the build process when libtensorflow is enabled.

Specific updates (required)

see commit history

How did you test each of these updates (required)

The CI builds + uploads work as expected.
https://build.livepeer.live/0.5.18-3aa0d8ca/livepeer-linux-tensorflow-amd64.tar.gz <-- Depends on libtensorflow.so - ldd livepeer | grep tensorflow
https://build.livepeer.live/0.5.18-3aa0d8ca/livepeer-linux-amd64.tar.gz <-- Standard build with no runtime dependency
Does this pull request close any open issues?

Fixes #1919

Checklist:

@jailuthra jailuthra force-pushed the jai/e2e-detection-testing branch 2 times, most recently from 5e3783b to edb5f31 Compare June 22, 2021 13:51
install_ffmpeg.sh Outdated Show resolved Hide resolved
install_ffmpeg.sh Show resolved Hide resolved
install_ffmpeg.sh Show resolved Hide resolved
@jailuthra jailuthra changed the title [TESTING] ci: non-tensorflow builds ci: Separate pipeline for tensorflow-enabled linux builds Jun 22, 2021
@jailuthra jailuthra marked this pull request as ready for review June 22, 2021 16:24
@jailuthra jailuthra requested a review from yondonfu June 22, 2021 16:24
Copy link
Member

@yondonfu yondonfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of my current understanding of the CI process:

  • Linux
    • Tensorflow Build
      • Always enables Tensorflow. User will need to download libtensorflow themselves to use with the binary
      • Only enables scene classification capability for non-semver tagged releases
    • Regular Build
      • Never enables Tensorflow
      • Only enables scene classification capability for non-semver tagged releases
      • Since Tensorflow is never enabled, scene classification can be enabled for a non-semver tagged release, but will not actually work
  • Docker
    • Always enables Tensorflow and libtensorflow is included in the image
    • Only enables scene classification capability for non-semver tagged releases
  • Windows
    • Never enables Tensorflow
    • Only enables scene classification capability for non-semver tagged releases
    • Since Tensorflow is never enabled, scene classification can be enabled for a non-semver tagged release, but will not actually work
  • Darwin
    • Never enables Tensorflow
    • Only enables scene classification capability for non-semver tagged releases
    • Since Tensorflow is never enabled, scene classification can be enabled for a non-semver tagged release, but will not actually work

.github/workflows/linux-tensorflow.yml Show resolved Hide resolved
Copy link
Member

@yondonfu yondonfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just want to make sure we're on the same page about this comment.

Other than that LGTM after squashing.

@jailuthra
Copy link
Contributor Author

Summary of my current understanding of the CI process:
..

Thanks for the summary, and yeah that sounds about right!
Keeping it functional only for linux + non-semver because the usecase (for now) is limited to our deployments and not to the wider community.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix run-time libtensorflow.so dependency for non-scene-classification node env
2 participants