Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[hma][py-tx] Unclean CI status for HMA and pytx #1568

Closed
Dcallies opened this issue Mar 22, 2024 · 4 comments · Fixed by #1602
Closed

[hma][py-tx] Unclean CI status for HMA and pytx #1568

Dcallies opened this issue Mar 22, 2024 · 4 comments · Fixed by #1602
Assignees
Labels
hma Items related to the hasher-matcher-actioner system python-threatexchange Items related to the threatexchange python tool / library

Comments

@Dcallies
Copy link
Contributor

Blocking some of the new PRs, throwing onto oncall work.

HMA: https://github.com/facebook/ThreatExchange/actions/runs/8331418249/job/22798281148?pr=1563

Looks like mypy updates picking up small brokenness.

py-tx: https://github.com/facebook/ThreatExchange/actions/runs/8395103331/job/22993645426?pr=1567

VPDQ is failing to build, looks like it might be related to #1331 cc @ianwal

@Dcallies Dcallies added python-threatexchange Items related to the threatexchange python tool / library hma Items related to the hasher-matcher-actioner system labels Mar 22, 2024
@Dcallies Dcallies self-assigned this Mar 22, 2024
@ianwal
Copy link
Contributor

ianwal commented Mar 22, 2024

Yes, it looks like the dependencies for vpdq are not installed. They are listed in vpdq README.md.

For now, we can just add the dependencies where FFmpeg is installed in the workflow to fix vpdq test.


Other thoughts:

Is there a proper package manager that could be added for vpdq that is used at Meta, such as Conan? That would allow vpdq to manage its own build dependencies, preferably automatically installing them as needed (just for build stage). vpdq has no runtime dependencies.

Alternatively, we could just publish wheels for vpdq to avoid having to even build in most cases. I think cibuildwheels would work and be fairly simple to setup as an action.

@Dcallies
Copy link
Contributor Author

Thanks a ton both @ianwal and @Jemeljanov for looking into this one.

I've merged #1569, and will do something shortly to confirm it fixed things.


In response to other thoughts:

Is there a proper package manager that could be added for vpdq that is used at Meta, such as Conan

This project is unfortunately limited by what tools I've used personally. So far, that has meant setup.py and pip, but even that is already outdated. I've had limited bandwidth so far this year to make improvements to our code due to competing priorities on my team at Meta, but making the repo more maintainable is a goal I'm tracking.

Alternatively, we could just publish wheels for vpdq to avoid having to even build in most cases. I think cibuildwheels would work and be fairly simple to setup as an action.

I'll also check this out as well, I'm going to open a new issue (#1571) for exploration of both of these.

@Dcallies
Copy link
Contributor Author

Confirmed that #1569 Fixed py-tx build!

@aryzle
Copy link
Collaborator

aryzle commented Jul 10, 2024

Since old runs linked above don't show the logs anymore, here is the latest one
py-tx: https://github.com/facebook/ThreatExchange/actions/runs/9308541738/job/25622160252

HMA looks fine

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hma Items related to the hasher-matcher-actioner system python-threatexchange Items related to the threatexchange python tool / library
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants