Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Baseline, examples] Migrate to Flax (or other NN libraries) #1059

Open
sotetsuk opened this issue Oct 4, 2023 · 3 comments
Open

[Baseline, examples] Migrate to Flax (or other NN libraries) #1059

sotetsuk opened this issue Oct 4, 2023 · 3 comments

Comments

@sotetsuk
Copy link
Owner

sotetsuk commented Oct 4, 2023

image
@lockwo
Copy link

lockwo commented Nov 4, 2023

How important is Flax to the migration? I've been exploring pgx and it seems really nice and I've started using it, but my go to is equinox (https://github.com/patrick-kidger/equinox, probably the second most popular behind flax). I would be down to work on this issue, as long as you are ok with equinox instead of flax.

@sotetsuk
Copy link
Owner Author

sotetsuk commented Nov 4, 2023

Thank you for your interest in pgx! 🙏

Whether it's Flax or not isn't particularly important. I simply considered it as the first candidate because I thought it's the framework used by the largest community.

I think Equinox is a good choice. The issue I had with Haiku was that batch normalization couldn't handle dimensions that were made by vmap, so inevitably, I had to initialize models that included batch dimensions. Equinox doesn't have this issue, so I think it's wonderful.

If you could send a PR, I would greatly appreciate it. If you send a PR, I'll check the following points on my end and then merge:

  • A decrease in NN inference speed due to the switch from Haiku to Equinox is acceptable up to about double the time.
  • It would be preferable to have similar learning performance with 400 iterations on Go 9x9 (although I think a few percent difference would be acceptable).

I believe this will be a relatively significant change, so from a review standpoint, it would be preferable if the PR could focus on changes to the NN module and keep other changes to a minimum.

@sotetsuk
Copy link
Owner Author

sotetsuk commented Nov 4, 2023

At the very least, there will be no members in our team working on this issue by the end of the year, so it's fine for you to proceed at your own pace 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants