Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify core encode decode functions to pass a dict to the prediction function #13

Open
JossWhittle opened this issue Mar 2, 2022 · 0 comments
Labels
enhancement New feature or request high priority

Comments

@JossWhittle
Copy link
Contributor

JossWhittle commented Mar 2, 2022

Currently the lowres inputs are passed directly to the prediction_fn as the only input.

  • Modify to accept a dict that has at least one key for the lowres input.

  • Provide boolean flag to also pass a positional encoding tensor along with the lowres which the model can use if needed.

  • Chunked encode decode will need to generate the correct chunks of the positional encoding for the current chunk.

  • Model can choose how to use positional encodings.

    • Image case would receive (B, H, W, 2) tensor containing the Y and X coordinates of each pixel in the trailing axis.
    • Volume case would receive (B, D, H, W, 3) tensor containing the Z, Y, and X coordinates of each voxel in the trailing axis.
@JossWhittle JossWhittle added feature enhancement New feature or request high priority and removed feature labels Mar 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request high priority
Projects
None yet
Development

No branches or pull requests

1 participant