Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Multi gpu tests and predict_tile error #646 #649

Closed
wants to merge 2 commits into from
Closed

Conversation

bw4sz
Copy link
Collaborator

@bw4sz bw4sz commented Apr 1, 2024

This is still a work in progress, closes #646 and #467.

  • create a test that reproduces and can bypass if not multi-gpu
  • identify source of error.

@bw4sz bw4sz changed the title Multi gpu tests and predict_tile error #646 [WIP] Multi gpu tests and predict_tile error #646 Apr 1, 2024
@bw4sz
Copy link
Collaborator Author

bw4sz commented Apr 3, 2024

Profile results of moving cpu() call to the end of the prediction epoch.

# Profile predict_tile
from deepforest import main
import time

# Profile predict_tile
path = "/orange/ewhite/NeonData/OSBS/DP3.30010.001/neon-aop-products/2023/FullSite/D03/2023_OSBS_7/L3/Camera/Mosaic/2023_OSBS_7_406000_3291000_image.tif"
m = main.deepforest()
m.use_release()

start_time = time.time()
m.predict_tile(path)
end_time = time.time()

execution_time = end_time - start_time
print(f"Execution time: {execution_time} seconds")

On HPC with 2 available GPUs.
Original: (fails due to multi-gpu)

PR: 43.12

@bw4sz bw4sz closed this Apr 3, 2024
@bw4sz bw4sz deleted the multi_gpu_tests branch April 3, 2024 18:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

predict_tile() output is incorrect when multiple GPUs are present
1 participant