Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add GPU acceleration in advanced settings #1646

Closed
Tracked by #905
imtuyethan opened this issue Jan 17, 2024 · 0 comments
Closed
Tracked by #905

feat: add GPU acceleration in advanced settings #1646

imtuyethan opened this issue Jan 17, 2024 · 0 comments
Assignees
Labels
engineering: Jan Inference Layer Jan can serve models locally: with correct data structs, APIs, multi-inference engines, multi-model roadmap: Observability System Monitor, logs, observability and hardware insights status: needs designs Needs designs type: feature request A new feature
Milestone

Comments

@imtuyethan
Copy link
Contributor

Epic: https://github.com/orgs/janhq/projects/5/views/16?pane=issue&itemId=46869443

Specs

Settings / Advanced Settings

  • GPU Acceleration: [ Enable / Disable ]
    • (Info button or tooltip:Enhance model performance by utilizing your device's GPU for acceleration.)
  • GPU Options: [ Nvidia / AMD ] (visible only when GPU Monitoring is enabled)
    • Default: Auto-select GPU with highest VRAM

GPU Monitoring Toggle:

  • When toggled on, reveal the GPU selection dropdown for Nvidia and AMD.
  • Default to the auto-selected GPU with the highest VRAM.

GPU Monitoring Warning:

  • If the user toggles on GPU monitoring, provide a brief explanation of how it can improve model performance.
  • Optionally, warn that enabling GPU monitoring may have a slight impact on system resources.

GPU Selection:

  • Use a dropdown to allow users to manually select Nvidia or AMD when GPU monitoring is enabled.
  • Default to the auto-selected GPU with the highest VRAM.

Device Compatibility Check:

  • Before applying changes, check if the user's device has a compatible GPU.
  • If not, display an error message and prevent the user from enabling GPU monitoring until a compatible GPU is detected.
    (Optional: provide a suggestion or link to update GPU drivers or install a compatible GPU).

Feedback:

  • Display a success message upon applying changes.
GPU monitoring has been successfully enabled.
  • Or when user disables GPU monitoring without any specific issues.
GPU monitoring has been successfully disabled.
  • In this case, changes don't require a restart.

Error scenarios:

1. User tries to enable GPU monitoring, but their device doesn't have a compatible GPU

GPU Monitoring cannot be enabled on your device.
Your current device does not have a compatible GPU for monitoring. To enable GPU monitoring, please ensure your device has a supported Nvidia or AMD GPU with updated drivers.

2. GPU Selection with Incompatible GPU

User wants to select a specific GPU for monitoring, but that GPU is not compatible -> incompatible GPU will be disable.

3. GPU Monitoring Turned Off with Compatible GPU

User attempts to turn off GPU monitoring when they have a compatible GPU.
Tooltip when hover warning icon:

Disabling GPU Acceleration may result in reduced performance. It's recommended to keep this enabled for optimal user experience.

Image

Out-of-scope:

  • Not support AMD for now.

Design

Figma link: https://www.figma.com/file/ytn1nRZ17FUmJHTlhmZB9f/Jan-App?type=design&node-id=2303-39294&mode=design&t=cyen6gHzxMZyZEER-4

Image

@imtuyethan imtuyethan added type: feature request A new feature engineering: Jan Inference Layer Jan can serve models locally: with correct data structs, APIs, multi-inference engines, multi-model roadmap: Observability System Monitor, logs, observability and hardware insights labels Jan 17, 2024
@imtuyethan imtuyethan added this to the v0.4.7 milestone Jan 17, 2024
@imtuyethan imtuyethan self-assigned this Jan 17, 2024
@imtuyethan imtuyethan added the status: needs designs Needs designs label Jan 17, 2024
@imtuyethan imtuyethan reopened this Jan 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
engineering: Jan Inference Layer Jan can serve models locally: with correct data structs, APIs, multi-inference engines, multi-model roadmap: Observability System Monitor, logs, observability and hardware insights status: needs designs Needs designs type: feature request A new feature
Projects
Archived in project
Development

No branches or pull requests

1 participant