Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scheduler: calculate max disk usage based on user prefs correctly. #4923

Merged
merged 1 commit into from Sep 28, 2022

Conversation

davidpanderson
Copy link
Contributor

If max_used_gb or max_used_pct are zero (in client prefs) treat as "no limit".

I haven't tested this.

If max_used_gb or max_used_pct are zero (in client prefs) treat as "no limit".
@codecov
Copy link

codecov bot commented Sep 15, 2022

Codecov Report

Merging #4923 (3a3ce9f) into master (52ccadd) will decrease coverage by 0.00%.
The diff coverage is 0.00%.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #4923      +/-   ##
============================================
- Coverage     10.43%   10.43%   -0.01%     
  Complexity     1046     1046              
============================================
  Files           278      278              
  Lines         35829    35831       +2     
  Branches       8114     8114              
============================================
  Hits           3739     3739              
- Misses        31719    31721       +2     
  Partials        371      371              
Impacted Files Coverage Δ
sched/sched_config.cpp 0.00% <0.00%> (ø)
sched/sched_send.cpp 0.00% <0.00%> (ø)

@AenBleidd
Copy link
Member

@lfield, would be nice if you could take a look

@lfield
Copy link
Contributor

lfield commented Sep 21, 2022

This branch has been built and the updated cgi binary has been updated for our Dev project.

@lfield
Copy link
Contributor

lfield commented Sep 27, 2022

It seems to be working fine on LHC@home dev. I am not sure how to test the actual fix as we don't have tasks requiring over 100GB of disk.

@Vulpine05
Copy link
Contributor

This may be a dumb question, but is it possible to generate a task with a lot of bloat to get to 100 gb? Probably easier said than done.

@lfield
Copy link
Contributor

lfield commented Sep 28, 2022

I updated the template of one of our apps so that rsc_disk_bound is greater that 100GB. This is reflected in the WU description.

Max Disk Usage | 104,904.17 MB

In the client I first set the 'Use no more than' option to 10GB and no task was received. I then set this value to zero and received a task. The client is shown bellow.

Wed 28 Sep 2022 04:06:15 PM CEST |  | No general preferences found - using defaults
Wed 28 Sep 2022 04:06:15 PM CEST |  | Reading preferences override file
Wed 28 Sep 2022 04:06:15 PM CEST |  | Preferences:
Wed 28 Sep 2022 04:06:15 PM CEST |  | max memory usage when active: 14064.96 MB
Wed 28 Sep 2022 04:06:15 PM CEST |  | max memory usage when idle: 14064.96 MB
Wed 28 Sep 2022 04:06:15 PM CEST |  | max disk usage: 10.00 GB
Wed 28 Sep 2022 04:06:15 PM CEST |  | (to change preferences, visit a project web site or select Preferences in the Manager)
Wed 28 Sep 2022 04:07:47 PM CEST | lhcathome-dev | update requested by user
Wed 28 Sep 2022 04:07:49 PM CEST | lhcathome-dev | Sending scheduler request: Requested by user.
Wed 28 Sep 2022 04:07:49 PM CEST | lhcathome-dev | Requesting new tasks for CPU
Wed 28 Sep 2022 04:07:50 PM CEST | lhcathome-dev | Scheduler request completed: got 0 new tasks
Wed 28 Sep 2022 04:07:50 PM CEST | lhcathome-dev | No tasks sent
Wed 28 Sep 2022 04:07:50 PM CEST | lhcathome-dev | Message from server: Theory Simulation needs 2012.14MB more disk space.  You currently have 5617.25 MB available and it needs 7629.39 MB.
Wed 28 Sep 2022 04:07:50 PM CEST | lhcathome-dev | Project requested delay of 61 seconds
Wed 28 Sep 2022 04:08:43 PM CEST |  | No general preferences found - using defaults
Wed 28 Sep 2022 04:08:43 PM CEST |  | Reading preferences override file
Wed 28 Sep 2022 04:08:43 PM CEST |  | Preferences:
Wed 28 Sep 2022 04:08:43 PM CEST |  | max memory usage when active: 14064.96 MB
Wed 28 Sep 2022 04:08:43 PM CEST |  | max memory usage when idle: 14064.96 MB
Wed 28 Sep 2022 04:08:43 PM CEST |  | max disk usage: 124.92 GB
Wed 28 Sep 2022 04:08:43 PM CEST |  | (to change preferences, visit a project web site or select Preferences in the Manager)
Wed 28 Sep 2022 04:08:55 PM CEST | lhcathome-dev | Sending scheduler request: To fetch work.
Wed 28 Sep 2022 04:08:55 PM CEST | lhcathome-dev | Requesting new tasks for CPU
Wed 28 Sep 2022 04:08:56 PM CEST | lhcathome-dev | Scheduler request completed: got 1 new tasks
Wed 28 Sep 2022 04:08:56 PM CEST | lhcathome-dev | Project requested delay of 61 seconds
Wed 28 Sep 2022 04:08:58 PM CEST | lhcathome-dev | Started download of 2390-1113300-292.run
Wed 28 Sep 2022 04:08:59 PM CEST | lhcathome-dev | Finished download of 2390-1113300-292.run
Wed 28 Sep 2022 04:08:59 PM CEST | lhcathome-dev | Starting task Theory_2390-1113300-292_0

@AenBleidd
Copy link
Member

@lfield, thank you for testing this. I believe now you can merge it

@lfield lfield merged commit b98d200 into master Sep 28, 2022
Server automation moved this from Backlog to Done Sep 28, 2022
@AenBleidd AenBleidd deleted the dpa_sched_disk branch August 15, 2023 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Server
  
Done
Development

Successfully merging this pull request may close these issues.

Tasks requiring more disk space despite over 300GB free on a dedicated drive
4 participants