Ticket26937: check for free disk space by juga0 · Pull Request #241 · torproject/sbws

juga0 · 2018-07-25T23:07:53Z

No description provided.

pastly

I kind of like the idea but I don't understand the size assumptions made in sbws_required_disk_space().

I also don't know why two specific log messages were picked as places to catch an exception when we think that maybe we ran out of disk space. Couldn't this happen every time we log? Shouldn't we therefore catch this exception at every log message? (Please do not write code to do that)

If I understood the logic in calculating the necessary disk space, agreed that it is reasonable logic, and sbws only checked at the top of main for having enough space, then I would like this PR better.

pastly · 2018-08-01T16:18:51Z

+    """
+    # Number of relays per line average size in Bytes
+    size_v3bw_file = 7500 * 220
+    num_v3bw_files = int(conf['general']['data_period'])


data_period is the number of days into the past that we consider measurements valid. Why does this line assume that the number of v3bw files will be the same as this value?

I think that i intended to calculate size of the minimum number of results files to keep, not v3bw files.
For v3bw files, it would be enough to keep only the last 2 ones that should have been generated in the last 2 hours.
This is going to depend a lot on the crontab that will generate/clean files and would be provided separately, but at least this would give an estimation on the minimum space required

pastly · 2018-08-01T16:21:12Z

+    # not counted rotated files and assuming that when it is not rotated the
+    # size will be aproximately 10MiB
+    size_log_file = (int(conf['logging']['to_file_max_bytes']) or 10485760) \
+        if conf['logging']['to_stdout'] == 'yes' else 0


Shouldn't this be to_file, not to_stdout? Also, you should do if conf.getboolean('logging', 'to_file') since there are more was to indicate a true value than the word "yes" https://docs.python.org/3/library/configparser.html#configparser.ConfigParser.getboolean

you're right, mistake

pastly · 2018-08-01T16:22:29Z

+    space_v3bw_files = size_v3bw_file * num_v3bw_files
+    # not counted rotated files and assuming that when it is not rotated the
+    # size will be aproximately 10MiB
+    size_log_file = (int(conf['logging']['to_file_max_bytes']) or 10485760) \


conf.getint('logging', 'to_file_max_bytes') instead of wrapping with int()

Since 10 MiB is specified in the config.default.ini and that file should not be edited by anyone, I do not think we should hardcode a fallback of 10 MiB here

makes sense

pastly · 2018-08-01T16:24:17Z

+
+
+def df(path):
+    """Return space left on device where path is."""


Please note that the units are MiB

pastly · 2018-08-01T16:26:53Z

+    size_log_file = (int(conf['logging']['to_file_max_bytes']) or 10485760) \
+        if conf['logging']['to_stdout'] == 'yes' else 0
+    # roughly...
+    space_result_files = space_v3bw_files


How do we know this will be close to true?

not sure now why i did that :), probably i should calculate the space they take separately, in the way commented in #241 (comment)

pastly · 2018-08-01T16:27:27Z

+    disk_avail_mb = df(conf['paths']['sbws_home'])
+    if disk_avail_mb < disk_required_mb:
+        log.warn("The space left on the device (%s MiB) is less than "
+                 "the minimum recommented to run sbws (%s MiB)."


"recommended"

also replace conf[''] for conf.getint/conf.getboolean

and add comment about MiB

juga0 · 2018-08-03T12:58:30Z

I also don't know why two specific log messages were picked as places to catch an exception when we think that maybe we ran out of disk space.

In concrete cases where i run out of space, they happened there.

Couldn't this happen every time we log?

Yes

Shouldn't we therefore catch this exception at every log message? (Please do not write code to do that)

Maybe, but we would need to change all logs, so probably we do not want that.

What about just leaving the logs in this PR?

juga0 · 2018-08-03T12:59:04Z

Fixed the issues you pointed out when calculating the space

pastly

One bug remaining, see inline comment.

I don't like the 2 logging changes where we catch OSError+print and where we make a guess in the message that we might have run out of disk space during a completely unrelated exception.

I'd merge this with the fixed bug and the 2 logging changes removed.

pastly · 2018-08-03T13:24:17Z

+    # not counted rotated files and assuming that when it is not rotated the
+    # size will be aproximately 10MiB
+    size_log_file = conf.getint('logging', 'to_file_max_bytes') or 10485760 \
+        if conf.getboolean('logging', 'to_stdout') else 0


This is supposed to be to_file.

sorry, forgot, fixed

This reverts commit 72b45d4.

This reverts commit ede9fed. An exception trying to write to log file when there is not disk space can happen anywhere in the code

juga0 · 2018-08-04T05:18:50Z

I don't like the 2 logging changes where we catch OSError+print and where we make a guess in the message that we might have run out of disk space during a completely unrelated exception.

In 72b45d4#diff-814bf53a84b89545a05dd2761603efc4R68, we are trying to close a circuit that we can not even get, i don't think it's totally unrelated exception. But removing it.

juga0 · 2018-08-04T05:19:40Z

If now it's fine for you, i can rebase to master and fix the conflicts. I'd do it in a different branch.

getpath is necessary so that ~ is expanded GH: closes #241 trac: implements #26937

juga0 added 7 commits July 25, 2018 22:58

Warn when exception might be caused by no disk space

72b45d4

Exit on OSError when building circuit

ede9fed

Add functions to check disk space

9ed2dec

Add unit tests for disk space functions

149a4f6

Check disk space before running scanner

8091ba6

Check disk space before running generate

c2d2815

Add changelog entry about disk space warning

cdd5e39

juga0 added the enhancement New feature or request label Jul 25, 2018

pastly suggested changes Aug 1, 2018

View reviewed changes

pastly changed the title ~~Ticket26937~~ Ticket26937: check for free disk space Aug 1, 2018

juga0 added 5 commits August 3, 2018 12:52

Calculate space for result files and tor dir

20d1af9

also replace conf[''] for conf.getint/conf.getboolean

Update the total

1c7f76c

and add comment about MiB

Specify the unit is MiB

cff5381

Fix typo

378194a

Increase space needed in test

813f82f

pastly suggested changes Aug 3, 2018

View reviewed changes

juga0 added 3 commits August 4, 2018 05:13

Size of log file, no stdout

5f29699

Revert "Warn when exception might be caused by no disk space"

c389ad9

This reverts commit 72b45d4.

Revert "Exit on OSError when building circuit"

8190ba8

This reverts commit ede9fed. An exception trying to write to log file when there is not disk space can happen anywhere in the code

pastly closed this in 5e363d8 Sep 5, 2018

pastly mentioned this pull request Sep 5, 2018

Ticket26937 02 (docs about system requirements) #249

Closed

pastly added a commit that referenced this pull request Sep 5, 2018

Use conf.getint and getpath

93e454e

getpath is necessary so that ~ is expanded GH: closes #241 trac: implements #26937



		def df(path):
		"""Return space left on device where path is."""

Conversation

juga0 commented Jul 25, 2018

Uh oh!

pastly left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juga0 commented Aug 3, 2018

Uh oh!

juga0 commented Aug 3, 2018

Uh oh!

pastly left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juga0 commented Aug 4, 2018

Uh oh!

juga0 commented Aug 4, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants