Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How many writes does the hourly sync cause? #64

Closed
StuartIanNaylor opened this issue Mar 12, 2019 · 2 comments
Closed

How many writes does the hourly sync cause? #64

StuartIanNaylor opened this issue Mar 12, 2019 · 2 comments

Comments

@StuartIanNaylor
Copy link
Contributor

StuartIanNaylor commented Mar 12, 2019

The service reload via the hourly cron job causes sync to disk on each hour

There is an absence of any check to drive usage and from the cp command the whole $RAM_LOG is written out to HDD.
Isn't this likely to cause vastly more HDD writes than just appending to normal logs in many situations?

It completely lacks any form of control, probably should run more frequently but with logical tests if log usage necessitates a sync, but just to overwrite everything each hour?

`syncToDisk () {
isSafe

if [ "$USE_RSYNC" = true ]; then
    rsync -aXWv --delete --links $RAM_LOG/ $HDD_LOG/ 2>&1 | $LOG_OUTPUT
else
    cp -rfup $RAM_LOG/ -T $HDD_LOG/ 2>&1 | $LOG_OUTPUT
fi

}`

I guess somrthing like sudo sed -i.bak '/^\srotate ./i olddir /var/log/oldlog' inputfile
But I am far from accomplished with sed

@azlux
Copy link
Owner

azlux commented Mar 12, 2019

This project has been made for SD card. In case of HDD, your aren't limited to write cycle.
With a SD card, Less you write, better it is. Even if you have more data to write.

@StuartIanNaylor
Copy link
Contributor Author

StuartIanNaylor commented Mar 12, 2019

HDD as apposed to Ram irrespective of if media is SD, Nand or Emmc still doesn't change the fact that writing out complete logs on any change every hour, especially as those logs increase in size, means yes frequency of write is lowered, but the write size is often huge in comparison.

It is dependent on log size but with the application of the sphere of log2ram of the general Pi & clone world of Arm SBC computers its debatable if L2R makes much difference in frequency in comparison to write Size.
I am not hear to debate the merits of block wear and the above was my first comment on the main question of why does it write every hour when its completely not needed.

The ramlog only needs to be written to hdd.log as it is programmatically named on stop.
I commented further in #65 as also ramlog doesn't prune and holds pointless old logs in memory which again with the oldlog directive of logrotate can be easily partitioned and moved when total ramlog usage is at a certain level.

Reducing hourly writes by sometimes as little as 3 or 4 but increasing data write size upwards of a 1,000 is likely no better and maybe even worse.
What makes that worse is there is absolutely no need to write every hour.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants