Skip to content

Conversation

@Al2Klimov
Copy link
Member

This allows specifying less than a day as threshold. That is especially useful if history doesn't mattter as much as disk space.

I want my large (ref/NC/820479) test setup to run over WE w/o running out of disk space. So I built this.

@Al2Klimov Al2Klimov added enhancement New feature or request area/history labels Jan 24, 2025
@cla-bot cla-bot bot added the cla/signed label Jan 24, 2025
olderThan := tick.Time
wholeDays, dayFraction := math.Modf(float64(days))

if wholeDays > 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be negative? Above in line 171/172 it is already checked if days < 1.

Btw, you may want to change the abort constraint to days < 0 as otherwise you cannot use half days, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be negative?

I guess you answered your own question:

Above in line 171/172 it is already checked if days < 1.

So no, it can't be.

(If I understand you correctly.)

Copy link
Member

@julianbrost julianbrost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I remember correctly, that config attribute only exists in this way instead of being a standard time.Duration because the largest unit time.ParseDuration is hours and you'd have to configure large non-obvious values for realistic use-cases. An alternative could be to allow more advanced duration strings like for example history: 1y, then the change would also provide some real-world improvement.

And another thing to consider: could allowing very small values conflict with writing the history? Like could this trigger foreign key errors because the referenced row was already deleted? The minimum of 1 day might actually provide some safety margin there.

Comment on lines 186 to 237
olderThan := tick.Time
wholeDays, dayFraction := math.Modf(float64(days))

if wholeDays > 0 {
olderThan = olderThan.AddDate(0, 0, -int(wholeDays))
}

if dayFraction > 0 {
olderThan = olderThan.Add(-time.Duration(dayFraction * float64(24*time.Hour)))
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this actually do something more useful than just days * 24 * time.Hour? Yes, there's a slight difference with DST changes, but does it even do something useful then?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, only a limited number of days fit into time.Duration. I'm working around that limitation here.

@Al2Klimov
Copy link
Member Author

The minimum of 1 day might actually provide some safety margin there.

E.g 1h safety margin would also do it.

@Al2Klimov
Copy link
Member Author

BUT then I'd tell users to increase the threshold.

This allows specifying less than a day as threshold. That is especially useful if history doesn't mattter as much as disk space.
@oxzi oxzi changed the title Retention: switch number of days type uint16 -> float32 Retention: switch number of days type uint16 -> float64 May 15, 2025
@oxzi
Copy link
Member

oxzi commented May 15, 2025

Allowing fractions for days seems wrong. While I understand the use case, I would suggest not misusing retention-days but change this to a more generic reteion-period field, expecting a Go duration. Of course, this would need some migration, potentially making retention-days deprecated first before removing it.

However, before starting to implement such a change, some other opinions may be useful.

@lippserd
Copy link
Member

Allowing fractions for days seems wrong. While I understand the use case, I would suggest not misusing retention-days but change this to a more generic reteion-period field, expecting a Go duration. Of course, this would need some migration, potentially making retention-days deprecated first before removing it.

However, before starting to implement such a change, some other opinions may be useful.

I wouldn't change anything and just close this PR until we have a specific feature request.

@oxzi
Copy link
Member

oxzi commented Jun 17, 2025

Thanks for your second opinion.

As having a retention period of less than a day is more a development than a production requirement and that the config either "feels" wrong or requires a breaking change, I would close this now.

Thanks for all your work, @Al2Klimov. If the needs arises, we can resurrect this PR.

@oxzi oxzi closed this Jun 17, 2025
@yhabteab yhabteab deleted the retention-subdays branch June 17, 2025 08:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants