Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

issue_64_fix #258

Closed
wants to merge 9 commits into from
Closed

issue_64_fix #258

wants to merge 9 commits into from

Conversation

scrichar
Copy link
Contributor

Fixes #64 for our use

@scrichar scrichar closed this Feb 19, 2018
@scrichar scrichar deleted the issue_64_fix branch February 19, 2018 20:00
@scrichar scrichar restored the issue_64_fix branch February 19, 2018 20:06
@scrichar
Copy link
Contributor Author

Have tested locally, this fixes #64

@@ -49,6 +50,7 @@ type Destination struct {
SpoolSyncPeriod time.Duration
SpoolSleep time.Duration // how long to wait between stores to spool
UnspoolSleep time.Duration // how long to wait between loads from spool
RouteName string // how long to wait between loads from spool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this comment doesn't make any sense

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, my bad, copied from line above, will remove that

@Dieterbe
Copy link
Contributor

this PR looks pretty good. have to analyze a bit deeper though.
is it backwards compatible? do users need to specify a spool name or will it default to what it was before?

@scrichar
Copy link
Contributor Author

You do not need to specify a spool name, the spoolName would now be derived from a combination of the routeName and the destination/port. Thus providing a way to have unique spool names across different routes with the same destination/port. When the carbon-relay starts up after an upgrade with this fix, the relay would create brand new spools, and essentially any spool that existed before the upgrade would be orphaned. Perhaps I could add a note in the ReadME, for example: "When upgrading to a release > v0.9.4, verify any critical data in the spools is emptied, or the data is not critical and can be orphaned as spool naming convention has changed."

@Dieterbe
Copy link
Contributor

sounds good. don't worry about the readme. we can put that in the release notes once we make a release

util/util.go Outdated
func SpoolKeyAddrToPath(s string, routeName string) string {
s = AddrToPath(s)
sList := []string{routeName, s}
return strings.Join(sList, "_")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is unnecessarily complicated
can just return routeName + "_" + AddrToPath(s)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed

@Dieterbe
Copy link
Contributor

note also that we currently set cleanAddr and base a lot of stats on that.
looks like the stats suffer the same problem as spool names (multiple dests with same addr clobbering each other's stats).
ditto for log calls, which use the dest.Addr property

so my suggestion is , rename the spoolKey property to simply 'Key', and use it for all stats and log calls.
remove the cleanAddr property.

this way we solve the problem for spool files as well as log messages as well as stats.

@scrichar
Copy link
Contributor Author

scrichar commented Mar 5, 2018

@Dieterbe , I have tested these changes locally, things are looking good on my fork. Logs and stats are now using the new property "key". cleanAddr is removed. Spooling is now using the combination of the route name and the destination/port combo. Can you review?

@Dieterbe
Copy link
Contributor

Dieterbe commented Mar 5, 2018

looks pretty good.
I did squash all your commits into one and applied some cleanups. see 7a664ab

thanks a lot for your contribution, this was a longstanding pain point and i'm glad you stepped up :)

@Dieterbe Dieterbe closed this Mar 5, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants