Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

in_tail: position_file: Define path based equalities on TargetInfo #3391

Closed
wants to merge 3 commits into from

Conversation

cosmo0920
Copy link
Contributor

@cosmo0920 cosmo0920 commented May 25, 2021

Signed-off-by: Hiroshi Hatake hatake@calyptia.com

Which issue(s) this PR fixes:
Related to #3365

What this PR does / why we need it:

In the previous Fluentd v1.11.x implementation uses path based tailing
keys.
We wouldn't rewrite tailing mechanism entirely.
And also we're implicitly assuming path based equality for TargetInfo in tailing logic.
We should handle TargetInfo with path based equality instead of path and
inode based equality.

Docs Changes:

No needed.

Release Note:

Same as title.

@cosmo0920 cosmo0920 requested review from ashie and kenhys May 25, 2021 04:44
@cosmo0920 cosmo0920 self-assigned this May 25, 2021
@cosmo0920 cosmo0920 force-pushed the define-target-info-equalities branch from 883eaa7 to 2c9b31f Compare May 25, 2021 04:49
@ashie
Copy link
Member

ashie commented May 25, 2021

Typo?

-In the previous Fluentd v1.11.x implementation uses patch based tailing keys.
+In the previous Fluentd v1.11.x implementation uses path based tailing keys.

In the previous Fluentd v1.11.x implementation uses path based tailing
keys.
We wouldn't rewrite tailing mechanism entirely.
And also we're implicitly assuming path based equality for TargetInfo in tailing logic.
We should handle TargetInfo with path based equality instead of path and
inode based equality.

Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
@cosmo0920 cosmo0920 force-pushed the define-target-info-equalities branch from 2c9b31f to e516268 Compare May 25, 2021 04:51
@cosmo0920
Copy link
Contributor Author

Typo?

-In the previous Fluentd v1.11.x implementation uses patch based tailing keys.
+In the previous Fluentd v1.11.x implementation uses path based tailing keys.

Yep. This is typo. I'd fixed it.

@ashie
Copy link
Member

ashie commented May 25, 2021

Hmm, we should release v1.12.4 with this patch before releasing v1.13?

@ashie
Copy link
Member

ashie commented May 25, 2021

I'll create v1.12 branch.

@cosmo0920
Copy link
Contributor Author

Hmm, we should release v1.12.4 with this patch before releasing v1.13?

Yes, please.

@ashie
Copy link
Member

ashie commented May 25, 2021

I'll release v1.12.4 in a few days.

end

def hash
self.path.hash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure follow_inodes feature but does it work well also on @follow_inodes?
I think that keys of @tails should be inode on @follow_inodes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or paths are updated by update_watcher on rotation so paths are never duplicate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll take a look on this.

Copy link
Member

@ashie ashie May 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still checking it too.
Probably above my concern is wrong, it should be path maybe.
Otherwise update_watcher can't discard old TailWather from @tails.

Copy link
Member

@ashie ashie May 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Memo:

Instead of overriding them, I thought using target_info.path as a key is more straightforward and understandable: @tails[target_info.path]

But it's also wrong.
TargetInfo is referred by each_keys so keys have to be TargetInfo.

Copy link
Member

@ashie ashie May 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, probably it's O.K. as a first aid but I think more refactoring is needed to make easy to understand in the future 🤔

Copy link
Contributor Author

@cosmo0920 cosmo0920 May 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of overriding them, I thought using target_info.path as a key is more straightforward and understandable: @tails[target_info.path]

But it's also wrong.
TargetInfo is referred by each_keys so keys have to be TargetInfo.

Yes! I'm also considering to use it to be made easier to understand, but overriding equality on TargetInfo is one of the simplest way to dispose discarded old TailWatcher correctly. (This strategy doesn't request more refactoring.)

t1.eql? t2
end
end

Copy link
Contributor

@kenhys kenhys May 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To emphasize the difference of eql? and equal? behavior, how about adding one more test case using assert_not_same t1, t2?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be done in ab36738 and 2f2214b.

Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
…sub_test_case

Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
@ashie ashie changed the base branch from master to v1.12 May 26, 2021 03:37
@ashie ashie changed the base branch from v1.12 to master May 26, 2021 03:37
@ashie
Copy link
Member

ashie commented May 26, 2021

I've created a pull request for v1.12 and merged already: #3393
I'll merge v1.12 branch to master after releasing v1.12.4, then close this PR.
Thank you for investigating it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants