Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Comment/Comment Removed points hooks don't work together as intended #221

Closed
JDGrimes opened this issue Nov 18, 2014 · 16 comments
Closed

Comment/Comment Removed points hooks don't work together as intended #221

JDGrimes opened this issue Nov 18, 2014 · 16 comments
Assignees
Labels
Milestone

Comments

@JDGrimes
Copy link
Member

The Comment points hook has been present since the plugin's first release. The Comment Removed points hook was split off from it in #46. They do not, and have never, behaved intuitively.

The original intent was to create a hook that awarded a user points when they received a comment. That's how the Comment points hook was born. Foresight dictated that there would need to be some way of handling spam. Users would need to be punished when they tried to game the system. This desire to mitigate spam originally resulted in the addition of a second points field to the Comment points hook. This was for the number of points to remove from the user if their comment was marked as spam or otherwise removed. This allowed the user to just remove the awarded points when a comment was marked as spam, or by increasing the amount, to actually remove more points than the user originally received thus penalizing them.

Eventually I decided that the best way to handle these two functions of a single points hook was to split the hook in two. Thus, the Comment Removed points hook was born.

The flow of events that the hook(s) were designed to cover would go something like this:

User leaves a comment » receives 20 points » comment marked as spam » 20 points removed

And they work great in that scenario. But there are other possible scenarios, like these:

User leaves a comment » receives 20 points » comment marked as spam » 20 points removed » comment reapproved » 20 points added

.

User leaves comment » Akismet marks as spam » 20 points removed » comment manually approved » 20 points added

While these types of scenarios where taken into account when the hooks were designed, they aren't handled well. For example, in both of the above cases the user has left a valid comment, but they haven't actually received an increase in points from it. It would be even worse if, e.g., 25 points were removed for spam comments. Then the user could leave a valid comment and end up losing 5 points. Ouch!

In both of the above scenarios the user should have ended up with +20 points. The question this ticket is here to explore it how to best make that happen.

The idea first conceived in #111 is to introduce automatic reversal of certain hooks. The Comment hook would reverse it's award when a comment is removed. In other words, it would be completely severed from the Comment Removed points hook, which would itself implement the same reversal feature.

The flow for the comments points hook would then look like this:

User leaves comment » 20 points awarded » Comment marked as spam » 20 points reversed

The problematic flows would look like this:

User leaves a comment » receives 20 points » comment marked as spam » 20 points reversed » comment reapproved » 20 points added

.

User leaves comment » Akismet marks as spam » comment manually approved » 20 points added

Now in both of these cases the user would have +20 points. The Comment Removed hook would operate separately, and is not included in the flows outlined above. With that hook configured as well the flows would look like this:

User leaves a comment » receives 20 points » comment marked as spam » 20 points reversed » 10 points removed » comment reapproved » 20 points added » 10 points reversed

.

User leaves comment » Akismet marks as spam » 10 points removed » comment manually approved » 20 points added » -10 points reversed


The main question at this point is whether these "reversals" should be logged or be done behind-the-scenes, deleting the related log information.

@JDGrimes JDGrimes added the bug label Nov 18, 2014
@JDGrimes JDGrimes added this to the 1.8.0 milestone Nov 18, 2014
@JDGrimes
Copy link
Member Author

Not logging the reversal is new territory, and may have unintended side affects. Although, a hook should have only one function. Logging the reversal might encourage the reversal feature to be used differently than intended.


When we go through with this, the Comment Removed hook should probably be renamed to the Comment Spam hook, and only run when a user leaves a spam comment, since reversals will already be removing points awarded if a comment is just deleted.

@JDGrimes JDGrimes self-assigned this Nov 18, 2014
@JDGrimes
Copy link
Member Author

As far as just deleting the logs goes, on the plus side, it means we don't have to mess with new log messages, and also, that will keep us from giving the user two messages when their comment is marked as spam if the Comment Removed hook is also in use. The con is that it could be confusing to a user to suddenly have some points revoked. However, if the user is spamming, what does it really matter? The only thing that makes me second guess it is for the new Comment Received points hook. The post author isn't a spammer, and yet they might get points revoked if a spam comment gets removed. So they could still be confused, and wouldn't be a guilty party. Of course, if they are themselves marking the comment as spam, they'll be able to figure out what happened pretty quickly. Also, that could discourage post authors from flagging spam comments on their posts. It's beyond the scope of this issue, and more in line with #111, but we could handle that in two ways:

  1. Add a hook to give a post author points when they mark a comment on their post as spam. This gives post authors the incentive to triage post comments and remove low quality ones.
  2. Don't reverse the post author hook at all. Sounds crazy on the one hand, but if we exclude the post author's own comments from awarding them points, there's no way they could game the system, other than if they are writing posts that are specifically attracting spam comments, or they are working together with other authors to game the system. (Which they could really do anyway, but if an administrator deleted the comments the points awarded would be automatically reversed.)

Again, beyond the scope of this, but something that still requires thought. I kind of like 1, but it's probably way beyond the scope of 1.8.0, unfortunately.

@JDGrimes
Copy link
Member Author

Oh, and option 3 here is to do reversals differently for the Comment Received hook, and instead of doing things behind-the-scenes, logging what is going on, at least for now. Probably the best course for 1.8.0.

@JDGrimes
Copy link
Member Author

We shouldn't attempt this for 1.8.0. We'll do only what is needed for #111.

@JDGrimes JDGrimes removed this from the 1.8.0 milestone Nov 18, 2014
@JDGrimes
Copy link
Member Author

Another possibility would be to integrate the two hooks so that there aren't two log entries when one is reversed. If the other one is present they would just hook into each other. That could be complicated though, and might not work exactly as intended when the hooks are configured for different post types (or one is configured for all post types).

@JDGrimes JDGrimes removed their assignment Nov 19, 2014
@JDGrimes JDGrimes modified the milestone: 1.9.0 Dec 1, 2014
@JDGrimes
Copy link
Member Author

JDGrimes commented Jan 6, 2015

Honestly, why can't we just remove the Comment Removed hook entirely. It won't really be needed any more, and it would make things simpler, I think.

@JDGrimes
Copy link
Member Author

JDGrimes commented Jan 8, 2015

We could even do something like this:

When we update the points hooks, check if the Comment Removed hook is being used, and if it has more points than the Comment points hook, we could let it stay on those sites. We'd hide it on the rest of the sites.

@JDGrimes
Copy link
Member Author

JDGrimes commented Jan 8, 2015

I'm thinking that we already punish spammers enough by having them waste their time—they wont end up with any extra points for leaving spam comments. Usually if a spammer is obnoxious I'd expect that the site's moderators will delete their account. Anyway, I'm thinking it is time to end the extra punishment thing. We'll maintain backwards compatibility for those that are using the Comment Removed points hook to punish.

@JDGrimes JDGrimes self-assigned this Jan 8, 2015
@JDGrimes
Copy link
Member Author

JDGrimes commented Jan 8, 2015

What about back-compat for users who don't use a corresponding Comment Removed hook for each Comment hook? We'd suddenly be removing points for removed comments.

I think the best backward compatible solution is this:

  • The Comment Removed hook gets hidden on new installs.
  • The Comment hook is given automatic reversals.
  • The Comment hook gets a checkbox option, "Automatically reverse when comment is removed?"

Then, on update:

  • For unpaired Comment hooks, we set the auto-reverse setting on all to false
  • We combine any Comment/Comment Removed hook pairs that have equal points values
  • We set the auto-reverse setting to false for the Comment hook in any hook pairs where the values are different

@JDGrimes
Copy link
Member Author

JDGrimes commented Jan 8, 2015

New troubles:

When we perform the auto-reversal (as it presently works), we just perform a logs query to pull out all of the logs matching that comment ID. We then loop through these and reverse the transactions.

The only metadata associated with a transaction is the comment ID. This means that there is no direct way to determine what instance it was that fired the original transaction.

However, the auto reverse setting would be saved as an instance setting. In other words, it is a per-instance thing. So we need to determine which instance logged the transaction before we can determine whether it should be reversed.

There are several possible approaches to solve this:

  1. We can analyze the comment to determine what type of post it was on. The problem with this approach is that we could run into conflicts between hooks, for example if a user has a Post hook and an All hook. However, we may just want to consider that a configuration issue where the onus is on the user. On the other hand, we allow the user to do this, and what's to say that some aren't? We'd want to try to maintain the same behavior as before in this case, although it is really undefined behavior at the moment.
  2. We can save an auto_reverse meta key for each transaction from now on, and use that to determine whether to automatically reverse a transaction. The issue here is that we'll continue to auto-reverse transactions even if a user has turned this off for the instance that performed them. That would be a bad user experience. It also wouldn't solve the issue when it comes to transactions that occurred before the update.
  3. We could add a unique identifier to each instance's settings. This would be saved as metadata for each transaction, which would then be used to check that same instance's settings when the transaction was reversed. The issue here is that if a user changes the instance from one post type to another, they will end up having reversals for other points types now obeying that instance's reversal settings.

Of these three options, I think that perhaps no. 1 is most promising. The issue with it, that of potential conflicts between different instances, is not vital. It can also be partly solved by saving a post_type meta key whose value is the post type setting of the instance performing the transaction. However, that still leaves the issue of what to do for previous transactions...

@JDGrimes
Copy link
Member Author

JDGrimes commented Jan 8, 2015

For the solution to no. 1, here is my proposal for how we determine the reversal setting to use:

We get the type of post the comment is on. If there is an instance specifically for the post type, we use the auto-reversal setting from it. Otherwise, if there is an instance with the All post type setting, we use the auto-reversal setting from it. Conflict resolved.

@JDGrimes
Copy link
Member Author

In regard to conflicts from multiple hooks affecting the same post type, this is moot: only one hook will fire per comment per points type no matter how many instances apply to it, because the comment meta key that holds the last status that the comment was awarded/reversed for is based only on the points type.

So even in the past, when there were multiple instances affecting the same post type, only one would award points.

@JDGrimes
Copy link
Member Author

We might want to consider recombining the Post and Post Delete points hooks and implementing auto-reversal as well. We don't currently have the same issues there, because the Post Delete hook only fires when a post is deleted, but there could be benefits.

@JDGrimes
Copy link
Member Author

Oh, and something interesting that I noticed. The Comment points hook awards the post author as well as everyone else. We decided not to do this in the Comment Received points hook. I don't know that it is something we should change in either case now, due to backward incompatibility. It would be nice to be consistent though. Part of my rationale for keeping the comment hook the way it is, is that it's nice to reward a post author for interacting with users who comment on their post, especially if they are answering questions.

@JDGrimes
Copy link
Member Author

I'm thinking about how far we should go in terms of limiting or maintaining the legacy aspects of this. I'm thinking we'll do it like this:

  • If there are any Comment Removed hooks left, we'll keep that hook type available, not just as a handler for the extant instances; we'll let them add new instances too.
  • If there are any unpaired Comment hooks we'll give the user the option to disable the auto-reversal for any Comment hook instance.

This could leave a user confused if there is a left-over Comment Removed hook but no un-paired Comment hooks. However, that is really an edge-case that isn't significant enough to consider.

I had thought about possibly restricting these legacy features as much as possible, only enabling them for the current hook instances that required them. However, I think this could easily confuse and frustrate the user. So the new plan is to enable these legacy features across the board when they are in use at all, and then we'll remove them in 2.0. That will give time that might allow many users to discover the change.

JDGrimes added a commit that referenced this issue Jan 17, 2015
We already know the comment ID, so we are just wasting resources
pulling it out of the points log meta table.

Working on #221
JDGrimes added a commit that referenced this issue Jan 17, 2015
JDGrimes added a commit that referenced this issue Jan 17, 2015
Previously the Comment points hook did not support auto reversal.
However, auto-reversal has been automatically handled by it’s parent
class since 1.8.0. This has been disabled for the Comment points hook
up to now by overriding the `reverse_hook()` method. This removes that
method, enabling auto-reversal.

The parent class is also updated to make it possible for child classes
to allow auto-reversal to be disabled per-instance by the user, and the
Comment hook class has been configured enable this feature.

See #221
JDGrimes added a commit that referenced this issue Jan 17, 2015
JDGrimes added a commit that referenced this issue Jan 17, 2015
We introduce the `$wordpoints_component` and `$previous_version`
properties, and use them in the `update_component()` method, making it
optional to supply its arguments.

See #221
JDGrimes added a commit that referenced this issue Jan 17, 2015
On update to 1.9.0, we combine any Comment/Comment Removed hook pairs
that have the same number of points. For the rest, we turn off
auto-reversal, for backward compatibility.

See #221
JDGrimes added a commit that referenced this issue Jan 17, 2015
Only on sites where disabling auto-reversal of the Comment hook is
necessary because of existing hooks do we allow it to be disabled as a
legacy feature.

We also introduce a new method to the base hook class, `set_option()`,
and include unit tests for it and related methods.

See #221
JDGrimes added a commit that referenced this issue Jan 17, 2015
We now only register it when it is required for legacy sites where it
is still used.

See #221
JDGrimes added a commit that referenced this issue Jan 18, 2015
The tests involving the Comment and Comment Removed hooks need to be
updated now that the Comment Removed hook is disabled by default.

We now need to manually register the Comment Removed hook in any test
that requires it. However, the only way to do that is by calling
`WordPoints_Points_Hooks::initialize_hooks()` again after calling the
`register()` method to register the hook’s class. This results in a
second set of hook objects being instantiated. This effectively causes
the hooks to be fired twice in the tests that do this. This state
doesn’t leak into other tests, because the actions are reset after each
test.

It does cause an issue with the Comment hook in regard to
auto-reversal. The auto-reversal feature can be disabled on legacy
installs by setting a certain hook type “option”. This is object
instance-specific, and is saved in a property. This causes problems
when we need to reinitialize the hooks, because then the objects
returned by the `get_handler_by_id_base()` method are different from
the objects which are hooked to the actions. This state of things will
continue to affect each test after a re-initialization occurs.

Because of this, I’ve found it necessary to isolate the Comment
Removed-related tests that re-initialize the hooks, and add them to a
new excluded `legacy` test group.

We don’t currently run this group on Travis, and I’m going to leave it
like that for now.

See #221
JDGrimes added a commit that referenced this issue Jan 18, 2015
@JDGrimes
Copy link
Member Author

I guess that's it.

JDGrimes added a commit that referenced this issue Jan 18, 2015
JDGrimes added a commit that referenced this issue Jan 18, 2015
JDGrimes added a commit that referenced this issue Jan 27, 2015
Now that the Comment Removed hook may be disabled on legacy installs,
we need to provide a separate function to generate the log text, in
case the logs get regenerated for some reason.

This adds a new function to do that, and it is now used by the Comment
Removed hook class’s `logs()` method.

Includes unit tests.

See #221
JDGrimes added a commit that referenced this issue Jan 27, 2015
We moved these to an excluded legacy group to deal with the issue of
the points hook normally only being available on legacy installs.
However, this isn’t really necessary, thanks to work in this line
regarding the Post Delete hook in #256.

See #221
JDGrimes added a commit that referenced this issue Jan 30, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant