Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-16825: Fix response serialization bug #1867

Conversation

gerlowskija
Copy link
Contributor

@gerlowskija gerlowskija commented Aug 28, 2023

https://issues.apache.org/jira/browse/SOLR-16825

Description

Request and response body POJOs in the 'api' module must implement 'ReflectWritable', a marker interface that indicates to JavabinCodec and other serialization classes that they have field information that can be discovered via reflection.

A prior commit added several POJOs to this module that didn't follow this rule, causing test failures in nightly runs.

Solution

This commit adds the needed marker interface to several classes in the 'api' submodule that were missing it.

Tests

Broken tests resume passing.

Checklist

Please review the following and check all that apply:

  • I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • I have created a Jira issue and added the issue ID to my pull request title.
  • I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
  • I have developed this patch against the main branch.
  • I have run ./gradlew check.
  • I have added tests for my changes.

@epugh
Copy link
Contributor

epugh commented Aug 28, 2023

This feels like a bit of a "sigh" change... So for now through the end of time every pojo gets a rather opaque interface that they implement? ;-).

@HoustonPutman
Copy link
Contributor

Hmm might be able to "fix" this...

@HoustonPutman
Copy link
Contributor

Ok we don't need to use ReflectWritable I think... Basically we can assume that any non-known class uses ReflectWritable...

By changing:

  • JavaBinCodec:276 -> writeMap(new DelegateMapWriter(val));
  • TextWriter:114 -> writeMap(name, new DelegateMapWriter(val));

I'm going to run the full test suite to make sure this works, but it would be very nice if it does. Also we can keep the ReflectWriter checks in the if-else statements, so that other types can be overriden to choose DelegateMapWriter instead of their default serialization method.

@gerlowskija
Copy link
Contributor Author

I considered that approach initially, but ended up scrapping the idea as I was worried that changing JavaBinCodec's default would be too broad and "dangerous".

But in hindsight maybe that was the wrong call - the JBC fallback behavior is so bad it's hard to imagine anything relying on it. If we're happy relying on tests to vet this for us, and you all think it's reasonable, I'm all for it. Lmk if I can do anything to help!

Therefore request/response types do not need to extend
anything in order to work with our serialization framework
@HoustonPutman HoustonPutman force-pushed the SOLR-16825-fix-response-serialization-bug branch from 584bd7c to 3ea8db1 Compare August 28, 2023 21:21
@HoustonPutman
Copy link
Contributor

Ok, rewrote this so that it will use a Reflection-based writer by default, but still use a string writer if no reflection fields are found. This means that we don't need to worry about the inheritance for new types, but we can still manually choose a "ReflectWritable" if there is ambiguity and we need to make sure.

Copy link
Contributor Author

@gerlowskija gerlowskija left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GH won't allow me to approve since I started the PR, but LGTM 👍

import java.io.IOException;
import org.apache.solr.common.util.Utils;

public class DelegateMapWriter implements MapWriter {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good thought to nuke this implementation 👍

@@ -88,7 +87,7 @@ default void writeVal(String name, Object val, boolean raw) throws IOException {
} else if (val instanceof MapWriter) {
writeMap(name, (MapWriter) val);
} else if (val instanceof ReflectWritable) {
writeMap(name, new DelegateMapWriter(val));
writeVal(name, Utils.getReflectWritable(val));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Q] What purpose does the call to getReflectWritable serve, since we know that "val" is already a ReflectWritable?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, the name is probably really bad here, but getReflectWritable() does not return a ReflectWritable class. It returns that DelegateReflectWriter. I'll rename this to getReflectWriter(), as it will be much less confusing. Good catch.

@HoustonPutman HoustonPutman merged commit 9a45385 into apache:main Aug 29, 2023
3 checks passed
HoustonPutman pushed a commit that referenced this pull request Aug 29, 2023
Unknown types will try to serialize using Reflection,
but will use a "toString()" serialization if no JSONProperty annotations
are found. This is backwards compatible.

Co-authored-by: Houston Putman <houston@apache.org>
(cherry picked from commit 9a45385)
@gerlowskija gerlowskija deleted the SOLR-16825-fix-response-serialization-bug branch August 31, 2023 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants