BatchExecutor flush commands NPE #2126
Comments
zhangjunqiang
changed the title
|
Thanks for the report. I have issues reproducing the issue. The |
mp911de
added
the
status: waiting-for-feedback
sorry, i don't get your mind. The RedisBatchQuery class is implement just like https://github.com/lettuce-io/lettuce-core/wiki/Redis-Command-Interfaces or #624. |
|
The redis connect init with guice like this. public class StatefulRedisClusterConnectionProvider implements Provider<StatefulRedisClusterConnection<String, String>> {
private Set<RedisURI> redisURIs;
public StatefulRedisClusterConnectionProvider(Set<RedisURI> redisURIs) {
this.redisURIs = redisURIs;
}
@Override
public StatefulRedisClusterConnection<String, String> get() {
final RedisClusterClient redisClusterClient = RedisClusterClient.create(redisURIs);
// https://github.com/lettuce-io/lettuce-core/wiki/Redis-Cluster#user-content-refreshing-the-cluster-topology-view
ClusterTopologyRefreshOptions options = ClusterTopologyRefreshOptions.builder()
.refreshPeriod(Duration.ofSeconds(60))
.enableAllAdaptiveRefreshTriggers()
.adaptiveRefreshTriggersTimeout(Duration.ofSeconds(30))
.refreshTriggersReconnectAttempts(5)
.build();
redisClusterClient.setOptions(ClusterClientOptions.builder()
.topologyRefreshOptions(options)
.autoReconnect(true)
.maxRedirects(5)
.disconnectedBehavior(ClientOptions.DisconnectedBehavior.DEFAULT)
.socketOptions(SocketOptions.builder()
.keepAlive(true)
.tcpNoDelay(true)
.build())
.validateClusterNodeMembership(false)
.build());
StatefulRedisClusterConnection<String, String> connection = redisClusterClient.connect();
connection.setReadFrom(ReadFrom.REPLICA_PREFERRED);
connection.setAutoFlushCommands(true);
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
connection.close();
redisClusterClient.shutdown();
}));
return connection;
}
} |
|
Having a similar issue. Heres my stacktrace, also caused by calling a batch flush:
Using lettuce version 6.1.6.RELEASE |
|
It looks like we are both getting an NPE on this line (cluster and standalone): |
|
it seems to me if getType() returns a non null, all name() implementations return a non-null. So the NPE is probably coming from getType() returning null. As far as I can see that only occurs with the PristineFallbackCommand Command |
|
I believe the issue is harder to find because it depends on whether or not assertions are enabled private List<RedisCommand<Object, Object, Object>> prepareForceFlush() {
List<RedisCommand<Object, Object, Object>> batch = new ArrayList<>(Math.max(batchSize, 10));
do {
RedisCommand<Object, Object, Object> poll = queue.poll();
assert poll != null;
batch.add(poll);
} while (!queue.isEmpty());
return batch;
}So this returns a list with a null command with no assertions. I'd suggest just skipping the flush in such a case, unless the intention is to error when flush is empty? |
|
You end up with a null command sent to the dispatch where you get the NPE: private List<? extends RedisCommand<?, ?, ?>> doFlush(boolean forcedFlush, boolean defaultFlush, int consume) {
List<RedisCommand<Object, Object, Object>> commands = null;
if (forcedFlush) {
commands = prepareForceFlush(); // COMMAND HAS 1 ITEM THAT IS NULL
} else if (defaultFlush) {
commands = prepareDefaultFlush(consume);
}
if (commands != null && !commands.isEmpty()) {
if (commands.size() == 1) {
connection.dispatch(commands.get(0)); // RETURNS NULL
} else {
connection.dispatch(commands);
}
return commands;
}
return Collections.emptyList();
} |
|
And then StatefulRedisConnectionImpl.java gets a null argument: protected <T> RedisCommand<K, V, T> preProcessCommand(RedisCommand<K, V, T> command) {
RedisCommand<K, V, T> local = command;
if (local.getType().name().equals(AUTH.name())) { |
|
From a spot check it seems this behavior is already solved in the latest: private List<RedisCommand<Object, Object, Object>> prepareForceFlush() {
List<RedisCommand<Object, Object, Object>> batch = newDrainTarget();
while (!queue.isEmpty()) {
RedisCommand<Object, Object, Object> poll = queue.poll();
if (poll != null) {
batch.add(poll);
}
}
return batch;
} |
Closing as it seems this is no longer a problem. Please reopen if you are having this issue. |
Bug Report
I create RedisBatchQuery with BatchExecutor API. I use the redisBatchQuery to sync lots of id to redis. but it will happend NPE Occasionally (at redisBatchQuery.flush() method).
After careful observation. when the ids size is divisible by @batchsize(1000), it will NPE. otherwise it will not happend.
so i think there is something wrong with SimpleBatcher.flush().
Current Behavior
Stack trace
Input Code
Input Code
Expected behavior/code
Environment
Possible Solution
Additional context
The text was updated successfully, but these errors were encountered: