[Bug report] URLFilterBolt seems to emit unanchored tuples

[lfoppiano]

Version

3.6.0 used through the Common Crawl news-crawler

Describe what's wrong

HI all, I'm not sure whether this is a bug, it seems that the URLFilterBolt.execute() emits unanchored tuples: it uses the two-argument collector.emit(String streamId, List<Object> tuple), which does not anchor the outgoing tuple to the incoming one. This happens at both of its emit sites:

core/src/main/java/org/apache/stormcrawler/bolt/URLFilterBolt.java:80-85 (the discoveredOnly
short-circuit):

if (discoveredOnly && !status.equals(Status.DISCOVERED)) {
    Values v = new Values(urlString, metadata, status);
    collector.emit(stream, v);          // <-- unanchored
    collector.ack(input);
    return;
}

and :99-101 (the normal path):

Values v = new Values(filtered, metadata, status);
collector.emit(stream, v);              // <-- unanchored
collector.ack(input);

Let me know, if this is a bug I can submit a PR right away 😄

Error message and/or stacktrace

If you run it long enough you will notice that the waitAck saturated util it goes in OOM (or at least this is what happend to me)

How to reproduce

Affected version: 3.6.0 (present in earlier releases too — the emit form is unchanged)
Component: stormcrawler-core, org.apache.stormcrawler.bolt.URLFilterBolt

Additional context

No response

[jnioche]

Good catch, thanks @lfoppiano
They should definitely be anchered, like in all the other bolts. A PR would be very welcome, thanks

[lfoppiano]

Sure.

Regarding the method that doesn't supply anchoring, is it actually useful to have it or should be deprecated or renamed to a more explicit name?

[jnioche]

Regarding the method that doesn't supply anchoring, is it actually useful to have it or should be deprecated or renamed to a more explicit name?

collector.emit(...) ?

that's in Apache Storm, not StormCrawler. I am pretty sure they are useful e.g. for side-effect emits (metrics, logging, debug streams) where losing that data shouldn't trigger replay of the original tuple.