open-q performance regression on 1.13?

[jarohen]

https://clojurians-log.clojureverse.org/crux/2020-12-25, from @nivekuil/@refset

I think there's some other sort of perf regression in 1.13.. the first query on something is taking consistently around 20-24 seconds, up from 500ms or so before. Note that this particular path is pretty idiosyncratic: I make ￱~￱50-100 open-q queries (using eql/project -- actually same behavior without it) in a loop then go through them lazily. I think it's just the initial part that's slow, fetching more data from the open-q iterators seems to be the same perf

Hmm, it sounds like this could be tough to debug without a profiler / flamegraph, and I'm not aware of any significant change to how open-q initializes. Do you see the same join order as before coming out of the crux.query DEBUG logging?

not sure, I'd have to spend some time upgrading my instrumentation to check but I don't think join order is the issue as it is literally the first query, i.e. start app, wait however long, query takes 20s, wait 24h, query takes 300ms. I also tried sending more requests while the ￱~￱20s query is taking place and they also took about the same duration. I know the JVM has to warm up and all but something must be off for such a big difference to now appear. I'm personally relegating this to "back of mind" urgency but I'll let you know if I take a deeper look

Also of note is that I can only see this in my prod-like environment, only difference should be jdbc tx/doc stores instead of in-memory/rocks. rocks for indexes on both

[nivekuil]

as an update, I saw this issue on JDBC, but now I've switched stuff over to kafka+s3 and can no longer reproduce this. Best guess is it was something specific to JDBC (dead/stale connections?)

[jarohen]

Ah, thanks @nivekuil, that's a good steer :)

[jarohen]

https://clojurians-log.clojureverse.org/crux/2021-01-28/1611864007.084600

related to #1403

Toyam Cox 
Fun fact, there is no real way to figure out that the dependency is wrong because it does in fact write to the configured postgresql database. Ask me how I know!

Toyam Cox
I tested by replacing the dep, spinning up a second instance, and hey, there was my original data still there! I couldn't just read the database directly, it's opaque to me

Kevin Liu
@jarohen:  I fell victim to this as well! I suspect you can safely close https://github.com/juxt/crux/issues/1364 and blame the dep

Thanks @nivekuil!