Optimized various tests.

[ngnpope]

See commit messages for explanations.

[felixxm]

Thanks 🚀 On Oracle:

test	before	after
test_include_materialized_views	148.0s	12.717s
test_include_views	150.0s	9.971s

[ngnpope]

That's exactly what I was hoping for!

[claudep]

That's great. however not the final word. I already got a failing test on a PR which could be related (cache.tests.PyLibMCCacheTests.test_touch, https://djangoci.com/job/pull-requests-bionic/database=postgis,label=bionic-pr,python=python3.6/8623/).

[ngnpope]

@claudep Indeed, but these failures did crop up on occasion before. I don't have a rate of how often, but these things are hard to measure reliably anyway.

I thought the saving made it at least worth trying, on the basis that we can review over the coming days/weeks and tweak or revert if we perceive it to be worse.

So thank you for the first report! 😄

In case you missed it, I discussed this above: #13496 (comment)

[timgraham]

I've just seen the same failure of test_touch on my PR.

[ngnpope]

Thanks for the heads-up @timgraham.

So here is the relevant bit of the test:

        # cache.touch() updates the timeout.
        cache.set('expire1', 'very quickly', timeout=1)
        self.assertIs(cache.touch('expire1', timeout=2), True)
        time.sleep(1.0)
        self.assertIs(cache.has_key('expire1'), True)
        time.sleep(1.5)
        self.assertIs(cache.has_key('expire1'), False)

So the flow is:

1. Set with a short timeout.
2. Update to a longer timeout while verifying the key is still set.
3. Wait long enough for the original timeout to have expired.
4. Then check the key is still set.
5. Wait long enough for the longer timeout to expire.
6. Then check the key is not set.

We're failing at step 4. For that to happen we must be losing a whole second somewhere… ☹️

I note that there is some guidance on troubleshooting timeouts. I wonder if anything crops up in the memcached stats related to maximum connections @felixxm?

[ngnpope]

Hmm. Just seen a lot more touch failures.

It is rather strange that we would need to wait for multiple seconds. Are all parallel test runs executed on the same memcached instance on CI? I presume that there is some sort of key prefixing if so, otherwise this is bound to fail. Or are too many parallel runs causing too many connections or too much load on memcached per the link in my previous comment.

Perhaps it is just easier to revert 177a49e, but it feels like there is some underlying problem…

[felixxm]

I presume that there is some sort of key prefixing if so, otherwise this is bound to fail.

Yes, based on a Python version, database and build name, so it's properly isolated.

Perhaps it is just easier to revert 177a49e, but it feels like there is some underlying problem…

Agreed let's revert it (#13565), there is no doubt that it caused more frequent failures. We need to investigate this more closely 🕵️