🛠 update lodash,knex and bookshelf to latest versions

[kirrg001]

refs #7189

• we had a memory leak after upgrading to bookshelf 0.10.1
• bookshelf has fixed the memory leak in 0.10.2

[ErisDS]

3 requests per second to a Ghost server which has lot of data in the database.
The memory with latest lodash, latest bookshelf and latest knex grows to a max memory of ~300mb and always falls down to 130-140mb.

Is that the same or different to the previous test? Can you maybe share a graph of both?

If I am remembering correctly, this same test did not reproduce the memory leak last time, and so this is not a good test to base an conclusion on?

I now have access to a trace account, so maybe we can setup a longer running test tomorrow ?

[kirrg001]

Is that the same or different to the previous test? Can you maybe share a graph of both?

Same. I don't have a graph.

If I am remembering correctly, this same test did not reproduce the memory leak last time, and so this is not a good test to base an conclusion on?

I was able to reproduce the memory leak with this exact test suite. And now it's gone with the new knex bump.

I now have access to a trace account, so maybe we can setup a longer running test tomorrow ?

If you want sure.

My ghost server is running now an hour and memory is good.

[ErisDS]

When I asked you if it was the same, I meant the results, not the test itself (you stated in your PR that the test was the same test). I'm asking how these results compare to the results you saw last time? I want to see side-by-side results so that I can understand the difference, if there is one.

[kirrg001]

I can make graphs tomorrow 👍 with knex 0.10 (which we are using now), knex 0.11 (which had a memory leak) and knex 0.12. (which solved the memory leak)

[ErisDS]

Either a graph or a table would be fab.

[kirrg001]

It turned out that knex 0.12 is not solving the memory problems.
I did another measuring this morning. Here are the results (tested with lodash 3 and 4, it doesn't matter)

---

Bookshelf 0.9.x
Knex 0.10.x

Bookshelf 0.9.x
Knex 0.12.x

(both setups result in the same graph)

---

Bookshelf 0.10.x
Knex 0.12.x

Bookshelf 0.10.x
Knex 0.10.x

(both setups result in the same graph)

Looks like the bookshelf version 0.10.x is causing the problems.

[tgriesser]

I'll need to take a look to see what's happening here

[tgriesser]

@kirrg001 is there a step-by-step I can follow to reproduce those graphs and get those results?

[tgriesser]

Just in looking through the commit log I believe the issue likely has to do with the __super__ reference in .extends in this commit.

If it is indeed the source of issue I can go ahead and revert the portions that change asap, will try and take a look a little later.

[kirrg001]

@tgriesser Thanks for getting involved.

To reproduce the problem, you need to checkout this branch and setup a ghost blog with a bigger database (i had 20 posts in my database). Then you can trigger some curl commands from shell and watch the memory. Write me a PM in slack, if you need help.

see also #7189

[tgriesser]

This is fixed in Bookshelf 0.10.2 🎉 - Thanks @kirrg001 for the help in profiling / tracking this one down!

[kirrg001]

I can confirm: the memory leak is fixed in 0.10.2.
Thank you @tgriesser!

[kirrg001]

ready for review && merge

[ErisDS]

@kirrg001 this needs rebasing please :)

[kirrg001]

@ErisDS 👍 👍 didn't see, will do now

[kirrg001]

@ErisDS did

[ErisDS]

I would like this same PR made to the lts branch if possible? I can do it, but I figured you might want to 😉

[kirrg001]

@ErisDS sure will create