Load test: sending to lobid-resources subject inboxes

[fsteeg]

• Every day, for every new lobid-resource, for each of its subjects, POST an LDN to the subject's inbox, e.g. to https://test.skohub.io/inbox?target=http://test.lobid.org/gnd/4020517-4
• Send the lobid-resource's full JSON as the POST body
• Spread the updates over the day to simulate live updates

While we don't have a live system behind lobid-resources, the idea here is to simulate the load created by multiple senders. To increase the amount of messages, we could change this from subjects to authors in the future.

[dr0i]

Wrote a gist https://gist.github.com/dr0i/72fdc385fc4e72a6930f1644523bbdc6.
When https://test.skohub.io/ inbox is up, it may be crontabbed.

[acka47]

When https://test.skohub.io/ inbox is up, it may be crontabbe

https://test.skohub.io/ is up now, see skohub-io/skohub-pubsub#12

[dr0i]

Up'n running. Crontabbed starting every day at 09:00.

[fsteeg]

Just a heads-up for @literarymachine: https://test.skohub.io/ should start receiving automated traffic starting tomorrow 09:00, see #986 (comment) (first comment in this issue).

[dr0i]

@literarymachine @fsteeg it's already started and sending. And will , from now on, start every day at 9:00.

[dr0i]

Create #992 to be able to see what's happening.
@fsteeg If this issue is good you can close it.

[acka47]

Talking to @literarymachine at ELAG2019, he said that it would make sense to make the load test a bit harder by sending out all the notifications at once as soon as the daily updates are in. Reopening to test this scenario for some days.

[dr0i]

Removed the sleep. This script will now work as fast as it can, I expect no more than 10/s.

[literarymachine]

Removed the sleep. This script will now work as fast as it can, I expect no more than 10/s.

I haven't closely monitored the server, but it is still up and running. Thinking about this, I should probably add some logging: skohub-io/skohub-pubsub#14

[literarymachine]

To increase the amount of messages, we could change this from subjects to authors in the future.

Why not both?

[dr0i]

Is this running test actually useful or can I stop if it's indeed unnecessarily producing load?

[acka47]

I guess you can stop it for now.

[acka47]

@dr0i have you stopped the load test already? I not, please do and close this issue when it is stopped.

[dr0i]

Yeah I stopped it already at 11th July. For reference, it can be started again enabling dispatchLobidResourcesToSkohubInbox.sh, see crontab. Closing.