Freeze the s3 APIs until the notification sub-system initializes completely

[Praveenrajmani]

Description

Currently, we will miss the events if MinIO server receives the API requests before the notification sub-system is initialized and the bucket target rules are populated.

This PR will freeze the APIs until these subsystems get completely initialized.

Motivation and Context

• No events should me missed on server restarts
• No interim events should be missed between the bootstrap and notification-subsystem initialization.

How to test this PR?

• Configure bucket notifications with or without queue_dir
• In master apply the following diff to simulate the config subsystem and bucket target rules time lag.

diff --git a/cmd/server-main.go b/cmd/server-main.go
index e212e13e2..4f5e047e8 100644
--- a/cmd/server-main.go
+++ b/cmd/server-main.go
@@ -417,6 +417,8 @@ func initServer(ctx context.Context, newObject ObjectLayer) error {
 
        r := rand.New(rand.NewSource(time.Now().UnixNano()))
 
+       time.Sleep(3*time.Minute)
+
        for {
                select {
                case <-ctx.Done():

• Start MinIO - it will be sleeping for 3 minutes.
• Copy few objects to MinIO - it will be successful but no events can be seen or persisted for those objects.
• With this change in the PR, the API's will freezed until the targets and configs are initialized and then unfreezes once done. Here, we won't miss any events.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Optimization (provides speedup with no functional changes)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• Fixes a regression (If yes, please add commit-id or PR # here)
• Unit tests added/updated
• Internal documentation updated
• Create a documentation update request here

[Praveenrajmani]

@harshavardhana Can we print the startup message once all the initializations return? Because, with this change, we can't see SQS ARNs as the config wouldn't have been loaded yet.

Can we display the startup message only after all the go-routines return (ie, wg.Wait)

[harshavardhana]

This is not yet correct @Praveenrajmani let me send an update to this branch.

[harshavardhana]

@Praveenrajmani see what I wanted to do here. Mostly this would work if you can give this a test.

[Praveenrajmani]

yes, this works @harshavardhana

[vadmeste]

LGTM

[poornas]

LGTM