[HUDI-3911][DOCS][WIP] Add async indexing doc

[codope]

[bhasudha]

@codope I read through the blog. Thanks for the great writeup. I have a couple of thoughts/questions from an external perspective.

• Should we name this blog to explicitly mention as Asynchronous metadata indexing using Hudi. This will avoid confusion with the current indexing concept here for users - https://hudi.apache.org/docs/indexing ?
• Also I see that we have a category of Services in our Docs. There is a page for clustering, compaction, etc in there. Since this is one of the table services, should we also add a page on metadata indexing under Services. It can have a brief description and point to this blog for detailed content.
• One last thing is, seeing the description, I also wanted to understand what happens if we disable hoodie.metadata.index.async . What is the behavior then ? Does it run inline ? What are the implications. It would be good to see those explained as well.

Let me know your thoughts.

[nsivabalan]

I like the motivation section 👏

[codope]

@codope I read through the blog. Thanks for the great writeup. I have a couple of thoughts/questions from an external perspective.

• Should we name this blog to explicitly mention as Asynchronous metadata indexing using Hudi. This will avoid confusion with the current indexing concept here for users - https://hudi.apache.org/docs/indexing ?
• Also I see that we have a category of Services in our Docs. There is a page for clustering, compaction, etc in there. Since this is one of the table services, should we also add a page on metadata indexing under Services. It can have a brief description and point to this blog for detailed content.
• One last thing is, seeing the description, I also wanted to understand what happens if we disable hoodie.metadata.index.async . What is the behavior then ? Does it run inline ? What are the implications. It would be good to see those explained as well.

Let me know your thoughts.

Good points @bhasudha

I have moved the setup and usage related sections to a separate page under Services tab, check out #5476
I need to do some more work on this blog. I will rename, add more design elements and diagram. If async indexing is disabled with any of the index types enabled then those metadata partitions will be created/updated synchronously. Added this line in caveat section in #5476

[codope]

Should I add the hudi stack figure from https://hudi.apache.org/blog/2021/07/21/streaming-data-lake-platform/#hudi-stack

[codope]

Closing this PR #5476 already merged