Add clould-centric docs about getting started wtih the stack

[dedemorton]

Right now, the stack getting started docs show how to install and run the OSS components of the stack. For users who are interested in cloud, though, the setup is much simpler. They can eliminate the first two parts of the setup (installing ES and Kibana) and simply set up a cloud account. Then when they configure Beats, they set the cloud.id and cloud.auth config options, and are quickly able to see results (with minimal effort).

The stack getting started guide does mention that there is a hosted Elastic Service available on cloud, but I'm wondering if a full tutorial that highlights the simplicity of cloud might be worthwhile. The new tutorial would be very similar to the full stack (OSS) tutorial, except:

• The sections about installing ES and Kibana would be replaced with some simple steps about activating a trial account.
• The info about setting up Beats would be shared across the two tutorials, with some of the config steps conditionally coded to include cloud-specific config info.
• The Logstash section might go away completely (TBD).

Having a tutorial at the same level as the stack getting started (with a similar layout) raises the visibility of cloud and shows how easy it is to get started quickly. This is also a quick win because most of the content can be shared/reused with the OSS version of the tutorial.

@nrichers @lcawl Opening this as an item for discussion. If you think this is a good idea, we talk about ownership. Probably makes the most sense for Lisa (or me, if she's too busy) to own the overall tutorial and for Nik to advise on the contents of the cloud part. WDYT?

[tbragin]

+1 on a cloud-centric tutorial as part of stack getting started documentation.

[nrichers]

I'm +1 on adding more information for our hosted cloud offering, but there might be a some additional considerations. I had the stack getting started up against the Cloud getting started for bit today, hopping back and forth, and my impression was this:

• A full tutorial would essentially duplicate much of the content we already have in our Cloud docs. Looking at Getting started with the Elastic Stack, the steps run you through setting up ES and Kibana, followed by setting up Beats and Logstash. In our Getting Started for Cloud, we have a very lean, short tutorial for getting a deployment with ES and Kibana up, in keeping with how simple the service is to set up, followed by separate docs for setting up Beats and Logstash with the Cloud ID (content that the three of us collaborated on a while back). The latter are linked to as next steps after creating a deployment. Compared to what DeDe is proposing here, the Cloud docs seem to cover an awful lot of the same ground, in not many more words.
• Our Elasticsearch Service itself has grown to have terminology of its own that needs to be understood. It's no longer the case that you just create an Elasticsearch cluster and then add Kibana with a fairly basic UI. You create a deployment and there are some thing around the templates that we offer that you need to understand in order to make use of the service effectively. The UI is now based on the one from ECE, with the added functionality provided by DNT. So you'd have to pull in additional info from the actual cloud docs to explain it all, but that just starts to look like more duplication.
• Our hosted Elasticsearch Service itself is not part of the Elastic Stack, and so having a full tutorial in the stack docs for something that is not actually a part of the stack (especially if there's very similar documentation for this available elsewhere), seems out of step with the guidance from Haley and others.

Based on these points, I would suggest that we not add a tutorial for Cloud. One method that other books have adopted is to add callouts in a few strategic locations that point to our Elasticsearch Service better, but I don't know if this would be acceptable to you? If you really do want a tutorial for Cloud in here somewhere, perhaps the next step would be for us to chat about how we can make this happen, but I'm out until the beginning of September.

@kellyemurphy @KOTungseth I don't mean to speak for all of Cloud in my comments. If you have any thoughts on this proposal, please add them?

[dedemorton]

@nrichers We already have pointers to the cloud docs, so I think we're covered there. I just think we are missing out on the opportunity to encourage adoption by funneling users through the quickest possible getting started experience (cloud + beats + ingest) right up front. Having the full experience documented in one place is pretty powerful. Whether that content belongs in the cloud docs or the stack docs is debatable. I'd love to hear other opinions on this question. If you want to monitor the performance of your servers and you've heard that Elastic software will do that for you, where do you go? Do you look for documentation about getting started with the stack, or do you know right away that you want to use our hosted service? As long as we are careful to introduce cloud as a commercial offering, I think we can have a brief section about cloud in a tutorial under the stack docs without causing confusion.

I opened this issue because I was struck by how simple it was to get started when you use cloud. I found the section about cloud ID in the cloud docs to be a bit over-engineered, and it had some issues (see https://github.com/elastic/cloud/issues/19214). The only thing I really needed to know was where to find the cloud ID and cloud auth info. Once I figured that out from skimming the cloud docs, I was able to send data to cloud successfully. That suggests to me that we can provide a leaner getting started experience. I'll throw in one unknown here, though...I went through the steps before DNT was released. I would be worried in general, though, if we are requiring users to know special terminology up front in order to set up and use a trial version of our products. :-/

I'm not really concerned about duplication because that's inevitable with a tutorial. In fact, the tutorial (and even the cloud getting started!) already duplicates info that you'll find in the Beats and Logstash documentation.

It's your call. I just wanted to bring up the idea because I felt we were making the setup seem overly complex for users who just want to try out the software.

[nrichers]

@dedemorton Thank you for your thoughts. I trust your judgement, so let's chat when I am back at work next week? I invited you to one of our Cumulonimbus! syncs with Kaarina and Kelly so that we can hash out the details.

I found the section about cloud ID in the cloud docs to be a bit over-engineered, and it had some issues (see elastic/cloud#19214).

Agreed, this needs to be fixed. The docs we have now were a collaboration between you, me, and Tanya, and the examples were tested by Jordan and Tudor. Let's also talk about how we should tackle this issue next week?

[nrichers]

We discussed this issue first during the @elastic/cloud-writers sync with @dedemorton and @lcawl, and then again with the larger group of Elastic writers.

Our discussion likely isn't quite done yet, but the interim action items we agreed on include:

• Single source a Cloud getting started for the stack GS, based on the current Cloud GS, but make it a quick start guide that excludes all but the absolutely essential details (similar to the ECE Quick Start)
• Remove Logstash from the current stack GS; likely not needed if you already use Beats.
• Remove Beats and Logstash steps from Cloud docs, leaving only the conceptual info; link to the steps elsewhere (stack GS or Beats/Logstash docs)
• Future: Consider removing other overlapping content from Cloud, such as the ES and Kibana GS content

[dedemorton]

@nrichers Just a couple of points because I left the meeting before we'd reach a consensus on all of your points:

• When I opened this issue originally, I proposed that we use shared content from the stack getting started and add cloud info to create a new getting started (see my original comment). Pulling in some content from the cloud GS (alongside content from the stack GS) might make sense, too, but I don't want to lose the platform-specific steps that are currently in the stack GS. The steps are as relevant for cloud as they are for users running ES and Kibana on their own hardware.

  After you've gotten an outline together, maybe we can meet to discuss the structure. I think there's a real benefit in using a parallel structure because it will underscore how much easier cloud makes the setup and configuration. If we can make the stack GS better by making it leaner, I would love to do that, too!

• I don't want to remove Logstash from the stack GS right now. We might in the future, but we should have a discussion with other stake holders first. So let's make the action item say that we will "consider removing" the content.

[dedemorton]

@lcawl FYI ^^ (just noticed there was a typo in your name earlier)

[dedemorton]

There's a cross-team effort to resolve the proliferation of getting started content (and improve the user experience), so I'm closing this old issue.