Update overviews with more detailed code

[jgeewax]

• BigQuery
• Bigtable
• Compute Engine
• Datastore
• DNS
• Language
• Logging
• Prediction
• Pub/Sub (including autoAck demonstration)
• Resource Manager
• Spanner
• Speech
• Storage
• Translate
• Vision

Right now, clicking on a service (ie, Datastore) in the docs brings me to a page that has a brief overview and then the stuff that happens to be at the top level (like... int and double).

Can we restructure this a smidge?

1. Move double and int somewhere else (at least in the docs?)
   On this page, I'm really not looking for such a low-level detail, I'm looking for high-level stuff. If I want to get an idea of how to store special types, I'd be looking for a like to "Types" or something along those lines.
2. The brief overview is fine, but it's bascially a 301 in that it says "you really want the dataset result... take a look at that". Can we make this act as a more friendly and (somewhat) comprehensive overview? Basically "here's the typical use case for [service], and how you write code to do it".

For example, with Datastore:

"""
The gcloud.datastore object gives you some convenience methods, as well as exposes a dataset function. This will allow you to create a dataset, which is the object from which you will interact with the Google Cloud Datastore.

The basic flow is:

1. Establish a connection to Cloud Datastore:

   var gcloud = require('gcloud')({
     projectId: 'grape-spaceship-123',
     keyFilename: '/path/to/keyfile.json'  // You can get this from the cloud console
   });
   
   var datastore = gcloud.datastore;
   var dataset = datastore.dataset();

   (Now that you have a dataset reference, take a look at the [link to dataset docs])

2. Add a TodoItem entity (if you don't know what an entity is, click here [link to entity concept on cloud.google.com]):

   var key = dataset.Key('TodoItem');
   var data = {title: "Buy milk", due: new Date('Tue May 12 2015 15:30:00 GMT-0400 (EDT)')};
   
   dataset.save({key: key, data: data}, function(err) {
     if (err) {
       console.log("Uh oh, something went wrong:", err);
     } else {
       console.log("Todo item saved:", key.path); // ['TodoItem', 5669468231434240]
   });

3. Retrieve your entity by it's key (if you don't know what a key is, click here [link to key concept on cloud.google.com]):

   var key = dataset.key(['TodoItem', 5669468231434240]);
   
   dataset.get(key, function(err, entity) {
     if (err) {
       console.log("Uh oh, something went wrong:", err);
     } else {
       console.log("Your entity is:", entity);
     });

The other things you might want to do are:

• Run queries ("Give me all TodoItems due tomorrow"): [link to query docs]
• Do multiple operations in one atomic group: [link to transactions docs]

"""

(And remember -- these code snippets must be tested so that if our code would break them, the tests fail. We can't have docs out there that don't work...)

[stephenplusplus]

I think this sounds like https://github.com/GoogleCloudPlatform/gcloud-common/issues/33#issuecomment-164104675 -- does the structure explained in that post align with the goals of this issue?

[callmehiphop]

@jgeewax I think this came up before and we moved it to https://github.com/GoogleCloudPlatform/gcloud-common/issues/41. Is node the only one who is experiencing this problem? If so, we could probably just add more examples to the constructor documentation.. otherwise we'll need a cross library solution.

@stephenplusplus the guides idea never really went anywhere (but I still think it is a great idea) and I have a feeling it would probably take a little while to actually get that implemented.

[stephenplusplus]

I think to implement this idea, it would be the same solution; versioned markdown files that live with the JSON output. I don't think a static overview.html holds up anymore for this use case.

[callmehiphop]

Per our spec, we actually have a field for an overview. Currently in node we didn't really have anything to populate it with (because we were using the static html files) but assuming we could find a way to finagle it in, this might be a fairly simple task..

[stephenplusplus]

Ah, cool. So this is how I see the navbar being re-structured:

• Datastore (<-- link that goes to overview page)
  • API
  • Datastore (<-- link that goes to API method dump)
  • Dataset (<-- link that goes to API method dump)
  • Query (<-- link that goes to API method dump)
  • Transaction (<-- link that goes to API method dump)

Does this force other changes on the JSON?

[jgeewax]

s/API/Overview/ ? API doesnt' really say "tell me about it in a non-scary way")

[stephenplusplus]

The bullets under "API" aren't overviews, they are all API docs. Clicking the top-most "Datastore" (Datastore (tell me about Datastore + gcloud in a non-scary way. Don't dump API docs on me yet, man)) would take you to an overview page.

[callmehiphop]

I don't think so, since we've agreed to go to a multi file approach then, we were going to need a table of contents JSON file to map the navigation to documentation JSON files.. (shaking up the navigation would then only be a config change)

There would be some change in generating the files, but I don't think it would be anything crazy. Our real problem is figuring out where to store the overview section.

[stephenplusplus]

Will the JSON be able to include headings/dividers?:

{
  nav: [
    {
      title: "Datastore",
      contents: "overview.md"
    },
    {
      title: "API",
      contents: [
        { title: "Datastore", contents: "datastore.json" },
        { title: "Dataset", contents: "dataset.json" },
        { title: "Query", contents: "query.json" },
        { title: "Transaction", contents: "transaction.json" }
      ]
    }
  ]
}

Or should we just force the hierarchy from my earlier post and just let the front end decide where to put things based on file type:

{
  nav: [
    { title: "Datastore", contents: "overview.md" },
    { title: "Datastore", contents: "datastore.json" }, // FE sees a JSON file, automatically considers it API docs
    { title: "Dataset", contents: "dataset.json" }, // FE sees a JSON file, automatically considers it API docs
    { title: "Query", contents: "query.json" }, // FE sees a JSON file, automatically considers it API docs
    { title: "Transaction", contents: "transaction.json" } // FE sees a JSON file, automatically considers it API docs
  ]
}

[callmehiphop]

I was actually thinking somewhere in the middle

{
  nav: [
    {
      title: 'Datastore',
      overview: 'datastore.md',
      contents: [
        {
          title: 'Query',
          contents: 'query.json'
        }
      ]
    }
  ]
}

[stephenplusplus]

Oops, forgot to nest an extra level. Just so we have an even comparison, here's my updated proposal:

{
  nav: [
    {
      title: "Datastore",
      contents: "overview.md",
      nav: [
        {
          title: "API",
          nav: [
            { title: "Datastore", contents: "datastore.json" },
            { title: "Dataset", contents: "dataset.json" },
            { title: "Query", contents: "query.json" },
            { title: "Transaction", contents: "transaction.json" }
          ]
        }
      ]
    }
  ]
}

The reason I like this approach is the ability to expand if we want extra overview files (like ones for Dataset, Query, etc):

{
  nav: [
    {
      title: "Datastore",
      contents: "overview.md",
      nav: [
        { title: "Dataset", contents: "dataset.md" },
        { title: "Query", contents: "query.md" },
        { title: "Transaction", contents: "transaction.md" },
        {
          title: "API",
          nav: [
            { title: "Datastore", contents: "datastore.json" },
            { title: "Dataset", contents: "dataset.json" },
            { title: "Query", contents: "query.json" },
            { title: "Transaction", contents: "transaction.json" }
          ]
        }
      ]
    }
  ]
}

[callmehiphop]

Got it, I totally dig that schema. I actually really like the separation between overviews and API docs as well.. reminds me of something we'd see upstream.

I really only see a few issues with this approach

1. We would need to author kind of a lot of content to support the overview sections
2. Using markdown files will slightly increase the complexity of testing the code snippets
3. We're going to have to remember to maintain the markdown files in the event of a code change, which historically we actually avoided this approach for that very reason (I think)

[stephenplusplus]

We would need to author kind of a lot of content to support the overview sections

Yeah, but I think anything is better than nothing. And if it's so simple that having a dedicated page just for it doesn't make sense, it can also go on the parent overview page.

Using markdown files will slightly increase the complexity of testing the code snippets

Definitely. But using .md files lets us merge words and code in a way that is the most easy to contribute to.

We're going to have to remember to maintain the markdown files in the event of a code change, which historically we actually avoided this approach for that very reason (I think)

Docs as close to the code as possible is the best, but there's no way around the distance this time :\

[callmehiphop]

I agree, I think the benefits outweigh the cons here for sure.. @jgeewax thoughts?

[jgeewax]

👍

[stephenplusplus]

@callmehiphop is this still relevant? I'm not sure of all that has evolved with the docs since we talked about this. Is it yet just a matter of us writing up some overview sections in MD files, or will this require more changes?

[jgeewax]

Just to clarify, the Ruby guys did a pretty nice job here:

• Ruby (nice pretty overview): http://googlecloudplatform.github.io/gcloud-ruby/docs/v0.6.2/Gcloud/Bigquery.html
• Node (not quite there yet): https://googlecloudplatform.github.io/gcloud-node/#/docs/v0.29.0/bigquery

I wonder though if we can write a single "article" (markdown) with included snippets where only the language changes... and then we'd basically have a mapping of sorts...

tasks = [{
  "service": "bigquery",
  "language": "node",
  "task": "datasets/list",
  "snippet": "<some spec pointing to a snippet of code somehow>"
},
...
]

[jgeewax]

@stephenplusplus Just bumping this back up. I went to the Datastore page on gcloud-node (https://googlecloudplatform.github.io/gcloud-node/#/docs/v0.30.1/datastore) and was sad.

I then went to the page for gcloud-ruby (http://googlecloudplatform.github.io/gcloud-ruby/#/docs/v0.7.2/gcloud/datastore), and was really happy...

Any idea when we can crank this out ?

Maybe @pcostell can help us point to a bunch of "tasks" that need to be documented, and we can then start pulling together some prose (shared) with some snippets (specific to a language) ?

[stephenplusplus]

I think we're just trying to figure out how the do this from a technical standpoint. The quickest resolution would be putting a bunch of HTML in the @example for the Datastore constructor.

@callmehiphop ideas on how to proceed?

[callmehiphop]

I believe the ruby team actually just dumps a bunch of HTML in their description tags. So we could definitely follow suit with some thing like this, or else treat all description fields as markdown, etc.

[stephenplusplus]

Took a shot at this in #1231 -- ptal!

[sduskis]

@stephenplusplus, this has not been active since 2016. What's the current state on this?

[callmehiphop]

I'm not sure this is still relevant. If anything we may want to address any clients where we did write an overview since they did not transition to the cloud site very well, e.g. Bigtable (scroll to the examples)

[JustinBeckwith]

I'm going to go ahead and close this out for now.