Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add appengine/bigquery sample #2

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions appengine/bigquery/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Accessing BigQuery from App Engine
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a short section with deployment instructions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


This code has been written to support this [blog post][1].

You can run this application locally or deploy it to App Engine app servers.
To do so you will need the `goapp` tool included in the [Go App Engine SDK][2].

- get all the dependencies for this code snippet

goapp get .

- deploy to App Engine servers

goapp deploy --application=[your-application-id] .

[1]: https://medium.com/@francesc/accessing-bigquery-from-app-engine-d01823de81ee
[2]: https://cloud.google.com/appengine/downloads?hl=en
77 changes: 77 additions & 0 deletions appengine/bigquery/app.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
// Copyright 2015 Google Inc. All rights reserved.
// Use of this source code is governed by the Apache 2.0
// license that can be found in the LICENSE file.

// This App Engine application uses its default service account to list all
// the BigQuery datasets accessible via the BigQuery REST API.
package sample

import (
"fmt"
"net/http"
"strings"

"golang.org/x/net/context"
"golang.org/x/oauth2"
"golang.org/x/oauth2/google"
"google.golang.org/api/bigquery/v2"
"google.golang.org/appengine"
"google.golang.org/appengine/urlfetch"
)

func init() {
// all requests are handled by handler.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this comment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comment above.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, it's not a very useful comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

http.HandleFunc("/", handle)
}

func handle(w http.ResponseWriter, r *http.Request) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if r.URL.Path != "/" {
// 404
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine like this, ANY url will be handled by this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Including /favicon.ico. It's wasteful to basically double up on requests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

// create a new App Engine context from the request.
c := appengine.NewContext(r)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want c -> ctx.

(ctx because we're using the actual context package now.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


// obtain the list of dataset names.
names, err := datasets(c)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}

w.Header().Set("Content-Type", "text")

if len(names) == 0 {
fmt.Fprintf(w, "no datasets visible")
} else {
fmt.Fprintf(w, "datasets:\n\t"+strings.Join(names, "\n\t"))
}
}

// datasets returns a list with the ids of all the Big Query datasets visible
// with the given context.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems like it'll list all the bigquery projects in the same project as the running App Engine app.

func datasets(c context.Context) ([]string, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c --> ctx

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

// create a new authenticated HTTP client over urlfetch.
client := &http.Client{
Transport: &oauth2.Transport{
Source: google.AppEngineTokenSource(c, bigquery.BigqueryScope),
Base: &urlfetch.Transport{Context: c},
},
}

// create the BigQuery service.
bq, err := bigquery.New(client)
if err != nil {
return nil, fmt.Errorf("create service: %v", err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could not create service:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

}

// obtain the current application id, the BigQuery id is the same.
appID := appengine.AppID(c)

// prepare the list of ids.
var ids []string
datasets, err := bq.Datasets.List(appID).Do()
if err != nil {
return nil, fmt.Errorf("could not list datasets for %q: %v", appID, err)
}
for _, d := range datasets.Datasets {
ids = append(ids, d.Id)
}
return ids, nil
}
10 changes: 10 additions & 0 deletions appengine/bigquery/app.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Copyright 2015 Google Inc. All rights reserved.
# Use of this source code is governed by the Apache 2.0
# license that can be found in the LICENSE file.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need for copyright header on app.yaml

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is that new? ok, done

version: 1
runtime: go
api_version: go1

handlers:
- url: /.*
script: _go_app