Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add appengine/bigquery sample #2

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
17 changes: 17 additions & 0 deletions appengine/bigquery/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Accessing BigQuery from App Engine
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a short section with deployment instructions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


This code has been written to support this [blog post][1].

You can run this application locally or deploy it to App Engine app servers.
To do so you will need the `goapp` tool included in the [Go App Engine SDK][2].

- get all the dependencies for this code snippet

goapp get .

- deploy to App Engine servers

goapp deploy --application=[your-application-id] .

[1]: https://medium.com/@francesc/accessing-bigquery-from-app-engine-d01823de81ee
[2]: https://cloud.google.com/appengine/downloads?hl=en
81 changes: 81 additions & 0 deletions appengine/bigquery/app.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
// Copyright 2015 Google Inc. All rights reserved.
// Use of this source code is governed by the Apache 2.0
// license that can be found in the LICENSE file.

// This App Engine application uses its default service account to list all
// the BigQuery datasets accessible via the BigQuery REST API.
package sample

import (
"fmt"
"net/http"
"strings"

"golang.org/x/net/context"
"golang.org/x/oauth2"
"golang.org/x/oauth2/google"
"google.golang.org/api/bigquery/v2"
"google.golang.org/appengine"
"google.golang.org/appengine/urlfetch"
)

func init() {
http.HandleFunc("/", handle)
}

func handle(w http.ResponseWriter, r *http.Request) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if r.URL.Path != "/" {
// 404
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine like this, ANY url will be handled by this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Including /favicon.ico. It's wasteful to basically double up on requests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

if r.URL.Path != "/" {
http.NotFound(w, r)
return
}

// create a new App Engine context from the request.
ctx := appengine.NewContext(r)

// obtain the list of dataset names.
names, err := datasets(ctx)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}

w.Header().Set("Content-Type", "text")

if len(names) == 0 {
fmt.Fprintf(w, "no datasets visible")
} else {
fmt.Fprintf(w, "datasets:\n\t"+strings.Join(names, "\n\t"))
}
}

// datasets returns a list with the ids of all the Big Query datasets visible
// with the given context.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems like it'll list all the bigquery projects in the same project as the running App Engine app.

func datasets(ctx context.Context) ([]string, error) {
// create a new authenticated HTTP client over urlfetch.
client := &http.Client{
Transport: &oauth2.Transport{
Source: google.AppEngineTokenSource(ctx, bigquery.BigqueryScope),
Base: &urlfetch.Transport{Context: ctx},
},
}

// create the BigQuery service.
bq, err := bigquery.New(client)
if err != nil {
return nil, fmt.Errorf("could not create service: %v", err)
}

// obtain the current application id, the BigQuery id is the same.
appID := appengine.AppID(ctx)

// prepare the list of ids.
var ids []string
datasets, err := bq.Datasets.List(appID).Do()
if err != nil {
return nil, fmt.Errorf("could not list datasets for %q: %v", appID, err)
}
for _, d := range datasets.Datasets {
ids = append(ids, d.Id)
}
return ids, nil
}
7 changes: 7 additions & 0 deletions appengine/bigquery/app.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
version: 1
runtime: go
api_version: go1

handlers:
- url: /.*
script: _go_app