rails tutorial with gem "google-cloud"

[dhirajbajaj]

Hi all,

I have been exploring GAE for rails and couldn't find a appropriate tutorial for clarity.
Either way, i am left a bit confused about using datastore, storage, service accounts, keyfile.json, setting up emulator, gem "gcloud" / "google-cloud" etc.

Can anyone share a repo or tutorial that can helps to test and get running quickly with datastore and other services based on the concept.

[quartzmo]

Hi @dhirajbajaj, did you try this in-depth tutorial?

https://cloud.google.com/ruby/getting-started/tutorial-app

You can chose to use Cloud Datastore on the second page. It also covers Storage, Logging and Pub/Sub.

[quartzmo]

We recently changed the name of this library from "gcloud" to "google-cloud" and the tutorial needs to be updated, but all of the examples should work with the current version of this library.

[dhirajbajaj]

Yes @quartzmo, i followed bookshelf tutorial, and i got Google::Cloud API error with cloud-data-store example which led me to fix it with setting up credentials and emulator for local env.
I got it working locally, but its raising 503 when gcloud app deploy which is cloudstore - access error.

It makes me to think that i am missing something when deploying. Thus the issue. :)
Can you share any sample repo/pointers that would clarify.

[quartzmo]

I don't know of any other resources, but I think it would be great to document what you experienced in detail here on this issue, for three reasons:

1. @blowmage and I can hopefully help you find a solution right away.
2. In time, the authors of the tutorial will be able to fix/update what's not working.
3. Anyone else experiencing the same problem will hopefully land on this issue, and find the solution.

Can you back up to where you first got an error while following the tutorial, and paste it here, along with any environmental details you can capture (actual gem versions)? Leave out anything sensitive like your auth details of course.

[bmclean]

@dhirajbajaj We have an example Rails app using 'google-cloud-datastore' located here which works on Heroku.

[dhirajbajaj]

@quartzmo yes this makes sense. I will share it by tomorrow. Allow me some time.

@bmclean thanks for sharing. But would be great if its deployable to GAE making it more manageable.
Btw, i followed this repo only to setup datastore.rb initializer for local env.

[quartzmo]

@dhirajbajaj Sounds good, thank you! Once you've posted here, I'll try to reproduce your issue as well.

[bmclean]

@dhirajbajaj OK, perfect. At one point we did have the Agrimatics Rails app running on the App Engine Managed VMs (I think it is called Flexible Environment now) so I might be able to help.

[dhirajbajaj]

@quartzmo

Some background: I am still exploring and coming from quick deployment mindset like capistrano, rubber to ec2 with clarity on configs, or to heroku setting up config/postgres service from cli.

I started with ruby documentation/quickstarts, installed cloud sdk and then moved to bookshelf tutorial to try it out.
I use ruby 2.3.1, rails 5, gcloud info

I ran the example with the instructions in the tute :

dhirajbajaj in /Volumes/SIGMA/_app/codebin/labs/yochk/ruby-test/2-cloud-datastore on master*
⚡ rails s
=> Booting WEBrick
=> Rails 4.2.6 application starting in development on http://localhost:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
[2016-09-20 12:53:16] INFO  WEBrick 1.3.1
[2016-09-20 12:53:16] INFO  ruby 2.3.1 (2016-04-26) [x86_64-darwin14]
[2016-09-20 12:53:16] INFO  WEBrick::HTTPServer#start: pid=37649 port=3000

Started GET "/" for ::1 at 2016-09-20 12:53:23 +0530
Processing by BooksController#index as HTML
Completed 500 Internal Server Error in 7006ms

Gcloud::Datastore::ApiError (API call to runQuery was not successful):
  app/models/book.rb:41:in `query'
  app/controllers/books_controller.rb:20:in `index'

  Rendered /Users/dhirajbajaj/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/actionpack-4.2.6/lib/action_dispatch/middleware/templates/rescues/_source.erb (4.7ms)
  Rendered /Users/dhirajbajaj/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/actionpack-4.2.6/lib/action_dispatch/middleware/templates/rescues/_trace.html.erb (2.7ms)
  Rendered /Users/dhirajbajaj/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/actionpack-4.2.6/lib/action_dispatch/middleware/templates/rescues/_request_and_response.html.erb (14.5ms)
  Rendered /Users/dhirajbajaj/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/actionpack-4.2.6/lib/action_dispatch/middleware/templates/rescues/diagnostics.html.erb within rescues/layout (75.9ms)

At this point, i had no clarity about setting up credentials, as it isn't mentioned in the tutorial docs, plus service account details and setting up emulator with beta components required for local development.

It came to me after digging from issues like : API error issue and also changing to google-cloud gem with @bmclean solution.

Also, got some other commit errors:

Started POST "/books" for ::1 at 2016-09-20 14:35:36 +0530
Processing by BooksController#create as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"V0bHlP0tUWtth+/STNRX/7SlCf/WnxT3cHjZgbv5wOE9hZgtQmOVYQZMyNLHOuYq3VuMx34xl4AfMLtxDEFfwg==", "book"=>{"title"=>"Awesome fawsome", "author"=>"fawsome", "published_on"=>"2015-11-29", "description"=>"dsjkda dk adkjad asdmnad asd"}}
Completed 500 Internal Server Error in 844ms

Gcloud::Datastore::ApiError (API call to commit was not successful):
  app/models/book.rb:83:in `save'
  app/controllers/books_controller.rb:65:in `create'

I set up the emulator, and credentials to setup env variables and cloud-data-store to get it running on local env using:

datastore_initializer.rb : 
   if Rails.env.development? 
      ENV['DATASTORE_EMULATOR_HOST'] = 'localhost:8111'
      ENV['GOOGLE_CLOUD_PROJECT'] = 'project-id'
   elsif... // test env 
   else // production
      ENV['GOOGLE_CLOUD_KEYFILE'] = "/config/project-config.json"
   end
    require 'google/cloud/datastore'
    @dataset = Google::Cloud.datastore(ENV['GOOGLE_CLOUD_PROJECT'])

dhirajbajaj in /Volumes/SIGMA/_app/codebin/labs/yochk/ruby-test/2-cloud-datastore on master*
⚡  gcloud beta emulators datastore start --host-port localhost:8111

While this works for local env. When i deploy this to production, it raises 503 server error. The log shows 500/301 raised by _ah/health VMruntimehealthchecker. so clearly there is some issue with code/config.

I will try to fix it. Any pointers will be great.

Some suggestion to tutorial would be to add :

• clarity about env variables, service accounts and where it has to be utilized as some-where it mentions app-engine and compute engine will automatically pickup credentials.
• clarity about querying data-store, emulator for local dev, setting entities, basics etc.
• clarity about logging/debugging for local env.
• clarity about deployment config etc.
• different naming convention Gcloud sdk, gem "gcloud" / "google-cloud-ruby" now.

Hope you find it useful.

@bmclean Please share.

[bmclean]

Regarding _ah/health, take a look at the Procfile from 6-task-queueing in getting-started-ruby. You need something that responds to those health requests, otherwise the instance will be classified as unhealthy and it won't receive any requests. The example uses health_check.ru. Alternatively you could create a controller to handle them:

class AppEngineController < ApplicationController
  def health
    render plain: 'OK'
  end
end

with a route like so:

get '_ah/health', to: 'app_engine#health'

If you are fighting with 301 redirects also take a look at this issue related to using SSL with Rails on the Flexible Environment. We had to disable SSL to get the Rails app to run.

[quartzmo]

@dhirajbajaj Thank you, this is great. I will try to follow along and reproduce today.

Do you have any feedback on our google-cloud-datastore Authentication docs?

And did you try connecting locally to Cloud Datastore (not the emulator) using just google-cloud-datastore? You could even do this outside of Rails, in IRB for example.

[dhirajbajaj]

@bmclean Thanks i will try it out. 👍

@quartzmo Just feel that authentication docs should be to be linked in the tutorial in prerequisite section to cloud-data-store or much better to include the setup step in the tutorial.

Yes, using only gem "google-cloud-datastore" runs in IRB. I tried it.
Now i see the problem was the env variables and bmclean pointers. Still far from a production level app deployment.

[quartzmo]

@dhirajbajaj

authentication docs should be to be linked in the tutorial in prerequisite section to cloud-data-store or much better to include the setup step in the tutorial.

This and your other bullet points, above, are valuable feedback for the authors of the tutorial, and we will do what we can to bring it to their attention. As far as I know the tutorial is not open-source.

different naming convention Gcloud sdk, gem "gcloud" / "google-cloud-ruby" now.

The sample application is open-source, so I assume the best way to update the gem name(s) from gcloud to google-cloud in the tutorial examples is to open a PR there. Although, since there is already one open for a previous version of gcloud Datastore (GoogleCloudPlatform/getting-started-ruby#55), maybe an issue is the best way to start.

Is there anything more we can do within the scope of this project? If not, can you close the issue?

[quartzmo]

@bmclean Like usual, thank you!!!

[quartzmo]

Thank you @dhirajbajaj for creating the issue for the Bookshelf tutorial, hopefully it will be updated soon with our new library and gem name(s).

Can I close this issue now?

[dhirajbajaj]

@quartzmo Yes, you can close it. I have got it deployed with @bmclean pointers.

Its working on production except for create/update/delete request are raising 422 error whereas in dev env its create/update/delete are working fine.

[quartzmo]

Closing, kudos to @bmclean.

[bmclean]

Thanks @quartzmo!