Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated Python3 Version #69

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,8 @@ tmp
import/*.csv
import/*.json
venv
.vscode/*
client_secrets.json
.gcloudignore
service_account_credentials.json
cron.yaml
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
language: python
python:
- "2.7"
- "3.7"
# command to install dependencies
install: "pip install tox"
# command to run tests
Expand Down
84 changes: 28 additions & 56 deletions app.yaml
Original file line number Diff line number Diff line change
@@ -1,58 +1,30 @@
application: yelplove
service: default
version: 1-0-0
runtime: python27
api_version: 1
threadsafe: true
runtime: python37
instance_class: F2

handlers:
- url: /api/.*
script: main.app
secure: always
- url: /keys/?.*
script: main.app
login: admin
secure: always
- url: /subscriptions/?.*
script: main.app
login: admin
secure: always
- url: /aliases/?.*
script: main.app
login: admin
secure: always
- url: /robots.txt
static_files: static/robots.txt
upload: static/robots.txt
secure: optional
- url: /static
static_dir: static
secure: always
- url: /_themes/(.*)/img/(.*)
static_files: themes/\1/static/img/\2
upload: themes/(.*)/static/img/(.*)
secure: always
login: optional
- url: .* # Anything not explicitly listed above
script: main.app
login: required
secure: always

builtins:
- remote_api: on

libraries:
- name: ssl
version: latest

skip_files:
- ^(.*/)?#.*#$
- ^(.*/)?.*/RCS/.*$
- ^(.*/)?.*\.py[co]$
- ^(.*/)?.*~$
- ^(.*/)?\..*$
- ^YelpLove.egg-info(/.*)?$
- ^config-example.py$
- ^google_appengine(/.*)?$
- ^tmp(/.*)?$
- ^virtualenv_.*$
- url: /api/.*
script: auto
secure: always
- url: /keys/?.*
script: auto
secure: always
- url: /subscriptions/?.*
script: auto
secure: always
- url: /aliases/?.*
script: auto
secure: always
- url: /robots.txt
static_files: static/robots.txt
upload: static/robots.txt
secure: optional
- url: /static
static_dir: static
secure: always
- url: /_themes/(.*)/img/(.*)
static_files: themes/\1/static/img/\2
upload: themes/(.*)/static/img/(.*)
secure: always
- url: .* # Anything not explicitly listed above
script: auto
secure: always
12 changes: 12 additions & 0 deletions config-example.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,15 @@
GRAVATAR = 'backup'

ORG_TITLE = 'All Company'

EMAIL_SERVICE_ACCOUNT = 'example@appspot.gserviceaccount.com'

# From address of love email address
EMAIL_DELEGATION_ADDRESS = 'test_email@example.com'

PROJECT_NAME = 'test-project'

DB_USER = 'user'
DB_PASSWORD = 'password'
DB_NAME = 'employees'
DB_CONNECTION_NAME = 'project:region:name'
8 changes: 4 additions & 4 deletions logic/__init__.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# -*- coding: utf-8 -*-
from datetime import timedelta
from itertools import izip_longest
from itertools import zip_longest

import pytz
from google.appengine.ext import ndb
from google.cloud import ndb
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very neat, I didn't know about this! This ndb library is not mentioned at all in the datastore migration documentation. Apparently it uses Cloud Firestore internally. Have you thought about how we're going to transfer the data? Or would we start with a new, fresh datastore?

Otherwise we could try out python-ndb, which should allow us to access the data we're currently using.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may be wrong but it appears the python-ndb is the same as google-cloud-ndb. The docs actually show to pip install google-cloud-ndb here.

Also I believe google migrated all legacy datastore db's to use firestore in datastore mode. Shootie was upgraded using the same libraries so we wouldn't need to transfer any data and will still use the same datastore.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! That's perfect then!



TIMESPAN_LAST_WEEK = 'last_week'
Expand All @@ -13,11 +13,11 @@
def chunk(iterable, chunk_size):
"""Collect data into fixed-length chunks or blocks (http://docs.python.org/2/library/itertools.html#recipes)"""
args = [iter(iterable)] * chunk_size
return izip_longest(*args)
return zip_longest(*args)


def to_the_future(dict):
for k, v in dict.iteritems():
for k, v in list(dict.items()):
if issubclass(v.__class__, ndb.Future):
dict[k] = v.get_result()

Expand Down
4 changes: 2 additions & 2 deletions logic/department.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@

META_DEPARTMENT_MAP = {
meta_department: set(departments)
for meta_department, departments in _departments.iteritems()
for meta_department, departments in _departments.items()
}


def get_meta_department(dept):
for meta_dept, dept_set in META_DEPARTMENT_MAP.iteritems():
for meta_dept, dept_set in META_DEPARTMENT_MAP.items():
if dept in dept_set:
return meta_dept
return DEFAULT_META_DEPARTMENT
52 changes: 44 additions & 8 deletions logic/email.py
Original file line number Diff line number Diff line change
@@ -1,24 +1,35 @@
# -*- coding: utf-8 -*-
from google.appengine.api.mail import EmailMessage

from util.email import get_name_and_email
import config
import logic.secret

from googleapiclient.discovery import build

from email.mime.text import MIMEText

import base64

from google.oauth2 import service_account
import json

if config.EMAIL_BACKEND == 'sendgrid':
# a bit of a hack here so that we can avoid adding dependencies unless
# the user wants them
import sendgrid

with open('service_account_credentials.json') as json_file:
CREDS = json.loads(json_file.read())


def send_appengine_email(sender, recipient, subject, body_html, body_text):
email = EmailMessage()
email.sender = sender
email.to = recipient
email.subject = subject
email.body = body_text
email.html = body_html
email.send()
message = create_message(
config.EMAIL_DELEGATION_ADDRESS, recipient, subject, body_html,
)

service = service_account_login()
# me is the magic word to reference the account credentials
send_message(service, 'me', message)


def send_sendgrid_email(sender, recipient, subject, body_html, body_text):
Expand Down Expand Up @@ -52,3 +63,28 @@ def send_email(sender, recipient, subject, body_html, body_text):
"""
backend = EMAIL_BACKENDS[config.EMAIL_BACKEND]
backend(sender, recipient, subject, body_html, body_text)


def create_message(sender, to, subject, message_text):
message = MIMEText(message_text, 'html')
message['to'] = to
message['from'] = sender
message['subject'] = subject
return {'raw': base64.urlsafe_b64encode(message.as_bytes()).decode()}


def send_message(service, user_id, message):
service.users().messages().send(userId=user_id, body=message).execute()
return message


def service_account_login():
SCOPES = ['https://mail.google.com/']
SERVICE_ACCOUNT_FILE = 'service_account_credentials.json'

credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES
)
delegated_credentials = credentials.with_subject(config.EMAIL_DELEGATION_ADDRESS)
service = build('gmail', 'v1', credentials=delegated_credentials)
return service
Loading