Monkey-patch os.env to be threadlocal and set up environment to mimic GAE

[andrewsg]

This is a rearchitecture of the environment variable code to reduce code size and complexity.

R: @apolito @mingzhaodotname @gbin CC: @isdal

[apolito]

what about wsgi_env.get('HTTP_X_APPENGINE_' + key, default)
?

And now I see you already had this conversation....

[andrewsg]

@apolito Okay, I've refactored so that the ordering of all steps is perfectly clear and in one place, in reset_environment_middleware, and worked out the ordering to match the old VME code. PTAL

[apolito]

this is new behavior right?

I think this is a mistake. People use app.yaml settings for things like keys and user accounts to use. You don't want a request header to be able to overwrite those.

[andrewsg]

This is my best understanding of the current functionality. See meta_app.py starting here: https://github.com/GoogleCloudPlatform/appengine-python-vm-runtime/blob/ad16b62f9911c5ddd81fd24d5ca04dea56dc8755/python_vm_runtime/google/appengine/ext/vmruntime/meta_app.py#L549

This happens late in the process, but it's if key not in os.environ:. I can only interpret that as "this is the very lowest-priority thing", because any other keys including request header keys will overwrite that. Of course, request header keys aren't just dropped in with their original header name -- they're prefixed with "HTTP_" by the time they arrive in wsgi_env. That magic naming convention strikes me as fragile, but I guess that's the precedent set by CGI and carried over into the earliest versions of GAE (and WSGI).

[apolito]

So to verify:

If I have an app.yaml user env FOO set to 'bar'
and a user makes a request with the header FOO: baz

my environment will be

FOO => bar
HTTP_FPP => baz?

[andrewsg]

Yes, that's right. wsgi_env is already full of specially named, mission-critical stuff and namespacing into HTTP_ is how it handles that conflict. (This is an antiquated practice, but I guess that's WSGI.)

[apolito]

LGTM

[andrewsg]

Thanks!

[andrewsg]

Squashed to df9368