New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metaissue: Untrusted mode: plv8u #222

Open
EvanCarroll opened this Issue May 2, 2017 · 11 comments

Comments

Projects
None yet
5 participants
@EvanCarroll
Copy link

EvanCarroll commented May 2, 2017

No definitive idea of when we'll get it, but anything using IO, or file-system reads requires an untrusted mode. We will dupe all tickets onto this issue, and place this in the queue as a milestone.

@EvanCarroll EvanCarroll changed the title Metaissue: Untrusted mode. Metaissue: Untrusted mode: plv8u May 2, 2017

@JerrySievert

This comment has been minimized.

Copy link
Contributor

JerrySievert commented May 2, 2017

some notes (going to go ahead and use this issue to write down some of the things that have been going through my head on this:

AWS/RDS

one thing that you'll see if you look at languages that are supported by RDS is that they're all trusted languages. in fact, none of the language extensions have the ability to be compiled untrusted. when you look at plpython and plperl, you'll see that they compile both extensions together by default. staying on RDS is very important, so in order to have an untrusted plv8, it should not compile both and make them both available.

this gives some options:

  1. separate repo, sharing the core, so that trusted and untrusted versions of the extension are not built together, this could come in handy to prevent repo bloat

  2. compilation difference make untrusted that would do the build and set up the sql and control files

Libraries

in order for this to work, there would need to be a light shim between Postgres and the functionality to open files and deal with sockets. most functionality should end up in javascript, likely living in a table in a plv8 schema. (see https://github.com/JerrySievert/plv8-modules for an example)

this will require a lot of writing and testing, as unlike plperl or plpython, these libraries are not built into the language itself, but have become de facto standards via nodejs.

require, and an npm or yarn infrastructure would need to be built out as well.

Testing

I've already written a testing environment that would help out quite a bit: https://legitimatesounding.com/blog/unit_testing_in_postgresql_with_plv8.html

there would need to be a very large number of tests written to pull this off well.

I am happy to move equinox under plv8, especially if it helps get more testing overall (plv8 itself will continue to use make installcheck, but it makes more sense to have libraries tested with something more suited).

Debugging

In process (this one is going to take me some time), but would be very useful for an untrusted extension.

lots to do overall, but nothing impossible, it's just going to take some time.

in the mean time, there are a couple of other initiatives in place that I'm trying to work on as well for plv8.

@bendiy

This comment has been minimized.

Copy link
Contributor

bendiy commented Jun 5, 2017

I think I've seen 5 or 6 different version of require support for PL/v8 on GitHub. I've got one that browserify's npm modules and sticks them in a node_modules table that require('foo') queries and loads from. We should consider adding it as a standard core feature to PL/v8. There are plenty of use for libraries that don't do any IO or file access and can still be trusted.

A note on debugging:
In the Add v8_inspector support... issue, support for a websocket server for the debugger is mentioned. Will that work with a trusted version of PL/v8? Will opening up a socket for the debugger to connect to require running an untrusted version?

@JerrySievert

This comment has been minimized.

Copy link
Contributor

JerrySievert commented Jun 9, 2017

opening a socket that can accept connections would fall under untrusted.

@JerrySievert

This comment has been minimized.

Copy link
Contributor

JerrySievert commented Jul 11, 2017

I've created https://github.com/plv8/plv8u and have started breaking everything apart. looking at which modules need to be supported in which order.

looking at:

  • require - plv8u.core_modules and plv8u.modules for core modules and local modules
  • fs
  • net - which would ultimately give us debugger support

I'm open to opinions.

@deinspanjer

This comment has been minimized.

Copy link
Contributor

deinspanjer commented Jul 13, 2017

Very cool to see there is work on plv8u although since we are currently living in RDS, it won't be of use to us.

Just wanted to check and make sure that the require stuff you are looking to support is going to be core, i.e. available in both trusted and un. I'm one of the many using a module loader with plv8 and it would be great to get first class support there.

@JerrySievert

This comment has been minimized.

Copy link
Contributor

JerrySievert commented Jul 13, 2017

Require will be core in untrusted, but I can likely port most of the core logic (sans file system support) into plv8. Unfortunately that still won't help until Debian is able to support modern v8, at which point you'd be able to use it in rds.

I do not plan to backport major functionality to unsupported versions (they receive critical bug updates, and security fixes but not new features).

@EvanCarroll

This comment has been minimized.

Copy link

EvanCarroll commented Jul 13, 2017

Food for thought, the shim that handles fs features, perhaps we should only permit that access to node_modules inside of the PostgreSQL superuser's home? I'm not sure if a distinction is needed between JS that is installed globally (to /usr/local/lib/node_modules/) and available via require, and JS only available to the superuser.

The two locations in question could be accessed via an npm install like this,

sudo npm install -g lib            # global install
sudo -u "postgres" npm install lib # PostgresSQL su install
@JerrySievert

This comment has been minimized.

Copy link
Contributor

JerrySievert commented Jul 13, 2017

I was planning on a core module directory where core modules live then a variable set for additional modules - like plv8u.module_directory - the question is: admin only or user settable

@bendiy

This comment has been minimized.

Copy link
Contributor

bendiy commented Jul 14, 2017

Please make sure to support a way for modules to be loaded from a table. Either some default in a plv8 schema or one set through a config setting. It is nice to be able to include your modules in a database backup or upgrade script and not have to touch the server.

@bendiy

This comment has been minimized.

Copy link
Contributor

bendiy commented Jul 14, 2017

When I created a node shim for another JavaScript engine, I found these modules helpful in adding some core node.js API features in pure JavaScript:

"dependencies": {
  "buffer": "^4.6.0",
  "buffer-browserify": "^0.2.5",
  "core-js": "^2.3.0",
  "events": "^1.1.0",
  "http-parser-js": "^0.4.2",
  "path": "^0.12.7",
  "process": "^0.11.2",
  "punycode": "^1.4.1",
  "querystring-es3": "^0.2.1",
  "stream-browserify": "^2.0.1",
  "string_decoder": "^0.10.31",
  "url": "^0.11.0",
  "util": "^0.10.3"
}

The core node.js API features I had to support in C++ were:

buffer
crypto
fs
https
timers
child_process
dns
http
net
_stream_transform
ws

I didn't cover everything, but it was enough to get me what I needed.

@jmealo

This comment has been minimized.

Copy link

jmealo commented Aug 26, 2017

Could we proxy the Chrome Developer Protocol though a foreign data wrapper (or a combination of custom wal messages/logical decoding, chunked notify/listen)? The PG community is often fond of using existing functionally to create new functionally. There are Lua and Go debuggers that are exposed though CDP so it's possible that a polyglot shim could bring debugging to trusted languages. It might be worth reaching out to the RDS team with a few proposals. A PG worker that exposes a CDP gateway to trusted languages using IAM/PG compatible authentication might tick the right boxes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment