Created extension with plpgsql functions #4

Open
wants to merge 6 commits into
from

Projects

None yet

3 participants

@keithf4
keithf4 commented Oct 26, 2015

I watched your talk from DevDay Poland and found it very informative! So I took up your request/challange to create this API in plpgsql. I'd been meaning to learn more about jsonb and this seemed a good project to do so.
I also turned it into an actual extension to make installing and maintaining it easier. I changed quite a bit about the functions themselves as well in addition to converting them to plpgsql, so I hope you don't mind. I've been writing extensions myself quite a lot (pg_partman being my biggest one), so I've developed my own conventions to make ongoing maintenance easier.
By default, running make creates the plpgsql version. If you'd like to create the plv8 version, you can pass PLV8=1 to make and it will create those instead. However, it does not create the plv8 language for you anymore, you have to do that manually beforehand. Couldn't figure out how to make that part optional.

Below are notes about the individual functions and how I changed things.

create_document
- Renamed to "create_document" to be more consistent with other function names
- Require full schema qualification. Did this with other functions as well. This allows the format() function to work properly in case object names need to be escaped/quoted (schema and table name have to be quoted separately).
- Use text type, not varchar (similar issue throughout functions). Postgres convention.
- Assumed table name will be "docs" in index creation. Fixed to use any table name.
- No need to name indexes. Postgres will name with column appropriately

save_document
- Doesn't work on an array input yet
- Original did not insert document if id value is given in json and does not exist.
- Requires UPSERT introduced in 9.5 to actually be transaction safe. Workaround with race conditions in place now.

find_document
- For the id search, the id column and the id in the document should match, so don't see any reason for a dedicated function to just do "WHERE id = #"
-- Or can just give criteria '{"id":#}'
- This is doing the same thing as filter but with an added order by. So just combined filter & find features into one find function with optional parameters
- @> operator requires right operand be jsonb. Tried doing a to_json conversion, but didn't work very well. Other functions have a json parameter, so this one having it isn't inconsistent. Seems to provide a more consistent and validated json search format anyway.

update_search
- Turned into a trigger function that is called on every document insert & update of the body field in any table. Keeps search up to date automatically.

@robconery
Owner

This... is extraordinary work! Honestly it should be in its own project; it's completely different than what i put together here (which is great!). I'd be happy to link over to it - but I really think this should be yours; its great work!

@keithf4
keithf4 commented Oct 29, 2015

Thank you! I'll try to think of a different, creative name :)

@terrisgit

Does the plpgsql version do fewer jsonb to string conversions? Other advantages?

@keithf4
keithf4 commented Oct 29, 2015

There's no explicit string conversions. I just leave it in the jsonb data type and output jsonb from all functions that output json as well. I do use jsonb_each(), but the docs say that only converts the key names to text, not the values.
Only other major advantage at this time is not requiring a third-party library. The other things I changed could easily be added to the PLV8 version.
I will admit the disadvantage at this point is fewer options for adding features to actually edit the json data without extracting it, editing it, and putting it back in.

@robconery
Owner

@terrisgit Postgres deals with jsonb as a string. It's stored in binary but when you move it around in a query or have to use it in SQL it's a string. So far it's not showing as a problem in terms of speed.

@keithf4
keithf4 commented Nov 5, 2015

I've created an extension called pg_doc_store as my alternative for this. Feel free to close this pull request.
Thanks again for sharing your work!
https://github.com/keithf4/pg_doc_store

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment