New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Missing pgdg packages for plv8 for postgres 11 #308
Comments
these packages are not provided by the plv8 project, they are provided by maintainers that work with ubuntu. it appears that ubuntu no longer supports v8, as it is not meant to be distributed as a shared library, and thus no longer supports distributing plv8, as it requires v8. |
This is a greatly disappointing development. Our production system relies on plv8 coming from postgresql repositories. This means we can't move forward upgrading beyond server version 10. |
if someone can provide pgdg packages for ubuntu/debian, I'd happily point documentation to them, but I don't have the resources to provide that across distributions, unfortunately. |
Hi, Community RPM packager speaking: Debian and Red Hat packagers removed plv8 intentionally, due to build dependencies that do not exist on each distribution. Current PL/v8 development path is not suitable for packaging. Regards, Devrim |
yup, @devrimgunduz is absolutely correct, and it's unfortunate. given that plv8 relies on v8, there are only a couple of ways that it could happen:
there are plenty of discussions available where all of these options are discussed, I and or @devrimgunduz could easily link to all of them. it's not a decision that they entered into lightly, as it is not just plv8 that is affected - you can see the results of the same decision on chrome and node.js: and you can see that bsd's support by bundling v8 (which is how v8 is designed to be distributed), and not attempting to distribute it as a shared library: |
This is really bad news. I want to install pl/v8 on Ubuntu with Postgresql 11.2. Is there any solution? |
I'm running pl/v8 on postgres 12 beta. Compiling it was super simple. Follow the guide in the docs and you're good to go. |
Hi, I'm part of the V8 team. I'm surprised that this is somehow a new situation. V8 does not use semantic versioning (and never has), so distributing V8 as shared library with the OS has never really worked, and we have always discouraged this practice. It sounds backwards to me that V8 would be distributed with the OS, which in turn causes pl/v8 to not be distributed. Both Node.js and Chrome bundle the version of V8. I'm hoping that this is something that would work for pl/v8 as well? Yang
So if you forked V8, called it V9, removed the ability to build a shared library from it, then it would be fine? |
@df7cb @devrimgunduz care to help answer questions from @hashseed ? there seems to be a mismatch between @hashseed's understanding of modern operating system and package management and the everyday realties of the pains of being a volunteer maintainer.
no, speaking as the maintainer of plv8, the better solution would be to simply rename "plv8" to "pljs" and replace the javascript engine with something that is acceptable to the users as far as functionality goes, the package maintainers as far as maintainability goes, and the maintainer (me) as far as dealing the changes when a critical bug is found, or a new version of macOS is released that causes v8 to stop building, or the constant pain of trying to keep v8 compiling on windows across very minor v8 version changes, and the 3-5gb of disk space and network usage for every attempted build of v8 across plv8 versions whenever any regressions are dealt with. |
I'm currently working on a open source lib to integrate nodejs and plv8 (called pgproxy) that'll allow functions to be called from node, but execute on pg (and vice versa via notification channel). I think the potential of plv8 (or pljs) is enormous, and I hope this issue is resolved soon. Trying to get plv8 building and installing on windows has been difficult enough that I gave up and installed an Ubuntu VM thinking it would be the path of least resistance. Went with PG11, because, why wouldn't I? Came here to discover postgresql-11-plv8 isn't and maybe won't be a thing? I love the work you all have done, I think we have a huge opportunity in the pg/js space with plv8/pljs - if there's anything I can do to help with this issue, please let me know, I'm happy to pitch in! Update: for the curious, the (early version) pgproxy: https://github.com/claytongulick/pgproxy |
I spent a couple of hours and have a working prototype of QuickJS implemented. so far it's missing:
otherwise, it's working. I've opened #364 for discussion about whether it's sensible to abandon v8 completely, and would love some community input. /cc @pkit @bendiy @alexandru-lazarev let's open this up for discussion. |
Hi @hashseed, to sanely package plv8 for Debian, we need a v8 package, ideally as a shared library, or maybe as a static library. The problem is that compiling v8 needs 30GB or something of random dependencies from the internet that would need to be packaged themselves for integration into Debian. |
Hi @claytongulick, pgproxy is a confusing choice for a name because there's already https://github.com/plproxy/plproxy. I suggest picking something that isn't as close, especially as your thing seems to be a "pl" (procedural language) as well. |
I see. Thanks for the context. Node.js maintains a gyp build, and may introduce a cmake build, which includes build configs for V8. Would that be a solution to this problem in the long term? |
Already did that. Works fine in both directions. Just make sure to secure it properly. By default notification channels are not secure at all. Anybody could write/read anything to/from any channel. And there is no GRANT/REVOKE for channels. @JerrySievert |
@hashseed node doesn't require downloading a full build system including compiler, linker, c++lib, clang-format (?!!?!) to be able to compile. they've maintained their own build system divorced from v8 since 3.14. that's the big difference, not that node uses gyp instead of ninja, or cmake. |
Do you mean that you already have a project that does this? Can you post a link? I don't want to recreate something that already exists. |
Hi, thanks for the suggestion, and I'm totally open to thinking it through. At first blush, I don't think there will be much confusion between the libs because they're consumed by completely different audiences - the pgproxy lib is a nodejs utility that calls plv8, it's not another "pl" lib, and the name describes it's function pretty clearly - i.e. it creates a proxy object for pg calls. However, like I said, totally open to changing to something else if you have suggestions - maybe we can take the conversation to the pgproxy repo? |
Yep. Closed source for now. May be open sourced in the future... |
I think it's disingenuous to pretend that moving that from V8 to a bytecode interpreted JS engine is not going to have wildly different and unexpected performance profile than a typical major version upgrade would have, not to mention the potential instability of other JS engines.
I'm trying to make it clear that it's the only option available if you're concerned about users and cloud providers dropping support as it seemed to me to be the premise of opening this discussion.
I've spent the time and it's not an insurmountable problem at all. ChakraCore1, Graal2 and YODAOS3 were all able to implement N-API so targeting a subset of N-API is one potential avenue. Mozilla's work on WASI4 and Wasmer's Anyway, I've got a branch that uses the Node.js build pipeline that alleviates all the concerns other than stable ABI issues: devinus@b1f6d3f Some notes:
|
So why not start bundling v8 with this extension? Not sure why would that not be possible, instead of using it as a shared library? Is this because it is hard to assure compilation? You mention "pain of trying to keep". Would a CI help here? I see that this project is using TravisCI but just Linux. GitHub now has actions which support running on Linux, macOS, and Windows, so you could apply for beta (just by clicking a button) and then setup CI to help here? To me it looks like this pain the main reason you are suggesting removal of v8. Other solutions seems doable with some more help by the community: conversion to bundled v8, conversion to node.js, use of CI to help detecting build regressions, having maybe more people helping with maintaining of this library and helping keeping up with releases? |
v8 is bundled, and has been for several years. package maintainers for the most part have not distributed updated versions of plv8 because of this, instead they have distributed a version with a 6 year old version of v8 that was able to be compiled as a shared library (3.14). the responses that I have received have been that if plv8 cannot use a shared v8, and requires an embedded copy, it will not be released and maintained by package maintainers or distributions. this is stated multiple times and you can follow other issues where there are links to official removals from debian and fedora/redhat. full stop. against policy. not going to happen. followed shortly by disliking the requirements that v8 installs its own build system (compiler, libraries, formatter, etc just to be able to compile) making it, in their opinion, unmaintainable - so even if it weren't against policy, it still would not be maintained. as for CI, travis has worked well, but I'm happy to set up CI in 100 different places if people think that it's using travis and not some other CI that is the issue.
been done for YEARS.
doesn't solve either of the two major distribution issues, and likely makes the former one worse given that those distributions aren't distributing latest node.js either.
we have CI, this doesn't seem to be a problem since it's not maintenance of plv8 itself that is the issue.
PR's have always been welcomed, and usually merged. and there aren't very many releases to do - the issue is package management. |
adding: plv8 is fairly stable at this point because postgres is pretty mature at this point. these are the reasons why plv8 gets updated at this point in time (plv8 would be considered pretty mature as well):
|
Hm, Debian seems to have node 12, which is current latest version. Interestingly, it seems like libv8-dev has been replaced with libnode-dev. So maybe changing dependency to libnode is maybe what should be done?
This is in conflict with what @hashseed is saying that both Chrome and node.js bundles v8, and it seems Chromium and node.js are packages for Debian and many other distributions. So it seems there cannot be a policy preventing plv8 to bundle v8? Maybe previous plv8 maintainers chose by their own decision to prefer shared dependency, and now that shared dependency has been removed, have removed plv8 as well.
So some other maintainers could step up?
I am just saying that GitHub Actions and CI is free and supports also testing on macOS and Windows. But I see that Travis CI also supports those since recently. So maybe there is no need to switch to GitHub Actions, just Travis CI could be configured to run there as well. |
What @JerrySievert said upthread about Debian Policy is a very good summary so I'm not repeating that here. Why Chromium gets away with doing bad things is because it's Chromium :(. Other packages like plv8 are not in the position to make the rest of the project bend its quality guidelines like the "no embedded code copies" rule. If there's a different JS library that is available as a shared library, we absolutely ship plv8 again.
It's not just about personal dislike, it's a Debian rule: https://www.debian.org/doc/debian-policy/. |
@df7cb Thanks for clarification. So v8 is not distributed as a shared library anymore, and all packages besides Chromium and node.js cannot use v8 anymore if they want to be distributed? So they have to rewrite and start using something else, even if not as good, as a consequence? This is really a strange situation. |
I do read the rule differently:
It seems v8 is intended to be bundled. So this rule does not apply anymore? The rule is clearly written to not prevent any software to be included. But your interpenetration of the rule looks like software can be excluded because of that rule. Only if v8 can be packaged as a shared library, then plv8 should be using it, but if not, then it can use the bundled code. |
I think this was true before, when shared v8 was available. But now that shared v8 is not available anymore, plv8 should be able to bundle it and be distributed. |
You missed the footnote:
Bundling is ok for things like autoconf that dumps stuff to "configure". Also ok would be gnulib which is producing (small amounts of) code that is meant to be embedded. For V8, it's not as simple as "upstream doesn't believe in shared libraries anymore, so that is ok now". |
I think for those we can cross-compile, but we would probably not be able to test/run them inside CIs I mentioned. True. |
here's the list of supported platforms:
note that v8 current doesn't compile on all of those platforms, but older versions of v8 support some of them. that's the target list. |
Maybe perfect should not be an enemy of the good in this case. I think adding CI to help with platforms CI can help would be useful. But yes, it would not be perfect. |
if you want to take on supporting of packages for platforms, great, it's sorely needed. |
As I said, I can take some time to help with the CI. But let's first see how this will develop so that I know what I should be packaging. :-) Edit: You can in meantime apply for GitHub actions beta. |
First, kudos to all maintainers. Coders of the future are grateful for your work. I echo this. The postgres v8 combo is da bomb. It would be very very sad for it to not be packaged on debian/ubuntu, which is becoming the new default, and avail to all over policy mismatch. I guess it's either build your own of stay with 10? here's what I get when I attempt to build on ubuntu 18.04 I tried this on Ubuntu 18.04, as root:
This will download and install the build tools and use up about 5GB extra on your install, the fetch and build process (make) takes a while, but then I get this error: /bin/sh: 1: ../../third_party/llvm-build/Release+Asserts/bin/clang++: not found IIUC the v8 build process basically involves downloading all the the tools and headers etc bypassing the package system? If so why is it that I did not get the clang++ tool? (I sincerely think a regular pre-compiled module is a must, building v8 really does not seem straightforward) EDIT -- SOLUTION: make fails in unclear ways as root on linux!
TIA! |
I will miss plv8 when I convert hundreds of lines to plpgsql. Building from source on Debian has become broken on 11 (even following comments from above), plus the security implications of downloading from google.com during builds. |
@dfcsoftware "become broken" doesn't describe the problem. |
On Linux 4.14.108-ti-r131 Debian 10 armv7l environment, since there is no longer a package, I am trying a build on version 8-2.3.13 gives me the following errors. The first thing strange I see is the build is for armv6l and this an arm71 host. Line 52 in build/depot_tools/cipd makes this decision. $ uname -m Then the build cannot find infra/3pp/tools for this version. |
@dfcsoftware looks like purely |
@dfcsoftware Ah, I see cpython3 is used in your build, IIRC |
@pkit looks like I am running python 2.7 on the arm box: Thanks a lot for your suggestions! Unfortunatly I am not well versed in this build environment. As an alternative I tried a build on x86_64 host. It went much further, compiling plv8.so, but it has this error: ~/plv8-2.3.13$ make ~/plv8-2.3.13$ c++ --version ~/plv8-2.3.13$ uname -a ~/plv8-2.3.13$ lsb_release -a |
@dfcsoftware Building PLV8 for MacOS or Linux has some specific requirements:
|
@pkit Thanks once again, I was missing libc++-dev and libc++abi-dev on this new host. I have a successful build and install, but create extension fails:
|
@dfcsoftware probably some of the required libraries cannot be found by the |
It looks like this symbol is a part of v8, and thus should be compiled into your binary. |
I removed the entire directory, un-tarred it, and built again with no build error (just warnings), removed the old .so, re-installed. ` ldd /usr/lib/postgresql/12/lib/plv8-2.3.13.so
` Only research I could find on (ZN2v84base12CallOnceImplEPlPFvPvES2): Also tried in vain to install plv8--3.0alpha, but get similar error creating extension: Wondering if this package is causing conflicts: |
@dfcsoftware hmm |
|
Undefined. |
you should not have libnode-dev installed. plv8 will fail if it is there, due to the missing implementation that you are seeing. |
Eureka! Thank you @pkit and @JerrySievert for your patience and support. plv8 extension works on Postgresql 12! 👍 Now I just need to figure out how to get Angular to work since removing libnode-dev removed npm.
|
@dfcsoftware install a local, non system-wide, nodejs. |
how did you resolve the error? Would you mind giving some details? |
@hmarzooq I followed @JerrySievert advice and uninstalled libnode-dev and @pkit to remove nodejs, then install a local version of nodejs. Honestly after this I converted all my PLV8 stored procedures to Pg/sql. With the newer json support it was not too hard and removes a large dependency within my project. So I don't have this environment to look back on anymore. |
@dfcsoftware fantastic! pl/pgsql is a great solution, especially if it fits your needs. |
hello everyone, Can somebody assist me? When I try to create Plv8 with Postgresql11 under Centos7, I Just Get The Following Error 2023-07-05 10:35:33.272 CEST [3614] ERROR: could not load library "/usr/pgsql-11/lib/plv8-2.3.13.so": /usr/pgsql-11/lib/plv8-2.3.13.so: undefined symbol: ZN2v84base12CallOnceImplEPlPFvPvES2 |
@GeorgesBAZ if you have a specific problem, please either open an issue in GitHub or connect via discord at https://discord.gg/5fJN52Se - you will need to include information about the environment you are using, and how you installed plv8. |
Hi there,
As best as we can tell, the
postgresql-11-plv8
packages were not released with postgres 11 on the usual postgres package repositories. Specifically we depend on the ones for ubuntu trusty and ubuntu xenial.Could you build and upload these packages?
Thanks
The text was updated successfully, but these errors were encountered: