Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing pgdg packages for plv8 for postgres 11 #308

Closed
tcrayford opened this issue Oct 31, 2018 · 79 comments
Closed

Missing pgdg packages for plv8 for postgres 11 #308

tcrayford opened this issue Oct 31, 2018 · 79 comments

Comments

@tcrayford
Copy link

Hi there,

As best as we can tell, the postgresql-11-plv8 packages were not released with postgres 11 on the usual postgres package repositories. Specifically we depend on the ones for ubuntu trusty and ubuntu xenial.

Could you build and upload these packages?

Thanks

@JerrySievert
Copy link
Member

these packages are not provided by the plv8 project, they are provided by maintainers that work with ubuntu.

it appears that ubuntu no longer supports v8, as it is not meant to be distributed as a shared library, and thus no longer supports distributing plv8, as it requires v8.

@cherio
Copy link

cherio commented Feb 18, 2019

This is a greatly disappointing development. Our production system relies on plv8 coming from postgresql repositories. This means we can't move forward upgrading beyond server version 10.

@JerrySievert
Copy link
Member

if someone can provide pgdg packages for ubuntu/debian, I'd happily point documentation to them, but I don't have the resources to provide that across distributions, unfortunately.

@devrimgunduz
Copy link

Hi,

Community RPM packager speaking:

Debian and Red Hat packagers removed plv8 intentionally, due to build dependencies that do not exist on each distribution. Current PL/v8 development path is not suitable for packaging.

Regards, Devrim

@JerrySievert
Copy link
Member

yup, @devrimgunduz is absolutely correct, and it's unfortunate. given that plv8 relies on v8, there are only a couple of ways that it could happen:

  • remove v8 as a dependency, and go with another embedded javascript library
  • back up to a version of v8 that supported shared library distribution (6 years ago)
  • convince the v8 team that their development goals need to change to allow v8 to be distributed as a shared library
  • change the rules that the package maintainers use to determine what can and cannot be packaged

there are plenty of discussions available where all of these options are discussed, I and or @devrimgunduz could easily link to all of them. it's not a decision that they entered into lightly, as it is not just plv8 that is affected - you can see the results of the same decision on chrome and node.js:

https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions-enterprise-linux-fedora-and-snap-packages

and you can see that bsd's support by bundling v8 (which is how v8 is designed to be distributed), and not attempting to distribute it as a shared library:

https://www.freshports.org/www/node

@ansarizafar
Copy link

This is really bad news. I want to install pl/v8 on Ubuntu with Postgresql 11.2. Is there any solution?

@jsommr
Copy link

jsommr commented Aug 24, 2019

I'm running pl/v8 on postgres 12 beta. Compiling it was super simple. Follow the guide in the docs and you're good to go.

@hashseed
Copy link

Hi,

I'm part of the V8 team. I'm surprised that this is somehow a new situation. V8 does not use semantic versioning (and never has), so distributing V8 as shared library with the OS has never really worked, and we have always discouraged this practice. It sounds backwards to me that V8 would be distributed with the OS, which in turn causes pl/v8 to not be distributed.

Both Node.js and Chrome bundle the version of V8. I'm hoping that this is something that would work for pl/v8 as well?

Yang

remove v8 as a dependency, and go with another embedded javascript library

So if you forked V8, called it V9, removed the ability to build a shared library from it, then it would be fine?

@JerrySievert
Copy link
Member

@df7cb @devrimgunduz care to help answer questions from @hashseed ? there seems to be a mismatch between @hashseed's understanding of modern operating system and package management and the everyday realties of the pains of being a volunteer maintainer.

So if you forked V8, called it V9, removed the ability to build a shared library from it, then it would be fine?

no, speaking as the maintainer of plv8, the better solution would be to simply rename "plv8" to "pljs" and replace the javascript engine with something that is acceptable to the users as far as functionality goes, the package maintainers as far as maintainability goes, and the maintainer (me) as far as dealing the changes when a critical bug is found, or a new version of macOS is released that causes v8 to stop building, or the constant pain of trying to keep v8 compiling on windows across very minor v8 version changes, and the 3-5gb of disk space and network usage for every attempted build of v8 across plv8 versions whenever any regressions are dealt with.

@claytongulick
Copy link
Contributor

claytongulick commented Aug 29, 2019

I'm currently working on a open source lib to integrate nodejs and plv8 (called pgproxy) that'll allow functions to be called from node, but execute on pg (and vice versa via notification channel).

I think the potential of plv8 (or pljs) is enormous, and I hope this issue is resolved soon.

Trying to get plv8 building and installing on windows has been difficult enough that I gave up and installed an Ubuntu VM thinking it would be the path of least resistance. Went with PG11, because, why wouldn't I?

Came here to discover postgresql-11-plv8 isn't and maybe won't be a thing?

I love the work you all have done, I think we have a huge opportunity in the pg/js space with plv8/pljs - if there's anything I can do to help with this issue, please let me know, I'm happy to pitch in!

Update: for the curious, the (early version) pgproxy: https://github.com/claytongulick/pgproxy

@JerrySievert
Copy link
Member

I spent a couple of hours and have a working prototype of QuickJS implemented.

so far it's missing:

  • json/jsonb
  • start scripts
  • function cache
  • context isolation
  • inline call handler
  • memory context isolation
  • SPI functionality

otherwise, it's working.

I've opened #364 for discussion about whether it's sensible to abandon v8 completely, and would love some community input.

/cc @pkit @bendiy @alexandru-lazarev

let's open this up for discussion.

@df7cb
Copy link

df7cb commented Sep 3, 2019

Hi @hashseed, to sanely package plv8 for Debian, we need a v8 package, ideally as a shared library, or maybe as a static library. The problem is that compiling v8 needs 30GB or something of random dependencies from the internet that would need to be packaged themselves for integration into Debian.
All in all, this is simply too fat to be manageable. (The web browser package maintainers don't like that either, but they have no other choice.)

@df7cb
Copy link

df7cb commented Sep 3, 2019

I'm currently working on a open source lib to integrate nodejs and plv8 (called pgproxy) that'll allow functions to be called from node, but execute on pg (and vice versa via notification channel).

Hi @claytongulick, pgproxy is a confusing choice for a name because there's already https://github.com/plproxy/plproxy. I suggest picking something that isn't as close, especially as your thing seems to be a "pl" (procedural language) as well.

@hashseed
Copy link

hashseed commented Sep 3, 2019

I see. Thanks for the context. Node.js maintains a gyp build, and may introduce a cmake build, which includes build configs for V8. Would that be a solution to this problem in the long term?

@pkit
Copy link
Contributor

pkit commented Sep 3, 2019

@claytongulick

I'm currently working on a open source lib to integrate nodejs and plv8 (called pgproxy) that'll allow functions to be called from node, but execute on pg (and vice versa via notification channel).

Already did that. Works fine in both directions. Just make sure to secure it properly. By default notification channels are not secure at all. Anybody could write/read anything to/from any channel. And there is no GRANT/REVOKE for channels.

@JerrySievert
will look into it

@JerrySievert
Copy link
Member

@hashseed node doesn't require downloading a full build system including compiler, linker, c++lib, clang-format (?!!?!) to be able to compile. they've maintained their own build system divorced from v8 since 3.14.

that's the big difference, not that node uses gyp instead of ninja, or cmake.

@claytongulick
Copy link
Contributor

@pkit

Already did that. Works fine in both directions.

Do you mean that you already have a project that does this? Can you post a link? I don't want to recreate something that already exists.

@claytongulick
Copy link
Contributor

claytongulick commented Sep 4, 2019

@df7cb

Hi @claytongulick, pgproxy is a confusing choice for a name because there's already https://github.com/plproxy/plproxy. I suggest picking something that isn't as close, especially as your thing seems to be a "pl" (procedural language) as well.

Hi, thanks for the suggestion, and I'm totally open to thinking it through. At first blush, I don't think there will be much confusion between the libs because they're consumed by completely different audiences - the pgproxy lib is a nodejs utility that calls plv8, it's not another "pl" lib, and the name describes it's function pretty clearly - i.e. it creates a proxy object for pg calls. However, like I said, totally open to changing to something else if you have suggestions - maybe we can take the conversation to the pgproxy repo?

@pkit
Copy link
Contributor

pkit commented Sep 5, 2019

@claytongulick

Do you mean that you already have a project that does this?

Yep. Closed source for now. May be open sourced in the future...

@devinus
Copy link

devinus commented Sep 14, 2019

trying to tie any sort of performance characteristics to a version of plv8 is really not advised, unless it is at a patch level, since v8 itself has drastically different performance characteristics across its versions.

I think it's disingenuous to pretend that moving that from V8 to a bytecode interpreted JS engine is not going to have wildly different and unexpected performance profile than a typical major version upgrade would have, not to mention the potential instability of other JS engines.

and you just made a bunch of assumptions - a) that a new package was never an option

I'm trying to make it clear that it's the only option available if you're concerned about users and cloud providers dropping support as it seemed to me to be the premise of opening this discussion.

if you spent time looking at the lack of commonality between javascript engines, you likely would not suggest this - there is less commonality between javascript engines than between different languages

I've spent the time and it's not an insurmountable problem at all. ChakraCore1, Graal2 and YODAOS3 were all able to implement N-API so targeting a subset of N-API is one potential avenue. Mozilla's work on WASI4 and Wasmer's wasm PG extension5 could also be a source of inspiration, because plv8's subset of types could easily be represented across the WASM call boundary.

Anyway, I've got a branch that uses the Node.js build pipeline that alleviates all the concerns other than stable ABI issues: devinus@b1f6d3f

Some notes:

  1. Using the Node.js pipeline to build V8 only requires gcc/g++/python/make so it will be possible to git clone git@github.com:plv8/plv8.git; cd plv8; make install with just those 4 dependencies and be done
  2. Node.js packages are pretty ubiquitous and availably on almost every platform I can think of about including Windows, so package maintainers should have no problem
  3. The V8 version for Node.js 12.3.0 is also 7.4.288
  4. Node.js takes care of ICU target/host endianness differences automagically

@mitar
Copy link

mitar commented Sep 17, 2019

replace the javascript engine with something that is acceptable to the users as far as functionality goes, the package maintainers as far as maintainability goes, and the maintainer (me) as far as dealing the changes when a critical bug is found, or a new version of macOS is released that causes v8 to stop building, or the constant pain of trying to keep v8 compiling on windows across very minor v8 version changes, and the 3-5gb of disk space and network usage for every attempted build of v8 across plv8 versions whenever any regressions are dealt with.

So why not start bundling v8 with this extension? Not sure why would that not be possible, instead of using it as a shared library? Is this because it is hard to assure compilation? You mention "pain of trying to keep". Would a CI help here? I see that this project is using TravisCI but just Linux. GitHub now has actions which support running on Linux, macOS, and Windows, so you could apply for beta (just by clicking a button) and then setup CI to help here?

To me it looks like this pain the main reason you are suggesting removal of v8. Other solutions seems doable with some more help by the community: conversion to bundled v8, conversion to node.js, use of CI to help detecting build regressions, having maybe more people helping with maintaining of this library and helping keeping up with releases?

@JerrySievert
Copy link
Member

So why not start bundling v8 with this extension? Not sure why would that not be possible, instead of using it as a shared library?

v8 is bundled, and has been for several years. package maintainers for the most part have not distributed updated versions of plv8 because of this, instead they have distributed a version with a 6 year old version of v8 that was able to be compiled as a shared library (3.14).

the responses that I have received have been that if plv8 cannot use a shared v8, and requires an embedded copy, it will not be released and maintained by package maintainers or distributions. this is stated multiple times and you can follow other issues where there are links to official removals from debian and fedora/redhat. full stop. against policy. not going to happen.

followed shortly by disliking the requirements that v8 installs its own build system (compiler, libraries, formatter, etc just to be able to compile) making it, in their opinion, unmaintainable - so even if it weren't against policy, it still would not be maintained.

as for CI, travis has worked well, but I'm happy to set up CI in 100 different places if people think that it's using travis and not some other CI that is the issue.

conversion to bundled v8

been done for YEARS.

conversion to node.js

doesn't solve either of the two major distribution issues, and likely makes the former one worse given that those distributions aren't distributing latest node.js either.

use of CI to help detecting build regressions

we have CI, this doesn't seem to be a problem since it's not maintenance of plv8 itself that is the issue.

having maybe more people helping with maintaining of this library and helping keeping up with releases?

PR's have always been welcomed, and usually merged. and there aren't very many releases to do - the issue is package management.

@JerrySievert
Copy link
Member

adding:

plv8 is fairly stable at this point because postgres is pretty mature at this point. these are the reasons why plv8 gets updated at this point in time (plv8 would be considered pretty mature as well):

  • javascript changes, so v8 changes - plv8 tracks major v8 changes which also require changes to plv8 because v8's API/ABI is not stable between releases
  • postgres internals change - this happens usually on major postgres versions, like 11 and 12
  • postgres adds a new feature - it's important that plv8 can support postgres features (see right above this for frequency)
  • ability to make a change that allows for some optimization - plv8 is pretty mature, but this still happens
  • regressions/bugs - plv8 is software written by humans, and sometimes it takes years for bugs to show up

@mitar
Copy link

mitar commented Sep 17, 2019

doesn't solve either of the two major distribution issues, and likely makes the former one worse given that those distributions aren't distributing latest node.js either.

Hm, Debian seems to have node 12, which is current latest version.

Interestingly, it seems like libv8-dev has been replaced with libnode-dev. So maybe changing dependency to libnode is maybe what should be done?

the responses that I have received have been that if plv8 cannot use a shared v8, and requires an embedded copy, it will not be released and maintained by package maintainers or distributions

This is in conflict with what @hashseed is saying that both Chrome and node.js bundles v8, and it seems Chromium and node.js are packages for Debian and many other distributions. So it seems there cannot be a policy preventing plv8 to bundle v8? Maybe previous plv8 maintainers chose by their own decision to prefer shared dependency, and now that shared dependency has been removed, have removed plv8 as well.

followed shortly by disliking the requirements that v8 installs its own build system (compiler, libraries, formatter, etc just to be able to compile) making it, in their opinion, unmaintainable - so even if it weren't against policy, it still would not be maintained.

So some other maintainers could step up?

as for CI, travis has worked well, but I'm happy to set up CI in 100 different places if people think that it's using travis and not some other CI that is the issue.

I am just saying that GitHub Actions and CI is free and supports also testing on macOS and Windows. But I see that Travis CI also supports those since recently. So maybe there is no need to switch to GitHub Actions, just Travis CI could be configured to run there as well.

@df7cb
Copy link

df7cb commented Sep 17, 2019

the responses that I have received have been that if plv8 cannot use a shared v8, and requires an embedded copy, it will not be released and maintained by package maintainers or distributions

This is in conflict with what @hashseed is saying that both Chrome and node.js bundles v8, and it seems Chromium and node.js are packages for Debian and many other distributions. So it seems there cannot be a policy preventing plv8 to bundle v8? Maybe previous plv8 maintainers chose by their own decision to prefer shared dependency, and now that shared dependency has been removed, have removed plv8 as well.

What @JerrySievert said upthread about Debian Policy is a very good summary so I'm not repeating that here. Why Chromium gets away with doing bad things is because it's Chromium :(. Other packages like plv8 are not in the position to make the rest of the project bend its quality guidelines like the "no embedded code copies" rule. If there's a different JS library that is available as a shared library, we absolutely ship plv8 again.

followed shortly by disliking the requirements that v8 installs its own build system (compiler, libraries, formatter, etc just to be able to compile) making it, in their opinion, unmaintainable - so even if it weren't against policy, it still would not be maintained.

So some other maintainers could step up?

It's not just about personal dislike, it's a Debian rule: https://www.debian.org/doc/debian-policy/.

@mitar
Copy link

mitar commented Sep 17, 2019

@df7cb Thanks for clarification. So v8 is not distributed as a shared library anymore, and all packages besides Chromium and node.js cannot use v8 anymore if they want to be distributed? So they have to rewrite and start using something else, even if not as good, as a consequence? This is really a strange situation.

@mitar
Copy link

mitar commented Sep 17, 2019

It's not just about personal dislike, it's a Debian rule

I do read the rule differently:

Debian packages should not make use of these convenience copies unless the included package is explicitly intended to be used in this way.

It seems v8 is intended to be bundled. So this rule does not apply anymore?

The rule is clearly written to not prevent any software to be included. But your interpenetration of the rule looks like software can be excluded because of that rule. Only if v8 can be packaged as a shared library, then plv8 should be using it, but if not, then it can use the bundled code.

@mitar
Copy link

mitar commented Sep 17, 2019

the responses that I have received have been that if plv8 cannot use a shared v8, and requires an embedded copy, it will not be released and maintained by package maintainers or distributions.

I think this was true before, when shared v8 was available. But now that shared v8 is not available anymore, plv8 should be able to bundle it and be distributed.

@df7cb
Copy link

df7cb commented Sep 17, 2019

I do read the rule differently:

Debian packages should not make use of these convenience copies unless the included package is explicitly intended to be used in this way.

It seems v8 is intended to be bundled. So this rule does not apply anymore?

You missed the footnote:

[17] For example, parts of the GNU build system work like this.

Bundling is ok for things like autoconf that dumps stuff to "configure". Also ok would be gnulib which is producing (small amounts of) code that is meant to be embedded.

For V8, it's not as simple as "upstream doesn't believe in shared libraries anymore, so that is ok now".

@mitar
Copy link

mitar commented Sep 17, 2019

you're missing the rest of the platforms

I think for those we can cross-compile, but we would probably not be able to test/run them inside CIs I mentioned. True.

@JerrySievert
Copy link
Member

here's the list of supported platforms:

x86, x86_64, IA64, PowerPC, PowerPC 64, S/390, S/390x, Sparc, Sparc 64, Alpha, ARM, MIPS, MIPSEL, M68K, and PA-RISC.

note that v8 current doesn't compile on all of those platforms, but older versions of v8 support some of them. that's the target list.

@mitar
Copy link

mitar commented Sep 17, 2019

Maybe perfect should not be an enemy of the good in this case. I think adding CI to help with platforms CI can help would be useful. But yes, it would not be perfect.

@JerrySievert
Copy link
Member

if you want to take on supporting of packages for platforms, great, it's sorely needed.

@mitar
Copy link

mitar commented Sep 17, 2019

As I said, I can take some time to help with the CI. But let's first see how this will develop so that I know what I should be packaging. :-)

Edit: You can in meantime apply for GitHub actions beta.

@francoisp
Copy link

francoisp commented Oct 4, 2019

r-cran-v8 uses libnode64 on Debian
Seems pretty cool to me.

First, kudos to all maintainers. Coders of the future are grateful for your work.

I echo this. The postgres v8 combo is da bomb. It would be very very sad for it to not be packaged on debian/ubuntu, which is becoming the new default, and avail to all over policy mismatch.

I guess it's either build your own of stay with 10? here's what I get when I attempt to build on ubuntu 18.04

I tried this on Ubuntu 18.04, as root:

apt update; apt upgrade -y; apt install -y postgresql-server-dev-11 make git pkg-config chromium-browser subversion clang apg ninja-build cmake libc++-dev libc++abi-dev

wget https://github.com/plv8/plv8/archive/v2.3.13.tar.gz
tar -xvzf v2.3.13.tar.gz
cd plv8-2.3.13
make

This will download and install the build tools and use up about 5GB extra on your install, the fetch and build process (make) takes a while, but then I get this error:

/bin/sh: 1: ../../third_party/llvm-build/Release+Asserts/bin/clang++: not found

IIUC the v8 build process basically involves downloading all the the tools and headers etc bypassing the package system? If so why is it that I did not get the clang++ tool?

(I sincerely think a regular pre-compiled module is a must, building v8 really does not seem straightforward)

EDIT -- SOLUTION: make fails in unclear ways as root on linux!

adduser --gecos "buildplv8" --disabled-password plv8;

sudo - plv8
wget https://github.com/plv8/plv8/archive/v2.3.13.tar.gz
tar -xvzf v2.3.13.tar.gz
cd plv8-2.3.13
make

TIA!
Francois

@dfcsoftware
Copy link

dfcsoftware commented Jul 31, 2020

I will miss plv8 when I convert hundreds of lines to plpgsql. Building from source on Debian has become broken on 11 (even following comments from above), plus the security implications of downloading from google.com during builds.

@pkit
Copy link
Contributor

pkit commented Aug 14, 2020

@dfcsoftware "become broken" doesn't describe the problem.
What's broken?

@dfcsoftware
Copy link

dfcsoftware commented Aug 14, 2020

On Linux 4.14.108-ti-r131 Debian 10 armv7l environment, since there is no longer a package, I am trying a build on version 8-2.3.13 gives me the following errors. The first thing strange I see is the build is for armv6l and this an arm71 host. Line 52 in build/depot_tools/cipd makes this decision.

$ uname -m
armv7l

Then the build cannot find infra/3pp/tools for this version.
...-> build errors <-
~/plv8-2.3.13$ make
mkdir -p build
cd build; git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
Cloning into 'depot_tools'...
remote: Sending approximately 43.99 MiB ...
remote: Counting objects: 3, done
remote: Finding sources: 100% (3/3)
remote: Total 39634 (delta 25315), reused 39633 (delta 25315)
Receiving objects: 100% (39634/39634), 43.98 MiB | 2.76 MiB/s, done.
Resolving deltas: 100% (25315/25315), done.
cd build; fetch v8; cd v8; git checkout 6.4.388.40; gclient sync ; cd build/config ; git cherry-pick 4287a0d364541583a50cc91465330251460d489a ; cd ../.. ; tools/dev/v8gen.py arm64.release -- is_component_build=false v8_static_library=true v8_use_snapshot=true v8_use_external_startup_data=false v8_enable_i18n_support=false
Errors:
failed to resolve infra/3pp/tools/git/linux-armv6l@version:2.28.0.chromium.6 (line 27): no such package: infra/3pp/tools/git/linux-armv6l
failed to resolve infra/3pp/tools/cpython/linux-armv6l@version:2.7.17.chromium.22 (line 21): no such package: infra/3pp/tools/cpython/linux-armv6l
failed to resolve infra/3pp/tools/cpython3/linux-armv6l@version:3.8.0.chromium.8 (line 24): no such package: infra/3pp/tools/cpython3/linux-armv6l
/data/don/plv8-2.3.13/build/depot_tools/bootstrap_python3: line 32: bootstrap-3.8.0.chromium.8_bin/python3/bin/python3: No such file or directory
cat: /data/don/plv8-2.3.13/build/depot_tools/python3_bin_reldir.txt: No such file or directory
[E2020-08-14T14:49:04.728022-04:00 11543 0 annotate.go:241] original error: fork/exec /data/don/plv8-2.3.13/build/depot_tools/python3: no such file or directory
[E2020-08-14T14:49:04.729074-04:00 11543 0 annotate.go:241]
[E2020-08-14T14:49:04.729922-04:00 11543 0 annotate.go:241] goroutine 1:
[E2020-08-14T14:49:04.731087-04:00 11543 0 annotate.go:241] #0 go.chromium.org/luci/vpython/python/interpreter.go:89 - python.(*Interpreter).GetVersion()
[E2020-08-14T14:49:04.731310-04:00 11543 0 annotate.go:241] #1 go.chromium.org/luci/vpython/venv/config.go:286 - venv.(*Config).resolvePythonInterpreter()
[E2020-08-14T14:49:04.731878-04:00 11543 0 annotate.go:241] reason: failed to determine Python version for: /data/don/plv8-2.3.13/build/depot_tools//python3
[E2020-08-14T14:49:04.732440-04:00 11543 0 annotate.go:241]
[E2020-08-14T14:49:04.733275-04:00 11543 0 annotate.go:241] #2 go.chromium.org/luci/vpython/venv/config.go:186 - venv.(*Config).makeEnv()
[E2020-08-14T14:49:04.734006-04:00 11543 0 annotate.go:241] reason: failed to resolve system Python interpreter
[E2020-08-14T14:49:04.734565-04:00 11543 0 annotate.go:241]
[E2020-08-14T14:49:04.735312-04:00 11543 0 annotate.go:241] #3 go.chromium.org/luci/vpython/venv/venv.go:150 - venv.With()
[E2020-08-14T14:49:04.735913-04:00 11543 0 annotate.go:241] reason: failed to initialize empty probe environment
[E2020-08-14T14:49:04.737029-04:00 11543 0 annotate.go:241]
[E2020-08-14T14:49:04.737228-04:00 11543 0 annotate.go:241] #4 go.chromium.org/luci/vpython/run.go:62 - vpython.Run()
[E2020-08-14T14:49:04.738734-04:00 11543 0 annotate.go:241] #5 go.chromium.org/luci/vpython/application/application.go:320 - application.(*application).mainImpl()
[E2020-08-14T14:49:04.739496-04:00 11543 0 annotate.go:241] #6 go.chromium.org/luci/vpython/application/application.go:406 - application.(*Config).Main.func1()
[E2020-08-14T14:49:04.740062-04:00 11543 0 annotate.go:241] #7 go.chromium.org/luci/vpython/application/support.go:46 - application.run()
[E2020-08-14T14:49:04.740812-04:00 11543 0 annotate.go:241] #8 go.chromium.org/luci/vpython/application/application.go:405 - application.(*Config).Main()
[E2020-08-14T14:49:04.741876-04:00 11543 0 annotate.go:241] #9 vpython/main.go:106 - main.mainImpl()
[E2020-08-14T14:49:04.742085-04:00 11543 0 annotate.go:241] #10 vpython/main.go:112 - main.main()
[E2020-08-14T14:49:04.742871-04:00 11543 0 annotate.go:241] #11 runtime/proc.go:203 - runtime.main()
[E2020-08-14T14:49:04.743420-04:00 11543 0 annotate.go:241] #12 runtime/asm_arm.s:868 - runtime.goexit()

@pkit
Copy link
Contributor

pkit commented Aug 14, 2020

@dfcsoftware looks like purely v8 on arm build problem iirc llvm had some arm7 build problems. And they do use llvm in ninja last time I've checked.
Anyway you can just go ahead and build v8 itself in isolation. When it builds successfully plv8 will also build. I suppose.

@pkit
Copy link
Contributor

pkit commented Aug 15, 2020

@dfcsoftware Ah, I see cpython3 is used in your build, IIRC v8gen.py uses python2
Try to make sure that your env is python2-based.

@dfcsoftware
Copy link

@pkit looks like I am running python 2.7 on the arm box:
~/plv8-2.3.13$ python --version
Python 2.7.16

Thanks a lot for your suggestions! Unfortunatly I am not well versed in this build environment.

As an alternative I tried a build on x86_64 host. It went much further, compiling plv8.so, but it has this error:

~/plv8-2.3.13$ make
...
g++ -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fno-omit-frame-pointer -fPIC -shared -o plv8-2.3.13.so plv8.o plv8_type.o plv8_func.o plv8_param.o coffee-script.o livescript.o -L/usr/lib/x86_64-linux-gnu -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -L/usr/lib/llvm-10/lib -L/usr/lib/x86_64-linux-gnu/mit-krb5 -Wl,--as-needed -Lbuild/v8/out.gn/x64.release/obj -Lbuild/v8/out.gn/x64.release/obj/third_party/icu -lv8_base -lv8_snapshot -lv8_libplatform -lv8_libbase -lv8_libsampler -lv8_libplatform -Lbuild/v8/out.gn/x64.release/obj -lrt -std=c++11 -lc++
/usr/bin/ld: cannot find -lc++
collect2: error: ld returned 1 exit status
make: *** [/usr/lib/postgresql/12/lib/pgxs/src/makefiles/../../src/Makefile.shlib:309: plv8-2.3.13.so] Error 1

~/plv8-2.3.13$ c++ --version
c++ (Ubuntu 9.3.0-10ubuntu2) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

~/plv8-2.3.13$ uname -a
Linux xxxx 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

~/plv8-2.3.13$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.1 LTS
Release: 20.04
Codename: focal

@pkit
Copy link
Contributor

pkit commented Aug 15, 2020

@dfcsoftware
https://plv8.github.io/#building

Building PLV8 for MacOS or Linux has some specific requirements:

Git
g++ or clang++
Python 2 (for v8)
pkg-config (linux only for v8)
libc++-dev (linux only)
libc++abi-dev (linux only)

@dfcsoftware
Copy link

@pkit Thanks once again, I was missing libc++-dev and libc++abi-dev on this new host.

I have a successful build and install, but create extension fails:
postgres=# create extension plv8; ERROR: could not load library "/usr/lib/postgresql/12/lib/plv8-2.3.13.so": /usr/lib/postgresql/12/lib/plv8-2.3.13.so: undefined symbol: _ZN2v84base12CallOnceImplEPlPFvPvES2_

~/plv8-2.3.13$ ls -l /usr/lib/postgresql/12/lib/plv8-2.3.13.so
-rwxr-xr-x 1 root root 15309416 Aug 15 10:02 /usr/lib/postgresql/12/lib/plv8-2.3.13.so

@pkit
Copy link
Contributor

pkit commented Aug 15, 2020

@dfcsoftware probably some of the required libraries cannot be found by the .so or other versions are found
ldd is your friend here :)

@pkit
Copy link
Contributor

pkit commented Aug 15, 2020

It looks like this symbol is a part of v8, and thus should be compiled into your binary.
You probably need to delete ./build directory in plv8 and re-build. It looks like it links with your arm build retries or some other previously built binaries.

@dfcsoftware
Copy link

I removed the entire directory, un-tarred it, and built again with no build error (just warnings), removed the old .so, re-installed.
Still exact same error trying to create extension. ldd is the same list on second build, just different hex values.

`

ldd /usr/lib/postgresql/12/lib/plv8-2.3.13.so

linux-vdso.so.1 (0x00007ffd07fb6000)
libnode.so.64 => /usr/lib/x86_64-linux-gnu/libnode.so.64 (0x00007fdb081e7000)
libc++.so.1 => /usr/lib/x86_64-linux-gnu/libc++.so.1 (0x00007fdb08125000)
libc++abi.so.1 => /usr/lib/x86_64-linux-gnu/libc++abi.so.1 (0x00007fdb080ed000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fdb07f0c000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fdb07dbd000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fdb07da2000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fdb07bae000)
/lib64/ld-linux-x86-64.so.2 (0x00007fdb0a396000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fdb07b92000)
libuv.so.1 => /usr/lib/x86_64-linux-gnu/libuv.so.1 (0x00007fdb07b61000)
libcares.so.2 => /usr/lib/x86_64-linux-gnu/libcares.so.2 (0x00007fdb07b4d000)
libnghttp2.so.14 => /usr/lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007fdb07b24000)
libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007fdb0784e000)
libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007fdb077b9000)
libicui18n.so.66 => /usr/lib/x86_64-linux-gnu/libicui18n.so.66 (0x00007fdb074ba000)
libicuuc.so.66 => /usr/lib/x86_64-linux-gnu/libicuuc.so.66 (0x00007fdb072d4000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fdb072ce000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fdb072ab000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fdb072a0000)
libicudata.so.66 => /usr/lib/x86_64-linux-gnu/libicudata.so.66 (0x00007fdb057dd000)

`

Only research I could find on (ZN2v84base12CallOnceImplEPlPFvPvES2):
couchbase/tlm@4191a2a

Also tried in vain to install plv8--3.0alpha, but get similar error creating extension:
undefined symbol: _ZN2v84base7ieee7543powEdd

Wondering if this package is causing conflicts:
dpkg -S /usr/lib/x86_64-linux-gnu/libv8.so
libnode-dev:amd64: /usr/lib/x86_64-linux-gnu/libv8.so

@pkit
Copy link
Contributor

pkit commented Aug 15, 2020

@dfcsoftware hmm
objdump -tT plv8-2.3.13.so | grep CallOnce
?

@dfcsoftware
Copy link

~/plv8-2.3.13$ objdump -tT plv8-2.3.13.so | grep CallOnce
0000000000000000 *UND* 0000000000000000 _ZN2v84base12CallOnceImplEPlPFvPvES2_
0000000000000000 D *UND* 0000000000000000 _ZN2v84base12CallOnceImplEPlPFvPvES2_

@pkit
Copy link
Contributor

pkit commented Aug 15, 2020

Undefined.
Not statically linked.
I don't even have that symbol in my plv8-2.x or 3.x
I think there's probably some dynamically linked stuff.
And instead of being inlined it's left undefined.
Can you check if that symbol there in your dynsmic libv8.so? And if it exists somewhere in .o files in build/v8 ?

@JerrySievert
Copy link
Member

you should not have libnode-dev installed. plv8 will fail if it is there, due to the missing implementation that you are seeing.

@dfcsoftware
Copy link

Eureka! Thank you @pkit and @JerrySievert for your patience and support. plv8 extension works on Postgresql 12! 👍

Now I just need to figure out how to get Angular to work since removing libnode-dev removed npm.

postgres=# select version();
version
---------------------------------------------------------------------------------------------------------------
PostgreSQL 12.2 (Ubuntu 12.2-4) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.3.0-8ubuntu1) 9.3.0, 64-bit
(1 row)

postgres=# SELECT plv8_version();
plv8_version
--------------
2.3.13
(1 row)

postgres=# select * from pg_extension ;
oid | extname | extowner | extnamespace | extrelocatable | extversion | extconfig | extcondition
-------+---------+----------+--------------+----------------+------------+-----------+--------------
13447 | plpgsql | 10 | 11 | f | 1.0 | |
17977 | plv8 | 10 | 11 | f | 2.3.13 | |
(2 rows)

@pkit
Copy link
Contributor

pkit commented Aug 15, 2020

@dfcsoftware install a local, non system-wide, nodejs.
And setup PATH accordingly.

@hmarzooq
Copy link

hmarzooq commented Dec 16, 2021

dfcsoftware
Eureka! Thank you @pkit and @JerrySievert for your patience and support. plv8 extension works on Postgresql 12! 👍

how did you resolve the error? Would you mind giving some details?

@dfcsoftware
Copy link

@hmarzooq I followed @JerrySievert advice and uninstalled libnode-dev and @pkit to remove nodejs, then install a local version of nodejs.

Honestly after this I converted all my PLV8 stored procedures to Pg/sql. With the newer json support it was not too hard and removes a large dependency within my project. So I don't have this environment to look back on anymore.

@JerrySievert
Copy link
Member

@dfcsoftware fantastic! pl/pgsql is a great solution, especially if it fits your needs.

@GeorgesBAZ
Copy link

hello everyone,

Can somebody assist me? When I try to create Plv8 with Postgresql11 under Centos7, I Just Get The Following Error

2023-07-05 10:35:33.272 CEST [3614] ERROR: could not load library "/usr/pgsql-11/lib/plv8-2.3.13.so": /usr/pgsql-11/lib/plv8-2.3.13.so: undefined symbol: ZN2v84base12CallOnceImplEPlPFvPvES2
2023-07-05 10:35:33.272 CEST [3614] STATEMENT: create extension if not exists plv8;

@JerrySievert
Copy link
Member

@GeorgesBAZ if you have a specific problem, please either open an issue in GitHub or connect via discord at https://discord.gg/5fJN52Se - you will need to include information about the environment you are using, and how you installed plv8.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests