Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PLEASE REMOVE THE ADVICE TO USE PGX FROM YOUR PAGE #1022

Open
uwauwa opened this issue Jan 12, 2021 · 6 comments
Open

PLEASE REMOVE THE ADVICE TO USE PGX FROM YOUR PAGE #1022

uwauwa opened this issue Jan 12, 2021 · 6 comments

Comments

@uwauwa
Copy link

uwauwa commented Jan 12, 2021

PLEASE REMOVE THE ADVICE TO USE PGX FROM YOUR PAGE


jackc/pgx#905 (comment)

we have migrated 250 instances web applications from lib/pq to pgx based on the performance promises

  1. pgxpool.Connect is not working and was throwing connections timeout, this caused instances to freeze and break nginx with gateway time out, no matter how much increase nginx timeout, postgres rds memory
  2. reverting back to pgx.Connect throw hundreds of busy connection messages and all concurrent queries, ajax services and API broken
  3. scan float64 is not possible, and forcing the float64 to scan pgx.Numeric on interfaces values broken the reporting for any numeric postgresql table
  4. using sql driver to scan float64 on database/sql NullFloat64 added extra complexity and extra loop check made the performance gain over pq useless

we had to revert back to lib/pq.
we believe this is not production ready and the performance statement should be amended

ubunut 20/aws ec2, postgresql12.5 /aws rds, go1.5,
gobwas/ws latest
lib/pq latest
pgx latest

@kataras
Copy link

kataras commented Feb 3, 2021

That's not fair...

  1. You getting timeout errors because of a context canceled or deadline exceeded errors, check it with:
// IsErrTimeout reports whether the given "err" was caused by a timeout;
// context.Canceled, context.DeadlineExceeded or a network timeout.
func IsErrTimeout(err error) bool {
	if IsErrCanceled(err) {
		return true
	}

	var netErr net.Error
	return errors.As(err, &netErr) && netErr.Timeout()
}

// IsErrCanceled reports whether the given "err" was caused by a
// context.Canceled or context.DeadlineExceeded timeout.
func IsErrCanceled(err error) bool {
	return errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded)
}
  1. ?
  2. scaning float64 is possible, you can use decimal type or the specific type from pgx: var param pgtype.Float8
  3. ?

Both pgx and pq are great libraries, I used pq at first but now I am using pgx on a production-level product with thousand of calls per second, works like a charm (both of them did).

Cheers and have fun!

@uwauwa
Copy link
Author

uwauwa commented Feb 4, 2021

If the PGX maintainers acknowledged "PGX statements can't handle float64"
Read here
jackc/pgx#150 (comment)
jackc/pgx#150
jackc/pgx#112

If the PGX Maintainers acknowledged the pool issues and limitations
Read here
jackc/pgx#494 (comment)

Where did you come up with your conclusion? You misleading people by comments does not relate to the issue being raised here.

Please try to understand and study the 4 points I mentioned above, please try to make relevant comments worth reading mate.

Thank you,

@Lekensteyn
Copy link

@uwauwa please try to be more civil, @kataras took their time trying to come up with a helpful response and at least the first response looks reasonable. I can't comment on the float64 point since I don't know about them.

Having that said, I looked briefly in lib/pq and pgx connection error handling since we ran into a production issue with it. Based on that I will try to answer your questions regarding that aspect:

  1. pgxpool.Connect is not working and was throwing connections timeout, this caused instances to freeze and break nginx with gateway time out, no matter how much increase nginx timeout, postgres rds memory

Perhaps you ran into a connection limit? pgxpool defaults to a maximum of four connections (or the number of CPUs, whichever is larger):
https://github.com/jackc/pgx/blob/v4.10.0/pgxpool/pool.go#L258-L261

  1. reverting back to pgx.Connect throw hundreds of busy connection messages and all concurrent queries, ajax services and API broken

pgx.Connect returns a pgx.Conn which is documented as not safe for concurrent usage, https://pkg.go.dev/github.com/jackc/pgx#Conn

Conn is a PostgreSQL connection handle. It is not safe for concurrent usage. Use ConnPool to manage access to multiple database connections from multiple goroutines.

I have not looked closely at how to use pgx in its best capacity so cannot make recommendations there, but those are questions to be filed with the pgx community.

@ghost
Copy link

ghost commented Aug 2, 2021

Hi,

Is there any progress here please?

@uwauwa can you please provide more information?

We have similar situation where we use this lib heavily in our agency.

Thank you,

@S-YOU
Copy link

S-YOU commented Nov 12, 2022

Having this post appears on Google is pretty annoying. Change the title and ALL CAPS or close or delete it.
Nobody need to know someone else personal opinion!

image

@afshanz
Copy link

afshanz commented Mar 6, 2023

PLEASE REMOVE THE ADVICE TO USE PGX FROM YOUR PAGE

jackc/pgx#905 (comment)

we have migrated 250 instances web applications from lib/pq to pgx based on the performance promises

  1. pgxpool.Connect is not working and was throwing connections timeout, this caused instances to freeze and break nginx with gateway time out, no matter how much increase nginx timeout, postgres rds memory
  2. reverting back to pgx.Connect throw hundreds of busy connection messages and all concurrent queries, ajax services and API broken
  3. scan float64 is not possible, and forcing the float64 to scan pgx.Numeric on interfaces values broken the reporting for any numeric postgresql table
  4. using sql driver to scan float64 on database/sql NullFloat64 added extra complexity and extra loop check made the performance gain over pq useless

we had to revert back to lib/pq. we believe this is not production ready and the performance statement should be amended

ubunut 20/aws ec2, postgresql12.5 /aws rds, go1.5, gobwas/ws latest lib/pq latest pgx latest

for case 2) i raised an issue with pgx jackc/pgx#1541

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants