-
Notifications
You must be signed in to change notification settings - Fork 88
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]: BCP import memory leak with decimals #2455
Labels
bug
Something isn't working
Comments
This was referenced Apr 1, 2024
shardgupta
pushed a commit
that referenced
this issue
Apr 8, 2024
`TdsTypeNumericToDatum` [allocates a buffer](https://github.com/babelfish-for postgresql/babelfish_extensions/blob/6606b1118978477186869c3dddd2c6d85fcc6387/contrib/babelfishpg_tds/src/backend/tds/tdstypeio.c#L1133) for parsing incoming numeric data, but does not release this buffer. So it remains in `MessageContext` until the end of BCP import call. Proposed patch frees this buffer before exiting `TdsTypeNumericToDatum`. ### Issues Resolved #2455, BABEL-4882 Signed-off-by: Alex Kasko <alex@staticlibs.net>
shardgupta
pushed a commit
that referenced
this issue
Apr 8, 2024
`TdsTypeNumericToDatum` [allocates a buffer](https://github.com/babelfish-for postgresql/babelfish_extensions/blob/6606b1118978477186869c3dddd2c6d85fcc6387/contrib/babelfishpg_tds/src/backend/tds/tdstypeio.c#L1133) for parsing incoming numeric data, but does not release this buffer. So it remains in `MessageContext` until the end of BCP import call. Proposed patch frees this buffer before exiting `TdsTypeNumericToDatum`. ### Issues Resolved #2455, BABEL-4882 Signed-off-by: Alex Kasko <alex@staticlibs.net>
staticlibs
added a commit
to wiltondb/babelfish_extensions
that referenced
this issue
Apr 14, 2024
…2456) `TdsTypeNumericToDatum` [allocates a buffer](https://github.com/babelfish-for postgresql/babelfish_extensions/blob/6606b1118978477186869c3dddd2c6d85fcc6387/contrib/babelfishpg_tds/src/backend/tds/tdstypeio.c#L1133) for parsing incoming numeric data, but does not release this buffer. So it remains in `MessageContext` until the end of BCP import call. Proposed patch frees this buffer before exiting `TdsTypeNumericToDatum`. ### Issues Resolved babelfish-for-postgresql#2455, BABEL-4882 Signed-off-by: Alex Kasko <alex@staticlibs.net>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What happened?
There seems to be a leak in decimal (numeric) conversion. Memory is allocated in
MessageContext
, so it is released after BCP import finishes. Still can be a problem when importing large files.Using the following table with a decimal column:
Export it with BCP:
Clear the table:
Run server under mprof:
Import exported data back with BCP:
BCP import backend process shows excessive memory usage:
Will submit a patch to
TdsTypeNumericToDatum
shortly.Version
BABEL_3_X_DEV (Default)
Extension
babelfishpg_tds
Which flavor of Linux are you using when you see the bug?
Fedora
Relevant log output
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: