Skip to content

Latest commit

 

History

History
19 lines (16 loc) · 12.9 KB

data-processing-postgresql-issues.md

File metadata and controls

19 lines (16 loc) · 12.9 KB

Data Processing-PostgreSQL Issues

Where Why How to resolve
Error message Error Location Error type Error identification (Cause) Probable resolution
ERROR: duplicate key value violates unique constraint "_pkey" Postgres logs in console Duplicates record are in Input file even after validation (Primary key constraint violation of table XYZ) When the primary key has same set of values for more than 2 records, the error will occur. The error mostly occurs while updating to aggregation tables, due to inconsitency/mapping issues in the data. The data needs to be mapped correctly to eliminate this issue.
ERROR: null value in column "XYZ" violates not-null constraint Postgres logs in console Null value records are in Input file even after NULL validation for column XYZ When an required field has null value in the data file. Null values need to be removed from the data file
ERROR: deadlock detected: DETAILS: Process 21601 waits for ShareLock on transaction 89371; blocked by process 21603 Postgres logs in console DEADLOCK situation because process 21601 is waiting until process 21603 is completed When simultaneous update is happening to the same record by two process. Check if there are two Job running simultaneously, if so stop one of them.
ERROR: could not extend file "base/16384/75771.2": No space left on device Postgres logs in console Not enough disk space in system When an operation performed doesn't have enough disk space to accomplish the task due to lack of memory. Check if there is enough disk space to accomplish the task.
"ERROR: out of memory DETAIL: Failed on request of size 324589128." Postgres logs in console Not enough main memory

When we consume more memory than available on our machine we can see out of memory errors within our Postgres logs, or in worse cases the OOM killer can start to randomly kill running processes to free up memory.
An out of memory error in Postgres simply errors on the query you’re running, where as the the OOM killer in linux begins killing running processes which in some cases might even include Postgres itself

Reduce the emission file size to avoid the issue.
ERROR: value too long for type character varying(100) Postgres logs in console Data field exceeds size exceeds the column size When the field has too long to accommodate within a column, we get this error Correct the data field which exceeds the size.
ERROR: value too long for type integer Postgres logs in console Data field exceeds size exceeds the column size When the field has too long to accommodate within a column, we get this error Correct the data field which exceeds the size.
ERROR: column "" does not exist at character Postgres logs in console Data field doesn't exist When the data fields are not according to the emission specification Correct the header in the emitted file
ERROR: there is no unique or exclusion constraint matching the ON CONFLICT specification Postgres logs in console Inconsistency/mapping issues in the data When mapping of the data has issues, we may get this issue The error mostly occurs while updating to aggregation tables, due to inconsistency/mapping issues in the data. The data needs to be mapped correctly to eliminate this issue.
FATAL: terminating connection due to administrator command Postgres logs in console When the data base was restarted

If the data base restarts during the running process,
we will get this issue

Database shouldn't be restarted during a process is running, check the sys logs to know why the database service was terminated.
LOG: received fast shutdown request Postgres logs in console When the data base was restarted

If the data base restarts during the running process,
we will get this issue

Database shouldn't be restarted during a process is running, check the sys logs to know why the database service was terminated.
ERROR: relation "" does not exist at character Postgres logs in console When the relation doesn't exist If the relation doesn't exist, will get this issue