Fails to parse output from pg_dump?

[bjmc]

I'm not sure if I'm using this wrong, but DDLParser() fails to parse the attached PostgreSQL schema output from pg_dump

schema.txt

Traceback:

~/.cache/pypoetry/virtualenvs/capgras-r404Yw5Z-py3.8/lib/python3.8/site-packages/simple_ddl_parser/parser.py in run(self, dump, dump_path, file_path, output_mode, group_by_type, json_dump)
    228             Dict == one entity from ddl - one table or sequence or type.
    229         """
--> 230         tables = self.parse_data()
    231         tables = result_format(tables, output_mode, group_by_type)
    232         if dump:

~/.cache/pypoetry/virtualenvs/capgras-r404Yw5Z-py3.8/lib/python3.8/site-packages/simple_ddl_parser/parser.py in parse_data(self)
    167                 if final_line or new_statement:
    168                     # end of sql operation, remove ; from end of line
--> 169                     statement = statement[:-1]
    170                 elif num != len(lines) - 1 and not skip:
    171                     # continue combine lines in one massive

TypeError: 'NoneType' object is not subscriptable

Please feel free to close this issue if you don't intend to support this use-case.

[xnuinside]

@bjmc hi, thanks for reporting the issue! I will work on it today & release a fix after.

[xnuinside]

@bjmc thanks for the sample - I added support for bunch cases based on your example. Now it is parsable and you can get output. Please, use version 0.22.6 it is released - https://pypi.org/project/simple-ddl-parser/ . I added also bunch of tests based on your sample - https://github.com/xnuinside/simple-ddl-parser/blob/main/tests/test_ddl_settings.py and https://github.com/xnuinside/simple-ddl-parser/blob/main/tests/test_sequences.py#L145

[bjmc]

Wow, thanks for that incredibly fast response! I'll give the new version a test tomorrow and if it works, I'll close this issue.

Thanks for your work on this project @xnuinside - it's very ambitious, and as far as I can tell, no one else is even attempting to offer something similar.

[bjmc]

This is a big help. It parses without throwing an exception now! I think there's still room for improvement in understanding the meaning of the Postgres schema. For example, the DDL contains

ALTER TABLE ONLY public.accounts ADD CONSTRAINT accounts_username_key UNIQUE (username);

But in the parsed output, we see:

   {'name': 'username',
    'type': 'character varying',
    'size': 50,
    'references': None,
    'unique': False, <======= should be True?
    'nullable': False,
    'default': None,
    'check': None},

But the bug I reported is fixed, so closing this issue.

[xnuinside]

@bjmc I created separate issue for this - #101 . I will fix it in future release.