Skip to content

Conversation

@Iain-S
Copy link
Collaborator

@Iain-S Iain-S commented Feb 27, 2023

We currently write the output of the make- commands to stdout but this doesn't work so well now that we also write other files (e.g. vocab .csv files during make-generators and, potentially, stats.yaml).

This is a big single commit where we:

  1. Write the ORM and SSG files directly to disk (rather than stdout)
  2. Have defaults for the ORM and SSG file names, which can be overridden with --ssg-file=myfilename.py, etc.
  3. sys.exit(1) if an ORM, SSG or vocab file we are trying to write already exists (up for debate)
  4. Import files from the local directory (before, I think we were importing them relative to main.py or make.py, which lead to some funny behaviour. We now require the file to be in the current working dir, which necessitated several test changes.
  5. Rename tests/tmp to tests/workspace and run our functional tests there.
  6. Swap with patch("") as mock_something context managers for @patch("") test function decorators for better readability (and quicker to create/delete patch calls this way).

For the reviewer:

  1. Does this work end-to-end for you?

@Iain-S Iain-S added the WIP label Feb 27, 2023
@Iain-S Iain-S force-pushed the make-commands-write-to-file branch from c4144fd to 011b855 Compare February 27, 2023 18:18
@Iain-S Iain-S force-pushed the make-commands-write-to-file branch from db914d1 to eb95448 Compare February 27, 2023 18:27
@Iain-S Iain-S force-pushed the make-commands-write-to-file branch from f0f828b to b1ce6e8 Compare March 1, 2023 12:11
@Iain-S Iain-S removed the WIP label Mar 1, 2023
Change import_module so that it imports files from the cwd
Error if any file we try to write already exists
@Iain-S Iain-S force-pushed the make-commands-write-to-file branch from b1ce6e8 to f15f673 Compare March 1, 2023 12:31
@Iain-S Iain-S requested a review from mhauru March 1, 2023 12:47
Copy link
Collaborator

@mhauru mhauru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor request and one general comment, good otherwise

run_psql("dst.dump")

def test_workflow(self) -> None:
os.chdir("tests/workspace")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to use a tmp_path pytest fixture for this, to use a temporary folder that shouldn't conflict with anything existing, but pytest fixtures don't play nicely with unittest.TestCase subclasses. (https://docs.pytest.org/en/6.2.x/unittest.html)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think there must be a nicer way to do this than chdir() in the test functions / in the setUp. I could imagine the current way might cause some interdependence between tests if one fails in the setup before the first chdir().

@Iain-S Iain-S force-pushed the make-commands-write-to-file branch from 18e6b65 to d648f99 Compare March 1, 2023 16:21
@Iain-S Iain-S merged commit ed8d721 into main Mar 1, 2023
@Iain-S Iain-S deleted the make-commands-write-to-file branch March 1, 2023 16:28
tim-band added a commit to tim-band/sqlsynthgen that referenced this pull request Jun 19, 2025
* sampled and suppressed choice generators

* Fixed problems found trying this out for real.

---------

Co-authored-by: Tim Band <t.b@ucl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants