Applying TDD on SELECT - Proof of Concept (disposable draft)

[jimmytty]

This draft is just a proof of concept about how to apply TDD on plain SELECTs statements. It's based on #217.

The main problems of this approach are:

• need to run the stub file two times, one to show the output and other to collect the queries' results.
• compare the tests table against the queries outputs without manual UPDATE.
• some queries will need an alias to make things works properly.
• it's not easy to find and compare a wrong statement / output.

[blackk-foxx]

Hi @jimmytty,

• need to run the stub file two times, one to show the output and other to collect the queries' results.

This does not seem like a big problem; would you agree? Unless I'm mistaken, this is also how the existing practice tests work.

For the other three problems you cited, can you elaborate a bit? Can you provide an example -- or even better, modify the existing example -- to demonstrate the issue?

If I understand the last one correctly, the challenge is to correlate the actual output with the expected output. What would you think about adding a row number to each row in the tests and results tables, and joining the tables on the row number.

[IsaacG]

Given that all the practice exercises have a pretty direct 1-1 mapping from input to output, and given the additional complexity this adds, I'm not sure this is something we'd necessarily want to use here.

The advantage this approach has is, (1) being able to remove the UPDATE part of the solution and (2) potentially being able to use this for exercises that SELECT multiple values. The additional UPDATE component of the solution isn't very much. It's a pretty basic operator and can be provided in the stubs for earlier concept exercises. The multiple row issue is a lot harder to work around. It can definitely be worked around, eg using a JSON array, but that's a lot messier.

I'd love to hear if the other members of @exercism/sqlite have thoughts here.

[jimmytty]

Hi @jimmytty,

• need to run the stub file two times, one to show the output and other to collect the queries' results.

This does not seem like a big problem; would you agree? Unless I'm mistaken, this is also how the existing practice tests work.

Yes, this solution uses the current testing framework but increases complexity and reduces accuracy. For concepts, it might be usable (or maybe not). But I thought it was important to show that it's possible and that it might be useful in some way, even though I already knew it might never be truly viable.

For the other three problems you cited, can you elaborate a bit? Can you provide an example -- or even better, modify the existing example -- to demonstrate the issue?

For example, for alias, this code:

-- This one will fail on purpose
SELECT 'Hello, world.' AS say_hi;

will fail because the tests are expecting the same code without the alias ("say_hi"), I have written this way. I.e, if the student create their own alias my approach will always fail.

For other cases, like expressions:

SELECT 32 * 21.1 / 3

it's better to use an predefined alias to make my approach find the solution in a more deterministic way.

If I understand the last one correctly, the challenge is to correlate the actual output with the expected output. What would you think about adding a row number to each row in the tests and results tables, and joining the tables on the row number.

Already has a implicit "row_number" we can use without modify the table schema and if the documentation instructs the student to follow the established order exactly, we could use it as "primary key" and make better tests in fail cases. Also it's possible, if needed, write a specific test case for each select to be tested.

Being able to use SELECT freely would be a great advantage for initial concept exercises. It's possible, but perhaps not feasible.

[jimmytty]

Given that all the practice exercises have a pretty direct 1-1 mapping from input to output, and given the additional complexity this adds, I'm not sure this is something we'd necessarily want to use here.

The advantage this approach has is, (1) being able to remove the UPDATE part of the solution and (2) potentially being able to use this for exercises that SELECT multiple values. The additional UPDATE component of the solution isn't very much. It's a pretty basic operator and can be provided in the stubs for earlier concept exercises. The multiple row issue is a lot harder to work around. It can definitely be worked around, eg using a JSON array, but that's a lot messier.

I agree. I liked the good explanation too.

[glennj]

A radical thought: use a language like Python that ships with a sqlite library. Open the db, send the user's code, collect the results and compare to the expected results.

[blackk-foxx]

A radical thought: use a language like Python that ships with a sqlite library. Open the db, send the user's code, collect the results and compare to the expected results.

Personally, I would be much more effective using a Python-based approach than trying to work within the constraints of SQLite for the test reporting logic. Would the maintainers be amenable to this sort of thing?

[IsaacG]

A radical thought: use a language like Python that ships with a sqlite library. Open the db, send the user's code, collect the results and compare to the expected results.

You're welcome to try and make it work. However, if the Python SQLite module is similar to the Microsoft SQL module, it is great at executing single queries but rather useless at handling inputs with comments and multiple queries. I've spent several hours during a recent day job trying to make that work before concluding it isn't feasible. But you're welcome to try and see if you can make it work ;)