Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make column type inference optional #2358

Closed
Tracked by #2346
dmos62 opened this issue Jan 25, 2023 · 2 comments · Fixed by #3050
Closed
Tracked by #2346

Make column type inference optional #2358

dmos62 opened this issue Jan 25, 2023 · 2 comments · Fixed by #3050
Assignees
Labels
type: enhancement New feature or request work: backend Related to Python, Django, and simple SQL work: frontend Related to frontend code in the mathesar_ui directory
Milestone

Comments

@dmos62
Copy link
Contributor

dmos62 commented Jan 25, 2023

Root issue: #2346

Currently, the frontend makes it obligatory to perform type inference during import. This can be a blocking bug if

  1. the type inference takes too long to perform (to be addressed by Make column type inference per-column #2359 and Make column type inference faster #2347),
  2. and type inference is a synchronious operation tied to a HTTP transaction that is subject to timeouts (it currently is) (tracked by Decouple column type inference from an HTTP transaction #2360).

We can improve Mathesar's inference speed, but we can't ever make 1 impossible, because a user can always just give you a workload that's too big. Fixing 2 would get rid of timeouts and would be a significant improvement, but the user could still present such a big workload that he might be stuck waiting for it to finish for a very long time. Therefore, making type inference during importing optional is essential to a robust import process.

Similar to

@dmos62 dmos62 changed the title Make column type inference optional, Make column type inference optional Jan 25, 2023
@dmos62 dmos62 added type: enhancement New feature or request work: design work: backend Related to Python, Django, and simple SQL work: frontend Related to frontend code in the mathesar_ui directory labels Jan 25, 2023
@dmos62 dmos62 added this to the Backlog milestone Jan 25, 2023
@dmos62
Copy link
Contributor Author

dmos62 commented Jul 13, 2023

@seancolsen would you care to pick this issue up?

This issue seems to be the most direct and easy way to make the import process robust, because it would be a workaround for the import timeout problem. It also seems to be largely frontend work.

In #2346 I track additional ways to improve imports. Making type inference a per-column operation (one HTTP transaction, one column type inferred) is the next thing I'd do, but that involves significantly more work and coordination.

@seancolsen
Copy link
Contributor

I'm working on this now.

(@dmos62 my approach has shifted a tiny bit since our call earlier today.)

I'm planning to open a draft PR soon which will have screenshots of the app withe UI that I'm proposing. We can continue the discussion about how best to solve the larger problems at that point because we'll have a concrete example of some UX to reference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement New feature or request work: backend Related to Python, Django, and simple SQL work: frontend Related to frontend code in the mathesar_ui directory
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

2 participants