Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate VGSL spec before loading training data #298

Closed
mittagessen opened this issue Oct 29, 2021 · 2 comments
Closed

Validate VGSL spec before loading training data #298

mittagessen opened this issue Oct 29, 2021 · 2 comments

Comments

@mittagessen
Copy link
Owner

Add a method to lib.vgsl.TorchVGSLModel to validate a (partial) spec, feeding a 'dummy' line/image into it depending on the input specification. This would allow aborting training before loading the training dataset when an invalid/unworkable spec is given to the KrakenTrainer constructors.

@anutkk
Copy link
Contributor

anutkk commented Nov 1, 2021

Why should this require adding a method to lib.vgsl.TorchVGSLModel ? In the KrakenTrainer constructors immediately after adding the first element to the dataset it is possible to feed the dataset to the network inside a try...except and throw a specific error. That would require a couple of lines of code.

@mittagessen
Copy link
Owner Author

A separate validator is preferable as it would allow tools that use the API like escriptorium to validate a user-provided spec. In the KrakenTrainer object we'd have to instantiate the model twice (once without the output layer to validate and once with it after having loaded the complete dataset so we can determine the codec alphabet). I'd like to avoid that as the constructors are already annoyingly large complex pieces of code and should really be slimmed down.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants