Validation Test Helper

[MathiasSJacobsen]

Is there an example that exists to demonstrate how to test the custom validation rules written for the language? Could only find #340, but that was only testing the parsing.

Currently working on my master thesis, so there are braking changes all the time, so could be nice to have some automated test so I don't have to manually check them each time I change something.

[msujew]

Hi @MathiasSJacobsen,

we expose some testing functionality in the framework, but none for validation so far. For a quick custom setup you could simply check whether the diagnostics returned from the DocumentValidator match whatever you expect. If you want a simple regression test, you could do something like this:

First iteration: Setting up the test

const services = createLanguageServices();
const helper = parseHelper<RootAST>(services);
const documentContent = "...";

(async function () {
  const doc = await helper(documentContent);
  const diagnostics = await services.validation.DocumentValidator.validateDocument(doc);
  console.log(JSON.stringify(diagnostics, undefined, 2)); // Store this logged string somewhere
})();

Second iteration: Using the printed diagnostics to identify regressions

const services = createLanguageServices();
const helper = parseHelper<RootAST>(services);
const documentContent = "...";

test('validation test', async () => {
  const expectation = [ /** array of printed diagnostics */ ];
  const doc = await helper(documentContent);
  const diagnostics = await services.validation.DocumentValidator.validateDocument(doc);
  expect(diagnostics ).toEqual(expectation);
});

It's a bit crude and only works for a whole document, but it's quick to set up and get going.

A question out of pure personal curiosity: What are you using Langium for in your thesis? I'm also in the process of writing my master thesis (on parsing and unbounded lookahead, see Chevrotain/chevrotain#1714)

[MathiasSJacobsen]

I'll take a look and try it out. Do you have any idea for when it will (or if it will) be a part of the framework?

Im developing tool support for a multiway dataflow constraint system library called HotDrink. We think languim are going to be used for moste of it, we are still in the beginning-face of the thesis, so it might change. At point of writing it will most defiantly be used for syn-tax highlighting, auto-completion and semantic code validation.

[spoenemann]

Do you have any idea for when it will (or if it will) be a part of the framework?

@MathiasSJacobsen please create an issue. I think we can include this in the 0.4.0 version (around April).

[msujew]

Yeah, we definitely plan to add more testing utilities, especially for validations.

I see, sounds like a great use case for the framework. Good luck with your thesis!

[MathiasSJacobsen]

Thank you, and thanks for fast response!

[MathiasSJacobsen]

Just a small typo-fix

const services = createLanguageServices();
const helper = parseHelper<RootAST>(services);
const documentContent = "...";

test('validation test', async () => {
  const expectation = [ /** array of printed diagnostics */ ];
  const doc = await helper(documentContent);
  const diagnostics = await services.validation.DocumentValidator.validateDocument(doc.document);
  expect(diagnostics ).toEqual(expectation);
});