integration test for the text classification

[dcfidalgo]

This PR adds an integration test using the TextClassification head.

On my machine it takes <1 min and the numbers are reproducible.
It covers only a small part of the functionality, but with this test we would have caught the embedding bug for example.
The idea is that with time we extend the test to cover more functionalities, and maybe it can serve as blue print for other integration tests.

[dcfidalgo]

Just realized that it this is useless for the CI, since i get the data from our S3 ... maybe we can discuss briefly where to put this data, since it is the same as for our tutorials.

[frascuchon]

Just realized that it this is useless for the CI, since i get the data from our S3 ... maybe we can discuss briefly where to put this data, since it is the same as for our tutorials.

I prefer to keep a local copy of data used in tests. Some metrics could change if we update the original data.

[dvsrepo]

We can keep a very small local sample of the data and assert that the model overfits. El dom., 24 may. 2020 17:51, Francisco Aranda <notifications@github.com> escribió:

…

Just realized that it this is useless for the CI, since i get the data from our S3 ... maybe we can discuss briefly where to put this data, since it is the same as for our tutorials. I prefer to keep a local copy of data used in tests. Some metrics could change if we update the original data. — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#216 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIOJJZML7LK5XVMV6WYA6LRTE7AZANCNFSM4NIAPLJA> .

[dcfidalgo]

We can keep a very small local sample of the data and assert that the model
overfits.

@dvsrepo The idea was to be able to detect even subtle changes that maybe go undetected when completely overfitting a small data set. Maybe the question is to find a good balance between data set size and being able to catch subtle changes.
@frascuchon What is the maximum size of the data set i should include as local copy in the test?

[frascuchon]

@frascuchon What is the maximum size of the data set i should include as local copy in the test?

Your filtered dataset could be enough since take no quite training time. If I remember was set in code to 2000 records, isn't it ?

[dcfidalgo]

Ok, i think this can go in.
@frascuchon i added a few asserts regarding the vocab, not sure if you had some more thorough checks in mind. We could add them here or in a follow-up PR.

It would be nice to programatically create the pipeline configuration PipelineConfiguration and the save as a yaml file.

@frascuchon If you are ok with it, i will add this test in a specific def test_pipeline_configuration in another PR, in order not to delay further this one ...