Refactoring tests to its specific tasks

[anaschwendler]

This issue aims at refactoring two existent tests that are working well, but need enhancements to keep tests independent of network connection and writing to/reading filesystem.

Tests that need to be refactored:
tests/test_federal_senate_dataset.py
tests/test_chamber_of_deputies_dataset.py

@cuducos would like to help in it :)

[cuducos]

Probably next week I can handle that. If anyone wants to pair on that, drop a line saying day/time it could work for you.

[cuducos]

cc @lipemorais

[cuducos]

Notes by @lipemorais in #53 about def __translate_file(self, csv_path):

This internal method looks a little big, could we break it to make the parts smaller and more meaningful and clear?

At this line looks that is a translation from portuguese to english but it's changing state of categories variable here. It is making this code more complex than it could be. :/

May it could be done in a function just to give it a name and help with clarity.
Show all reviewers

(this last comment about line 93)

[anaschwendler]

We we're thinking about testing:

• Check the translations
• The clean() method unify all the senate datasets, maybe test if the reimbursements are getting concatenated (this is against TDD, but ¯_(ツ)_/¯ )
• The fetch() method does not check if the senate website is available to fetch the datasets, maybe test it or implement something should be good :)

[cuducos]

this is against TDD, but

In Brazilian Portuguese TDD means teste depois do deploy, right?

[lipemorais]

I'm run the tests and I just realized one thing, because it's taking more time than I expect.

Test this PR would even easier with unit tests. Just fetch needs integration, translate and clean could use only use unit test with a example files. :/

[anaschwendler]

@lipemorais Yes, I was talking about that tests taking so long to run yesterday in your daily meeting, and I didn't know what to do, do you have any suggestion for us? Something like a light or something? I really doesn't know what to do :T

I just really tested what I wanted to do, and didn't know how to properly refactor it

[lipemorais]

I can think now in two ways:

1. I would isolate the integration point, in this case is fetch and unit test translate and clean with a mocked example files.

2. In a second way I would unit test the three steps using mock while transform the integration test into a journey test, that should test this workflow without any mocks and let it run in Travis CI.

[lipemorais]

I'm working on it. :)

[lipemorais]

Hey!
Do we consider it half done with the PR #68, now it should be done for CEAP also.

[anaschwendler]

Ok, I'll close it ;)