Testing suggestions

[epogrebnyak]

to_csv: is not tested at all, function in its current state is hard to test as it creates file for the given
path you should consider passing open file to this function this way in tests you can pass mocked file
and check what was writen in that file by the function.

from_csv: is not tested explicitly but it is covered through some tests that use read_csv function

read_csv: is covered but not tested explicitly probably for the same reason as to_csv

get_tables_from_rows_segment: is not tested explicitly but it is extensively used both in code and it is used in
pipeline tests so to my mind it is vital to test this function thoroughly

    fix_multitable_units, check_required_labels: are not tested explicitly but are used in
    get_tables_from_rows_segment only so it wont be necessarily to test them separately if that function
    is tested well enough.

is_year: not tested explicitly but very straightforward and no need to test it.

split_to_tables: looks very complex but it is not tested explicitly.

month_end_day: not tested explicitly but very straightforward and no need to test it.

Row.__str__ is tested but __repr__ is not

RowStack.is_found is not covered fully (path leading to false is not covered)

RowStack.pop_segment is not tested explicitly it is covered by tests of pop method though.

Table.echo_error_table_not_valid: not tested, better take a second look echo in name by intuition means
printing something but this method throws whole bunch of errors is it really a expected behaviour?

Header.set_varname and Beader.set_unit have complex logic but not tested in isolation. (need more tests)

DictMaker is not tested explicitly but it is straightforward class and maybe it is enough that
it is implicitly covered (m_dict method is not covered though)



By the way you can omit scripts from coverage using --omit=(coma separated list of regex patterns) parameter
like so: coverage report -m --omit=test_*.py,tst_draft.py > coverage.output

[epogrebnyak]

"""
General Notes:

    "sad paths" should be tested as well as "happy paths"

    Related tests should be grouped in a TestClass as it makes tests more readable,
    grouped meaningfully and one can easily notice if tests are missing or duplicated.

    Mocks should be used to test each entity in isolation from its dependencies

splitter.py:
    this are mostly stateless functions and can be tested with traditional
    "give input expect output approach".

files.py:

    As far as the coverage report shows it is tested fairly well
    5 or 6 tests and it will be covered 100%. Tests missing for:
        - filter_date sad path
        - init_dirs
        - InterimDataFolder.get_latest_folder
        - etc.
    Maybe some refactoring (grouping of tests etc.)

parse.py:
    following are the test skeletons for parse module
    first of all existing tests should be split and added in appropriate stub methods
    Then remaining stubs should be implemented. Tests should be added or removed accordingly,
    during implementation.
"""


# label handling tests
class TestLabelHandling(object):
    def test_make_label(self):
        pass

    def test_split_label(self):
        pass

    def test_extract_varname(self):
        pass

    def extract_unit(self):
        pass


# csv file access tests
class TestCsvFileAccess(object):
    def test_to_csv(self):
        pass

    def test_from_csv(self):
        pass

    def test_read_csv(self):
        pass


class TestTables(object):
    def test_get_all_without_additional_definitions(self):
        pass

    def test_get_all_with_additional_definitions(self):
        pass

    def test_get_defined(self):
        pass


class TestYearFunctions(object):
    # Happy path
    def test_get_year_with_match(self):
        pass

    # Sad path
    def test_get_year_withouth_match(self):
        pass

    # Happy path
    def test_get_year_greater_or_equal_1991(self):
        pass

    # Sad path
    def test_get_year_less_than_1991(self):
        pass

    # Happy path
    def test_is_year_on_valid_year(self):
        pass

    # Sad path
    def test_is_year_on_invalid_year(self):
        pass


class TestRow(object):
    pass


class TestRowStack(object):
    def test_is_matched_on_empty_pat(self):
        pass

    # Happy path
    def test_is_matched_on_textline_starting_with_pat(self):
        pass

    # Sad path
    def test_is_matched_on_textline_not_starting_with_pat(self):
        pass

    def test_is_found_on_matching_text(self):
        pass

    def test_is_found_on_not_matching_text(self):
        pass

    # TODO: add some more test_pop_* functions to cover whole method
    def test_pop(self):
        pass

    # TODO: add some more test_pop_segment_* functions to cover whole method
    def test_pop_segment(self):
        pass


class TestSplitToTables(object):
    # TODO: add methods to cover split_to_tables method
    pass


class TestTable(object):
    # TODO: add methods to test parse.Table class
    pass


class TestHeader(object):
    # TODO: add methods to test parse.Header class
    pass


class TestFixMultiTableUnits(object):
    pass


class TestCheckRequiredLabels(object):
    pass


class TestGetTablesFromRowsSegment(object):
    pass


class TestToFloat(object):
    pass


class TestDictMaker(object):
    pass


class TestEmitter(object):
    pass


class TestDatapoints(object):
    pass


class TestDataframeDatesHandling(object):
    def test_month_end_day(self):
        pass

    def test_get_date_month_end(self):
        pass

    def test_get_date_quarter_end(self):
        pass

    def test_get_date_year_end(self):
        pass


class TestFrames(object):
    pass


# This will probably be integration test of all the previously defined entities
class TestVintage(object):
    pass


class TestCollection(object):
    pass

[epogrebnyak]

• http://geosoft.no/development/unittesting.html
• https://medium.com/pacroy/why-most-unit-testing-is-waste-tests-dont-improve-quality-developers-do-47a8584f79ab
• https://stackoverflow.com/questions/32569732/correct-way-of-unit-testing-repr-with-dict
• https://stackoverflow.com/a/3322167: I'm not against code coverage, but it's too easy to fall into using code coverage as the metric to know when testing is complete. I think that would be a mistake.

[epogrebnyak]

https://archive.org/details/pyvideo_310___tests-and-testability-188

[epogrebnyak]

#Layers:
#
# tables.py
# - csv file
# - rows
#   - year, month
#   - rowstack
# - tables
#
# vintage.py
#   - datapoints
#   - emitter
#   - frames

# -------------------------------------------

# read_csv will read csv with filter for empty rows

csv_data = Rows(year, month, loader=read_csv)
csv_segment = csv_data.pop(pdef)
csv_rem = csv_data.remaining

# Fixtures:
# - csv content  
# - generate rows with different loader mocking read_csv
# - generate tables from rows  

# Fixture types:
# - splitting tables from rows
# - detecting labels in rows
# - reading table

[epogrebnyak]

See mock CSV in #9

[epogrebnyak]

An oldie with 'dirty hybrids': http://blog.stevensanderson.com/2009/08/24/writing-great-unit-tests-best-and-worst-practises/