Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[frozen] pyDKB.dataflow.stage tests #196

Open
wants to merge 28 commits into
base: pyDKB
Choose a base branch
from
Open

[frozen] pyDKB.dataflow.stage tests #196

wants to merge 28 commits into from

Commits on Nov 16, 2018

  1. 5 Configuration menu
    Copy the full SHA
    4c1c44a View commit details
    Browse the repository at this point in the history

Commits on Nov 19, 2018

  1. Updated a check_args method.

    Now the name of an argument that causes assertion failure should be
    obtainable.
    Evildoor committed Nov 19, 2018
    2 Configuration menu
    Copy the full SHA
    e1343c5 View commit details
    Browse the repository at this point in the history
  2. Reworked mode tests.

    This includes mode dictionary creation, removal of 'mode' member
    from args_to_add dictionary and repositioning of the latter for
    the sake of consistency.
    Evildoor committed Nov 19, 2018
    3 Configuration menu
    Copy the full SHA
    1022253 View commit details
    Browse the repository at this point in the history
  3. 14 Configuration menu
    Copy the full SHA
    088e180 View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2018

  1. Improved check_args method.

    Evildoor committed Nov 20, 2018
    Configuration menu
    Copy the full SHA
    339c3ca View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1a56cd0 View commit details
    Browse the repository at this point in the history
  3. Reworked modes handling.

    This way the code should be more flexible and easier to read.
    Evildoor committed Nov 20, 2018
    Configuration menu
    Copy the full SHA
    dcbdeab View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    3eaa413 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    1565684 View commit details
    Browse the repository at this point in the history
  6. Added input files test.

    Evildoor committed Nov 20, 2018
    Configuration menu
    Copy the full SHA
    be922c9 View commit details
    Browse the repository at this point in the history
  7. Removed short version of override tests.

    Both short and long versions of defining an argument are doing
    the same thing and are processed by argparse. Therefore, it is
    presumed that testing both versions of defining each argument
    is enough and overriding tests can be done just for one version.
    Evildoor committed Nov 20, 2018
    Configuration menu
    Copy the full SHA
    aa90f7c View commit details
    Browse the repository at this point in the history
  8. Moved override_hdfs_mode definition into a loop.

    Note: new version includes mode args check that was missing.
    Evildoor committed Nov 20, 2018
    Configuration menu
    Copy the full SHA
    cf56b0c View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2018

  1. Improved readability in testing hdfs overrides.

    In current version the line of code is pointless because arg
    is either 'source' or 'dest' which are redefined in the next few
    lines. However, the code reader may not know that. Furthermore,
    this line may be of use if it would be necessary to test
    some other arguments' interaction with hdfs.
    Evildoor committed Nov 21, 2018
    Configuration menu
    Copy the full SHA
    8a37c7d View commit details
    Browse the repository at this point in the history

Commits on Nov 23, 2018

  1. Moved repeating args definition into setUp().

    tearDown() and check_args() were modified to accomodate
    the changes.
    Evildoor committed Nov 23, 2018
    1 Configuration menu
    Copy the full SHA
    324978c View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2018

  1. Added tests for config argument.

    The argument differs from the others because its type is not
    string - therefore, argparse performs some manipulations with
    the supplied file, and these manipulations have to be taken
    into consideration.
    Evildoor committed Dec 14, 2018
    8 Configuration menu
    Copy the full SHA
    fdb498a View commit details
    Browse the repository at this point in the history

Commits on Dec 17, 2018

  1. Improved config tests.

    Argparse module throws SystemExit when something is wrong with the
    supplied arguments, regardless of the problem's type. Implemented a
    function to intercept such behaviour and retrieve argparse' message.
    Analysis of the message is used to check that the encountered
    problem is the expected one.
    Evildoor committed Dec 17, 2018
    Configuration menu
    Copy the full SHA
    42877e2 View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2019

  1. Added 'incorrect' tests for mode, source and dest.

    These arguments have a fixed sets of allowed values. The tests
    are checking that the code correctly displays error information
    when an unexpected argument value is supplied.
    Evildoor committed Jan 23, 2019
    Configuration menu
    Copy the full SHA
    9dc9aa0 View commit details
    Browse the repository at this point in the history
  2. Improved the functions with 'short' argument.

    The functions now perform 'short'-related manipulations by
    defining varialbes and using them in functions instead of
    having two different calls of the same function.
    Evildoor committed Jan 23, 2019
    Configuration menu
    Copy the full SHA
    19ca409 View commit details
    Browse the repository at this point in the history
  3. Updated tests for correct config.

    Now the tests check that the parameter in question is a file,
    rather than not None.
    Evildoor committed Jan 23, 2019
    Configuration menu
    Copy the full SHA
    c3e4810 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2019

  1. Configuration menu
    Copy the full SHA
    c81a096 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2019

  1. Adapt tests to pyDKB changes.

    - Replace stage.parse_args() with stage.configure().
    - Update tests to set input_dir if source is either stream or hdfs.
    - Add an input file name into most tests - 'file' is now a default source, and
      it demands an input file. Exceptions: test_i/test_input_dir - all files in
      the input dir will be taken, so input file is not mandatory; all tests
      including --hdfs; test_default - its role is to simulate an execution with
      no command line arguments.
    - Change test_default() to catch the error about missing input file.
    - Change add_override_mode() - file source/destination is not allowed in
      map-reduce mode, meaning that such combination raises an error that must be
      caught.
    - Define stage.configure() arguments in list args in some tests to follow
      the 80 symbols line length restriction without stretching a single code line
      into 3+ lines.
    Evildoor committed Apr 26, 2019
    Configuration menu
    Copy the full SHA
    f795e24 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2019

  1. Add empty EOM tests.

    If EOM is set to empty string, EOP will be set to '\n', unless overridden.
    Evildoor committed Apr 29, 2019
    Configuration menu
    Copy the full SHA
    77ef71b View commit details
    Browse the repository at this point in the history
  2. Change value used to trigger an error.

    It is possible that value 'i' will become valid in the future, while
    'incorrect' is much 'safer' in this regard - all of the values are
    single letters.
    Evildoor committed Apr 29, 2019
    Configuration menu
    Copy the full SHA
    1d4feb2 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    b5d6842 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    38a383e View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    c99f4c6 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    e1df893 View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2019

  1. Divide test_ProcessorStage.py per how-to: tests.

    - Move common function to separate file lib.py.
    Evildoor committed Apr 30, 2019
    Configuration menu
    Copy the full SHA
    7dad261 View commit details
    Browse the repository at this point in the history