Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration tests by compiling and calling Ghostferry from Ruby #64

Merged
merged 6 commits into from
Jan 21, 2019

Commits on Jan 15, 2019

  1. Moved Go test files from test/*.go to test/go/*.go

    Makes room to make the tests more like ruby tests, which will be
    employed heavily.
    shuhaowu committed Jan 15, 2019
    Configuration menu
    Copy the full SHA
    ea7428b View commit details
    Browse the repository at this point in the history
  2. Integration test running Ghostferry binary

    Instead of running integration tests directly in Go, we run it in Ruby
    by calling Ghostferry as a subprocess. The idea is that we will be able
    to test the interrupt/resume work more easily as well as making
    integration tests easier to write.
    
    In order to allow the ruby tests to inject race conditions for testing
    purposes, we need a way for the ruby code to pause and restart the
    execution of Go code at strategic locations. This is done with a local
    Unix socket. At locations of interests, the Go code will send some
    string via the Unix socket to the Ruby code. The ruby code will perform
    some sort of callback (such as injecting data, or locking a row) and
    then send a command back to the Go code via the same Unix socket,
    allowing it to continue executing. Note that each time we send some
    status to the ruby server, we do it in a new connection. This is to
    avoid race conditions if one shared connection was used between all the
    goroutines we employ inside Ghostferry.
    
    Stdout and stderr are captured and they can be asserted
    against/examined, more helper methods can be created later to give
    better structures for the stdout/stderr output such as being able to
    recognize a panic, or recognize particular stages of execution from
    within the logs.
    
    Lastly, the intention is to replace all integration tests with this
    framework in the future as it would likely be more robust and cause less
    issues such as goroutine leaks during the go test run. However, for the
    time being we will have both integration tests in Go and Ruby.
    shuhaowu committed Jan 15, 2019
    Configuration menu
    Copy the full SHA
    cbdb3a2 View commit details
    Browse the repository at this point in the history
  3. Added some helpers to handle the database.

    The DataWriterHelper is a straight port from data_writer.go, which is a
    multithreaded data writer that writes random data to the database.
    
    The DbHelper helps connects to the database, seed it, and verify that
    the source matches the target so that tests don't have to implement it
    over and over again.
    shuhaowu committed Jan 15, 2019
    Configuration menu
    Copy the full SHA
    5942c65 View commit details
    Browse the repository at this point in the history
  4. Added some integration tests

    Added some trivial integration tests and some interrupt/resume tests.
    
    Note the interrupt and resume works on master as of this commit without
    reconciler because the schema didn't change.
    shuhaowu committed Jan 15, 2019
    Configuration menu
    Copy the full SHA
    f1465a7 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    bd60675 View commit details
    Browse the repository at this point in the history

Commits on Jan 21, 2019

  1. Updated for comments

    shuhaowu committed Jan 21, 2019
    Configuration menu
    Copy the full SHA
    b8fb91a View commit details
    Browse the repository at this point in the history