Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update moto to 1.x milestone version #2471

Merged
merged 4 commits into from
Aug 16, 2018

Conversation

dlstadther
Copy link
Collaborator

@dlstadther dlstadther commented Jul 23, 2018

Description

Update moto version from 0.x milestone version to 1.x.

Motivation and Context

Updating moto is a step in the direction of verifying/updating Luigi for Python 3.7.0.

There might be breaking changes (as noted by moto 1.0.0).

Related to #2466

Have you tested this? If so, how?

Letting Travis do this.

@dlstadther
Copy link
Collaborator Author

To avoid scrolling back through the errors: py27-nonhdfs:

ERROR: test_read_iterator_long (contrib.s3_test.TestS3Target)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/spotify/luigi/test/contrib/s3_test.py", line 106, in test_read_iterator_long
    lines = [line for line in read_file]
  File "/home/travis/build/spotify/luigi/luigi/contrib/s3.py", line 607, in __iter__
    chunk = next(key_iter)
  File "/home/travis/build/spotify/luigi/luigi/contrib/s3.py", line 85, in read
    return self.streaming_body.read(size)
  File "/home/travis/build/spotify/luigi/.tox/py27-nonhdfs/lib/python2.7/site-packages/botocore/response.py", line 76, in read
    chunk = self._raw_stream.read(amt)
  File "/home/travis/build/spotify/luigi/.tox/py27-nonhdfs/lib/python2.7/site-packages/urllib3/response.py", line 413, in read
    data = self._fp.read(amt)
  File "/opt/python/2.7.14/lib/python2.7/StringIO.py", line 127, in read
    _complain_ifclosed(self.closed)
  File "/opt/python/2.7.14/lib/python2.7/StringIO.py", line 40, in _complain_ifclosed
    raise ValueError, "I/O operation on closed file"
ValueError: I/O operation on closed file

* upstream-master:
  S3 client refactor (spotify#2482)
  Rename to rpc_log_retries, and make it apply to all the logging involved
  Factor log_exceptions into a configuration parameter
  Fix attribute forwarding for tasks with dynamic dependencies (spotify#2478)
  Add a visiblity level for luigi.Parameters (spotify#2278)
  Add support for multiple requires and inherits arguments (spotify#2475)
  Add metadata columns to the RDBMS contrib (spotify#2440)
  Fix race condition in luigi.lock.acquire_for (spotify#2357) (spotify#2477)
  tests: Use RunOnceTask where possible (spotify#2476)
  Optional TOML configs support (spotify#2457)
  Added default port behaviour for Redshift (spotify#2474)
  Add codeowners file with default and specific example (spotify#2465)
  Add Data Revenue to the `blogged` list (spotify#2472)
@dlstadther
Copy link
Collaborator Author

dlstadther commented Aug 14, 2018

@Tarrasch Is this adequate validation that the next milestone version of moto works for us?

Only 1 test failed, originally. The issue appeared to be due to the lack of a final new line. The addition of the aforementioned NL corrected the test.

@dlstadther dlstadther changed the title [WIP] Update moto to 1.x milestone version Update moto to 1.x milestone version Aug 16, 2018
@Tarrasch Tarrasch merged commit 9b610b4 into spotify:master Aug 16, 2018
@Tarrasch
Copy link
Contributor

Nice fix! Thanks for doing this!

@dlstadther dlstadther deleted the feature/moto-version-update branch August 16, 2018 10:41
dlstadther added a commit to dlstadther/luigi that referenced this pull request Aug 16, 2018
* upstream-master:
  Remove long-deprecated scheduler config variable alternatives (spotify#2491)
  Bump tornado milestone version (spotify#2490)
  Update moto to 1.x milestone version (spotify#2471)
  Use passed password when create a redis connection (spotify#2489)
  S3 client refactor (spotify#2482)
  Rename to rpc_log_retries, and make it apply to all the logging involved
  Factor log_exceptions into a configuration parameter
  Fix attribute forwarding for tasks with dynamic dependencies (spotify#2478)
  Add a visiblity level for luigi.Parameters (spotify#2278)
  Add support for multiple requires and inherits arguments (spotify#2475)
  Add metadata columns to the RDBMS contrib (spotify#2440)
  Fix race condition in luigi.lock.acquire_for (spotify#2357) (spotify#2477)
  tests: Use RunOnceTask where possible (spotify#2476)
  Optional TOML configs support (spotify#2457)
  Added default port behaviour for Redshift (spotify#2474)
  Add codeowners file with default and specific example (spotify#2465)
  Add Data Revenue to the `blogged` list (spotify#2472)
  Fix Scheduler.add_task to overwrite accepts_messages attribute. (spotify#2469)
  Use task_id comparison in Task.__eq__. (spotify#2462)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants