New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix compatibility with unreleased changes to stdlib tokenizer #131

Merged
merged 1 commit into from Oct 1, 2018

Conversation

Projects
None yet
4 participants
@mwhudson
Contributor

mwhudson commented Oct 1, 2018

python/cpython@c4ef489
(not yet in any released version, but it's been backported to all
versions of Python in git, even 2.7!) changed the behaviour in the
stdlib's tokenize module to emit a synthetic NEWLINE token even if the
file does not end with a newline. This was causing a spurious
"mixed-line-endings" warning to be emitted, but luckily the synthetic
token is easy to test for (the token text is "").

fix compatibility with unreleased changes to stdlib tokenizer
python/cpython@c4ef489
(not yet in any released version, but it's been backported to all
versions of Python in git, even 2.7!) changed the behaviour in the
stdlib's tokenize module to emit a synthetic NEWLINE token even if the
file does not end with a newline. This was causing a spurious
"mixed-line-endings" warning to be emitted, but luckily the synthetic
token is easy to test for (the token text is "").

@mwhudson mwhudson force-pushed the mwhudson:tokenizer-compat branch from b86c646 to 4f53bba Oct 1, 2018

@njsmith

This comment has been minimized.

Member

njsmith commented Oct 1, 2018

Thanks! I should fix those tests, but it looks like the failures are just deprecation warnings, so I'm going t omerge anyway...

@njsmith njsmith merged commit f64e780 into pydata:master Oct 1, 2018

1 check failed

continuous-integration/travis-ci/pr The Travis CI build could not complete due to an error
Details
@gpshead

This comment has been minimized.

gpshead commented on 4f53bba Oct 24, 2018

FYI - These changes triggering the need for this show up in at least Python 3.6.7.

This comment has been minimized.

taleinat replied Oct 24, 2018

This is being reverted in Python 2.7, but is already released in 3.6.7 and 3.7.1.

This comment has been minimized.

Contributor

mwhudson replied Oct 24, 2018

This is being reverted in Python 2.7

Oh that's a relief to hear...

clrpackages pushed a commit to clearlinux-pkgs/patsy that referenced this pull request Oct 29, 2018

[update to 0.5.1] Add Term, EvalFactor to import in order for code to…
… run

Andrey Portnoy (1):
      Minor typo edits in 'How formulas work'.

Katrin Leinweber (1):
      Hyperlink DOIs against preferred resolver

Max Humber (1):
      Add Term, EvalFactor to import in order for code to run

Michael Hudson-Doyle (1):
      fix compatibility with unreleased changes to stdlib tokenizer

Nathaniel J. Smith (6):
      Bump version to 0.5.0+dev
      document that 0.5.0 is released
      Tell ipython directive to ignore deprecation warnings from numpy
      Fix a bit of ReST syntax
      Release notes for v0.5.1
      Bump version to 0.5.1

thequackdaddy (1):
      MAINT: Travis fixes

v0.5.1
------

* The Python 3.6.7 and 3.7.1 point releases changed the standard
  tokenizer module in a way that broke patsy. Updated patsy to work
  with these point releases. (See `#131
  <https://github.com/pydata/patsy/pull/131>`__ for details.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment