Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: sql support with SQLAlchemy - follow-up #6292

Closed
jorisvandenbossche opened this issue Feb 6, 2014 · 9 comments

Comments

Projects
None yet
6 participants
@jorisvandenbossche
Copy link
Member

commented Feb 6, 2014

#5950 is now merged! (Thanks to @mangecoeur)

But there were still some issue to flesh out/fix:

  • release notes / 0.14.0 whatsnew (Here is the section in whatsnew for when you are ready): http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0140-sql -> #7120
  • figure out user API (#6300) and update api.rst/docs accordingly
    • functional api (read_table/read_sql) (fixed in #6867)
    • other public functions? (execute, has_table)` (see #6300 and #6987)
    • OO API (PandasSQLAlchemy?)
  • get travis working (not skipping the tests)
    MySQL was taken out of the 2.7 build, for some reason was failing the build process: 043d165, and failure here (see install.sh): https://travis-ci.org/jreback/pandas/jobs/18477137 (note Joris: I think this is OK now with pymysql)
  • get MySQL working (failing tests, see #6316 (comment)) -> fixed in #6613
  • handle OperationsError (e.g. have PyMySQL installed but server is not running) -> #6651
  • lower bar on sql alchemy version -> moved to #6340, #7022
  • add postgresql tests (#6316) (and other flavors?)
  • move DataFrame.to_sql to generic and update it with new functionality -> fixed in #6598
  • timezone issues (see also #5950 (comment)) -> moved to #7139
  • check if relevant parts of old legacy tests pass with master -> #6987
  • check coverage of the test suite
  • add vbench (esp. vs old code)
  • columns argument in read_table not working
  • use itertuples rather than iterrows #6509
  • error when having integer column names (fixed in #6902)
  • read and write a MultiIndex (so index_col should take a list as well as a scalar) (fixed in #6735)
  • Allow mapping as parameters for SQL DBAPI2 #6708 (fixed in #6709)
  • Conversion of invalid column names is broken (you get column with None's) #6796, fixed in #6902
  • Data written with to_sql legacy mode (sqlite/mysql) is not persistent (not written, only in memory) #6846, fixed in #6875

There is also a todo list in #4163, but I thought it is cleaner to start a new one (but maybe we should copy some over).
Complement the list if you think of others!

@tony

This comment has been minimized.

Copy link

commented Feb 8, 2014

@jorisvandenbossche: Is there a minimum sqlalchemy version this is targeting?

Edit: I see v0.8.1 (https://github.com/pydata/pandas/pull/5950/files#diff-f2202167afc438a107cf9f73ac4f9fb1R98).

It may be helpful to know if it's a requirement / recommendation. If it's a larger project and the sqlalchemy version is locked and Pandas can be upgraded, they may still want to know if it's possible for them to utilize this at an older version.

@jreback

This comment has been minimized.

Copy link
Contributor

commented Feb 8, 2014

@tony this should work for pretty much all versions of SQLAlchemy; it uses mainly the connection features

@stonebig

This comment has been minimized.

Copy link
Contributor

commented Mar 5, 2014

Hello,

With the new sql interface, will the 'classic' DB2API still works with params of a sql being a dictionnary ?

typical example = "execute('SELECT * FROM ATABLE WHERE AFIELD=%(x)s', {'x':x})"

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

commented Mar 12, 2014

I added 'read and write a MultiIndex' to the list.

I (finally) have a need to use SQL so I'm going to be using sqlalchemy and the new pandas sql stuff pretty extensively in the next few weeks. I'll report what I find. Doing a df.to_sql() with a MultiIndex was the first thing I tried.

@jorisvandenbossche

This comment has been minimized.

Copy link
Member Author

commented Mar 13, 2014

@TomAugspurger Super! Certainly report (and fix? :-) everything, bugs, annoyances, suggestions. There are still a lot of issue to report and fix.

@stonebig I think so. Would you be able to test it with?

@jreback

This comment has been minimized.

Copy link
Contributor

commented May 5, 2014

@jorisvandenbossche

if you can move residual issues that are not likely to be closed in 0.14 to 0.14.1 (or later)

release notes / v0.14.0.txt needed as well

not pushing, but lmk when you think you are 'ready' enough on this

@jorisvandenbossche

This comment has been minimized.

Copy link
Member Author

commented May 5, 2014

@jreback I will try to finish some things of this thursday of friday. Is that OK?

@jreback

This comment has been minimized.

Copy link
Contributor

commented May 5, 2014

sure

thxs

@jsexauer jsexauer referenced this issue Feb 8, 2016

Open

DEPR: deprecations from prior versions #6581

0 of 95 tasks complete

@jreback jreback referenced this issue Jul 24, 2016

Open

DEPR: deprecations log for removed issues #13777

120 of 120 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.