WIP: Create a Django backend adapter for CockroachDB

[bdarnell]

Start following the Django tutorial and try to get the resulting app running on CockroachDB. This doesn't work yet, but it's a start. Only the last commit in this branch is really interesting.

Progress is currently blocked on cockroachdb/cockroach#4357

This branch also deletes the old SQL driver now that we've switched to the postgresql protocol.

[don-mums]

I tried to make it work but cockroachdb/cockroach#5194 is not only blocker for Django ORM. I tried to make it work and schema changes in transactions are also a problem. Only turning off transactions during migrations could make Django work with cockroach.

[pirate]

@don-mums there are several other blocking issues:

• CREATE DATABASE "test__1" WITH TEMPLATE "test_" sql: support creating a database as duplicate of another ("create database ... with template ...") cockroach#10151
• "DEFERRABLE INITIALLY DEFERRED" for constraints sql: support "DEFERRABLE INITIALLY DEFERRED" for constraints cockroach#9897
• column type changes sql: Support (some) column type changes cockroach#5950
• pg_table_is_visible sql: support pg_table_is_visible cockroach#12538
• UUID type sql: support a UUID type which text-marshals to valid UUID syntax cockroach#8719

See: https://gist.github.com/pirate/f2931acd97d52242756d85d52b42e8bd

[eldos-dl]

Some of the features that need to be supported by cockroachdb inorder to work with django-orm using postgresql_psycopg2 backend.

• uuid, time, inet datatypes
• Function pg_catalog.pg_table_is_visible()
• Directly interpreting timestamp literals from strings.
• extract() function in extract('day' FROM '2010-02-20 00:00:00') format, 'dow' for 'dayofweek'.
• date_trunc() function
• Deferrable constraint checks.
• Savepoints
• Tablespaces
• Schema changes inside transactions
• Sequences, setval() function
• Cursors
• ALTER COLUMN TYPE
• Operator patterns for varchar and text indexes.

[fwip]

From a quick glance, it looks like CockroachDB now supports the UUID datatype.

[yogurtnaturalny]

is any update with that ? Is any way how we could help ?

[bdarnell]

We've made progress on some of the features that django needs, but some of the big ones are still open (including column type changes and deferrable constraints). If you want to help out the issues linked in #14 (comment) would be a good place to start.

[theonewolf]

Is anyone actively looking at this? It would make CockroachDB immensely more usable and capable of powering thousands of Django dashboards.

[maxiplux]

Thanks for this amazing work , I would like know what is the current state or status about this driver to django ?.
Thanks

[pirate]

I spoke with someone at CockroachDB about Django support recently and it sounds like due to the sheer volume of work on their plate, they're focusing on paid customer feature requests first (which is understandable for a growing startup). If any of your companies really want Django support in CockroachDB, try to advocate to your managers to pay for the startup/enterprise versions to help speed things along!

[maxiplux]

I spoke with someone at CockroachDB about Django support recently and it sounds like due to the sheer volume of work on their plate, they're focusing on paid customer feature requests first (which is understandable for a growing startup). If any of your companies really want Django support in CockroachDB, try to advocate to your managers to pay for the startup/enterprise versions to help speed things along!

Thanks for your comments .
Have a nice day.

[omidraha]

@pirate , @bdarnell
Do you have any new news?
Support for this, causing a lot of django users to come and swarm to this wonderful database.

[bdarnell]

@omidraha No news. Cockroach Labs has no plans to work on this any time soon. In general, we don't have the resources to support every open-source framework out there, and we expect support for most platforms to be driven by the community instead of built by Cockroach Labs.

[dowenrei]

Any updates for this?
I'm eager to make this work with django

[bdarnell]

@rkruze has been looking into this.

[Alir3z4]

we expect support for most platforms to be driven by the community instead of built by Cockroach Labs.

Thank you for the great open source software and all the excellent work put into this.
I hope my comment doesn't come as rude or make any attack or a way to underestimate of the effort of every single individuals into this project.

However, when I read

PostgreSQL-compatible driver or ORM

I'd expect to anything from postgresql be working with the cockroachdb out of the box.
The first thing that got me watching cockroachdb development was the fact about being compatible with PostgreSQL drivers or ORM, but statments such as

we expect support for most platforms to be driven by the community instead of built by Cockroach Labs.

will make me to think such work is being put on the framework or app developers, the app/framework devs are simply using a PostgreSQL driver to build the ORM and if cockroachdb is compatible with PostgreSQL then they should not have anything else to work on.

I believe Django does uses very specific PostgreSQL features and has a great recommendation for such database, but things like #14 (comment) seems to be pretty standard PostgreSQL stuff that people would expect from a "PostgreSQL-compatible driver or ORM".

In any case, again I hope my comment doesn't come aggressive or showing any undermining the great effort on this project.

If Django gets supported, I'll be happy to do beta testing on some of my projects.
I hope to see more resources and support from cockroachlabs into such feature/problem.

[rkruze]

@Alir3z4 I have an initial version of a Django backend that is doing some slight modifications on top of the Postgres backend. You can find that backend here: https://github.com/cockroachlabs/cockroach-django I was able to complete the initial tutorial with this backend. Let me know if you run into any issues. Thanks!

[don-mums]

@rkruze Great job. I tried it on one small project and except one refactoring it works great and one issue. cockroachdb/django-cockroachdb#1

Since you cannot nest transaction.atomic and I like to use those as if they should go out of style I think I would encounter those many more times in something bigger.

[timgraham]

This can be closed in favor of https://github.com/cockroachdb/cockroach-django.

[bdarnell]

@timgraham Yep, we can close this PR in favor of the other repo

We should also think about whether it makes sense to continue to have this cockroachdb-python package that has all-in-one cockroach support or if it should be renamed cockroach-sqlalchemy. The original idea was all-in-one but now that seems like it just makes dependencies messier (and it seems unlikely that there would be code sharing between cockroach-django and cockroach-sqlalchemy)

[pirate]

(+1 for renaming this repo to cockroach-sqlalchemy, as a user of both sqlalchemy and django in separate projects, they are rarely mixed in the python ecosystem so imo it makes sense to keep them as explicitly separate packages.)

[jawabuu]

Hey @bdarnell , I've been looking at your approach for achieving compatibility with django and I think it can be applied for another 'newsql' db. Would you be interested in checking this out?