-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
initial commit of MongoDB backend #2
Changes from all commits
6a86eb1
01a2ed1
2e0d21a
35c0871
34e6e64
f99017c
28f2a34
50782dd
e646e2b
0b6ec2e
0106ad1
09da727
28f52b1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
DATABASES = { | ||
"default": { | ||
"ENGINE": "django_mongodb", | ||
"NAME": "djangotests", | ||
}, | ||
"other": { | ||
"ENGINE": "django_mongodb", | ||
"NAME": "djangotests-other", | ||
}, | ||
} | ||
DEFAULT_AUTO_FIELD = "django_mongodb.fields.MongoAutoField" | ||
PASSWORD_HASHERS = ("django.contrib.auth.hashers.MD5PasswordHasher",) | ||
SECRET_KEY = "django_tests_secret_key" | ||
USE_TZ = False |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,39 +1,65 @@ | ||
# MongoDB backend for Django | ||
|
||
This library is in the early stages of development, and so it's possible the API may change in the future - we definitely want to continue expanding it. We welcome your feedback as we continue to explore and build this tool. | ||
This backend is in the pre-alpha stage of development. Backwards incompatible | ||
changes may be made without notice. We welcome your feedback as we continue to | ||
explore and build. | ||
|
||
## Install and usage | ||
|
||
Use the version of `django-mongodb` that corresponds to your version of | ||
Django. For example, to get the latest compatible release for Django 5.0.x: | ||
|
||
`pip install django-mongodb==0.1.*` | ||
`pip install django-mongodb==5.0.*` | ||
|
||
The minor release number of Django doesn't correspond to the minor release | ||
number of django-mongodb. Use the latest minor release of each. | ||
|
||
While django-mongodb only has pre-releases (alphas or betas), you'll see an | ||
error with a list of the available versions. In that case, include `--pre` to | ||
allow `pip` to install the latest pre-release. | ||
|
||
For example, if django-mongodb 5.0 alpha 1 is the latest available version | ||
of the 5.0 release series: | ||
|
||
``` | ||
$ pip install django-mongodb==5.0.* | ||
ERROR: Could not find a version that satisfies the requirement | ||
django-mongodb==5.0.* (from versions: ..., 5.0a1) | ||
|
||
$ pip install --pre django-mongodb==5.0.* | ||
... | ||
Successfully installed ... django-mongodb-5.0a1 ... | ||
``` | ||
|
||
Configure the Django `DATABASES` setting similar to this: | ||
|
||
```python | ||
DATABASES = { | ||
"default": { | ||
"ENGINE": "django_mongodb", | ||
"NAME": "MY_DATABASE", | ||
"SCHEMA": "MY_SCHEMA", | ||
"WAREHOUSE": "MY_WAREHOUSE", | ||
"NAME": "my_database", | ||
"USER": "my_user", | ||
"PASSWORD": "my_password", | ||
"ACCOUNT": "my_account", | ||
}, | ||
} | ||
``` | ||
|
||
## Known issues and limitations | ||
|
||
TODO | ||
- The following `QuerySet` methods aren't supported: | ||
- `aggregate()` | ||
- `distinct()` | ||
- `extra()` | ||
- `select_related()` | ||
|
||
## Troubleshooting | ||
- Queries with joins aren't supported. | ||
|
||
### Debug logging | ||
## Troubleshooting | ||
|
||
TODO | ||
|
||
## Credits | ||
|
||
This project began by borrowing code from Django non-rel's | ||
[MongoDB Engine](https://github.com/django-nonrel/mongodb-engine), | ||
abandoned since 2015 and Django 1.6 (2-clause BSD license). |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
__version__ = "5.0a0" | ||
|
||
# Check Django compatibility before other imports which may fail if the | ||
# wrong version of Django is installed. | ||
from .utils import check_django_compatability | ||
|
||
check_django_compatability() |
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,121 @@ | ||
from django.core.exceptions import ImproperlyConfigured | ||
from django.db.backends.base.base import BaseDatabaseWrapper | ||
from django.db.backends.signals import connection_created | ||
from pymongo.collection import Collection | ||
from pymongo.mongo_client import MongoClient | ||
|
||
from . import dbapi as Database | ||
from .client import DatabaseClient | ||
from .creation import DatabaseCreation | ||
from .features import DatabaseFeatures | ||
from .introspection import DatabaseIntrospection | ||
from .operations import DatabaseOperations | ||
from .schema import DatabaseSchemaEditor | ||
|
||
|
||
class Cursor: | ||
"""A "nodb" cursor that does nothing except work on a context manager.""" | ||
|
||
def __enter__(self): | ||
pass | ||
|
||
def __exit__(self, exception_type, exception_value, exception_traceback): | ||
pass | ||
|
||
|
||
class DatabaseWrapper(BaseDatabaseWrapper): | ||
data_types = { | ||
"AutoField": "int", | ||
"BigAutoField": "long", | ||
"BinaryField": "binData", | ||
"BooleanField": "bool", | ||
"CharField": "string", | ||
"DateField": "date", | ||
"DateTimeField": "date", | ||
"DecimalField": "decimal", | ||
"DurationField": "long", | ||
"FileField": "string", | ||
"FilePathField": "string", | ||
"FloatField": "double", | ||
"IntegerField": "int", | ||
"BigIntegerField": "long", | ||
"GenericIPAddressField": "string", | ||
"NullBooleanField": "bool", | ||
"OneToOneField": "int", | ||
"PositiveIntegerField": "long", | ||
"PositiveSmallIntegerField": "int", | ||
"SlugField": "string", | ||
"SmallIntegerField": "int", | ||
"TextField": "string", | ||
"TimeField": "date", | ||
"UUIDField": "string", | ||
} | ||
|
||
vendor = "mongodb" | ||
Database = Database | ||
SchemaEditorClass = DatabaseSchemaEditor | ||
client_class = DatabaseClient | ||
creation_class = DatabaseCreation | ||
features_class = DatabaseFeatures | ||
introspection_class = DatabaseIntrospection | ||
ops_class = DatabaseOperations | ||
|
||
def __init__(self, *args, **kwargs): | ||
super().__init__(*args, **kwargs) | ||
self.connected = False | ||
del self.connection | ||
|
||
def get_collection(self, name, **kwargs): | ||
return Collection(self.database, name, **kwargs) | ||
|
||
def __getattr__(self, attr): | ||
""" | ||
Connect to the database the first time `connection` or `database` are | ||
accessed. | ||
""" | ||
if attr in ["connection", "database"]: | ||
assert not self.connected | ||
self._connect() | ||
return getattr(self, attr) | ||
Comment on lines
+76
to
+79
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why would we want There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think you may have read it wrong. AttributeError is raised for any attribute besides connection and database. The purpose of this code is the connect to the database whenever |
||
raise AttributeError(attr) | ||
|
||
def _connect(self): | ||
settings_dict = self.settings_dict | ||
|
||
options = settings_dict["OPTIONS"] | ||
# TODO: review and document OPERATIONS: https://github.com/mongodb-labs/django-mongodb/issues/6 | ||
self.operation_flags = options.pop("OPERATIONS", {}) | ||
if not any(k in ["save", "delete", "update"] for k in self.operation_flags): | ||
# Flags apply to all operations. | ||
flags = self.operation_flags | ||
self.operation_flags = {"save": flags, "delete": flags, "update": flags} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. iiuc, this will completely override ignore any flags for operations outside of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This code was copied uncritically from Django MongoDB Engine. See documentation: https://django-mongodb-engine.readthedocs.io/en/latest/reference/settings.html#acknowledged-operations. If it looks deficient, we can omit it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. TL;DR:
Got it. I don't know the best strategy here yet, so we'll pencil it for now. I think propagating the same flags to each write operation is fine, it's really the fact that we can propagate other flags and would want to keep them. The docs you linked (and thank you for linking them) show me that this was really meant for just save/delete/update operations at first. |
||
|
||
self.connection = MongoClient( | ||
host=settings_dict["HOST"] or None, port=int(settings_dict["PORT"] or 27017), **options | ||
) | ||
db_name = settings_dict["NAME"] | ||
if db_name: | ||
self.database = self.connection[db_name] | ||
|
||
user = settings_dict["USER"] | ||
password = settings_dict["PASSWORD"] | ||
if user and password and not self.database.authenticate(user, password): | ||
raise ImproperlyConfigured("Invalid username or password.") | ||
Comment on lines
+102
to
+103
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. NIT: Since mongodb usually takes a URI and aforementioned URI generally has the user & password already provided, I'm wondering if having these pieces of information are even worthwhile because any secure cloud connection needs the username and password already in the URI. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Okay. This code is copied from Django MongoDB Engine. We can discuss it more later. Django doesn't normally accept a URL for its configuration, but there's a third-party package to allow it: https://pypi.org/project/dj-database-url/ |
||
|
||
self.connected = True | ||
connection_created.send(sender=self.__class__, connection=self) | ||
|
||
def _commit(self): | ||
pass | ||
|
||
def _rollback(self): | ||
pass | ||
|
||
def close(self): | ||
if self.connected: | ||
del self.connection | ||
del self.database | ||
self.connected = False | ||
|
||
def cursor(self): | ||
return Cursor() | ||
Comment on lines
+120
to
+121
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Keeping this comment here just as a note for later. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This was a quick hack to prevent the need to override some Django internals where a "nodb" cursor is created but (for MongoDB purposes) not used. It may or may not stay around long-term. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm trying to find more information on cursors in django, any good documentation reference? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's up to the database driver (psycopg, mysqlclient, etc.) to implement, but the interface is described in PEP 249. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
import signal | ||
|
||
from django.db.backends.base.client import BaseDatabaseClient | ||
|
||
|
||
class DatabaseClient(BaseDatabaseClient): | ||
executable_name = "mongo" | ||
|
||
@classmethod | ||
def settings_to_cmd_args_env(cls, settings_dict, parameters): | ||
raise NotImplementedError | ||
Comment on lines
+9
to
+11
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's raise an issue for this ticket as well. Not sure to what extent this will get used, but I do see this as the proxy for making any command line calls. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
|
||
def runshell(self, parameters): | ||
sigint_handler = signal.getsignal(signal.SIGINT) | ||
try: | ||
# Allow SIGINT to pass to mongo to abort queries. | ||
signal.signal(signal.SIGINT, signal.SIG_IGN) | ||
super().runshell(parameters) | ||
finally: | ||
# Restore the original SIGINT handler. | ||
signal.signal(signal.SIGINT, sigint_handler) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not getting the point. I think it should be 'if' instead of 'assert'. Why should it fail if I call 'self.connection' twice?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
__getattr__()
is only called the first time when the attribute doesn't exist.