Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding list_fields command #1262

Merged
merged 20 commits into from Apr 7, 2020
Merged

Conversation

JackAtOmenApps
Copy link
Contributor

When working on projects with complex relations it can get confusing to remember how to refer to these relationships in queries. Additionally, sometimes it's annoying to dig through the models.py to find the field name of a particular field. The 'list_fields' command lists out all fields for each model within each installed app.

Particularly helpful when you need to use related fields, but don't remember how to refer to the relation in your queries.
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.01%) to 55.535% when pulling 88e50dc on OmenApps:master into 4bafa82 on django-extensions:master.

@coveralls
Copy link

coveralls commented Oct 22, 2018

Coverage Status

Coverage decreased (-3.1%) to 67.878% when pulling ba77c9d on OmenApps:master into 97c578a on django-extensions:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.01%) to 55.535% when pulling 88e50dc on OmenApps:master into 4bafa82 on django-extensions:master.

@JackAtOmenApps
Copy link
Contributor Author

Well, I'm trying to be helpful, but I'm not passing checks. I initially thought it was because I left a blank line at the end. But now it's failing even more checks after deleting it.

Anyone familiar with Travis CI have recommendations?

@trbs
Copy link
Member

trbs commented Oct 24, 2018

Your are missing a newline at the end of the file. (see W391)
A single empty newline at the end of the should fix that build error.

@JackAtOmenApps
Copy link
Contributor Author

@trbs my original commit had a newline, but failed one check. I'll try again.

@JackAtOmenApps
Copy link
Contributor Author

It says failing for flake8, but when I run flake8 locally I see no problems.

@trbs
Copy link
Member

trbs commented Oct 24, 2018

It says failing for flake8, but when I run flake8 locally I see no problems.

Depends on the version of flake8 :-)
The failures are now related to your PR and should be fixed in master soon.

@trbs
Copy link
Member

trbs commented Oct 24, 2018

Seems there is still a bit of trailing whitespace in docs/command_extensions.rst

@trbs
Copy link
Member

trbs commented Oct 25, 2018

Great !

One last question, could you please add some unittests for this command ?
This helps us a lot maintaining them.

Thanks !

@JackAtOmenApps
Copy link
Contributor Author

I'll work on adding this within the next couple days :)

@trbs
Copy link
Member

trbs commented Oct 28, 2018

Great !

Please also use self.stdout and friends, this makes testing a lot easier.
Think the existing models in the test suite are more then enough to test with so hopefully writing tests is easy :-)

@yeojin-dev
Copy link
Contributor

Could you merge this function first? Then I will apply the test case and stdout within March. If you don't have any plans for this month's release, that would be fine. @trbs

@trbs
Copy link
Member

trbs commented Mar 18, 2020

@yeojin-dev think we can wait another month :-)
Maybe you can take control by forking @OmenApps patches and submitting a new PR ?
(Unless @OmenApps wants to pick this up / work together with @yeojin-dev on this)

@JackAtOmenApps
Copy link
Contributor Author

@yeojin-dev & @trbs, I have a slightly improved version of this command I have been using in my projects. I'll update in the next day or two, and would love to work with @yeojin-dev on improvements/tests to get this ready for merging.

@JackAtOmenApps
Copy link
Contributor Author

@yeojin-dev & @trbs, I updated with the promised improvements. I'm glad someone else thinks this will be useful. My project is nearing 125 models, and it can be challenging to remember which models have what fields/methods. I use this script at least weekly, so hopefully it will benefit others.

I'll get started on tests in a bit. Any help from @yeojin-dev will be much appreciated!

@trbs
Copy link
Member

trbs commented Mar 20, 2020

Slightly updated version with some of my thoughts in them, take from it what you will :-)

Some things I changed:

  • Removed ast, I could not figure out the reason for it versus iterating over it directly which also gives us the possibility to do things like print python class name or database column type for fields.
  • Add option to show full argument signature of methods
  • Use self.stdout for easier testing later
  • Make everything more dense (personal preference)

Food for thought:

  • How about removing django default methods like save(), save_base(), full_clean(), etc ?
  • Being able to configure the cli options through a settings.py key would be nice (otherwise I would always run: ./manage.py list_model_info --field-class --signature
  • Use colorized output
  • Also allow tabular form, or something similar to ./manage.py export_emails --format ..., this would make it far easier to do something like ./manage.py list_model_info | grep testapp.UniqueTestAppModel and get all fields + methods listed.

Tests:

As for tests I won't expect anything crazy here... let us just have some simple tests based on the output that can be generated from the django-extensions test suite... checking that a couple of those models list all the defined fields + methods should be enough.

(Note: I don't think we have many methods defined on models in our tests so we might need to add a couple)

My suggestion for list_model_info:

# -*- coding: utf-8 -*-
# Author: OmenApps. http://www.omenapps.com
import inspect

from django.apps import apps as django_apps
from django.core.management.base import BaseCommand
from django.db import connection
from django_extensions.management.utils import signalcommand

TAB = "        "
HALFTAB = "    "


class Command(BaseCommand):
    """A simple management command which lists model fields and methods."""

    help = "List out the fields and methods for each model"

    def add_arguments(self, parser):
        super(Command, self).add_arguments(parser)
        parser.add_argument(
            '--field-class',
            action='store_true', default=False,
            help='show class name of field.'
        )
        parser.add_argument(
            '--db-type',
            action='store_true', default=False,
            help='show database column type of field.'
        )
        parser.add_argument(
            '--signature',
            action='store_true', default=False,
            help='show database column type of field.'
        )

    def list_model_info(self, options):
        model_list = sorted(
            django_apps.get_models(), key=lambda x: (x._meta.app_label, x._meta.object_name), reverse=False
        )
        for model in model_list:
            self.stdout.write(model._meta.app_label + "." + model._meta.object_name)
            self.stdout.write(HALFTAB + "Fields:")

            for field in model._meta.get_fields():
                if options['field_class']:
                    self.stdout.write(TAB + field.name + " - " + field.__class__.__name__)
                elif options['db_type']:
                    self.stdout.write(TAB + field.name + " - " + field.db_type(connection=connection))
                else:
                    self.stdout.write(TAB + field.name)

            self.stdout.write(HALFTAB + "Methods (non-private/internal):")

            for method_name in dir(model):
                try:
                    method = getattr(model, method_name)
                    if callable(method) and not method_name.startswith("_") and not method_name[0].isupper():
                        if options['signature']:
                            signature = inspect.signature(method)
                        else:
                            signature = "()"
                        self.stdout.write(TAB + method_name + str(signature))
                except AttributeError:
                    pass

            self.stdout.write("\n")

        self.stdout.write("Total Models in Project: %d" % len(model_list))

    @signalcommand
    def handle(self, *args, **options):
        self.list_model_info(options)

@JackAtOmenApps
Copy link
Contributor Author

Awesome @trbs, I love these ideas and the help. I'll get working on some of this over the next week.

@JackAtOmenApps
Copy link
Contributor Author

JackAtOmenApps commented Mar 26, 2020

@trbs: Alright, I made a bunch of changes:

  • Implemented your recommended code improvements
  • Added a bit of color
  • Made it possible to select a single model rather than all models
  • Added settings, which are overridden by their equivalent commandline arguments
  • Included a list of django's default methods to compare against when optionally filtering out private/default methods
  • Added more documentation
  • Still no tests 🙁

@JackAtOmenApps
Copy link
Contributor Author

JackAtOmenApps commented Mar 27, 2020

@trbs, I added tests, but the TravisCI config includes three different databases, so the --db-field option fails on some tests no matter how I write the assertions due to the different representations in different databases.

Any recommendations on an approach? It seems like tests for all variants could get complex really quickly.

Modified command name and added listing of each model's methods


Corrected order for entry of command in the docs


Removed extra spaces on line endings.
Corrected flake 8 issues and formatted with black.


Fixed line endings.


Made it possible to use both db-type and field-class together


Added tests


Fixed trailing whitespace


Corrected for representation in TravisCI
@trbs
Copy link
Member

trbs commented Apr 7, 2020

Very awesome !
Thanks !

@trbs trbs merged commit 3d3c75d into django-extensions:master Apr 7, 2020
@trbs
Copy link
Member

trbs commented Apr 8, 2020

As for tests.. I would think it would be okay to test on SQLite only...
At least we know that everybody should be able to run those tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants