Skip to content

Latest commit

 

History

History
946 lines (658 loc) · 64.8 KB

Django development in 2019.md

File metadata and controls

946 lines (658 loc) · 64.8 KB
Raw

Project RAM, Django and websites

Mentor

Design (Design Language)

django membership

Django membership luckydraw / lottery

What is a Web Framework

Model-Template-View

Python Web Framework

Django, Flask, Masonite, Vue.js, FastAPI, Sanic

Setup Development Environment

如何打造更好的科技新創 (Startup) 工作環境和組織文化？

The server

The Python env

Q: 2020-06-07: What if python3.8, which I installed with sudo apt install python3.8, doesn't not come with pip3 ?

Q: 2019-07-26: What if running Django on Docker? (Just like running Wordpress on Docker)

-  Google: PHP docker -> Google: wamp docker -> [Devilbox](http://devilbox.org/) (is a modern and highly customisable **dockerized PHP stack** supporting full **LAMP** and **MEAN** and running on all major platforms.) -> Google: django stack docker

Q: 2019-09-12: What if we have a using database? ( integrate Django into legacy databases)

Guide on Building with Django

Tutorial / Example in prototyping with Django (single page, & SPA)

Tutorial on Django (fullstack)

Tango with Django, Django Girls, MDN, Microsoft's Visual Studio Code, webpack tw -> django tw, django cn

Layout Django the right way

Best Practice

Templates

Static Files

Database Migration

2019-10-14: After moving model class around, I get an error occurs during python manage.py migrate:

Try running migrations one by one for every model. This way you can debug the app you are facing problem with :

python manage.py migrate location # that I delete class Location, and move it to mysite.
python manage.py migrate mysite   # that I move 2 classes in: Location, Status.
python manage.py migrate profiles # that it has models link to Location and Status.

2019-10-14: Should I commit migration files in repo?

Even though I can re-gerenate migration in production via python manage.py makemigrations, it is better to commit them to version control in order to keep track of database schema changes (esp. having data in db).

in local ENV: it is OK to run migrate:

python manage.py makemigrations 
python manage.py migrate

Now commit these newly created files, something like below.

git add app/migrations/...
git commit -m 'add migration files' app/migrations/...

In PRODUCTION: it is better not run makemigrations: I should not run makemigrations in production at all. I can run migrate in production and you will see the migrations are applied from the migration file that you committed from local. This way you can avoid all conflicts.

python manage.py migrate

Models

2019-10-14: https://docs.djangoproject.com/en/2.2/ref/models/fields/ to see what can we do when define models.

2019-10-12: django 2 use model from another app.

App profile doesn't see app location? (In app profile, there is a model class Profile having a ForeignKey location, which is defined in app location.)

    "INSTALLED_APPS." % (module, name)
RuntimeError: Model class location.models.Location doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.

It seems talking about namespace / app_name.

So I define a app_name in "location\urls.py":

app_name = "location"

But defines app_name doesn't help.

People claim that delete __init__.py helps, and the article shows that the bug is caused by __ init__.py.

The reason why you get these exceptions is that you structured your admin and/or models as packages and imported the individual admin and/or model classes in their respective __init__.py files.

So, what's the correct way to use __ init__.py?

And we go back to the question: django 2 use model from another app.

And about the error message: "INSTALLED_APPS." % (module, name)

OK, I find the answer finally: I forget to register the app location in setting.py + correct understanding of Python package & module. What I need to do is 2 things:

# revenue_engine/setting.py
INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    ...
    'mysite',
    'profiles',
    'location',
]
# revenue_engine/profiles/models.py
from django.db import models
...
from location.models import Location
# Create your models here.

location.models == package location has a module models.py == folder location has a file models.py.

2019-10-23: During models in app jobs up (5 models in jobs has FK to each other): Running makemigrations and migrate has no problems. when I run python manage.py createsuperuser , there is django.db.utils.IntegrityError: FOREIGN KEY constraint failed first and django.db.utils.InternalError: (1060, "Duplicate column name 'created_by_id'") error later. (I have just created models in app jobs. )

Google: django.db.utils.IntegrityError: FOREIGN KEY constraint failed sqlite createsuperuser

Google: django.db.utils.InternalError: (1060, "Duplicate column name

Since makemigrations is OK/no problems, than I can run migrate with --fake option:

python manage.py makemigrations
python manage.py migrate --fake

And then I can run createsuperuser with no problem.

Django Custom User Model (2019-10-08)

Custom user model should be admin-compliant. Generally speaking, there are 4 different ways to extend the existing User model #:

  1. Using a Proxy Model
  2. Using One-To-One Link with a User Model (UserProfile)
  3. Creating a Custom User Model Extending AbstractBaseUser
  4. Creating a Custom User Model Extending AbstractUser

there is no best solution. It will really depend on what you need to achieve. Keep it simple and choose wisely.

  • Proxy Model: You are happy with everything Django User provide and don’t need to store extra information.
  • User Profile: You are happy with the way Django handles the auth and need to add some non-auth related attributes to the User.
  • Custom User Model from AbstractBaseUser: The way Django handles auth doesn’t fit your project.
  • Custom User Model from AbstractUser: The way Django handles auth is a perfect fit for your project but still you want to add extra attributes without having to create a separate Model.

Using a reference custom User model from How to Extend Django User Model | Simple is Better Than Complex, when I login to admin, there is RelationObjectDoesNotExist exception occurs. (I take the #2 User Model)

If you define a OneToOneField that does not mean that every object of the targeted model contains an object from the referencing model. A OneToOneField in essence is a ForeignKey with a unique=True (and some extra logic to do proper naming).

Here it thus means that although every Profile has a related user, not every user has a Profile. It is possible that there are Users for which no Profile exists.

If you thus query some_user.profile there are two scenario's that can unfold:

  1. there is a related Profile object that is the fetched, and returned; or
  2. there is no such object, and then it raises a RelatedObjectDoesNotExist error.

There have been some proposals to return None in the latter case, but due to backwards compatibility, this will probably not be implemented in the (near) future, or implemented at all.

So you probably have a user for which there is no profile. Based on the full traceback, the error happens on the line instance.profile.save(). We can fix this by creating a profile in case there is no such profile:

from django.core.exceptions import ObjectDoesNotExist

@receiver(post_save, sender=User)
def create_user_profile(sender, instance, created, **kwargs):
    try:
        instance.profile.save()
    except ObjectDoesNotExist:
        Profile.objects.create(user=instance)

We thus check if the instance has a .profile that leads to a Profile object. In case it has not, it will raise an exception, and then we create one.

This is also more in the Easier to Ask Forgiveness than Permission (EAFP) spirit of Django.

Class Based Views (2019-10-03)

Building API

Settings (static files, media, etc) (2019-09-30)

L18N

Send email / Mailing / Mailer

Follow Django examples

3rd party

Awesome Resource for Django

  • Awesome Django - curated list of awesome Django resources
  • DjangoX - starter framework for new Django projects
  • DRFX - starter framework for new Django REST Framework projects
  • SQLjs - online SQL interpreter
  • HTML Escape Tool - automatically escape/unescape HTML code

Resources

Google, github, YouTube

Guide on Building with Flask

Tutorial in prototyping with Flask

Guide on Building API with FastAPI

Guide on Frontend

Guide on Vue.js

Guide on Data Visualization

Youtube

TODO: read them

Tooling - SQL client

  • Sqlite

Tooling - GIT

提升效率 (提升, 效率, 效能)

Guide on Deploying Django

Something interesting

中文手册 / 入门指南

Tech note

2019-10-22: To load .sql file into MySQL:

root@aaron-300V3A:/home/aaron/Syncthing/Sites4/Python/revenue-engine/utilities# mysql -u root itcs_dev < itcsdev.v1.0.1.sql

Google: mysql how to load .sql file -> command line - How to run SQL script in MySQL? - Stack Overflow

  • use the MySQL command line client: mysql -h hostname -u user database < path/to/test.sql
  • Install the MySQL GUI tools and open your SQL file, then execute it
  • Use phpmysql if the database is available via your webserver

--

Or, try to run it via script:

commit b4a89fbc3f3d230a5de18712e08aad0f74610148 Author: ringk-itcs 56421115+ringk-itcs@users.noreply.github.com Date: Mon Oct 21 15:49:28 2019 +0800

MySQL Database Population scripts

1) itcsdev_drop_tables.sql --- Drop all Website related tables (excl.  Django Tables)
2) itcsdev.v1.0.1.sql --- Create tables/objects and insert rows
3) MySQL_load_sql_file.sh --- Shell wrapper to run the above scripts.

su - adminitcs
cd <your_local_root>/revenue-engine/utlities
. ./MySQL_load_sql_file.sh itcsdev_drop_tables.sql    #--- Password is same as /etc/mysql/revengine.cnf
. ./MySQL_load_sql_file.sh itcsdev.v1.0.1.sql               #--- Password is same as /etc/mysql/revengine.cnf

--

2019-10-23:

Q: django.db.utils.IntegrityError: FOREIGN KEY constraint failed during python manage.py migrate.

Google: django.db.utils.IntegrityError: FOREIGN KEY constraint failed

--

2019-10-23:

Once Models are up, we can use the Django admin application can use your models to automatically build a site area that you can use to create, view, update, and delete records.

# admin.py
from django.contrib import admin

# Register your models here.
from catalog.models import Author, Genre, Book, BookInstance

admin.site.register(Book)
admin.site.register(Author)
admin.site.register(Genre)
admin.site.register(BookInstance)

To change how a model is displayed in the admin interface you define a ModelAdmin class (which describes the layout) and register it with the model.

# Define the admin class
class BookAdmin(admin.ModelAdmin):
    pass

# Register the admin class with the associated model
admin.site.register(Book, BookAdmin)

And instead, we will use the @register decorator to register the models (this does exactly the same thing as the admin.site.register() syntax):

# Register the Admin classes for Book using the decorator
@admin.register(Book)
class BookAdmin(admin.ModelAdmin):
    pass

# Register the Admin classes for BookInstance using the decorator
@admin.register(BookInstance) 
class BookInstanceAdmin(admin.ModelAdmin):
    pass

Further, we can configure list views by using list_display to add additional fields to the view.

class BookAdmin(admin.ModelAdmin):
    list_display = ('title', 'author', 'display_genre')

Google:django admin usage

  • [Django Tutorial Part 4: Django admin site - Learn web development | MDN](https //developer.mozilla.org/en-us/docs/web/css/padding) for further configure.

New to PHP: Starting RAM locally (XAMPP)

2019-07-24: As I am new to PHP (from Python), there is some questions about developing in PHP:

Google: php development on mac

Google: python virtual environment -> Google: php virtual environment

Q: 2019-07-26: What if I running PHP on docker? (Just like running Wordpress on Docker) -> Google: wamp docker -> Devilbox (is a modern and highly customizable dockerized PHP stack supporting full LAMP and MEAN and running on all major platforms.)

New to MacOS

2019-07-25: As I am new to Mac, there is some questions about using Mac & install software on Mac:

Google: what software to install on a new Mac

Google: homebrew

Google: where should syncthing located on Mac

Google: zhihu mac

Questions:

./database/Tables and Relational Schema.png

  • About DB design: Could you explain the entities of "Status":
    • ScheduleStatus
    • SettingsStatus
    • AccountStatus

./mvc

  • Are we using popular MVC framework (Code Igniter / CakePHP / Laravel, etc), or are we writing ourselves tidy MVC framework?

Target

  1. Current: Help to develop Project-Beaver.
  2. Current: Concern about performance issue (esp. on update schedule).
  3. Future: Possibly shift the backend of Project-Beaver to Django framework.
  4. Future: Implement Project-Beaver with Vue.js.

Plan / Approach

Stage 1

Aaron setup a local WAMP server to execute Project-Beaver on.

Aaron ignore the frontend side (esp. JavaScript) and focus on backend side:

Understand how Project-Beaver internal works.

Study on Django web framework and make a simple site as a demo for technical review (for Ricky.) ->

Study on JavaScript (I have no frontend background) -> Reading a book titled "Headfirst JavaScript Programming".

Stage 2

Aaron try to understand how the frontend components in Project-Beaver works.

Infrastructure

  • Version control
  • Python package dependency
  • Database schema control (a.k.a. Database migration)
  • Theme hirecature
  • Theme:
    • Use of Bootstrap 4
  • Localization
    • Time zone
    • I18N
    • L10N
  • Sub-system: Learning_logs (~ blogging)
    • Topic
    • Entry
  • Sub-system: Users system (membership)
    • Login
    • Logout
    • Registration
    • Login required (a.k.a. Page protection / Page restriction)
    • Extending Django's user object
  • API-ify
    • Data exchange with frontend (e.g. with Vue.js)
    • Frontend-backend decoupling (e.g. use Django Restful: DRF)

Approach

Setup (2019-09)

Install python 3.7

sudo apt install python3.7

Setup virtual env for Django

python3.7 -m venv django-revenue-env
source django-revenue-env/bin/activate

Install Django in virtual env. (Django version>=2.1)

python -m pip install django>=2.1

or, update requirements.txt and then install packages from it on other machines #

python -m pip freeze > requirements.txt
python -m pip -r requirements.txt

(Remark: Use brew instead of apt on Mac. )

Run test server

python manage.py runserver

About Django 3.0 (2020-05)