[qa] Added check migration filename script #16

atb00ker · 2018-09-30T21:26:35Z

Implement function to check migration file name is not in default pattern
Write Tests

Closes #15

coveralls · 2018-09-30T21:30:18Z

Coverage increased (+1.06%) to 95.455% when pulling 1e73c4f on atb00ker:issues/15 into 3cb1af8 on openwisp:master.

nemesifier

Great progress 👍
see my comments below

nemesifier · 2018-10-01T07:38:55Z

openwisp_utils/ci.py

+import re
+
+
+def CheckMigrationName():


keep in mind camel case notation should be used only for classes

nemesifier · 2018-10-01T07:40:07Z

openwisp_utils/ci.py

+    name; if default name pattern is found, exit with 1
+    """
+    MIGRATION_PATH = os.environ["MIGRATION_PATH"]
+    PROJECT_NAME = os.environ["TRAVIS_REPO_SLUG"]


since this is a function you don't need uppercase variable names (which is a convention to mean constants and global constants), you should use all lowercase variable names

Oops, i started this task with a shell script and copied all the uppercase variables from there!
I am on fixing this! Thanks! 😄

nemesifier · 2018-10-01T07:43:22Z

openwisp_utils/ci.py

+
+    # Get INGORE_SET
+    if "openwisp-users" in PROJECT_NAME:
+        IGNORE_SET = set(["0002_auto_20180508_2017.py"])


I don't remember how many migrations contain auto in the openwisp2 modules, this would become hard to maintain.
I think we can do something simpler which will still help us out: check only the last <n> files (in alphabetical order) and raise an issue only if one of that file contains. The <n> can be passed to the script as an argument and would have the default of 1. What do you think?

Oh! that sounds better than manually making the ignore list for all repositories!

However, i think instead of checking the last <n> migrations, we can send <n> as number of migrations to ignore.
Why? -- because if a PR has more than <n> migration files, the prior system would fail, however, since we already know the number of existing migrations, the latter <n> is easy to send and is independent of number of migrations in the pull request. What do you think? 😊

ok, initial migrations to ignore 👍

nemesifier · 2018-10-01T07:43:51Z

tests/test_project/tests/test_ci.py

+        environment = EnvironmentVarGuard()
+        environment.set('MIGRATION_PATH', './tests/test_project/migrations/')
+        open('./tests/test_project/migrations/0002_auto_20181001_0421.py', 'w').close()
+        CASES = ['test', 'openwisp-users']


same here regarding uppercase variable names

atb00ker · 2018-10-01T11:02:46Z

I am trying it out on openwisp-users

Update: While testing on openwisp-users, i realised my initial plan was not good.
I was planning to use check_migration_name() in .travis.yml directly, this is not good because the developers can't check for mistakes on their machine, i think including it with runtests.py will solve that problem. I am changing code for the same! 😄

Update: Done, i am testing it on openwisp-users again!

Update: It's working. this PR is ready for review. 😄

rohithasrk · 2018-10-02T05:43:00Z

openwisp_utils/ci.py

+        which checking should begin, say, if checking needs to
+        start after `0003_auto_20150410_3242.py` value will be `3`
+    """
+    migrations_to_ignore = migrations_to_ignore


Why this line @atb00ker

Oops, it was unintentional. Thanks! 😊

nemesifier

@atb00ker the code is good!

I don't understand this problem of not being able to run checks on our machine.
Can't we simply run the check as we currently do with isort and flake8?
If we have to modify every runscript.py we have it will be painful :-(

Moreover, I like the separate script allows to do the checks in separate phases of development, usually these improvements come as last thing before committing changes, initially it's totally ok to make dirty changes to make code work :-)
The unix philosophy says "make it work first, refine and optimize once it works" :-)

Did you know that setup.py gives two ways of registering scripts or functions that can become available from the command line? See https://python-packaging.readthedocs.io/en/latest/command-line-scripts.html-

Check how flake8 does it:

In netjsonconfig in the past I used a simpler but somewhat dirtier approach (because it needs a separate executable script file):

https://github.com/openwisp/netjsonconfig/blob/master/setup.py#L83

If you could do something like this the result would be very clean and in the near future we could extract this feature and make it available as a standalone python package and promote it to a wider audience (we could write all sort of django related QA checks for example).

atb00ker · 2018-10-03T17:36:38Z

@nemesisdesign

I don't understand this problem of not being able to run checks on our machine.

After reading the comment about removing code from a repo/package, i was trying to implement the solution provided there for this problem. However, the problem with that approach is that testing will happen only when .travis.yml runs and not on the workbench(like flake8). That's what i meant.

Keeping this point in mind, i changed that code.
The new idea was running the tests with runtests.py as implemented here which would enable us to test the errors on our workbench as well.

I like the separate script allows to do the checks in separate phases of development

The solution of running it like flake8 and isort sounds much better than adding it in the test suit. it will allow us to run these quality checks separately & making it a standalone python package sounds great.
Thanks for the resources linked. I'll get started with implementing it! 😊

nemesifier · 2018-10-04T09:01:20Z

@atb00ker great

nemesifier

Looks very good, will try tomorrow.

Closes openwisp#15

atb00ker · 2018-10-06T11:55:22Z

@nemesisdesign Great! Thanks! 😄
I tried it with openwisp-users: https://github.com/openwisp/openwisp-users/pull/35/files
It seems to work! 😊

nemesifier

👍

- do not raise a python exception when the shell script check fails because the python stack trace in that case is not useful information but a distraction - improved error message to handle singular and plural form, as well as separate the migrations by comma

atb00ker changed the title ~~[enhancement] Check migration filename not in default pattern~~ [WIP][enhancement] Check migration filename not in default pattern Sep 30, 2018

atb00ker force-pushed the issues/15 branch 7 times, most recently from 272f4b5 to d3ec08f Compare October 1, 2018 07:22

nemesifier reviewed Oct 1, 2018

View reviewed changes

atb00ker force-pushed the issues/15 branch 2 times, most recently from b5f5c44 to ab863a9 Compare October 1, 2018 10:56

atb00ker changed the title ~~[WIP][enhancement] Check migration filename not in default pattern~~ [enhancement] Check migration filename not in default pattern Oct 1, 2018

atb00ker force-pushed the issues/15 branch 5 times, most recently from 4c27c27 to c538b7e Compare October 1, 2018 11:24

atb00ker changed the title ~~[enhancement] Check migration filename not in default pattern~~ [WIP- Don't Merge][enhancement] Check migration filename not in default pattern Oct 1, 2018

atb00ker changed the title ~~[WIP- Don't Merge][enhancement] Check migration filename not in default pattern~~ [enhancement] Check migration filename not in default pattern Oct 1, 2018

atb00ker changed the title ~~[enhancement] Check migration filename not in default pattern~~ [WIP][enhancement] Check migration filename not in default pattern Oct 1, 2018

atb00ker force-pushed the issues/15 branch 2 times, most recently from 5ded890 to 44a40c4 Compare October 1, 2018 13:17

atb00ker changed the title ~~[WIP][enhancement] Check migration filename not in default pattern~~ [enhancement] Check migration filename not in default pattern Oct 1, 2018

atb00ker mentioned this pull request Oct 1, 2018

[qa] Include migration file name test openwisp/openwisp-users#35

Merged

rohithasrk reviewed Oct 2, 2018

View reviewed changes

atb00ker force-pushed the issues/15 branch from 44a40c4 to 1124610 Compare October 2, 2018 06:09

nemesifier reviewed Oct 2, 2018

View reviewed changes

atb00ker force-pushed the issues/15 branch 5 times, most recently from f96e1e6 to 9bcdd91 Compare October 5, 2018 21:09

nemesifier reviewed Oct 5, 2018

View reviewed changes

atb00ker force-pushed the issues/15 branch 2 times, most recently from b1e2475 to 0666d9e Compare October 6, 2018 08:22

[enhancement] Check migration filename not in default pattern

1e73c4f

Closes openwisp#15

atb00ker force-pushed the issues/15 branch from 0666d9e to 1e73c4f Compare October 6, 2018 08:28

nemesifier changed the title ~~[enhancement] Check migration filename not in default pattern~~ [qa] Check migration filename not in default pattern Oct 6, 2018

nemesifier changed the title ~~[qa] Check migration filename not in default pattern~~ [qa] Added check migration filename script Oct 6, 2018

nemesifier approved these changes Oct 6, 2018

View reviewed changes

nemesifier merged commit fa6f34a into openwisp:master Oct 6, 2018

atb00ker deleted the issues/15 branch October 6, 2018 14:55

nemesifier mentioned this pull request Nov 18, 2018

[qa] Add commit message style check #20

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[qa] Added check migration filename script #16

[qa] Added check migration filename script #16

atb00ker commented Sep 30, 2018 •

edited

coveralls commented Sep 30, 2018 •

edited

nemesifier left a comment

nemesifier Oct 1, 2018

nemesifier Oct 1, 2018

atb00ker Oct 1, 2018 •

edited

nemesifier Oct 1, 2018

atb00ker Oct 1, 2018

nemesifier Oct 1, 2018

nemesifier Oct 1, 2018

atb00ker commented Oct 1, 2018 •

edited

rohithasrk Oct 2, 2018

atb00ker Oct 2, 2018

nemesifier left a comment

atb00ker commented Oct 3, 2018

nemesifier commented Oct 4, 2018

nemesifier left a comment

atb00ker commented Oct 6, 2018

nemesifier left a comment

		import re


		def CheckMigrationName():

[qa] Added check migration filename script #16

[qa] Added check migration filename script #16

Conversation

atb00ker commented Sep 30, 2018 • edited

coveralls commented Sep 30, 2018 • edited

nemesifier left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atb00ker Oct 1, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atb00ker commented Oct 1, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nemesifier left a comment

Choose a reason for hiding this comment

atb00ker commented Oct 3, 2018

nemesifier commented Oct 4, 2018

nemesifier left a comment

Choose a reason for hiding this comment

atb00ker commented Oct 6, 2018

nemesifier left a comment

Choose a reason for hiding this comment

atb00ker commented Sep 30, 2018 •

edited

coveralls commented Sep 30, 2018 •

edited

atb00ker Oct 1, 2018 •

edited

atb00ker commented Oct 1, 2018 •

edited