Script to test index.html for validity #341

philippbayer · 2014-02-25T02:15:56Z

For issue #320

This is by no means exhaustive but I think a good start.

…ome common mistakes.

rgaiacs · 2014-02-25T18:20:21Z

setup/swc-index-test.py

+    7. instructor list should be a valid Python/Ruby list
+'''
+
+import sys as _sys


Is the first time that I see this.

If you have plans to use a variable named sys you should use sys_ following PEP8:

single_trailing_underscore_: used by convention to avoid conflicts with Python keyword

Good that you're as confused about this as I am!

setup/swc-installation-test1.py and setup/swc-installation-test2.py do this, so I thought this was some kind of internal standard that I knew nothing about, so I copied it: cargo-cult programming. Will delete this in the next commit.

rgaiacs · 2014-02-25T18:49:02Z

@DrSnuggles, Thanks for the contribution. =)

…eaks up some more long lines.

philippbayer · 2014-02-25T23:57:40Z

Cool! Check the newest commit, I put in all of your comments except that I left the check for countries like it is (for now?). The script also now checks for that template header and warns the user if it still exists.

Edit: Also, currently the script assumes that there is a email address, but the Jekyll code in index.html puts in the site's contact if the email address isn't specified. Is that behavior still wanted? Then I'd have to change the categories rule a bit.

rgaiacs · 2014-02-26T00:28:37Z

setup/swc-index-test.py

+        filename = args[1]
+
+    sys.stderr.write('Testing file "%s".\n' %filename)
+    index_fh = open(filename)


You forgot to close the file. What about use with?

rgaiacs · 2014-02-26T00:40:55Z

@DrSnuggles Thanks for all the help with this script.

Could you add the check function for all the categories, including layout (just check if the value is bootcamp)?

The script also now checks for that template header and warns the user if it still exists.

Nice.

Edit: Also, currently the script assumes that there is a email address, but the Jekyll code in index.html puts in the site's contact if the email address isn't specified. Is that behavior still wanted? Then I'd have to change the categories rule a bit.

@gvwilson @amyrbrown @arlissc What is the wanted behavior?

philippbayer · 2014-02-26T01:35:39Z

I think that I now got all categories, and it now uses the "with" operator.

If someone could test this under Windows I'd be grateful - so far only tested on Fedora 19, python 2.7.5 and 3.3.2

rgaiacs · 2014-03-05T12:54:08Z

@DrSnuggles Sorry for the delay to do another review round.

What about move the script from setup/swc-index-test.py to bin/validate_index.py? Look like that the setup folder is use for script that help students when configuring their machines.
The bootcamp metadata is in Yaml format. What about using PyYAML to handle it? The get_header() function could return the result of PyYAML's load function instead of just a string. This approach has a few advantages, e.g. is the approach used when build our site.
Will be great if we have some test functions. I can write the tests as part of the review process and send it as a PR for you if you OK with that.

philippbayer · 2014-03-06T00:10:22Z

Hi,

thanks for the comments! No worries about the tiny delay, I'm a bit busy right now anyway.

I'll implement the suggestions, pyYAML is a way better idea since then we wouldn't have to worry about white-spaces etc.

if you want to write tests, go ahead :)

philippbayer · 2014-03-06T01:29:52Z

OK, done! I'm now using pyyaml.load() instead of pyyaml.safe_load(), since it has a few more features (for example, a valid date is automatically a datetime.date object), and I don't think this index-checker is in any way security relevant. I had to adjust a few tests a little bit.

- Add unit test to all check functions - Add unit test to file handler Changes in swc_index_validator.py - Remove some calls to `strip()` They aren't necessary any more because we use PyYAML. - Changes in `check_contry()` Use only one conditional - Change in `check_intructor()` Check if the list isn't empty. TODO: - Fix the unit test that fail.

rgaiacs · 2014-03-06T14:59:37Z

@DrSnuggles I just send you a PR with the unit tests.

There is two last thing that I would like before accept this PR:

There is two test that fail and will be great if they pass.

$ nosetests bin/test_swc_index_validator.py
...
FAILED (failures=2)

Do you use the old string formatting operator. What about move it to the new string format method?

Add unit tests

… compability.

philippbayer · 2014-03-07T00:19:06Z

Cool, thank you!

I've introduced extra-checks to fix the tests (humandate's year has to be all-numbers, humandate's day has to contain some numbers, humantime has to contain some numbers), and I fixed a minor issue with the test's StringIO behaving differently under Python2, now it should work under both 2 and 3 (at least it does on my Fedora).

By the new string formatting method, do you mean to replace all instances of

"My name is %s" %(foo)

by

 "My name is {0}".format(foo)

?

…mber

rgaiacs · 2014-03-08T21:17:13Z

@DrSnuggles The answer to your question is YES.

Sorry for the delay to reply your question. The email notification send by GitHub don't have your question. (Did you edit/update your last comment? I would like to discovery why this happen to avoid it happen again.)

And thanks to make it work with Python2.

gvwilson · 2014-03-17T11:31:55Z

bin/swc_index_validator.py

+
+def check_layout(layout):
+    '''Checks whether layout equals "bootcamp".'''
+    if layout != 'bootcamp':


return layout == 'bootcamp'

…ls until rule is defined.

rgaiacs · 2014-03-27T01:14:40Z

bin/swc_index_validator.py

+
+def check_humantime(time):
+    '''A valid humantime contains at least one number'''
+    return bool(re.match(HUMANTIME_PATTERN, time.replace(" ","")))


You don't need the bool conversion.

The problem is (I think) that the AND operator I use on the returned function expects a boolean variable:

TypeError: unsupported operand type(s) for &=: 'bool' and '_sre.SRE_Match'

I could wrap the re.match into an if-statement, but that is a bit ugly:

if re.match(re.match(HUMANTIME_PATTERN, time.replace(" ","")): return True return False

So I thought that the bool-conversion is the "laziest" and nicest way.. could I make it nicer?

Script to test index.html for validity

philippbayer added 2 commits February 25, 2014 12:07

Adds simple Python script which checks the header of index.html for s…

9418dd8

…ome common mistakes.

Checks whether file is actually a proper index.html.

40e4a85

rgaiacs reviewed Feb 25, 2014
View reviewed changes

Implements comments of @r-gaia-cs. Adds check for template header. Br…

4e26338

…eaks up some more long lines.

rgaiacs reviewed Feb 26, 2014
View reviewed changes

Reorders function to follow index.html. Adds checks for all categories.

a4d7cee

Here's the correct index.html

0afa7f1

Uses pyyaml for header parsing. Moves script to bin/

113c06c

philippbayer and others added 2 commits March 7, 2014 10:02

Merge pull request #1 from r-gaia-cs/validator

a9570a5

Add unit tests

Fixes two broken tests. Fixes a problem in tests with Python2/Python3…

2874f46

… compability.

Slightly faster, more Pythonic way to check whether humanyear is a nu…

dd352c7

…mber

gvwilson reviewed Mar 17, 2014
View reviewed changes

philippbayer added 3 commits March 18, 2014 15:41

Implements most of @gvwilson's comments. humantime-test currently fai…

222cf37

…ls until rule is defined.

Adds @r-gaia-cs' regex for humantime

22e17d1

Fixes regex, adds more tests for humantime

f29b014

rgaiacs reviewed Mar 27, 2014
View reviewed changes

Implements @r-gaia-cs's suggestions about nicer handlers and messages

fc05c2a

gvwilson pushed a commit that referenced this pull request Apr 5, 2014

Merge pull request #341 from drsnuggles/master

76585ed

Script to test index.html for validity

gvwilson merged commit 76585ed into swcarpentry:master Apr 5, 2014

wking pushed a commit to wking/swc-boot-camps-v2 that referenced this pull request Sep 8, 2014

Merge pull request swcarpentry#341 from drsnuggles/master

5c1844d

Script to test index.html for validity

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Script to test index.html for validity #341

Script to test index.html for validity #341

philippbayer commented Feb 25, 2014

rgaiacs Feb 25, 2014

philippbayer Feb 25, 2014

rgaiacs commented Feb 25, 2014

philippbayer commented Feb 25, 2014

rgaiacs Feb 26, 2014

rgaiacs commented Feb 26, 2014

philippbayer commented Feb 26, 2014

rgaiacs commented Mar 5, 2014

philippbayer commented Mar 6, 2014

philippbayer commented Mar 6, 2014

rgaiacs commented Mar 6, 2014

philippbayer commented Mar 7, 2014

rgaiacs commented Mar 8, 2014

gvwilson Mar 17, 2014

rgaiacs Mar 27, 2014

philippbayer Mar 31, 2014

Script to test index.html for validity #341

Script to test index.html for validity #341

Conversation

philippbayer commented Feb 25, 2014

rgaiacs Feb 25, 2014

Choose a reason for hiding this comment

philippbayer Feb 25, 2014

Choose a reason for hiding this comment

rgaiacs commented Feb 25, 2014

philippbayer commented Feb 25, 2014

rgaiacs Feb 26, 2014

Choose a reason for hiding this comment

rgaiacs commented Feb 26, 2014

philippbayer commented Feb 26, 2014

rgaiacs commented Mar 5, 2014

philippbayer commented Mar 6, 2014

philippbayer commented Mar 6, 2014

rgaiacs commented Mar 6, 2014

philippbayer commented Mar 7, 2014

rgaiacs commented Mar 8, 2014

gvwilson Mar 17, 2014

Choose a reason for hiding this comment

rgaiacs Mar 27, 2014

Choose a reason for hiding this comment

philippbayer Mar 31, 2014

Choose a reason for hiding this comment