Grab old UDL files into new location #11

pryrt · 2020-02-04T20:32:48Z

closes #7

populate UDLs from old UDL list (fixing names/ids to match our requirements)
update JSON and Markdown lists to match

* DONE: removed the autocomplete's that were in that list (that I've found so far) * IN PROGRESS: into the M's in commun-udl-list.json * TODO: other-udl-list

… more effort on my part. Need to decide whether to give up on those, or download and do the local renames like I did on the `commun-udl-list.json`

…import the separate json files into the official list

pryrt · 2020-02-04T21:22:33Z

@chcg ,

With the schema checks, are we just disallowing unicode characters completely? Or is there some other way that I don't know of: I tried UTF-8, UTF8-with-BOM, and \uXXXX encoding them, and all failed the check.

(the original authors of the ancient UDLs sometimes had unicode characters in their names, so I was trying to maintain their own name spelling in the author field.)

UDLs/readme.txt

donho · 2020-02-04T23:30:27Z

udl-list.json

+			"version": "Sat, 11 Aug 2012 12:04:24 GMT",
+			"repository": "",
+			"description": "Snort",
+			"author": "http://www.tropismgroup.org/2012/08/02/snort-user-defined-language-udl-in-notepad/ <http://www.tropismgroup.org/2012/08/02/snort-user-defined-language-udl-in-notepad/>"


The content of URL is not readable.
Let's put Anonymous, and remove the URL for the moment.

URL removed. Since I had called it Snort_by-tropismgroup, I decided to use same identifier as the author.
pryrt@6f8f59e

I don't even think tropismgroup it's accurate.
http://www.tropismgroup.org/ is a lottery website and people need to login for some operation.
So let's put Anonymous until people come to claim it.

pryrt@e70fd6c switched to anonymous

donho · 2020-02-04T23:31:23Z

udl-list.md

+| [Skyrim Papyrus language](./UDLs/Skyrim-Papyrus_byJohnKostrzewski.xml) | Skyrim Papyrus language | John Kostrzewski |
+| [SmallWorld Magik](./UDLs/SmallWorldMagik_byChristianTaton.xml) | SmallWorld Magik | Christian Taton |
+| [Smarty](./UDLs/Smarty_byHelge-deVries.xml) | Smarty | Helge de Vries |
+| [Snort](./UDLs/Snort_by-tropismgroup.xml) | Snort | http://www.tropismgroup.org/2012/08/02/snort-user-defined-language-udl-in-notepad/ |


The content of URL is not readable.
Let's put Anonymous, and remove the URL for the moment.

URL removed. Since I had called it Snort_by-tropismgroup, I decided to use same identifier as the author.
pryrt@6f8f59e

pryrt@e70fd6c switched to anonymous

donho · 2020-02-04T23:50:47Z

udl-list.json

+			"version": "Sun, 03 Feb 2013 10:32:25 GMT",
+			"repository": "",
+			"description": "AVISynth",
+			"author": "\u041c\u0443\u0441\u0430-\u0410\u0445\u0443\u043d\043e\0432 \u0420\u0443\u0441\u0442\u0430\u043c <mailto:rustam-musa@rambler.ru>"


The JSON validation failed here.
@pryrt Are you sure it's a valid Unicode string?
Without the knowledge of this encoding, I saw the format \uXXXX but these 2 \043e & \0432 don't start with u
@chcg If it's a valid Unicode string, is it possible to modify the build script to validate also Unicode string?

@donho, thanks for finding those two. When they were fixed, it moved past the unicode parsing for the validation. I'll work on fixing the other validation errors next.

as of pryrt@7f2f5ba, validates correctly

donho · 2020-02-05T00:09:37Z

Awesome! Thank you @pryrt for your tremendous effort to make this list available!

One thing I'm worrying about, it's the agreement of author to make their UDLs available in this repository. So we should add a notice for people who prefer to erase the files (even the refs from the list) from this repository. But this notice note could be added after the merge of PR.

pryrt · 2020-02-05T00:12:16Z

But this notice could be added after the merge of PR.

Should it go in README, CONTRIBUTING, or both?

Normally I would have said CONTRIBUTING, but if we want it to be more visible, it could go in README or both.

pryrt · 2020-02-05T00:14:45Z

@chcg,

I can easily go in and fix the missing homepage attribute (and probably will). But with the phrasing on the CONTRIBUTING page (emphasis added):

The homepage attribute could be a link to your GitHub repository for the UDL language or your GitHub user page.

Did we want to make that attribute required, or optional?

donho · 2020-02-05T00:44:47Z

Did we want to make that attribute required, or optional?

IMO it should be optional.

chcg · 2020-02-05T22:09:19Z

@pryrt Do you think the schema is ok or is it necessary to change something? In general UTF-8 should be possible for json.

pryrt · 2020-02-05T23:45:42Z

@pryrt Do you think the schema is ok or is it necessary to change something? In general UTF-8 should be possible for json.

Well, it seemed to be getting hung up on this entry:

{
	"name": "npp-UDL-list",
	"version": "1.0",
	"UDLs": [
		{
			"id-name": "AVISynth",
			"display-name": "AVISynth",
			"version": "Sun, 03 Feb 2013 10:32:25 GMT",
			"repository": "",
			"homepage": "",
			"description": "AVISynth",
			"author": "Муса-Ахунов Рустам <mailto:rustam-musa@rambler.ru>"
		}
	]
}

... where it wouldn't work, whether that JSON was saved as UTF8 or UTF8-with-BOM in Notepad++, it gave that same error. When I used the \uXXXX notation, all of those characters worked and it moved on to validating properties instead of complaining about unicode.

If you think it should handle UTF-8, then there is something that needs to be fixed.

chcg · 2020-02-10T18:25:48Z

@pryrt See #15 which corrects the utf8 encoding issue

pryrt added 9 commits February 2, 2020 16:53

* DONE: Unzipped any zips

fa22956

* DONE: removed the autocomplete's that were in that list (that I've found so far) * IN PROGRESS: into the M's in commun-udl-list.json * TODO: other-udl-list

finished commun-udl-list.json; TODO = other-udl-list.json

f784c7e

most of the other-udl-list.json won't easily conform, and will take…

dc84b34

… more effort on my part. Need to decide whether to give up on those, or download and do the local renames like I did on the `commun-udl-list.json`

prepare other-udl-list for auto-download

88f75ab

downloaded the renamed others; TODO = update the udl-list.md; TODO = …

3289a3c

…import the separate json files into the official list

debugging the gen-md script; fixing filenames as I go (in progress)

a04bc96

okay, added to udl-list.md

290f4b4

and merge json's into udl-list.json

8790e80

clean out the scripts and temporary files

6a46116

pryrt requested a review from a team February 4, 2020 20:33

pryrt added 2 commits February 4, 2020 12:36

switch json to utf8-bom, see if that helps the validation

50f2b98

manually replace all unicode with json \uXXXX escapes...

27f337c

pryrt requested a review from chcg February 4, 2020 21:22

donho reviewed Feb 4, 2020

View reviewed changes

UDLs/readme.txt Outdated Show resolved Hide resolved

donho reviewed Feb 4, 2020

View reviewed changes

pryrt added 3 commits February 4, 2020 16:00

delete readme.txt from one of the unzips

e85e4bb

fix the tropismgroup author from url to just the shorthand

6f8f59e

fix the \04** to \u04**, see if it will validate now...

9c00e50

pryrt added 3 commits February 4, 2020 16:33

add missing homepage attributes

ab0e858

change Snort to Snort_byAnonymous_in2012

e70fd6c

populate missing repository attributes

202b65d

fix non-unique display-names

7f2f5ba

pryrt mentioned this pull request Feb 5, 2020

validation: make homepage attribute optional #12

Closed

pryrt mentioned this pull request Feb 5, 2020

allow authors to remove their UDL or link from repo #13

Closed

donho assigned pryrt Feb 5, 2020

donho added the accepted label Feb 5, 2020

donho approved these changes Feb 5, 2020

View reviewed changes

pryrt merged commit e3e486a into notepad-plus-plus:master Feb 5, 2020

pryrt deleted the GrabOldUdl2 branch February 6, 2020 14:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Grab old UDL files into new location #11

Grab old UDL files into new location #11

pryrt commented Feb 4, 2020

pryrt commented Feb 4, 2020

donho Feb 4, 2020

pryrt Feb 5, 2020

donho Feb 5, 2020

pryrt Feb 5, 2020

donho Feb 4, 2020

pryrt Feb 5, 2020

donho Feb 5, 2020

pryrt Feb 5, 2020

donho Feb 4, 2020 •

edited

Loading

pryrt Feb 5, 2020

pryrt Feb 5, 2020

donho commented Feb 5, 2020 •

edited

Loading

pryrt commented Feb 5, 2020

pryrt commented Feb 5, 2020

donho commented Feb 5, 2020

chcg commented Feb 5, 2020

pryrt commented Feb 5, 2020

chcg commented Feb 10, 2020

Grab old UDL files into new location #11

Grab old UDL files into new location #11

Conversation

pryrt commented Feb 4, 2020

pryrt commented Feb 4, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

donho Feb 4, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

donho commented Feb 5, 2020 • edited Loading

pryrt commented Feb 5, 2020

pryrt commented Feb 5, 2020

donho commented Feb 5, 2020

chcg commented Feb 5, 2020

pryrt commented Feb 5, 2020

chcg commented Feb 10, 2020

donho Feb 4, 2020 •

edited

Loading

donho commented Feb 5, 2020 •

edited

Loading