New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore commas inside wiki links #698

Closed
kocio-pl opened this Issue Oct 20, 2017 · 17 comments

Comments

Projects
None yet
4 participants
@kocio-pl

kocio-pl commented Oct 20, 2017

Currently comma inside the wikilink is treated as any other comma and it breaks some calendar entries, for example:

Where | What | When | Country
Colorado | Boulder]] | State of the Map U.S. 2017, [[Boulder | 2017-10-19-2017-10-22

https://osmbc.openstreetmap.de/calendar/preview

@TheFive

This comment has been minimized.

Show comment
Hide comment
@TheFive

TheFive Oct 21, 2017

Owner

The problem is, if you reference

[[Boulder, Colorado|Boulder]]

I haven't had that syntax in mind while parsing. I will fix it.

As a quickfix i have changed the wiki, and now it is working.

[[Boulder%2C_Colorado|Boulder]]
Owner

TheFive commented Oct 21, 2017

The problem is, if you reference

[[Boulder, Colorado|Boulder]]

I haven't had that syntax in mind while parsing. I will fix it.

As a quickfix i have changed the wiki, and now it is working.

[[Boulder%2C_Colorado|Boulder]]
@TheFive

This comment has been minimized.

Show comment
Hide comment
@TheFive

TheFive Oct 21, 2017

Owner

P.S. @kocio-pl Thanks for remarking here, i havn't recognized that.

Owner

TheFive commented Oct 21, 2017

P.S. @kocio-pl Thanks for remarking here, i havn't recognized that.

@TheFive

This comment has been minimized.

Show comment
Hide comment
@TheFive

TheFive Oct 21, 2017

Owner

Looks like the Dark Knight has come and did not accept the change. Sorry for that, I have tried to send him an email, as i do not like to start an edit war.
Christoph

Owner

TheFive commented Oct 21, 2017

Looks like the Dark Knight has come and did not accept the change. Sorry for that, I have tried to send him an email, as i do not like to start an edit war.
Christoph

@TheFive TheFive added the bug label Oct 21, 2017

@TheFive

This comment has been minimized.

Show comment
Hide comment
@TheFive
Owner

TheFive commented Oct 21, 2017

@kocio-pl

This comment has been minimized.

Show comment
Hide comment
@kocio-pl

kocio-pl Oct 21, 2017

Thanks for quick response! This bug is visible, but not critical, and people are using this form for different places (see also Rome and San Juan on preview), so fixing the code would be preferable anyway.

kocio-pl commented Oct 21, 2017

Thanks for quick response! This bug is visible, but not critical, and people are using this form for different places (see also Rome and San Juan on preview), so fixing the code would be preferable anyway.

@TheFive

This comment has been minimized.

Show comment
Hide comment
@TheFive

TheFive Oct 21, 2017

Owner

I have decided to reduce the amount of software i have to maintain and will stop the OSMBC Calendar Service (public review and conversion for weeklyOSM). I hope the team will find someone, to do the conversion automated (manual it is a lot of - better: to much - work) in the next two month.

The wiki calendar is now supporting hCalendar microformats, so an automated parser should be much simpler to create than mine was.

Owner

TheFive commented Oct 21, 2017

I have decided to reduce the amount of software i have to maintain and will stop the OSMBC Calendar Service (public review and conversion for weeklyOSM). I hope the team will find someone, to do the conversion automated (manual it is a lot of - better: to much - work) in the next two month.

The wiki calendar is now supporting hCalendar microformats, so an automated parser should be much simpler to create than mine was.

@TheFive TheFive closed this Oct 21, 2017

@TheFive TheFive added the wontfix label Oct 21, 2017

@fredao
@verdy-p

This comment has been minimized.

Show comment
Hide comment
@verdy-p

verdy-p Oct 21, 2017

Which problem ? The fact that your tool has problems paring the basic wiki syntax correctly (the links delimtied by [braces] are standard since always and are unbreakable: you cannot break in the middle of the left part which is the target). The wiki was there long before (years) your tool doing incorrect parsing.
We've made a way to parse it correctly with standardized tools, and still you don't want it.
Just remember that what you use is just a helper in order to create your bulletin but this still requires rereading. I've never broken the wiki and even made sure it was using a coherent and clean syntax everywhere.
And your tool is not the only user of the wiki. There's now support for microformats that are perfectly usable too for publishing on other sites where this format is now extremely common on many blogs.
That same microformat allows also producing other formats (RSS, iCal, HTML...)

verdy-p commented Oct 21, 2017

Which problem ? The fact that your tool has problems paring the basic wiki syntax correctly (the links delimtied by [braces] are standard since always and are unbreakable: you cannot break in the middle of the left part which is the target). The wiki was there long before (years) your tool doing incorrect parsing.
We've made a way to parse it correctly with standardized tools, and still you don't want it.
Just remember that what you use is just a helper in order to create your bulletin but this still requires rereading. I've never broken the wiki and even made sure it was using a coherent and clean syntax everywhere.
And your tool is not the only user of the wiki. There's now support for microformats that are perfectly usable too for publishing on other sites where this format is now extremely common on many blogs.
That same microformat allows also producing other formats (RSS, iCal, HTML...)

@TheFive

This comment has been minimized.

Show comment
Hide comment
@TheFive

TheFive Oct 21, 2017

Owner

Hi @verdy-p,
Looks like your hCalendar interpreter creates similar problems:
https://glennjones.net/tools/microformats/?url=http%3A%2F%2Fwiki.openstreetmap.org%2Fwiki%2FCurrent_events&callback=&dateformat=auto

"items": [{
        "type": ["h-event"],
        "properties": {
            "category": [""],
            "start": ["2017-10-19"],
            "end": ["2017-10-23"],
            "name": ["State of the Map U.S. 2017, Boulder, Colorado, United States"]
        }
    }, {
        "type": ["h-event"],
        "properties": {
            "category": [""],
            "start": ["2017-10-20"],
            "end": ["2017-10-22"],
            "name": ["Design the Smart Mobility Hackathon, Hamburg, Germany"]
        }
    }, {

If you try to intprete the "category of an entry" by comma, it is very hard to handle 4 commata.

The problem is not, that my tool has a bug, the problem is your unflexibility in handling that temporarily. But as i have suggested already: Please support the weeklyOSM team with a calendar (markdown or html).
As i have stated, my time is limited.

Owner

TheFive commented Oct 21, 2017

Hi @verdy-p,
Looks like your hCalendar interpreter creates similar problems:
https://glennjones.net/tools/microformats/?url=http%3A%2F%2Fwiki.openstreetmap.org%2Fwiki%2FCurrent_events&callback=&dateformat=auto

"items": [{
        "type": ["h-event"],
        "properties": {
            "category": [""],
            "start": ["2017-10-19"],
            "end": ["2017-10-23"],
            "name": ["State of the Map U.S. 2017, Boulder, Colorado, United States"]
        }
    }, {
        "type": ["h-event"],
        "properties": {
            "category": [""],
            "start": ["2017-10-20"],
            "end": ["2017-10-22"],
            "name": ["Design the Smart Mobility Hackathon, Hamburg, Germany"]
        }
    }, {

If you try to intprete the "category of an entry" by comma, it is very hard to handle 4 commata.

The problem is not, that my tool has a bug, the problem is your unflexibility in handling that temporarily. But as i have suggested already: Please support the weeklyOSM team with a calendar (markdown or html).
As i have stated, my time is limited.

@verdy-p

This comment has been minimized.

Show comment
Hide comment
@verdy-p

verdy-p Oct 21, 2017

I'd would have loved using a more precise perty in microformat, but because your tool was still continuing to parse the source wikicode (stil lincorerctly) it was impossible to change it more.
without your parser, this calander would have used a template to detail all fields more precisely, including with better geolocation.
For now the calendar has ALWYAS used since the begining an wikitable with three columns, and we have had lot of ficculties to change the format BECAUSE of your existing parser that appeared later and assumed too many false things and was not correctly interpreting the wiki syntax.

So if you want a better microformat with more properties than just "name", I can continue on it, but be ready to adapt your tool when we'll have more properties (hCalendar is still not used as precisely as we could, because I really took are of maintenaing some high compatiblity with your wikicode parser.

But definitely we could have BOTH:

  • more precise geotagging
  • better handling of icons
  • better integration of links
  • flexibility for the presentation (not all events will contain a "city", there has always been some nationwide events, or even intenrational events online, with events not geolocated at all)

verdy-p commented Oct 21, 2017

I'd would have loved using a more precise perty in microformat, but because your tool was still continuing to parse the source wikicode (stil lincorerctly) it was impossible to change it more.
without your parser, this calander would have used a template to detail all fields more precisely, including with better geolocation.
For now the calendar has ALWYAS used since the begining an wikitable with three columns, and we have had lot of ficculties to change the format BECAUSE of your existing parser that appeared later and assumed too many false things and was not correctly interpreting the wiki syntax.

So if you want a better microformat with more properties than just "name", I can continue on it, but be ready to adapt your tool when we'll have more properties (hCalendar is still not used as precisely as we could, because I really took are of maintenaing some high compatiblity with your wikicode parser.

But definitely we could have BOTH:

  • more precise geotagging
  • better handling of icons
  • better integration of links
  • flexibility for the presentation (not all events will contain a "city", there has always been some nationwide events, or even intenrational events online, with events not geolocated at all)
@verdy-p

This comment has been minimized.

Show comment
Hide comment
@verdy-p

verdy-p Oct 21, 2017

Note: the category in your example should never be empty, it is fed correctly by the first column containing the event type (shown as the leading icon on wiki)

verdy-p commented Oct 21, 2017

Note: the category in your example should never be empty, it is fed correctly by the first column containing the event type (shown as the leading icon on wiki)

@verdy-p

This comment has been minimized.

Show comment
Hide comment
@verdy-p

verdy-p Oct 21, 2017

Also I don't have any personal microformat interpreter, the doc contains an example of conforming parser found as an example on the hCalendar documentation and reference site.
And there's a link on the doc which already does that parsing: that parser is NOT mine.

http://pin13.net/mf2/?url=http%3A%2F%2Fwiki.openstreetmap.org%2Fwiki%2FTemplate%3ACalendar

verdy-p commented Oct 21, 2017

Also I don't have any personal microformat interpreter, the doc contains an example of conforming parser found as an example on the hCalendar documentation and reference site.
And there's a link on the doc which already does that parsing: that parser is NOT mine.

http://pin13.net/mf2/?url=http%3A%2F%2Fwiki.openstreetmap.org%2Fwiki%2FTemplate%3ACalendar

@verdy-p

This comment has been minimized.

Show comment
Hide comment
@verdy-p

verdy-p Oct 21, 2017

And anyway if you expect to have only two commas on an event description this has never been fixed. There are good reasons why one would need to locate a city within a region before the country, and there may be commas anywhere before. Some events will also not be located precisely in a single city, but a region, or seomtimes several regions or countries simultaneously.
The hCalendar format can support this, but for reading it in the small constrained width of ther table shown on the wiki, the presentation is simplified or there is a need to alter the order.
So even if you use your tool, it can only be a timesaver to help you reformat what you need in your weekly bulletin, but you still need to review what it generates and see how to refit the content.
We won't be able to automate things as long as you use your too limited wiki parser and blindly use what it detects (incorrectly).
We should have been able to use templates since long to make things easier to understand and edit without errors, but it was for now impossible BECAUSE of your existing parser.
I made the initial effort to have it now working correctly with hCalendar readers, we can then progress by using templates and more capabilities. But this requires testing. I've not broken more the existing format when introducing microformats, it should have remained compatible with any basically compliant wikiparser and without complicating too much the addition of events.
I documented everything, with examples for the most common cases. But it was not easy to preserve the compatibility with the rest of the wiki.

verdy-p commented Oct 21, 2017

And anyway if you expect to have only two commas on an event description this has never been fixed. There are good reasons why one would need to locate a city within a region before the country, and there may be commas anywhere before. Some events will also not be located precisely in a single city, but a region, or seomtimes several regions or countries simultaneously.
The hCalendar format can support this, but for reading it in the small constrained width of ther table shown on the wiki, the presentation is simplified or there is a need to alter the order.
So even if you use your tool, it can only be a timesaver to help you reformat what you need in your weekly bulletin, but you still need to review what it generates and see how to refit the content.
We won't be able to automate things as long as you use your too limited wiki parser and blindly use what it detects (incorrectly).
We should have been able to use templates since long to make things easier to understand and edit without errors, but it was for now impossible BECAUSE of your existing parser.
I made the initial effort to have it now working correctly with hCalendar readers, we can then progress by using templates and more capabilities. But this requires testing. I've not broken more the existing format when introducing microformats, it should have remained compatible with any basically compliant wikiparser and without complicating too much the addition of events.
I documented everything, with examples for the most common cases. But it was not easy to preserve the compatibility with the rest of the wiki.

@TheFive

This comment has been minimized.

Show comment
Hide comment
@TheFive

TheFive Oct 21, 2017

Owner

Sorry tldr.

You can develop the microformat interpreter as you want. We can switch off werklyOsM Calendar presentation from now, if you like.

Owner

TheFive commented Oct 21, 2017

Sorry tldr.

You can develop the microformat interpreter as you want. We can switch off werklyOsM Calendar presentation from now, if you like.

@fredao

This comment has been minimized.

Show comment
Hide comment
@fredao

fredao Oct 21, 2017

Contributor

@verdy-p

  • long roguish licentious speeches - as always.
  • do what you want with YOUR wiki-calendar. weeklyOSM will cancel this service without replacement.
  • So you are free to develop for the other users as it is necessary, without discussing changes beforehand.
Contributor

fredao commented Oct 21, 2017

@verdy-p

  • long roguish licentious speeches - as always.
  • do what you want with YOUR wiki-calendar. weeklyOSM will cancel this service without replacement.
  • So you are free to develop for the other users as it is necessary, without discussing changes beforehand.
@verdy-p

This comment has been minimized.

Show comment
Hide comment
@verdy-p

verdy-p Oct 21, 2017

Now I've been insulted gratuitously and personnaly in my wiki talk page. With additional false statements.

No, I've made things in plain respect of your work and many many times before. I was perfectly concious of your existence, otherwiser the changes would have been more radical, and I've already made many changes to make sure your tool would continue to work.

But it's a fact that instead of fixing a small result of your tool where it bugs sometimes, or fixing the tool, you just paralyze the wiki which was never intended to follow your rules. I've already proposed various solutions to help advance here because I'm also n ot satisified by the paralysy of the current format and the fact it cannot be precise enough to locate events, or give more precision on its scheduling.

NO, there's NEVER been any requirement to use the form "cityname, country" name at end of these events, it was only suggested in the documentation, with several examples of what was already done in the past, and never contested.

As well there's never been any requirement that this should be in English, or a requirement that links should not be used there, or about which link targets are suitable (internal wikilinks or links to OSM map coordinates or link to OSM object browser, or link to external site). They always have been very liberal. And not all events will require a city name (this has occured many times in the past).

I have made many corrections in the past that allowed the calendar to remain usable by all, or correctly refer the events, or to keep it maintainable and as clean as possible to allow automatic parsing without complex manual handling or dangerous "match-all" regexps. Many events would have never been visible without this, or would point to nowhere (because the links provided did not work, or the syntax itself was not working at all), or because the pnctuation was inconsistant (including for your own tool which requires some commas, frequently forgotten, where it will n ot recognize full stops, or words like "in").

Microformats were not proposed by me, but it was an excellent suggestion years ago, but not implemented, and I've made them real by taking into accoutn everything that had been done to not break things and complicate things further. I could have been much more radical and would have used templates everywhere if I had ignored the presence of your parser.

But as your parser is supposed to recognoze pure wiki syntax, it should never break like it did by breaking in the middle of links just on a comma found in the middle of the target part (and note that this can occur as well when instead of a wikilink we have an external link with URL containing such comma: the link here is also unbreakable).

Finally the calendar has always been multilingual: the ASCII comma "," is wrong for various languages using several scripts, its presence to split what was meant since the begining to be readable in natural language (not necessarily English) has NEVER been required, as well as it was never required to fix the sentence order. Once again using microformats is the only solution to "parse" these sentences written in natural language with automated tools.

verdy-p commented Oct 21, 2017

Now I've been insulted gratuitously and personnaly in my wiki talk page. With additional false statements.

No, I've made things in plain respect of your work and many many times before. I was perfectly concious of your existence, otherwiser the changes would have been more radical, and I've already made many changes to make sure your tool would continue to work.

But it's a fact that instead of fixing a small result of your tool where it bugs sometimes, or fixing the tool, you just paralyze the wiki which was never intended to follow your rules. I've already proposed various solutions to help advance here because I'm also n ot satisified by the paralysy of the current format and the fact it cannot be precise enough to locate events, or give more precision on its scheduling.

NO, there's NEVER been any requirement to use the form "cityname, country" name at end of these events, it was only suggested in the documentation, with several examples of what was already done in the past, and never contested.

As well there's never been any requirement that this should be in English, or a requirement that links should not be used there, or about which link targets are suitable (internal wikilinks or links to OSM map coordinates or link to OSM object browser, or link to external site). They always have been very liberal. And not all events will require a city name (this has occured many times in the past).

I have made many corrections in the past that allowed the calendar to remain usable by all, or correctly refer the events, or to keep it maintainable and as clean as possible to allow automatic parsing without complex manual handling or dangerous "match-all" regexps. Many events would have never been visible without this, or would point to nowhere (because the links provided did not work, or the syntax itself was not working at all), or because the pnctuation was inconsistant (including for your own tool which requires some commas, frequently forgotten, where it will n ot recognize full stops, or words like "in").

Microformats were not proposed by me, but it was an excellent suggestion years ago, but not implemented, and I've made them real by taking into accoutn everything that had been done to not break things and complicate things further. I could have been much more radical and would have used templates everywhere if I had ignored the presence of your parser.

But as your parser is supposed to recognoze pure wiki syntax, it should never break like it did by breaking in the middle of links just on a comma found in the middle of the target part (and note that this can occur as well when instead of a wikilink we have an external link with URL containing such comma: the link here is also unbreakable).

Finally the calendar has always been multilingual: the ASCII comma "," is wrong for various languages using several scripts, its presence to split what was meant since the begining to be readable in natural language (not necessarily English) has NEVER been required, as well as it was never required to fix the sentence order. Once again using microformats is the only solution to "parse" these sentences written in natural language with automated tools.

@TheFive

This comment has been minimized.

Show comment
Hide comment
@TheFive

TheFive Oct 22, 2017

Owner

As some of the last comments, including mine, are off topic I close this issue finally. I think the right people will find the right place to discusd the support for the community with calendar data in weeklyOSM, the one and only motivation i have followed with my tools.

Owner

TheFive commented Oct 22, 2017

As some of the last comments, including mine, are off topic I close this issue finally. I think the right people will find the right place to discusd the support for the community with calendar data in weeklyOSM, the one and only motivation i have followed with my tools.

Repository owner locked and limited conversation to collaborators Oct 22, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.