Literal and Its Sub-Classes

[tajmone]

@thoni56, I have a few questions about the literal class and its subclasses.

The manual could be expanded a bit on this topic, especially by providing exmaples. So, I'll expose here my doubts and questions about them, and if you can help me cllarify the topic I could then try to edit the Manual accordingly (in a separate branch).

Classes and Keywords

One of the things that initially puzzled me most on literals is that while they all belong to the predefined classes not all of them have corresponding keywords:

As a matter a fact, I propose to change the image of the above diagram to use color and style to provide further information:

• Yellow BG for inheritable classes (entity, thing, location, object, actor)
• Grey BG for non-inheritable classes (literal, string, integer)
• italics for classes whose name is not a keyword (literal, integer)

For, if I've understood correctly, while you can ADD TO the string class, you can't do that on literal and integer.

The diagram is shown in Ch.2's The Predefined Classes, and adding the proposed color/style changes, and a description of their meaning in the image's caption, could help new users memorize better the base classes.

This is a first draft I was working on (but here I used red borders instead of italics for non-keyword classes):

But I think that using italics instead of red borders is better (especially for a printable version, where the dark red would be lost in print).

What do you think about the classes diagram tweaks? Should we use in the Manual?

(the image above is just a draft, I'll create a good SVG image).

Styling Issues

In various places I have problems when it comes to styling literals, since they are not Alan keywords in the strict sense I haven't styled them as inline code, but as bold. The problem is that when the text mentions all the predefined classes it becomes a mixture of inline-code and bold styles:

• In Ch.2's The Predefined Classes I haven't yet decide how to style this paragraph, and just left all classes in bold in this context:

  They are entity, thing, location, actor, object, literal, string and integer and have the relationship, inheritance tree, shown in the figure above.

  ... because if I were to follow strictly the conventions, it would like a bit hugly:

  They are entity, thing, location, actor, object, literal, string and integer and have the relationship, inheritance tree, shown in the figure above.

I think that styling convention should make like easier to the reader, but in some contexts we should also consider text elegance. As a general rule, when a keyword occurs multiple times in a paragraph, I only style it as inline code in its first occurence, or in those places where it's important to underline that we're dealing with the keyword, not just the general concept. Wherever the context is obvious, or when the keyword and its noun counterpart in the natural language overlap, I prefer not to use styles.

But in the above list of native classes, distinguishing between classes that have and don't have a keyword counterpart might actually help the reader, who might otherwise try to use ADD TO with literal and integer — here, styling them in bold will make them contrast with the other classes, although I admit it looks hugly.

Any suggestions on this?

Examples Needed

The problem in the manual is that section 3.6. Instances/Literals is vague on the topic:

The classes literal, string and integer cannot be instantiated explicitly. Instead, you might say that they are implicitly instantiated when the player inputs a literal. For example:

[...]

It is possible to add Verbs to literal and its sub-classes. This way it is possible to create verbs that take strings and integers as parameters.

The last sentece really confuses me. Some examples here (or elsewhere, and just add a cross reference here) would help, otherwise the reader/learner is left with huge question mark floating in his head.

I've seen examples of verb additions to the string class (in the StdLib), but the above sentence clearly states that they can be added "to literal and its sub-classes".

Could you help me understand better this point?

I think that it's worth adding to the manual a couple of examples right after that sentence. So far, I can't recall finding in the manual practical examples on literal classes, but I might be wrong on this.

[thoni56]

Following up on the discussion on colouring schemes in #17 where we decided to remove all classes, even predefined ones, from the list of keywords. There is only two notations in the diagram that are required:

• inheritable classes
• non-instantiable classes

The last sentece really confuses me. Some examples here (or elsewhere, and just add a cross reference here) would help, otherwise the reader/learner is left with huge question mark floating in his head.

I've seen examples of verb additions to the string class (in the StdLib), but the above sentence clearly states that they can be added "to literal and its sub-classes".

Yes, it says that, and only that. It does not say that you can subclass any of them. But literal already has two pre-defined sublclasses, literal and string. And you can Add verbs to any of them.

But, yes, that could probably be explained further and exemplified.

Are we keeping issues of this kind in this project or in the Alan tracker?

[tajmone]

Thanks for the clarification.

Are we keeping issues of this kind in this project or in the Alan tracker?

Here the issue concerns the Manual content updates (I'm adding them as commented TODOs in the Manual source files). I'm not sure what you mean by "the Alan tracker", you mean at Pivotal?

[thoni56]

Yes I meant Pivotal. It gets a little confusing to have multiple issue trackers for similar types of issues.

Since changes to the language or IDE etc. will force updates to the documentation too. I'm thinking that issues here would primarily be around layout, formatting etc. But I also see that "bugs" in the documentation might go here.

The tracker has been the one and only place for me to keep track of all Alan issues, but I suppose that to enable more collaborators around the documentation we should keep issues that are strictly about documentation here.

Issues that change something which needs to be documented should start out in the tracker, and could later be transferred here, if the documentation was not done in direct conjunction to the implementation. (Technical Debt ;-) which we don't want...)

[tajmone]

If I'm not mistaken, Pivotal is not free. I remember looking into it when I discovered that Alan used it for tracking development, but then the registration form mentioned "Free for 30 days", so I desisted.

I think that separating code and documentation issues might make sense, after all there are no real "bugs" in documents, only things to correct or improve (but a document won't crash). Possibly, contributors to documentation might get overwhelmed by bug tracking being in the same place.

[thoni56]

Actually, I think a problem with documentation is as much a bug as a problem with running applications, it's just a matter of what effects the "bug" will have. And that is a gray-scale if anything is. Bugs in documentation might have more severe consequences than code bugs.

I surely don't pay anything ;-) And here's what they say:

Pivotal Tracker is free for:
Small teams & Project Viewers: Small teams (see the Free plan details above), read only project members (viewers), and members of public or archived projects do not count toward project collaborator limits.

Qualified non-profits and academic institutions: We have special plans available for registered, tax-exempt non-profits (such as charities) and accredited academic organizations. If you are a full-time employee of either (students, please see this), to request a sponsored plan for your account, please fill out this form.

Public projects: Public projects do not count toward any plan limits. You can create as many publicly visible projects as you desire, with no limits on the number of collaborators, completely free.

But let's not jump the gun, we'll see how this evolves.

[tajmone]

Thanks for the info on Pivotal, it wasn't very clear from mere website navigation. I'll read the full details and open an account then, and try to get a grip on how it works.

As for the documentation bugs in this repo vs Pivotal, another difference would be that this project will host also documents that don't belong to Alan's official documentation (the AlanIDE Guide is just one example), so the two project don't overlap 100%.

problem with documentation is as much a bug as a problem with running applications, it's just a matter of what effects the "bug" will have.

Then probably severe problems in the documentation should also be tracked on Pivotal, and in any case it's always possible to cross reference issues on the two platforms.

The point here is that as far as documentation contributors are concerned, they'll need a GitHub account to contribute to the documents, while creating a Pivotal account would not benefit them in terms of contributing contents or editing (for example, I'll be creating a Pivotal account solely because Alan uses it to track issues). Not all Alan users are programmers, and for many of them asking them to use Git might be already a demanding task (can't say Git is a user friendly tool at all).

[thoni56]

You can probably get a fair grip of Pivotal by just navigating the Alan project, as it is open and you can see most fields, buttons and such (but not do much).

We'll probably work it out once we see which issues show up.

[tajmone]

I've tried some tests with literals, and here is a working example:

The basement IsA location.
  Description "What a dark ang gloomy place!"
End The.

The vampire IsA actor At basement.
End The.

The coffin IsA object At basement.
  Container taking thing.
End The.

ADD TO EVERY thing
  VERB examine
    Does
      "You examine $+1, but find nothing unusual."
  END VERB.
END ADD TO.

ADD TO EVERY string
  VERB 'say'
    Does
      "You say ""$$" Say This. "$$""!"
  END VERB.
END ADD TO.

ADD TO EVERY integer
  VERB shout
    Does
      "You shout ""$$" Say This. "$$"" out loud!"
  END VERB.
END ADD TO.

ADD TO EVERY literal
  VERB whisper
    Does
      "You whisper ""$$" Say This. "$$""!"
  END VERB.
END ADD TO.

--------------------------------------------------------------------------------

Start at basement.

From this I realized that literal is a valid Alan keyword, and it should be added to the list of keywords in the Manual and in the syntax highlighters definitions!

This omission in the documentation is definitely a bug! Because of it, I never realized you could use literal in the code, and this affected the creation of all the syntax highlighters definitions for Alan which I created (and the Sublime Alan package too).

Is there a file in the Alan source code that contains all the keywords and reserved words, which I could reference to double check that no keyword is left of in the highlighters?

I was about to update the Alan syntax for Highlight (to move actor, location and string to the predefined classes group) when I decided I wanted to test the whole thing — luckily I did, for I would have had to re-fix the syntax again and add literal to it.

I'll wait for confirmation from you before updating the Highlight syntax and the XSLHL syntax used for creating the PDF Manual, just to ensure I get it all right this time.

[tajmone]

I have one further question about syntax highlighting Alan code, which I would like your advise on for I'm about to update also the Highlight syntax definition (but I would like to also apply this to the XSLHL syntax).

Include hero in Predefined Classes Group?

As mentioned, in the Alan syntax definition I also add a group to capture the predefined classes, which of course the user can disable by simply assigning to that group the same style as normal text. But leaving the opportunity open to those who would like to use them is and added benefit — since the XSHL syntax is also going to be used by other documents in the project!

In the current Alan syntax for Highlight, hero belongs to the same group of the predefined classes:

--------------------------------------------------------------------------------
--                 Predefined Alan Classes and Special Entities                 
--------------------------------------------------------------------------------
  { Id = 2,
    List = {
      -- Built-in usable classes:
      "actor", "entity", "literal", "location", "object", "string", "thing",
      -- Special entity for protagonist:
      "hero"
    }

(note that string and literal are not presen in the current definition, but I've added them to my local copy which I'm about to publish soon).

My reasoning is that hero being a predefined instance in Alan (and a rather important one too) it deserved being highlighted (or the possibility to do so). I've placed it together with the predefined classes, but on second though it might seem better to add a new special group for it, so that end users could decide if they want to show it in the highlighted code or hide it, without this choice affecting the predefined classes. Again, through styling is possible to control how and if it will be shown (like predefined classes, or differentely, or hidden).

The same applies to the XSLHL Alan syntax, where I can assign hero to a new capturing group (in XSHL called style) and then we can control if it should be visible or not.

Also bare in mind that we might wish to turn on and off coloring of predefined classes or the hero on specific code examples, by using a custom AsciiDoc role which could be associated to an alternative sytlesheet configuration, tweaked to disable/enable styling of these optional groups. In some context we might want to draw the attention to them.

What do you think about this, should I keep hero together with the predefined classes or add a new group for it (ie, both in my upcoming Highlight update and for the XSLHL syntax we use for the PDF).

As a general rule, would you like the pred.classes and/or the hero to be highlighted in the Manual? (even a slightly different color, or using bold).

A New Alan Syntax for Rouge

I'll also be using this guidelines that we've discussed here to create the new Alan syntax for Rouge — I've looked into integrating Highlight in the HTML toolchain, and even though it can be done we might have problems with callouts and substitutions (which are needed in the Manual).

So it's probably best to just create another syntax using Rouge, the native Ruby highlighter of choice for Asciidoctor. Alternatively, I could create a syntax for Highlight.js (which I've already worked with), which could then work directly in the browser (but I'm not sure if callouts would work) and could be used also on forums and websites. In any case, the current syntaxes can easily be adapted to these other tools, and adding Alan to different highlighters is a desirable goal.

[thoni56]

Literal

In you wrote that you realized that literal is an Alan keyword. Literal is not a keyword. It is a predefined class exactly like "object" or "location".

I think you "miss-wrote" since in the last comment you say "pre-defined" classes, which is correct.

Just to emphasize, as discussed previously, we should not highlight the pre-defined classes in the same way as keywords. In code they should probably be treated exactly like author defined identifiers.

In running text they should probably have some highlighting, but I think they should not have the same highlighting as keywords. (Have to look up examples of how actual keywords are used in running text...)

You might view all the predefined classes and behaviours as existing in a "base library" which makes them available to the author. Thus they cannot be redefined, since they are already defined. But they are not keywords in the sense that they are part of the language syntax (except for the idiosyncrasy of Current <identifier>).

Hero

herois also not a keyword, it is also a predefined identifier, but this time an instance identifier. So it should also go with the pre-defined classes and other identifiers.

[thoni56]

Here's a complete list of all tokens defined by the Alan language grammar. It currently includes actor and location which should, as discussed previously, not be keywords.

   0    0 Unknown                        main
   1    1 EndOfText                      main
   2    2 Identifier                     main
   3    3 Integer                        main
   4    4 ','                            main
   5    5 '.'                            main
   6    6 '{'                            main
   7    7 '}'                            main
   8    8 '='                            main
   9    9 ':'                            main
  10   10 '('                            main
  11   11 ')'                            main
  12   12 '*'                            main
  13   13 '!'                            main
  14   14 '+'                            main
  15   15 '-'                            main
  16   16 '/'                            main
  17   17 '>'                            main
  18   18 '<'                            main
  19   19 'to'                           main
  20   20 'is'                           main
  21   21 '=>'                           main
  22   22 'at'                           main
  23   23 'in'                           main
  24   24 'it'                           main
  25   25 'an'                           main
  26   26 'no'                           main
  27   27 'by'                           main
  28   28 'if'                           main
  29   29 'on'                           main
  30   30 'do'                           main
  31   31 'or'                           main
  32   32 'of'                           main
  33   33 '<>'                           main
  34   34 '=='                           main
  35   35 '>='                           main
  36   36 '<='                           main
  37   37 'the'                          main
  38   38 'end'                          main
  39   39 'not'                          main
  40   40 'and'                          main
  41   41 'isa'                          main
  42   42 'add'                          main
  43   43 'are'                          main
  44   44 'has'                          main
  45   45 'can'                          main
  46   46 'say'                          main
  47   47 'set'                          main
  48   48 'for'                          main
  49   49 'use'                          main
  50   50 'off'                          main
  51   51 'max'                          main
  52   52 'min'                          main
  53   53 'sum'                          main
  54   54 'verb'                         main
  55   55 'exit'                         main
  56   56 'when'                         main
  57   57 'quit'                         main
  58   58 STRING                         main
  59   59 'else'                         main
  60   60 'meta'                         main
  61   61 'does'                         main
  62   62 'form'                         main
  63   63 'name'                         main
  64   64 'with'                         main
  65   65 'then'                         main
  66   66 'step'                         main
  67   67 'wait'                         main
  68   68 'here'                         main
  69   69 'near'                         main
  70   70 'list'                         main
  71   71 'show'                         main
  72   72 'play'                         main
  73   73 'from'                         main
  74   74 'make'                         main
  75   75 'last'                         main
  76   76 'word'                         main
  77   77 'into'                         main
  78   78 'each'                         main
  79   79 'stop'                         main
  80   80 'look'                         main
  81   81 'save'                         main
  82   82 'this'                         main
  83   83 'only'                         main
  84   84 'every'                        main
  85   85 'event'                        main
  86   86 'start'                        main
  87   87 'check'                        main
  88   88 'where'                        main
  89   89 'count'                        main
  90   90 'after'                        main
  91   91 'until'                        main
  92   92 'style'                        main
  93   93 'empty'                        main
  94   94 'strip'                        main
  95   95 'first'                        main
  96   96 'words'                        main
  97   97 'elsif'                        main
  98   98 'score'                        main
  99   99 'actor'                        main
 100  100 'syntax'                       main
 101  101 'script'                       main
 102  102 'option'                       main
 103  103 'import'                       main
 104  104 'prompt'                       main
 105  105 'visits'                       main
 106  106 'opaque'                       main
 107  107 'taking'                       main
 108  108 'limits'                       main
 109  109 'header'                       main
 110  110 'nearby'                       main
 111  111 'locate'                       main
 112  112 'cancel'                       main
 113  113 'depend'                       main
 114  114 'system'                       main
 115  115 'random'                       main
 116  116 'before'                       main
 117  117 'options'                      main
 118  118 'message'                      main
 119  119 'article'                      main
 120  120 'entered'                      main
 121  121 'pronoun'                      main
 122  122 'extract'                      main
 123  123 'include'                      main
 124  124 'exclude'                      main
 125  125 'between'                      main
 126  126 'restore'                      main
 127  127 'restart'                      main
 128  128 'current'                      main
 129  129 'synonyms'                     main
 130  130 'definite'                     main
 131  131 'negative'                     main
 132  132 'directly'                     main
 133  133 'describe'                     main
 134  134 'schedule'                     main
 135  135 'increase'                     main
 136  136 'decrease'                     main
 137  137 'contains'                     main
 138  138 'location'                     main
 139  139 'mentioned'                    main
 140  140 'container'                    main
 141  141 'character'                    main
 142  142 'depending'                    main
 143  143 'attributes'                   main
 144  144 'indefinite'                   main
 145  145 'initialize'                   main
 146  146 'indirectly'                   main
 147  147 'characters'                   main
 148  148 'transcript'                   main
 149  149 'description'                  main
 150  150 'transitively'                 main

[tajmone]

Three Missing Keywords!

Thanks, this helped me a lot. After comparing the list you provided with the one in the Manual, I've actually found three missing keywords:

• meta
• transitively
• indirectly

So I'll add these three to all the highlighters definition and also amend the keywords list in the Manual.

... you wrote that you realized that literal is an Alan keyword. Literal is not a keyword. It is a predefined class exactly like "object" or "location".

I think you "miss-wrote" since in the last comment you say "pre-defined" classes, which is correct.

Yes, I must have got confused. Sometimes I try to think of them as "reserved words", at least when working with syntax definition where I have to consider any native token of the language.

This also shows that my view on this is a bit biased by the highligters' work — I tend to look at the list in the Manual with my mind focused on the needs of creating a syntax definition, and I tend to forget that it's meant for readers who are only interested in learning the language.

Keywords in Manual: CVS in Table

By the way, as a side note, now the keywords list in the Manual is handled via a table that takes a CSV list and auto-formats it into columns and rows:

[[table-of-keywords]]
.List of Alan Language's Keywords
[format=csv,cols="5*m"]
|======================
actor
Add
After

As an added benefit, we could control the number of columns by a custom attribute, to make the table larger or thinner according to output format (eg, when targeting eBook Readers use 3 columns instead of 5).

Also, we could import the keywords list from a CSV file instead of having the list inline — this would allow the CVS file to be placed in the assets folder and be reusable by other docs too.

[tajmone]

Highlight Syntax Updated

I've update the Alan syntax for Highlight accordingly, and created a merge request; so these changes should be integrated into the next release. Ultimately, I've moved hero into a keyword group of its own, and let the end user decide if to style it (and predefined classes) differently or not — leaving the option open.

This afternoon I'll do the same with XSLHL syntax, and then try these styling changes:

• predefined classes and hero: some color as normal text, but bold (just to remind the reader they are special).
• operators: same style as keywords (I've tested it locally and it look good, especially with parameters as it makes them easier to spot). After all, operators belong to the keywords list technically speaking.

The goal is to allow the reader to distinguish in the examples between what is hard-coded into Alan and what isn't, and using bold for predefined classes/instance could be a visually non intrusive reminder of that.

Let's see how the final results looks; we have freedom to test various combinations until we're in the draft stage.

It's a pity that XSHL doesn't supported escape sequences and interpolation inside strings (you can define an escape symbol, but it won't be styled in any way). In Highlight I implemented dollar special characters as escape sequences, and dollar parameters refernces as interpolation (did the same in Sublime Alan), and this makes strings look nicer, and also distinguished between special characters used for formatting from parameters subsitutions. I think that this is a quite cool feature, because the dollar special chars can make it hard to read strings, especially when there are many of them in one place.

Hopefully we'll have that in the HTML format, via Rouge (or whatever other highlighter we end up using).

[tajmone]

OK, I've fixed all the keywords and predefined classes in both the XSLHL syntax and in the Manual (not the keywords list of "D.2. Keywords" is stored in an external CSV file which is included into the document.

I've also added hero as an independent highlighting group, but it doesn't get styled in the Manual (it's there, in case some tutorials would like to use it).

Also, I've added to the Syntax Highlighting PDF document a code example containing all predefined classes and the hero, to both test that the syntax is working correctly and to preview how they will look like.

All previous test commits have now been merged into master, and the test branches won't be updated further (I think I'll just delete them that they are no longer needed). So the above links to branch specific contents will either stop working or not be up-to-date with master.

[thoni56]

Nice. In the Syntax Highlighting PDF I noticed that there was a "fiver" (5) that was coloured red.

I would consider literal values and constants equal to author defined identifiers. They are values that can be used in expressions.

I have noted that some colour schemes colour integers in a separate colour, maybe this was the case here? It wasn't exactly the same colour as keywords?

[tajmone]

I noticed that there was a "fiver" (5) that was coloured red.

Yes, I've added the possibility to highlight integers with a custom color, but I can just set it to normal color and hide it in the documentation.

I have noted that some colour schemes colour integers in a separate colour, maybe this was the case here?

Yes, indeed usually numerals are given a color of their own, it makes it easier to spot them in very long source files. But for the purposes of the Manual's example this might not be needed.

It wasn't exactly the same colour as keywords?

No, it's a bright red.

Anyhow, as soon as I open again the XSL files I'll change it to normal.

Then I'll start working on the Predefined classes diagram — I have drafted an SVG version using Mermaid, which produced the colored diagram image which I've posted above, but I'm not 100% happy with it, might just recreate it using Inkscape.

[tajmone]

Predfined Classes Diagram Ready!

I've found an amazing open source tool for drawing diagrams, called Dia Diagram Editor. It's cross platform and comes with a nice GUI to draw lots of different types of diagrams, and it's also user extensible and there many more diagrams types contributed by users as plugins. (there is so much good to be sayed about this app that I better stop here, otherwise I could go on praysing it for ever! it really saved my day).

I've used it to recreate the classes diagram, here is the SVG converted image:

The coolest thing is that Dia can also be invoked as a CLI tool to automate conversion of the source diagrams to different output formats (and it really supports many formats!).

In fact, I've added scripts to automate conversion and deployment of the diagram:

• https://github.com/alan-if/alan-docs/tree/master/_assets-src/images/predefined-classes

Also, SVG images created via Dia show up nicely in PDF documents — unlike many of the Asciidoct Diagram tools, like mermaid, which I've tried to use but the SVG image they produced wouldn't show up correctly in the PDF file (corrupted colors, missing text).

So, what you think of the new diagram? Does it look nice?

Of course, being a vector image it can really scale well, losslessly.

If you like the image as it is, the next step is to include it in both the Manual and the Beginner's Guide (both use this image). But I probably won't be able to do it before tomorrow.

[tajmone]

Updated Classes Diagram in Manual

I've added the SVG version of the predefined classes to the Manual (Live HTML link). It looks really nice in both the HTML and PDF versions.

I've also added to the caption a note about the usage of different bg-colors in the diagram, but I'm not fully satisfied with the text:

Figure 2. Relationships between the predefined classes (only classes shown with yellow background are inheritable and can be explicitly instantiated).

For a caption text it might be too long, but at the same time is too short to explain the issue properly. In the PDF version the caption text uses a very big font compared to the HTML document.

I've also fixed all occurences of literal classes to inline code style (whereas before I was rendering them in bold, thinking there were not reckognized).

[tajmone]

Since the original topic of the diagram has been solved, and all the keywords issues have been resolved both in the Manual as in the syntax highlighters, I've created a separate issue (#22) for the implementation of more examples on using literals. So this issue can be closed.