Implement repeats, variables & modules

[daveyarwood]

(None of these are yet defined in the grammar or implemented in the parser behavior.)

EDIT 9/30/15: See below for updated syntax ideas.

Repeats

Self-explanatory. e.g. c1(~1)x7 should expand to c1~1~1~1~1~1~1~1 at compile time.

Variables

Variables should work like they do in Lilypond. Any piece of code can be enclosed in brackets and defined as a variable, then called in the code any number of times, at any point in time.

Desired syntax:

# Defining a variable: 
riffA = [f8 f g+ a > c c d c <]
riffB = [b-8 b- > c+ d f f g f <]
riffC = [> c8 c d+ e g g a g < ]
riffD = [f8 f g+ a > c c d < b > | c c < b- b- a a g g]

# calling a variable
$riffA # expands to "f8 f g+ a > c c d c <"

# using variables + repeats
($riffA)x4 
($riffB)x2  ($riffA)x2
$riffC $riffB $riffD

# bonus: conglomerate variables into other variables!
rockinRiff = 
  [ ($riffA)x4 
    ($riffB)x2  ($riffA)x2
    $riffC $riffB $riffD    ] 

guitar/saxophone: 
    ($rockinRiff) x9999999

Modules

Take the variable concept and apply it to entire files.

For example, each instrument's part could be written in a separate file in the same directory as a main file, then in the main file (say, "score.alda"), you could write something like:

&clarinet.alda
&trombone.alda
&tuba.alda
&banjo.alda
&piano.alda
&drums.alda

and each of these statements would be replaced by the contents of each file at compile time.

This would be especially cool in that you could define custom instruments (e.g. custom synth instruments) as standalone files and then load them in as modules in any piece of music in which you'd like to use them.

NOTE: Modules might be a little tricky to implement and preserve line numbers for error reporting. Ideally we'd be able to report the line/column number and name of the file where the error occurred.

[oz-codes]

Absolutely brilliant concept. +1 to this, would increase and improve the fork and brevity of this concept. Like shia said, JUST DO IT

[daveyarwood]

Thank you! This is fairly high up on my list of priorities, and I think should be fairly easy to implement. Hopefully will get to it soon.

[Deveritas]

For repeats, what about using the sheet music symbols?
e.g. ||: a b4. c8 d :|| is treated as | a b4. c8 d | a b4. c8 d |?
And for multiple repetitions, maybe ||: a b c d :4||?

Or perhaps syntactic repetition should be kept separate from melodic repetition.

[daveyarwood]

The idea with repeats is that you'll be able to repeat any arbitrary Alda code, not just notes. You'll be able to encase an entire section of code (or even your entire score) in parentheses and stick a x4 after it, and it will be repeated 4 times.

That being said, though, I don't see any reason why we can't have it both ways. We can include both syntaxes for repeats in the grammar.

[crisptrutski]

@daveyarwood guessing the (<alda-code-here>)x<number> syntax needs to change now. Perhaps let reuse [] for those groupings also?

Seems like time is ripe for this feature now 😄

[daveyarwood]

@crisptrutski Oh yeah, good point. We'll have to update our approach now that inline Clojure code is a thing. I think we should implement #104 (at least the alda-code part) and leverage it for repeats.

I like the idea of representing things between [ square brackets ] as "Alda code objects" that we can do things with via Alda syntax. They would perhaps be equivalent to (and transform into) (alda.lisp/alda-code "string of alda code"), and we could use them for both variables and repeats:

# repeat
[ c d e f g ] x4

# variable
five-notes = [ c d e f g ]
$five-notes x4

I'm not really 100% sure how the syntax should look. Maybe there should be brackets around $five-notes too? I'm open to suggestions for this.

Maybe the xN syntax could parse as "repeat the thing before it N times", so that [ c d e f g ] x4 would be transformed by the parser into something like (alda.lisp/repeat-code 4 " c d e f g "), and then repeat-code could just call (alda.lisp/alda-code " c d e f g c d e f g c d e f g c d e f g ")

For modules, I never really liked the &filename.alda syntax I came up with. Now that we can leverage inline Clojure code, maybe it should just be (load-module "/path/to/file.alda") and/or (load-score "/path/to/file.alda")

[crisptrutski]

Have a suspicion that using a macro to repeat the nested code (and parsing repeat contents in the main grammar) will be simpler and more performant than leveraging alda-code

[crisptrutski]

re: extra brackets, think it would be nice not to require that. $five-notes x4 and c x4 would both be nice and legible.

[daveyarwood]

Have a suspicion that using a macro to repeat the nested code (and parsing repeat contents in the main grammar) will be simpler and more performant than leveraging alda-code

Oh yeah! You're probably right. It would be easier to just do the repetition as a transform step in the parser. 👍

[MillerMark]

Suggestions:
0. I agree with the use of square brackets for defining sequences, or a group of notes.

1. Eliminate the need for the "$" when referencing variables. This makes it easier on the user, reduces code complexity and makes Alda files easier to read. Just simply reference the variable after you've declared it. To avoid collision with notes, rests and the octave letter keywords, add a rule that says variable names must start with a letter and be at least two characters long.
2. Disallow the "-" from variable names and markers. This character should be used to shift pitch, and should not be part of marker or variable names. So "five-notes" would not be okay, but "fivenotes", "fiveNotes", and "five_notes" would all be okay.
3. Use the asterisk ( * ) for multiplication. It eliminates funky spacing (e.g., turns "fivenotes x4" into "fivenotes*4" or "fivenotes * 4" if you prefer).
4. Add two more built-in operators for pitch shifting the entire group. The multiplication operator and the pitch-shift operators would behave consistently with the individual note pitch shift operators (examples below).

# variable declaration looks like this:
melody = [04 c4 d e f g]
fullMelody = [melody a b >c melody]
rightHandMelody = [c e g a b-]
leftHandMelody = [b- a g e c]
upDownInC = [c e g a b- a g e c]

# variable reference looks like this (simple):
piano: melody
piano: V1: leftHandMelody V2: rightHandMelody V0: c4 c4 c4

# multiplier operator
melody*3

# these two are equivalent:
[c d e f g]*3
[c d e f g]***

# pitch shift up operator (shifts up semi-tones)
[c d e f g]+
[c d e f g]+3
melody+5
upDownInF = [upDownInC+5]
upDownInG = [upDownInF++]

# these two are equivalent:
[c d e f g]++++
[c d e f g]+4

# pitch shift down operator (shifts down semi-tones)
melody-5

# these two are equivalent:
[ c d e f g ]--
[ c d e f g ]-2

# operators can be combined (order doesn't matter - same results).
[ c d e f g ]+2*2
melody*2-2

# these two are equivalent:
melody*3+4
melody***++++

To simplify parsing code, I suggest disallowing spaces around operators.

[daveyarwood]

I'm working on implementing repeats now -- writing tests first so I can plan out exactly how they will work.

@Deveritas hinted at the idea that maybe the two proposed syntax ideas for repeats ([ this ] and ||: this :||) should represent two separate ideas -- syntactic repetition vs. musical repetition. I like this idea. I think maybe the first could expand to this this this and the second could expand to | this | this | this |. This will make for a more intuitive experience when we implement sheet music-rendering and barlines become significant.

Open question: should the number after the musical repetition operator mean "repeat this N times" or "play this N times total"? For example, should this: ||: c d e f :|| x1 be equivalent to c d e f c d e f or c d e f? The musician in me says the former, but the programmer in me says the latter. 😕 For reference, syntactic repeats will work the latter way ([ c d e ] x1 = c d e)

[daveyarwood]

I also think the syntax should be |: c d e :| rather than ||: c d e :|| -- although I suppose we could support both.

[daveyarwood]

@MillerMark:

1. Eliminate the need for the "$" when referencing variables. This makes it easier on the user, reduces code complexity and makes Alda files easier to read. Just simply reference the variable after you've declared it. To avoid collision with notes, rests and the octave letter keywords, add a rule that says variable names must start with a letter and be at least two characters long.

I like this idea, too. We currently have a similar rule for naming instruments -- a valid name must start with at least two letters. I was planning on using the same rules for variables. It might be slightly tricky to get the parser to recognize variable names as different from instrument names (e.g. piano:), but should be doable, I think.

[MillerMark]

Apologies for jumping in. While I see the value in creating a syntax that follows the user model of sheet music, it may also be useful to consider a syntax that is simple to type (which may correlate with faster music composition times).

I've played with syntax ideas for variables, note groups, repetition, sequence reversals, and pitch shift operations. All five of these ideas appear to be syntactically interconnected. After playing with this and actually creating a subset of the Alda in Windows, here is what I have found is simplest (e.g., the fewest characters), from a syntactical and key-pressing perspective:

1 . Variable names must start with two letters, and may optionally contain underscores ("_") and digits (as well as more letters) after the first two. This helps eliminate conflict with notes and octave specifiers (e.g., "o4", "c16", etc.). Variable names may not contain the dash, or minus character ("-").

2 . Notes and variables can be grouped together and treated as a sequence. Grouping starts with a "[" and ends with a "]". For example:

[spanUp o3 d+ d c middle d g f spanDown]

3 . Groups can be embedded/nested inside other groups. For example:

[o3 d+8 [c d+ g] >c d+ d c < b g b > d g f d+ d d+ [c d+ g]]
[a b > c cRun [c < b a]]

4 . Groups can span multiple lines if necessary.

5 . Variable assignment syntax looks like this:

variableName = {a group, or any sequence of notes or variables up to the end of the line}

Examples:

crun = o3 c4 e g a a+ a g e
cChord = [o2 c4/e/g r64 
          c1~1~1/e/g]

Notice the definition of cChord uses a group of notes that spans two lines. To keep things simple syntactically, variable assignments are restricted to a single line. If you need more than one line to define what's stored in a variable, use the group delimiters as demonstrated here.

6 . Pitch modifiers ("+" and "-") optionally follow a group or variable and adjust the pitch of all the notes in that variable or note group. Pitch-shift modifiers can be followed by a number (e.g., "-7"), or the "+" and "-" operators can be repeated in any combination for as many semitones you need to adjust (just like Alda's syntax for flat/sharp semitone adjustment).

Examples:

crun = o3 c4 e g a a+ a g e
frun = crun+5
grun = frun++
gChord = o2 g4/b/>d r64 <g1~1~1/b/>d
fChord = gChord--
cChord = fChord-5

7 . Multipliers ("*") repeat the sequence of all the notes held in a variable or defined by a note group by the number of times specified. Just like the pitch operators, multipliers can be repeated (so three asterisks in a row means repeat the group or variable three times).

Examples:

gChord = o2 g4/b/>d r64 <g1~1~1/b/>d
gChordTwice1 = gChord*2
gChordTwice2 = [o3 g4 b > d e f e d < b]*2
gChordTwice3 = gChord**

I feel strongly that the multiplier operator character should be an asterisk (and not the "x" character as suggested elsewhere) to avoid requiring the space between variable names and the multiplier. IOW, if the asterisk is my multiplier operator, I can write "cStomp*2". But if the "x" is my multiplier operator, I must write "cStomp x2". Forcing the space is inconsistent with Alda's note pitch-shift syntax (e.g., I can write "c+", and I'm never forced to write "c +").

8 . Sequence reversals are indicated with the "!" operator and reverse the sequence of all the notes held in a variable or note group.

Examples:

piano: [a b >c]!

runUp = o4 c8. d e f g
runDown = runUp!
updown = runUp a runDown <b
piano: updown*2

9 . Multipliers, reversals, and pitch operators can appear in any order after a variable or note group. Examples:

gChord = [o2 c4/e/g]+7
# the following variables should all hold the same note sequences:
fChordThreeTimes1 = gChord-2*3
fChordThreeTimes2 = gChord*3-2
fChordThreeTimes3 = [o2 c4/e/g r]+5*3
fChordThreeTimes4 = [o2 c4/e/g r]***+++++
fChordThreeTimes5 = [o2 c4/e/g r]+++++***
fChordThreeTimes6 = [o2 c4/e/g r]*3+5

Using this suggested syntax, it would be possible to create relatively sophisticated songs with a just few lines of Alda code:

cUp = o3 e g a
cDown = cUp!
cRun = o3 c4 cUp a+ cDown
fRun = cRun+5
gRun = fRun+2

cStomp = [o2 [c8/g]*2 [c/a]*2]*4
fStomp = cStomp+5
gStomp = fStomp+2

BrightAcousticPiano:  cStomp*2 fStomp*2 cStomp*2 gStomp fStomp cStomp o1 c1
trombone: cRun*2 fRun*2 cRun*2 gRun fRun cRun o4 c1

I also find that using pitch-shift operators, multipliers, and reversal operators, along with meaningful variable names makes it easier to see the structure of the song.

[daveyarwood]

@MillerMark Wow, there are a lot of great ideas here! It is a lot to take in at the moment, and will require further discussion and extended dev time to implement the ideas we want to bring in, but I do want to keep this conversation open. When I get a minute, I will go through your lists and make separate issues out of each idea, so we can discuss and/or implement them. I would prefer to just implement repeats, variables and modules in the scope of this particular issue.

That being said, there is stuff here that is definitely worth talking about as I implement repeats, variables and modules. I like the idea of trying to leave out brackets when possible, e.g. when defining variables (on a single line) and then referencing them later.

I'm beginning to warm up to the idea of * as a dedicated "multiplication operator" in Alda. Would make parsing easier, at least. The alternative would be requiring that the user either use brackets [likeThis]x2 or force them to place a space likeThis x2. likeThis*2 does look more elegant to me, and it visually stands out more against notes and octave changes, which also look like a single letter + a number. I'd welcome more input on this, if anyone else has a preference.

As a Lisp enthusiast, I'm trying to convince myself that it's a good idea to disallow hyphens in variable names :)

[daveyarwood]

Bear with me -- I'm breaking out each idea in this thread (including each of repeats, variables and modules) into a separate issue, to help keep things organized :)

[daveyarwood]

OK, I've broken out this issue into no less than 5 separate issues! When I opened this issue way back in January, I had no idea it would be so wide in scope -- too wide, I think, to discuss sanely in a single issue.

Hopefully I've done a good job of summarizing where we are on each issue. Please feel free to chime in on the individual issues if you have any additional insights. I'm really enjoying all the ideas this is generating!