Add String concatenation and String Interpolation

[abh80]

Added:

String concatenation

• Adds Binop.Add(...) support to String type.

constant c = "a" + "b" + "c" // gives "abc"
const a = "hello " + c  + "!" // gives "hello abc!"

String interpolation

• Desugars interpolated strings into series of string concatenation, using Binop.Add implementation from above

constant c = "a" + "b" + "c" // gives "abc"
const a = "hello $c!" // gives "hello abc!"

Closes #893

[Kronos3]

Thanks for the contribution! This is missing some tests. Take a look at compiler/tools/fpp-check and fpp-to-cpp.

We'll probably need tests for various forms of error/ok cases in fpp-check and a nominal case for the fpp-to-cpp.

[bocchino]

Thanks for this pull request! It is a good start.

To contribute a new feature to FPP, you will need to use this process: https://github.com/nasa/fpp/wiki/Adding-New-Features-to-FPP. If this sounds OK, we will make a feature branch, and you can make PRs to that.

The first thing we will need to do is to agree on the spec. The approach to string concatenation seems good. For interpolation, I think we will need to define an interpolation syntax that is parsed and interpreted during analysis, similar to the way that string interpolation works in Scala or Python. For instance, we probably want to allow this:

constant s1 = "def"
constant s2 = "abc${s1}ghi"

It might make sense to add concatenation separately, since it seems simpler.

[bocchino]

We will also decide whether we want to allow string expressions inside an interpolation, like this:

constant b = "b"
constant c = "c"
constant abcd = "a${b + c}d"

In this case we could just write

constant abcd = "a$b$cd"

or

constant bc = b + c
constant abcd = "a${bc}d"

so maybe it's not that useful. However, for consistency, it might be good to allow it.

All of this has to be worked out in the spec before we proceed to implementation.

[abh80]

Thanks for the contribution! This is missing some tests. Take a look at compiler/tools/fpp-check and fpp-to-cpp.

We'll probably need tests for various forms of error/ok cases in fpp-check and a nominal case for the fpp-to-cpp.

@Kronos3 Thank you for your review. I understand that testing is needed. I will add more test cases; here are the test files that I have identified as needing modification for robust test cases:

• Interpolation Tests:

  • lib/src/test/input/syntax/lexer/ok/literals.fpp -> added interpolated string literal tests (review needed)
  • [New]: lib/src/test/input/syntax/lexer/error/interpolated_strings.fpp -> added bad interpolation tests (review needed)
  • lib/src/test/scala/syntax/Parser.scala -> add interpolated string parser tests
  • lib/src/test/scala/syntax/Parser.scala -> add interpolated string parser tests
  • [New]: tools/fpp-check/test/expr/string_interp_ok.fpp -> add interpolation ok tests
  • [New]: tools/fpp-check/test/expr/string_interp_ok.ref.txt -> corresponding empty file

It looks like we will be trying a different approach for string interpolation.

• Concatenation Tests:
  • lib/src/test/scala/semantics/ValueSpec.scala -> add string + string add cases
  • [New]: tools/fpp-check/test/expr/string_concat_ok.fpp -> add string concatenation ok tests
  • [New]: tools/fpp-check/test/expr/string_concat_ok.ref.txt -> corresponding empty file
  • [???]: lib/src/test/input/syntax/lexer/ok/literals.fpp -> Add a string + string example; however, the lexer will accept it regardless. I think we can skip this.
  • tools/fpp-check/test/expr/tests.sh -> add test files definition for both concatenation and interpolation.

Please review and let me know if any changes.

[abh80]

Thanks for this pull request! It is a good start.

To contribute a new feature to FPP, you will need to use this process: https://github.com/nasa/fpp/wiki/Adding-New-Features-to-FPP. If this sounds OK, we will make a feature branch, and you can make PRs to that.

@bocchino Thanks, I understand. Please make the feature branch; I will work on that with wiki guidelines.

The first thing we will need to do is to agree on the spec. The approach to string concatenation seems good. For interpolation, I think we will need to define an interpolation syntax that is parsed and interpreted during analysis, similar to the way that string interpolation works in Scala or Python. For instance, we probably want to allow this:

constant s1 = "def"
constant s2 = "abc${s1}ghi"

I understand. That approach is cleaner architecturally, but it would require extending the analyzer to resolve interpolated variables. I initially tried handling interpolation in the lexer/parser to keep the implementation localized and avoid modifying later compilation phases. I’m happy to refactor it toward the cleaner architecture.

It might make sense to add concatenation separately, since it seems simpler.

Absolutely, I will update the tests needed for concatenation and make a separate pull request (after closing this one). Please let me know what do you think.

[bocchino]

Sounds good, thanks for agreeing to develop this feature! I made a feature branch here:

https://github.com/nasa/fpp/tree/feature/string-concat

Since string concatenation is simpler, and since concatenation is already a complete solution for configuring FPP code phases, let's start with concatenation only. Can you make a pull request to the feature branch that follows the procedure on the wiki for adding string concatenation? Once that's done we can proceed to string interpolation.

[abh80]

Closing this now