HIVE-15820: comment at the head of beeline -e

[ujc714]

What changes were proposed in this pull request?

1. Don't check if a line is a comment in Beeline.dispatch(). Instead, remove the comments from the line.
2. Replace removeComments(String, int[]) with removeComments(String) in Commands.handleMultiLineCmd().

Why are the changes needed?

1. The queries in '-e' parameter is passed to Beeline.dispatch() as a single line although there could be multiple lines. If the first line is a comment, the rest lines are ignored. We should pass the query strings to Commands.execute().
2. HiveStringUtils.removeComments(String, int[]) is used for a single line. In this method. If we use it to check a multiple line string and there is one comment line, the lines after this comment line will be discarded. In fact, HiveStringUtils.removeComments(String) splits a multiple line string to several single line strings and calls HiveStringUtils.removeComments(String, int[]) to process each single line. So HiveStringUtils.removeComments(String) is what we should use in Commands.handleMultiLineCmd().

Does this PR introduce any user-facing change?

No.

How was this patch tested?

1. org.apache.hive.beeline.testLinesEndingWithComments is used to test HiveStringUtils.removeComments(String).
2. org.apache.hive.beeline.cli.testSqlFromCmdWithComments* are used to test queries passed via '-e' option. And testSqlFromCmdWithComments2 is used to test the query after a comment line.

[ujc714]

The function trim() is called by HiveStringUtils.removeComments(String, int[]). As HiveStringUtils.removeComments(String, int[]) process a multiline statement as a single line string, only the leading spaces in the first line and the trailing spaces in the last line are removed. After I replaced HiveStringUtils.removeComments(String, int[]) with HiveStringUtils.removeComments(String), the leading spaces and trailing spaces in each line are trimmed. It's why 6 tests failed.

Rather than change HiveStringUtils.removeComments(String, int[]), I changed the test files as the new behaviour is expected.

[ujc714]

I moved trim() from HiveStringUtils.removeComments(String, int[]) to HiveStringUtils.removeComments(String) so that I only need to change TestHiveStringUtils.java instead of 6 other test result files.

In my option, HiveStringUtils.removeComments(String, int[]) shouldn't trim the spaces. Otherwise, we'll lose the indents in a formatted SQL statement. Trimming the leading and trailing spaces of the whole statement should be enough.

[nrg4878]

changes look good. I just requested an additional test scenario. Could you also include some of your test output, manually done, in the comments? Thank you

[nrg4878]

Could you please add a test scenario where a multiline query strings has comments in between fragments of query? just as an example
"select col1,
--partitioned year column
year,
--partitioned month column
month,
--partitioned date column
date
from test_table
where
--for a particular user
username = 'foo';"

should return something equivalent to
"select col1, year, month, date from test_table where username = 'foo';"

[ujc714]

Thanks for having a look! I added your example. It contains both separate comment lines and inline comment.

[ujc714]

Made the change as you advised. :)

[ujc714]

Thanks for having a look! I added your example. It contains both separate comment lines and inline comment.

[nrg4878]

Thanks for the changes. Change looks good to me. +1 for me.