[Submarine Spark Security] Different masking policies for different tables fail in the case of union

[qiuliang988]

Hi, We ran into a problem with Submarine's Spark Security for Column Masking
For example,
select col_1, col_2 from tbl_1 union select col_3, col_4 from tbl_2
If masking policies for col_2 and col_4 are different, col_4 will use col_2's policy instead of its own
because col_4 is not in "plan.output"

[Junyewu]

I also encountered the same problem.

@yaooqinn

[harsha-tenneti]

What is the expected output in this case?

[turtoise]

Hi,I was also using spark security plugin to authorize SparkSQL,but when i execute command "show tables " or any sql command ,it report below errors:

22/01/11 10:23:26 WARN PolicyRefresher: cache file does not exist or not readable '/etc/ranger/sparkServer/policycache/sparkSql_sparkServer.json'

I have no idea why would this happen even though i configured a sparkService in hive in ranger?

[xunliu]

The [Submarine Spark Security] functionality has been moved to the apache/incubator-kyuubi standalone project.