Make BQ annotations serializable

[anish749]

I was trying to write Scollections to BQ, and for some generic derivations where a nested type contained a class with a big query annotation, I was finding that the annotations are not serializable. I was thinking of making them seralizable because I think that is an easy fix and hopefully wont break anything.

My use case is something like this:

//.groupBy(r => r.userId) - usual Scio code.
.bqTrace("gcpproj.dataset.error_records", predicate = _.size > 1) // Trace errors to BQ.
//.flatMap(...) // usual scio code.

now my SCollection is of type [(String, Iterable[CaseClassWithBQAnnotation])]

[codecov]

Codecov Report

Merging #1773 into master will decrease coverage by 2.49%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master    #1773     +/-   ##
=========================================
- Coverage   71.67%   69.17%   -2.5%     
=========================================
  Files         176      176             
  Lines        5402     5402             
  Branches      410      388     -22     
=========================================
- Hits         3872     3737    -135     
- Misses       1530     1665    +135

Impacted Files	Coverage Δ
.../spotify/scio/bigquery/client/BigQueryConfig.scala	0% <0%> (-70.59%)	⬇️
...scala/com/spotify/scio/bigquery/client/Cache.scala	0% <0%> (-66.67%)	⬇️
.../spotify/scio/bigquery/BigQueryPartitionUtil.scala	0% <0%> (-58.98%)	⬇️
...scala/com/spotify/scio/bigquery/BigQueryUtil.scala	50% <0%> (-50%)	⬇️
...la/com/spotify/scio/bigquery/client/QueryOps.scala	0.82% <0%> (-47.11%)	⬇️
...com/spotify/scio/bigquery/types/BigQueryType.scala	37.5% <0%> (-25%)	⬇️
...la/com/spotify/scio/bigquery/client/TableOps.scala	0% <0%> (-22.37%)	⬇️
...la/com/spotify/scio/bigquery/client/BigQuery.scala	27.02% <0%> (-10.82%)	⬇️
...n/scala/com/spotify/scio/bigquery/BigQueryIO.scala	24.29% <0%> (-3.74%)	⬇️
...com/spotify/scio/bigquery/types/TypeProvider.scala	76.59% <0%> (-2.13%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8181a11...c5f5f07. Read the comment docs.

[jto]

Hi @anish749. Can you add a test to ensure that the 2 annotations do indeed serialize ?

[regadas]

@anish749 as @jto said a test here would be super 😄

[anish749]

sure.. I will add them.. Thanks for pointing out.

[anish749]

I moved these classes to the companion object because these annotated classes should be in a companion object (?)

[anish749]

I moved these to the companion object because the class SchemaProviderTest extends FlatSpec with Matchers wont be serializable because org.scalatest.FlatSpec isn't serializable. Not sure if this is the right way to do this. What are your thoughts.

I was getting the following exception if I had in in the class and not in the companion object:

[info]   Cause: java.io.NotSerializableException: org.scalatest.Assertions$AssertionsHelper
[info]  - field (class "org.scalatest.FlatSpec", name: "assertionsHelper", type: "class org.scalatest.Assertions$AssertionsHelper")
[info]  - object (class "com.spotify.scio.bigquery.types.TypeProviderTest", TypeProviderTest)
[info]  - field (class "com.spotify.scio.bigquery.types.TypeProviderTest$S1", name: "$outer", type: "class com.spotify.scio.bigquery.types.TypeProviderTest")
[info]  - root object (class "com.spotify.scio.bigquery.types.TypeProviderTest$S1", S1(1))
[info]   at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1182)

I enabled -Dsun.io.serialization.extendedDebugInfo=true to trace this.

[jto]

LGTM

[anish749]

@regadas @jto I added a test, but I am not very sure it is in the right class. Please let me know if this is otherwise, I would change it accordingly. I am not very familiar with this part of the code.

[jto]

LGTM