2022-01-04-check_spelling_dl_en

[jsl-models]

No description provided.

[maziyarpanahi]

@egenc @josejuanmartinez

we have these pipelines for spark 2.4 already, the spark 3.0 is missing. These pipelines must be saved/uploaded for spark_version: 3.0 (not just choose 3.0, but actually by using spark 3.0)

[egenc]

@maziyarpanahi , that is exactly what I did. Created these pipelines using pyspark==3.1.2

[maziyarpanahi]

The metadata says spark_version: 2.4 and it saved them as 2.4 on S3. It won't download these if a user is on spark3.x, so probably the spark version was manually changed in Models Hub UI.

[egenc]

so I created these pipelines using pyspark==3.1.2 . However, when I upload the model, I picked spark compatibility as 2 otherwise these models will not be compatible with spark 2..x

[maziyarpanahi]

The original issue was that these pipelines are missing for Spark 3.x: #2738

[egenc]

ok, pushing them again. thanks!

[egenc]

#6709

[maziyarpanahi]

Thanks @egenc
it's the same for the other 2 pipelines I closed.

[egenc]

totally got it. Fixing them all