Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added forced truncation of WKT types in Markdown/HTML rendering. #408

Merged
merged 1 commit into from Nov 8, 2019
Merged

Added forced truncation of WKT types in Markdown/HTML rendering. #408

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 5 additions & 2 deletions core/src/main/scala/org/locationtech/rasterframes/util/DataFrameRenderers.scala
View file
Open in desktop
Expand Up @@ -24,12 +24,14 @@ package org.locationtech.rasterframes.util
import geotrellis.raster.render.ColorRamps
import org.apache.spark.sql.Dataset
import org.apache.spark.sql.functions.{base64, concat, concat_ws, length, lit, substring, when}
import org.apache.spark.sql.jts.JTSTypes
import org.apache.spark.sql.types.{StringType, StructField}
import org.locationtech.rasterframes.expressions.DynamicExtractors
import org.locationtech.rasterframes.{rfConfig, rf_render_png, rf_resample}
import org.apache.spark.sql.rf.WithTypeConformity

/**
* DataFrame extensiosn for rendering sample content in a number of ways
* DataFrame extension for rendering sample content in a number of ways
*/
trait DataFrameRenderers {
private val truncateWidth = rfConfig.getInt("max-truncate-row-element-length")
Expand All @@ -47,8 +49,9 @@ trait DataFrameRenderers {
lit("\"></img>")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if the line above this one

 base64(rf_render_png(rf_resample(resolved, 0.5), ColorRamps.Viridis))

should be changed to use ColorRamps.greyscale(256), and then apply a color ramp in the python API for notebooks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I am not sure about what we can do on the notebook / python side if the image is already b64 encoded in HTML... Some css stuff? idk

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Damn. Didn't think of that.

)
else {
val isGeom = WithTypeConformity(c.dataType).conformsTo(JTSTypes.GeometryTypeInstance)
val str = resolved.cast(StringType)
if (truncate)
if (truncate || isGeom)
when(length(str) > lit(truncateWidth),
concat(substring(str, 1, truncateWidth), lit("..."))
)
Expand Down
8 changes: 6 additions & 2 deletions core/src/test/scala/org/locationtech/rasterframes/ExtensionMethodSpec.scala
View file
Open in desktop
Expand Up @@ -39,7 +39,7 @@ import scala.xml.parsing.XhtmlParser
class ExtensionMethodSpec extends TestEnvironment with TestData with SubdivideSupport {
lazy val rf = sampleTileLayerRDD.toLayer

describe("DataFrame exention methods") {
describe("DataFrame extension methods") {
it("should maintain original type") {
val df = rf.withPrefixedColumnNames("_foo_")
"val rf2: RasterFrameLayer = df" should compile
Expand All @@ -49,7 +49,7 @@ class ExtensionMethodSpec extends TestEnvironment with TestData with SubdivideSu
"val Some(col) = df.spatialKeyColumn" should compile
}
}
describe("RasterFrameLayer exention methods") {
describe("RasterFrameLayer extension methods") {
it("should provide spatial key column") {
noException should be thrownBy {
rf.spatialKeyColumn
Expand Down Expand Up @@ -124,6 +124,10 @@ class ExtensionMethodSpec extends TestEnvironment with TestData with SubdivideSu

val md3 = rf.toMarkdown(truncate=true, renderTiles = false)
md3 shouldNot include("<img")

// Should truncate JTS types even when we don't ask for it.
val md4 = rf.withGeometry().select("geometry").toMarkdown(truncate = false)
md4 should include ("...")
}

it("should render HTML") {
Expand Down