Skip to content

Added forced truncation of WKT types in Markdown/HTML rendering. #408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
metasim merged 1 commit into from
Nov 8, 2019
Merged

Added forced truncation of WKT types in Markdown/HTML rendering. #408

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions core/src/main/scala/org/locationtech/rasterframes/util/DataFrameRenderers.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,14 @@ package org.locationtech.rasterframes.util
import geotrellis.raster.render.ColorRamps
import org.apache.spark.sql.Dataset
import org.apache.spark.sql.functions.{base64, concat, concat_ws, length, lit, substring, when}
import org.apache.spark.sql.jts.JTSTypes
import org.apache.spark.sql.types.{StringType, StructField}
import org.locationtech.rasterframes.expressions.DynamicExtractors
import org.locationtech.rasterframes.{rfConfig, rf_render_png, rf_resample}
import org.apache.spark.sql.rf.WithTypeConformity

/**
* DataFrame extensiosn for rendering sample content in a number of ways
* DataFrame extension for rendering sample content in a number of ways
*/
trait DataFrameRenderers {
private val truncateWidth = rfConfig.getInt("max-truncate-row-element-length")
Expand All @@ -47,8 +49,9 @@ trait DataFrameRenderers {
lit("\"></img>")
Copy link
Copy Markdown
Member Author

@metasim metasim Nov 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if the line above this one

 base64(rf_render_png(rf_resample(resolved, 0.5), ColorRamps.Viridis))

should be changed to use ColorRamps.greyscale(256), and then apply a color ramp in the python API for notebooks.

Copy link
Copy Markdown
Member

@vpipkt vpipkt Nov 6, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I am not sure about what we can do on the notebook / python side if the image is already b64 encoded in HTML... Some css stuff? idk

Copy link
Copy Markdown
Member Author

@metasim metasim Nov 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Damn. Didn't think of that.

)
else {
val isGeom = WithTypeConformity(c.dataType).conformsTo(JTSTypes.GeometryTypeInstance)
val str = resolved.cast(StringType)
if (truncate)
if (truncate || isGeom)
when(length(str) > lit(truncateWidth),
concat(substring(str, 1, truncateWidth), lit("..."))
)
Expand Down
8 changes: 6 additions & 2 deletions core/src/test/scala/org/locationtech/rasterframes/ExtensionMethodSpec.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ import scala.xml.parsing.XhtmlParser
class ExtensionMethodSpec extends TestEnvironment with TestData with SubdivideSupport {
lazy val rf = sampleTileLayerRDD.toLayer

describe("DataFrame exention methods") {
describe("DataFrame extension methods") {
it("should maintain original type") {
val df = rf.withPrefixedColumnNames("_foo_")
"val rf2: RasterFrameLayer = df" should compile
Expand All @@ -49,7 +49,7 @@ class ExtensionMethodSpec extends TestEnvironment with TestData with SubdivideSu
"val Some(col) = df.spatialKeyColumn" should compile
}
}
describe("RasterFrameLayer exention methods") {
describe("RasterFrameLayer extension methods") {
it("should provide spatial key column") {
noException should be thrownBy {
rf.spatialKeyColumn
Expand Down Expand Up @@ -124,6 +124,10 @@ class ExtensionMethodSpec extends TestEnvironment with TestData with SubdivideSu

val md3 = rf.toMarkdown(truncate=true, renderTiles = false)
md3 shouldNot include("<img")

// Should truncate JTS types even when we don't ask for it.
val md4 = rf.withGeometry().select("geometry").toMarkdown(truncate = false)
md4 should include ("...")
}

it("should render HTML") {
Expand Down