Update geomaps to allow dataframes as input. #1187

MichaelMauderer · 2021-02-09T15:22:53Z

Pull Request Description

Allow Tables to feed the Geo Maps visualization. Uses the columns longitude, latitude and label.
Extracts the data through an Enso preprocessor.

Important Notes

Also, some refactoring of the Geo Maps code.

Checklist

Please include the following checklist in your PR:

The CHANGELOG.md was updated with the changes introduced in this PR.
The documentation has been updated if necessary.
All code conforms to the Rust style guide.
~~All code has automatic tests where possible.~~
All code has been profiled where possible.
All code has been manually tested in the IDE.
~~All code has been manually tested in the "debug/interface" scene.~~
All code has been manually tested by the PR owner against our test scenarios.
All code has been manually tested by at least one reviewer against our test scenarios.

farmaazon · 2021-02-11T14:51:13Z

src/rust/ide/view/graph-editor/src/builtin/visualization/java_script/geoMap.js

+ * Can also consume an array of `Geo_Point` structures, or a dataframe that has the columns
+ * `latitude`, `longitude` and `label`.


Do we have a Geo_Point structure in Enso stdlib? Some time ago we didn't.

No, we don't It has to be user defined. This is not new, just explicitly stating the current behaviour.

Ok, so let's put the description how the Geo_Point should look like.

We should remove Geo_Point support in this PR. It was a hack before maps support for tables. Please confirm it with Sylwia, but it should be dropped already

Is removed.

farmaazon · 2021-02-11T14:52:46Z

src/rust/ide/view/graph-editor/src/builtin/visualization/java_script/geoMap.js

+                'df ->\n' +
+                    "    columns = df.select ['label', 'latitude', 'longitude'] . columns\n" +
+                    "    serialized = columns.map (c -> ['df_' + c.name, c.to_vector])\n" +
+                    '    Json.from_pairs serialized . to_text'


How it is working for "old" datatypes like array of GeoPoints, or just json string?; As I see it assumes that df has select and map method.

I miss checking if type of df is a Table.

Right now if the expression fails, we get the original data. Once we want to support multiple data types we can use case to transform depending on the input type.

Ok, but that is an obscure feature I would say, and I feel it could change easily. So I would put here the case for Table explicitly and make df.to_json.to_text in other cases.

Is refactored.

CHANGELOG.md

Co-authored-by: Adam Obuchowicz <adam.obuchowicz@luna-lang.org>

CHANGELOG.md

wdanilo · 2021-02-11T16:07:23Z

src/rust/ide/view/graph-editor/src/builtin/visualization/java_script/geoMap.js

+ * Can also consume an array of `Geo_Point` structures, or a dataframe that has the columns
+ * `latitude`, `longitude` and `label`.


We should remove Geo_Point support in this PR. It was a hack before maps support for tables. Please confirm it with Sylwia, but it should be dropped already

src/rust/ide/view/graph-editor/src/builtin/visualization/java_script/geoMap.js

…n review feedback.

farmaazon · 2021-02-12T16:37:47Z

On my machine the example with json string does not work.

farmaazon

And when I create such table, I don't see any points on the map.

farmaazon · 2021-02-12T14:58:33Z

src/rust/ide/view/graph-editor/src/builtin/visualization/java_script/geoMap.js

@@ -10,7 +10,6 @@
 */
 const TOKEN =
    'pk.eyJ1IjoiZW5zby1vcmciLCJhIjoiY2tmNnh5MXh2MGlyOTJ5cWdubnFxbXo4ZSJ9.3KdAcCiiXJcSM18nwk09-Q'
-const GEO_POINT = 'Geo_Point'
 const GEO_MAP = 'Geo_Map'


As I remember, Geo_Map we should remove as well?

wdanilo · 2021-02-15T09:43:51Z

src/rust/ide/view/graph-editor/src/builtin/visualization/java_script/geoMap.js

+/**
+ * Extract the visualisation data from a dataframe.
+ */
+function extractFromDataFrame(parsedData, preparedDataPoints, accentColor) {


I think the name of the function could be better. The docs tell about extracting VISUALIZATION DATA. Lets add it to the name of the function as the name doesn't tell what we are extracting there.

MichaelMauderer · 2021-02-15T14:23:10Z

And when I create such table, I don't see any points on the map.

This is actually correct. These points are really far apart. And the default radius is to small to be seen at that distance.

On my machine the example with json string does not work.

The example is broken. It has the coordinates for the viewport and the point to display switched. This has been fixed.

Allow Tables to feed the Geo Maps visualization. Uses the columns longitude, latitude and label. Extracts the data through an Enso preprocessor. Original commit: enso-org/ide@1ec89ee

feat: Update geomaps to allow dataframes as input.

78f2551

MichaelMauderer changed the title ~~feat: Update geomaps to allow dataframes as input.~~ Update geomaps to allow dataframes as input. Feb 9, 2021

MichaelMauderer added 2 commits February 9, 2021 16:26

doc: Update CHANGELOG.md.

f22cc7c

doc: Update visualisation doc string.

d9f6444

MichaelMauderer self-assigned this Feb 9, 2021

MichaelMauderer added Category: Visualizations Visualizations embedded in Enso IDE Difficulty: Core Contributor Should only be attempted by a core contributor Priority: High Should be scheduled as soon as possible Type: Enhancement An enhancement to the current state of Enso IDE labels Feb 9, 2021

MichaelMauderer added 2 commits February 10, 2021 14:32

feat: Allow labels to be undefined.

27eb6d9

lint: Reformat file.

0b1935e

MichaelMauderer marked this pull request as ready for review February 10, 2021 13:52

MichaelMauderer requested a review from wdanilo as a code owner February 10, 2021 13:52

MichaelMauderer requested a review from farmaazon February 10, 2021 13:52

farmaazon reviewed Feb 11, 2021

View reviewed changes

MichaelMauderer and others added 2 commits February 11, 2021 15:10

Fix typo in CHANGELOG.md

4015bb6

Co-authored-by: Adam Obuchowicz <adam.obuchowicz@luna-lang.org>

Merge branch 'develop' into wip/mm/ide-1055

655410a

MichaelMauderer requested a review from farmaazon February 11, 2021 15:12

wdanilo reviewed Feb 11, 2021

View reviewed changes

MichaelMauderer added 6 commits February 11, 2021 17:38

refactor: Update enso preprocessor to match on type.

8fa39bf

refactor: Update enso preprocessor to match on type. Refactor based o…

39e4923

…n review feedback.

doc: Update CHANGELOG.md.

592f3af

doc: Update CHANGELOG.md.

fddbe2c

Merge branch 'develop' into wip/mm/ide-1055

52e434a

style: Prettier.

cfdc548

MichaelMauderer requested a review from wdanilo February 12, 2021 15:30

farmaazon reviewed Feb 12, 2021

View reviewed changes

wdanilo approved these changes Feb 15, 2021

View reviewed changes

fix: Update example and center point calculation.

956a563

MichaelMauderer added 4 commits February 15, 2021 15:17

refactor: Expand function names.

a159cf1

refactor: Remove Geo_Map.

3e0958b

style: Prettier.

eece107

revert: Debug point size.

39f9838

MichaelMauderer requested a review from farmaazon February 15, 2021 14:36

MichaelMauderer added 2 commits February 16, 2021 09:35

fix: Remove console logging.

aba0eab

Merge branch 'develop' into wip/mm/ide-1055

5382e88

MichaelMauderer force-pushed the wip/mm/ide-1055 branch from c9b3e53 to 5382e88 Compare February 16, 2021 08:46

farmaazon approved these changes Feb 16, 2021

View reviewed changes

Merge branch 'develop' into wip/mm/ide-1055

7fc9be2

farmaazon merged commit 1ec89ee into develop Feb 16, 2021

farmaazon deleted the wip/mm/ide-1055 branch February 16, 2021 10:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update geomaps to allow dataframes as input. #1187

Update geomaps to allow dataframes as input. #1187

MichaelMauderer commented Feb 9, 2021 •

edited by farmaazon

farmaazon Feb 11, 2021

MichaelMauderer Feb 11, 2021

farmaazon Feb 11, 2021

wdanilo Feb 11, 2021

MichaelMauderer Feb 12, 2021

farmaazon Feb 11, 2021

MichaelMauderer Feb 11, 2021

farmaazon Feb 11, 2021

MichaelMauderer Feb 12, 2021

wdanilo Feb 11, 2021

farmaazon commented Feb 12, 2021

farmaazon left a comment

farmaazon Feb 12, 2021

MichaelMauderer Feb 15, 2021

wdanilo Feb 15, 2021

MichaelMauderer Feb 15, 2021

MichaelMauderer commented Feb 15, 2021

		* Can also consume an array of `Geo_Point` structures, or a dataframe that has the columns
		* `latitude`, `longitude` and `label`.

Update geomaps to allow dataframes as input. #1187

Update geomaps to allow dataframes as input. #1187

Conversation

MichaelMauderer commented Feb 9, 2021 • edited by farmaazon

Pull Request Description

Important Notes

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

farmaazon commented Feb 12, 2021

farmaazon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MichaelMauderer commented Feb 15, 2021

MichaelMauderer commented Feb 9, 2021 •

edited by farmaazon