Per-feature post-processing #6476

bagnell · 2018-04-20T23:05:24Z

NOTE this is targeting the post-processing-1.0 branch.

Adds per-feature post-processing. Each post-process stage has a selectedFeature property that is an array of primitives (Model, Primitive, Billboard, etc.) or Cesium3DTileFeatures. The only two stages with built-in support are the silhouette and black and white stages. Any classification primitives are not supported.

See the Per-Feature Post Processing and 3D Tiles Feature Picking Sandcastle examples.

TODO:

~~Add support for entities.~~
Fix tests.
Add tests.
Update CHANGES.md

…stle example for picking.

…ample.

cesium-concierge · 2018-04-20T23:05:25Z

@bagnell, thanks for the pull request! Maintainers, we have a signed CLA from @bagnell, so you can review this at any time.

⚠️ I noticed that CHANGES.md has not been updated. If this change updates the public API in any way, fixes a bug, or makes any non-trivial update, please add a bullet point to CHANGES.md and comment on this pull request so we know it was updated. For more info, see the Pull Request Guidelines.

I am a bot who helps you make Cesium awesome! Contributions to my configuration are welcome.

🌍 🌎 🌏

bagnell · 2018-04-20T23:09:05Z

@mramato Is there a way to get a primitive from an entity? I'd like to support adding an entity or a *Graphics object to the selectedFeatures array.

lilleyse · 2018-04-26T21:44:14Z

Another suggested TODO, but could be in another PR:

Remove silhouetting from Model.js because an edge post process can be used instead.

lilleyse · 2018-04-26T22:18:26Z

I got about half way through reviewing and should finish tomorrow.

But ignoring the actual post processing aspect, which I haven't looked at yet, I really love this PR so far. The fact that we no longer have to worry about creating separate draw and pick commands/shaders for everything is really nice!

My one question so far is whether there are concerns about performance, like if we had any pick shaders that were optimized to do picking fast. I didn't notice any right away but you might have a better idea. I suppose if we needed the fast path in the future we could have a GLSL define for pick pass vs. draw pass (or something like that).

bagnell · 2018-04-27T16:46:00Z

My one question so far is whether there are concerns about performance, like if we had any pick shaders that were optimized to do picking fast.

There are two possible pick shaders that can be generated. If the fragment shader has any discards, it's the same shader with the color changed to the pick color. Otherwise, a new shader is generated just writes the pick color.

bagnell · 2018-05-11T19:47:06Z

Is it easy to add a configurable silhouette edge length?

Not really. Edge detection finds the edge. We could add another pass to expand the edges, but the filter would be twice the length. For example, if you want a width of 5 pixels, that would be a 10x10 filter or 100 texture reads per fragment.

…re not supported in the Sandcastle example.

…ssing Sandcastle example.

bagnell · 2018-05-11T23:28:03Z

Would it be possible to add silhouette support to translucent objects?

I need to think about this. It doesn't work right now because translucent geometry doesn't write depth.

bagnell · 2018-05-11T23:30:31Z

@lilleyse Updated.

Everything besides translucency, treating a whole tileset as a feature, and edge length has been fixed. The whole tileset as a feature and maybe translucency are doable, but I would open issues for them.

bagnell · 2018-05-15T19:27:43Z

@lilleyse I re-targeted this to merge in master since the post-processing branch was merged.

lilleyse · 2018-05-18T19:02:16Z

Everything besides translucency, treating a whole tileset as a feature, and edge length has been fixed. The whole tileset as a feature and maybe translucency are doable, but I would open issues for them.

Ok, yeah open issues for those two. Edge length doesn't sound worth it.

The recent fixes work well. Given the large architectural changes let's merge after the June release.

bagnell · 2018-06-04T19:09:03Z

@lilleyse This is ready. After this is merged, I'll open the issues.

EDIT: Opened #6656 and #6657.

lilleyse · 2018-06-04T21:40:49Z

Just did a quick check to make sure everything was still functioning. Thanks @bagnell.

pjcozzi · 2018-06-09T19:06:02Z

Late to the party, but still going to add a few comments here.

pjcozzi · 2018-06-09T19:06:43Z

Are the dots expected on my Macbook/Intel?

http://localhost:8080/Apps/Sandcastle/index.html?src=3D%20Tiles%20Feature%20Picking.html

pjcozzi · 2018-06-09T19:08:11Z

Is the smearing here expected?

pjcozzi · 2018-06-09T19:11:37Z

Is this Sandcastle example - and silhouettes in general - supposed to work in 2D? It's OK if it isn't, but let's document it if not already done.

pjcozzi · 2018-06-09T19:14:18Z

Apps/Sandcastle/gallery/Per-Feature Post Processing.html

+    }
+});
+
+var collection = viewer.scene.postProcessStages;


collection?

Be precise.

What should be the canonical name used throughout? stages, right?

pjcozzi · 2018-06-09T19:15:45Z

Apps/Sandcastle/gallery/Per-Feature Post Processing.html

+var blackAndWhite = collection.add(Cesium.PostProcessStageLibrary.createBlackAndWhiteStage());
+blackAndWhite.uniforms.gradations = 5.0;
+
+if (!silhouette.isSupported(viewer.scene)) {


Isn't it odd that a user needs to create an instance to check isSupported? Is there clean and obvious way to avoid this with static isSupported somewhere appropriate like GroundPrimitive.isSupported?

How else would it work? There isn't a class for each of the stages. If there was, we could do something like Silhouette.isSupported.

Maybe:

if (!Cesium.PostProcessStageLibrary.isSilhouetteSupported(viewer.scene)) {

Slightly more coupled, but cleaner for the end user.

@bagnell do you plan on adding this in #6680?

Yup, db5cb75

pjcozzi · 2018-06-09T19:17:23Z

Apps/Sandcastle/gallery/Per-Feature Post Processing.html

+    handler.setInputAction(function(movement) {
+        var pickedObject = viewer.scene.pick(movement.endPosition);
+        if (Cesium.defined(pickedObject)) {
+            stage.selectedFeatures = [pickedObject.primitive];


Are we sure selectedFeatures is the right term? What all can this be? A primitive, entity, 3D tileset feature?

Maybe just selected or something like appliesTo?

pjcozzi · 2018-06-09T19:20:02Z

CHANGES.md

+### 1.47 - 2018-07-02
+
+##### Additions :tada:
+* `PostProcessStage` has a `selectedFeatures` property which is an array of primitives used for selectively applying a post-process stage. In the fragment shader, use the function `bool czm_selected(vec2 textureCoordinates` to determine whether or not the stage should be applied at that fragment.


In the fragment shader, use the function bool czm_selected(vec2 textureCoordinates to determine whether or not the stage should be applied at that fragment.

This is too much detail for CHANGES.md. Just have this documented where needed. Maybe a Sandcastle example with a custom stage as well.

pjcozzi · 2018-06-09T19:21:09Z

CHANGES.md

+
+##### Additions :tada:
+* `PostProcessStage` has a `selectedFeatures` property which is an array of primitives used for selectively applying a post-process stage. In the fragment shader, use the function `bool czm_selected(vec2 textureCoordinates` to determine whether or not the stage should be applied at that fragment.
+   * The black-and-white and silhouette stages have per-feature support.


Provide the specific name of the stage in CesiumJS.

Also, what is the plan for per-feature bloom - isn't that where we want to be? What is stopping us?

I don't think per-feature bloom makes sense. With HDR, you could change the luminance so certain geometry has bloom while other don't.

If a user wants to implement mouse-over glow for an object that bleeds into the scene on the edges, how do they do it? We expose luminance as part of the material and they just change it? Does it need a blur post-process?

pjcozzi · 2018-06-09T19:49:41Z

Source/Scene/Scene.js

+            return;
+        }
+
+        if (scene._logDepthBuffer && defined(command.derivedCommands.logDepth)) {


Minor, but why not var derivedCommands = command.derivedCommands to remove a lot of redundancy below?

bagnell · 2018-06-12T20:42:55Z

Is the smearing here expected?

The smearing is expected.

Are the dots expected on my Macbook/Intel?

The dots are not.

pjcozzi · 2018-06-12T21:31:42Z

Are the dots expected on my Macbook/Intel?

The dots are not.

If this is specific to Mac/Intel, probably OK to just submit an issue.

lilleyse · 2018-06-14T13:36:23Z

I see some dots as well. Seems to be from some of the small seams in the geometry.

I wonder... could the silhouette shader be changed so that it outlines only the edge of the feature and not the interior sections? If the post process shader has access to the id texture it could ignore checking depths for pixels that share the same id as the current pixel. This would fix both the dots and the smearing.

lilleyse · 2018-06-19T20:07:37Z

@bagnell curious if you have any thoughts on this approach.

bagnell added 24 commits February 5, 2018 15:01

Per-feature post-processing WIP.

601841c

Per-feature post-process WIP.

0875b17

Per-feature post-process [skip ci].

43d2b45

Merge branch 'post-processing-1.0' into per-feature-post-process

6a159eb

Remove render state override.

c6781a3

Add ability to have multiple silhouette colors.

1766464

Add black and white selected feature option.

77826df

Fix silhouette artifact. Add mouse over black and white example.

d36507e

Fix single silhouetted example. Add Sandcastle example.

8c4d392

Only render pick id when post process has selected features.

82bf70b

Use derived pick commands for billboards.

8408c7f

Update picking for 3D Tiles.

bca8eda

If command is not pickable, write depth.

1599d31

Remove more picking code in favor of derived commands. Add dev Sandca…

e5ca775

…stle example for picking.

Disable blending for pick render state.

5616502

Use derived pick commands for vector tile polylines.

6879b96

Use derived commands for primitives (not classification).

cb38405

Merge branch 'post-processing-1.0' into per-feature-post-process

d1dd710

Fixes after merge.

4daa234

Update ModelInstanceCollection WIP. Update picking test sandcastle ex…

98f280b

…ample.

Add fix and example for model instancing.

8f2d660

Fix point cloud picking and add example.

eb66871

Add vector tile picking examples.

fc19067

Add private pick id getter to primitives.

cefe1d8

bagnell mentioned this pull request May 2, 2018

Post processing #5615

Merged

12 tasks

bagnell added 3 commits May 11, 2018 16:06

Fix silhouettes with alpha and add console warning when silhouettes a…

8959e7c

…re not supported in the Sandcastle example.

Fix crash when switching between stages in the per-feature post-proce…

96d3916

…ssing Sandcastle example.

Fix crashes and remove unneeded code.

f6ccb68

bagnell changed the base branch from post-processing-1.0 to master May 15, 2018 19:04

bagnell added 3 commits May 15, 2018 15:07

Merge branch 'master' into per-feature-post-process

d88c2ee

Fix after merge.

d01201d

Update CHANGES.md.

664f72f

Merge branch 'master' into per-feature-post-process

d38bbbf

Merge branch 'master' into per-feature-post-process

428abde

This was referenced Jun 4, 2018

Add support for silhouette on translucent objects #6656

Open

Add per feature post processing to an entire 3D Tileset #6657

Open

lilleyse merged commit 65a382c into master Jun 4, 2018

lilleyse deleted the per-feature-post-process branch June 4, 2018 21:40

pjcozzi reviewed Jun 9, 2018

View reviewed changes

bagnell mentioned this pull request Jun 12, 2018

Post process updates #6680

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Per-feature post-processing #6476

Per-feature post-processing #6476

bagnell commented Apr 20, 2018 •

edited

Loading

cesium-concierge commented Apr 20, 2018

bagnell commented Apr 20, 2018

lilleyse commented Apr 26, 2018

lilleyse commented Apr 26, 2018

bagnell commented Apr 27, 2018

bagnell commented May 11, 2018

bagnell commented May 11, 2018

bagnell commented May 11, 2018

bagnell commented May 15, 2018

lilleyse commented May 18, 2018

bagnell commented Jun 4, 2018 •

edited

Loading

lilleyse commented Jun 4, 2018

pjcozzi commented Jun 9, 2018

pjcozzi commented Jun 9, 2018

pjcozzi commented Jun 9, 2018

pjcozzi commented Jun 9, 2018

pjcozzi Jun 9, 2018

pjcozzi Jun 9, 2018

bagnell Jun 12, 2018

pjcozzi Jun 12, 2018

lilleyse Jun 14, 2018

bagnell Jun 15, 2018

pjcozzi Jun 9, 2018

pjcozzi Jun 9, 2018

pjcozzi Jun 9, 2018

bagnell Jun 12, 2018

pjcozzi Jun 12, 2018

pjcozzi Jun 9, 2018

bagnell commented Jun 12, 2018

pjcozzi commented Jun 12, 2018

lilleyse commented Jun 14, 2018

lilleyse commented Jun 19, 2018

Per-feature post-processing #6476

Per-feature post-processing #6476

Conversation

bagnell commented Apr 20, 2018 • edited Loading

cesium-concierge commented Apr 20, 2018

bagnell commented Apr 20, 2018

lilleyse commented Apr 26, 2018

lilleyse commented Apr 26, 2018

bagnell commented Apr 27, 2018

bagnell commented May 11, 2018

bagnell commented May 11, 2018

bagnell commented May 11, 2018

bagnell commented May 15, 2018

lilleyse commented May 18, 2018

bagnell commented Jun 4, 2018 • edited Loading

lilleyse commented Jun 4, 2018

pjcozzi commented Jun 9, 2018

pjcozzi commented Jun 9, 2018

pjcozzi commented Jun 9, 2018

pjcozzi commented Jun 9, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bagnell commented Jun 12, 2018

pjcozzi commented Jun 12, 2018

lilleyse commented Jun 14, 2018

lilleyse commented Jun 19, 2018

bagnell commented Apr 20, 2018 •

edited

Loading

bagnell commented Jun 4, 2018 •

edited

Loading