ParticleFX2 tasks

[darksylinc]

• Installation sets msHasParticleFX2Plugin
• Setting msHasParticleFX2Plugin updates all Hlms
• Implement:
  • Point
  • Oriented common
  • Oriented self
  • Perpendicular common
  • Perpendicular Self
• Rotation Type
• Support colour
• Port Emitters
• Port Affectors
• Test extractS16 & extractS8
• Parse Particle scripts
• Port C & NEON Mathlib
• ArrayVector3::collapseMin is buggy on NEON
• Remove ArrayInterQuaternion
• Unlit works on all APIs
• Works on PBS
• Actually multithread and handle RNG from multiple threads.
• Use lower slot for SSBO since it has a bug on Adreno 650
• Port multiplier mode to ScaleAffector in PFX1.
• Old plugin kept a list of ParticleSys, Emitters and Affectors in their factories. Must be removed.
• Add sample
• initEmittedParticles ignores ParticleSystem2's transform at the time of emission
• There is much vertex waste at the beginning and end of each drawcall. See if it can be optimized.
• Blue Noise works on all APIs
• Works fine with shadows
• Sort ParticleSystems by distance to camera
• Planar Reflections is broken by BlueNoise
• ScaleInterpolatorAffector2 needs to know default scale
• Works fine with multiple lights
• BillboardSet can be created via SceneManager
• BillboardSet appears on screen
• ParticleSystemManager2::_getSharedIndexBuffer does not account that BillboardSet's quota may be set later.

[cryham]

Looks awesome, very promising.
What do you think of adding a final demo/sample with Soft Particles implemented? Would be useful for all right?

[darksylinc]

My aim is the following:

1. Provide a near functional replacement for the current ParticleFX that can parse existing scripts. However some features may be hard to implement in this PFX2 version, thus the "old" ParticleFX plugin won't be deprecated or removed (unless I somehow manage to implement all features, what looks particularly difficult is the local_space property, individual particle sorting, and emitting emitters)
2. Multithreaded and SIMD-friendly
3. Insane level of batching. The old PFX plugin ends up with far too many batches (basically 1 per particle system). This system will have 1 draw call per type of particle
4. Quotas work differently. On PFX1 the quota is per particle system. On PFX2 the quota is total and shared for all instances of the same type.
   • There is no need for frustum culling.
   • Particle System instances will be sorted by distance to camera and given priority for emission
   • Particle Systems that are far away eventually run out of particles as the last ones they emitted have died; and can't replenish more because the instances closer to camera are exhausting (hoarding) all the quota
   • This means deactivation is no longer a concern
   • If the quota has not been exhausted, then far away particles will continue to be emitting normally regardless of distance
   • This is not a problem: the quota is supposed to mean how much the system can handle. If you set a quota of 1000, then the system should be able to handle 1.000 particles, whether they're off-screen or on-screen, or whether it's 3 instances using 300 each, or 100.000 instances using the first available 1.000.
5. We use vertex pulling and generate the geometry on the vertex shader procedurally, which means we only have to upload the data once per frame, heavily reducing BW and RAM needed
   • PFX1 must reupload the particles on every pass; because (except for BBT_PERPENDICULAR_COMMON & BBT_PERPENDICULAR_SELF) their geometry is relative to camera
   • We may still need to have a fallback path to do this on the CPU (PFX1 style) depending on how it performs on mobile (I already hit a bug on Adreno 650)
6. Stretch goal: Be able to emit other particle FXs (which is way better than emitting emitters)
7. Surprising benefit: during this task I've noticed some useless inefficiencies in PFX1 (they're listed above) that cause O(N²) degradation when removing particles; this can be fixed in PFX1.

The idea is that you should be able to replace most particle FXs with the new system, and those that can't just run it on the older system.

The new system is an extreme take on efficiency, while keeping backwards compatibility with 90-95% of the functionality.

I may improve it further (one of the reasons I didn't want to enhance PFX1 was its massive performance limitations); but for now soft particles are not one of the goals.

Probably the biggest missing feature that PFX1 has will be individual particle sorting. Perhaps this can be countered with OIT techniques like WBOIT.

[cryham]

Okay good to know. I'm glad there will be less batches and more particles possible.
But you didn't actually answer my question about Soft particles implementation (like standard thing fixing hard edges since years), which I don't exactly know how to implement and would benefit greatly if it was already done in a sample/demo.

Well no individual particle sorting is sad for me, it means dust and clouds will look bad. But if OIT was already included in sample then great. Otherwise I doubt I could even think of that in future.
I would be thrilled if there was even particle sorting across particle systems. This would fix some issues I had on few maps that had a few of different smoke/dust areas overlapping and visibly switching then.
I guess I should go into 3d volumetric (clouds etc) but I don't even know how to start with this: do I render a cube with lots of slices, how do I know its front and back distance at once in shader?

[darksylinc]

But you didn't actually answer my question about Soft particles implementation

I... did? I said "...but for now soft particles are not one of the goals."

Well no individual particle sorting is sad for me, it means dust and clouds will look bad
But if OIT was already included in sample then great. Otherwise I doubt I could even think of that in future.

Indeed, we'll cross that bridge when the implementations has the bare fundamentals running :)

guess I should go into 3d volumetric (clouds etc) but I don't even know how to start with this: do I render a cube with lots of slices, how do I know its front and back distance at once in shader?

There are many ways. Currently the most efficient ones are using SDF (Signed Distance Fields) and Compute Shaders. Not basic topics though.

[cryham]

for now soft particles are not one of the goals.

Ah I see, pity, without this my dust will look bad in any system.

SDF (Signed Distance Fields) and Compute Shaders. Not basic topics though. Not basic topics though.

Right. But SDF so far I know is to create the clouds themself, but IDK how to even put that cloud's bounding box in scene with other 3D stuff. Or maybe it would be without geometry (meshes) even? So all shaders having new code for it like fog, and shapes computed with SDF so all shaders just reading e.g. sphere position, radius for cloud areas? Hmm. Well, either way it's probably costly to render.

[darksylinc]

Right. But SDF so far I know is to create the clouds themself,

No, SDF is how you render them. You use ray marching and SDF just accelerates the ray march. Here's a (very advanced) example.

Here's a more simple shader where a sphere is rendered via SDF (no geometry, no vertices; all raymarching and SDF)

[darksylinc]

The main difference between that sample and an actual SDF cloud renderer is that in the sphere example the SDF value comes from GetMinSceneDistanceFromPoint(t) (since the SDF of a sphere can be easily derived mathematically) while the SDF of a cloud system comes from a precalculated 3D texture.

[cryham]

Right. Thanks for info. I think I'll try it someday, I enjoyed those demos since a while, it would be cool to drive through such clouds.

And for soft particles? Do I need to split rendering into 2 workspaces, e.g. first draw all (or only depth?) except particles, then draw particles on top using that distance to fade them close? Is this right, and is it costly?

[cryham]

Hi, just curious: what is the status on this? Is it far from finish and merge?
I checked out the demo and it was working fine for me, on Debian with GL3+.

I also saw some very nice new tutorials:
Add Hlms tutorial 02
Add Hlms tutorial
strangely done on this branch, why not on master?

BTW, wanted to also ask: is it possible to have random size particles at start?
There are width and height in

particle_system Cloud1
{
	material			Cloud1
	quota				1000
	particle_width		122
	particle_height		122
	..

What I'm missing here is some range e.g. factors scale_min, scale_max that would randomly scale width and height when emitting. I know of affector Scaler but it works later.

[darksylinc]

Hi, just curious: what is the status on this? Is it far from finish and merge?

I am waiting on two things:

1. I need to release 3.0 so I can make warm_up the new master branch, and current master be renamed to 3.0 branch.
2. A client project which is now focusing on something else (which is using warm_up). The plan is to eventually integrate PFX2 so it can be better tested.

I also saw some very nice new tutorials:
Add Hlms tutorial 02
Add Hlms tutorial
strangely done on this branch, why not on master?

Hlms has had some API changes in warm_up to add multithreaded shader generation and PSO generation (which btw I need to document), and I didn't want to have to deal with the changes (basically merge conflicts).

What I'm missing here is some range e.g. factors scale_min, scale_max that would randomly scale width and height when emitting. I know of affector Scaler but it works later.

On PFX2 it should be easy to support.
I don't know about PFX1, because it has two modes of operation (all particles have same size vs each particle has its own size). It shouldn't be impossible though.

[cryham]

Hi. Any update, on status of this?
I thought it'd take a couple of weeks but it's been already few months 😟 .