Update BulletSharp to latest version #289

Eideren · 2018-12-10T18:08:51Z

Content of this post where edited for future references, you can find previous records on the edit history located inside the header of this post.

Original Description:
Our version of BulletSharp is a modified, extremely old, pre-github version of https://github.com/AndresTraks/BulletSharpPInvoke , this PR should fix dependencies and the likes to allow us to pull from a clean and up to date nuget version of BulletSharpPInvoke instead.
Xenko's version contains a bunch of methods and fields which will have to be replaced with workarounds, some of which don't have any straightforward substitute.

Location of the sources, build scripts, tools and instructions used for this PR's libs located over here:
https://github.com/Eideren/BulletSharpPInvoke

Kryptos-FR · 2018-12-10T22:33:49Z

I think our version is based on https://github.com/sinkingsugar/BulletSharpPInvoke which was indeed a fork of AndresTraks code. We might also use the mobile version from https://github.com/sinkingsugar/bullet2-sharp-mobile.

Maybe kindly ping @sinkingsugar 😄

xen2 · 2018-12-11T01:08:30Z

sources/core/Xenko.Core.Mathematics/Matrix.cs

+        [MethodImpl(MethodImplOptions.AggressiveInlining)]
+        public static implicit operator BulletSharp.Math.Matrix(Matrix value)
+        {
+            unsafe { return *(BulletSharp.Math.Matrix*)&value; }


Looking at https://github.com/AndresTraks/BulletSharpPInvoke/blob/master/BulletSharp/Math/Matrix.cs#L58
and https://github.com/xenko3d/xenko/blob/master/sources/core/Xenko.Core.Mathematics/Matrix.cs#L64

Xenko has a different memory layout than Bullet (M21 vs M12)

sinkingsugar · 2018-12-11T09:13:32Z

Hmm the API is slightly different, many things were done towards Xenko perf/optimization.
Can't remember all but like @xen2 said, Xenko matrices are different.
And SharpMotionState was a big one to optimize perf and collisions detection.
Lot's of changes were also in the C++ library itself!

This is just a wrapper, not sure you get any benefit by updating it unless the C++ part is also updated ( and has outstanding changes )

So unless the native library is also updated and properly tested on all platforms I would pay attention.

Eideren · 2018-12-11T14:08:24Z

Well, no idea how I ended up looking at xenko's matrix and miss the difference in their order.

Just to clarify, this PR wasn't made just to stay up to date with the wrapper, the main push behind this PR is the fact that the version used within xenko doesn't support multiple features of bullet, the main one being concave mesh collision.

From what I gather you guys don't have the sources for the C# and C++ side anymore, I can't do much about that without them so unless you do end up finding them I'll just continue on and we'll see where we end up.

xen2 · 2018-12-11T14:21:10Z

Sorry, didn't realize you were looking for the old source (esp. after @Kryptos-FR mentionned a repository which was almost the proper one, I thought you had it and were more focusing on how to rebuild from new sources!)

Here you go, it was there all along!
https://github.com/xenko3d/bullet2-sharp-mobile/tree/master/src
It contains both C++ and C#

xen2 · 2018-12-11T14:23:06Z

Now I understand better why you did a compare with decompiled code (at first I thought you had the sources but wanted to see everything formatted the same way... should have understood that earlier!)
Sorry for the misunderstanding!

xen2 · 2018-12-11T14:30:08Z

BTW, while I still have it on top of my head, some remarks about how those native libs are compiled so far:

We were usually building the sources for libbulletc and some other native libraries that needs target OS in manually in various computer/VM (hard for people to do it themselves from scratch since those VM had lot of specific software installed).

It would be great if we were switching to something more reproducible and easier to test.
For example, something that can either (1) run in CI, or (2) something that can setup a full docker instance with all dependencies (for Linux/Android, or similar system for Mac), or a combination of both.

That's probably something we should do for all those dependencies that can't be built using our C++ native layer directly in Xenko projects (which support only a very limited subset of the standard library) but instead need to be built on the target OS.

tebjan · 2018-12-11T14:34:19Z

Xenko matrices are different.

that is indeed a bit problematic. since directx as well es opengl and i guess bullet have exactly the same memory layout. i wonder what was leading to that decision and which action was taken. my feeling is that there was a couple of wrong assumptions of how matrix transposition and coordinate system handedness works.

xen2 · 2018-12-11T14:50:18Z

@tebjan moved Matrix discussion and my comments in #291

tebjan · 2018-12-11T14:59:24Z

thanks @xen2 , will need some time to get deeper into it. hope i will have time for this soon.

Eideren · 2018-12-11T16:43:27Z

Hey, thanks a bunch for the source @xen2 , 'BvhTriangleMeshShape' is already implemented on that repo, as well as a ton of other missing classes, could the dll included with xenko be out of date ?

xen2 · 2018-12-11T16:46:09Z

Doesn't seem to be included in the csproj:
https://github.com/xenko3d/bullet2-sharp-mobile/blob/master/src/BulletSharpPInvoke/BulletSharpPInvoke.csproj

Eideren · 2018-12-14T17:13:17Z

So, I ended up choosing to work on optimizing the main repo of BulletSharpPInvoke instead of keeping our own up to date, I already have posted a PR over there, we'll see how it works out.

All contact data in xenko is in world space right ?

xen2 · 2018-12-17T07:47:08Z

@Eideren Took a quick look at our code and bullet code and yes, it looks like all ContactPoint data is in world coordinates.

GutterLab · 2019-01-08T07:27:48Z

May want to consider using https://github.com/bepu/bepuphysics2 instead of bullet altogether.

tebjan · 2019-01-21T15:12:25Z

@GutterLab bepuphysics doesn't look as mature as bullet, from their readme:

It notably does not yet include:

Convex hulls, cylinders, cones, or other complex shapes

Dedicated heightmap terrain meshes

Mesh-mesh collision detection

Bounciness, other than the frequency/damping ratio tuning

Continuous collision detection, other than the speculative margin

Many useful constraint types

Eideren · 2019-01-31T19:05:09Z

I'm still working on this but BulletSharpPInvoke's maintainer seems pretty busy right now, we might have to fork it instead, we'll see.

Flavelius · 2019-02-01T08:03:54Z

To chime in on the different engine suggestions, PhysX could be a good consideration too, potentially providing some speedups though GPU support.

Kryptos-FR · 2019-02-01T08:31:32Z

potentially providing some speedups though GPU support

Isn't that only on NVidia graphic cards. If so, we'll have a kind of vendor-lock.

Flavelius · 2019-02-01T16:22:11Z

Afaik, it is. But having that optional, according to
https://store.steampowered.com/hwsurvey/videocard/ or
https://www.statista.com/chart/15172/the-pc-graphics-card-market-share-among-gamers-on-steam/
it could atleast benefit a good share of users. Although not all, yes.

xen2 · 2019-06-17T12:11:53Z

Quick update: after 3 long weeks of delays due to customs, I should receive all my stuff tomorrow.
Crossing finger that I have that "get/setdata with no command list" branch lying somewhere.

xen2 · 2019-06-18T22:30:40Z

OK, got the computer today!
I could extract the branch, rebase it (git didn't play so nice since it was on old Xenko repo without common ancestor and before all the renaming) and fix a few compile errors.
It really had the GetData and SetData without CommandList, so at least that part I remembered well!
Here it is: https://github.com/xenko3d/xenko/tree/d3d_multithreading

I didn't investigate (yet) the details of the PR as well as if & how to merge it.
Please note that there is a few things that are probably broken still (i.e. I didn't see a grid in the Game Studio).

Eideren · 2019-06-19T16:01:49Z

@xen2 You didn't get back to me on the issue with extracting vertex data from disk, I did prod around a bit to see if I could figure it out myself and from what I understood the data is uploaded to the GPU as soon as it is deserialized ? Without any proper way to hook ourselves into the system ?

I would love to help you out with the commandlist stuff but I don't have much experience with graphics api, so I won't be able to contribute significantly.

I think I should put the editor-side mesh collider code in a separate PR so that we can finally merge this one if you're ready to do so ?

xen2 · 2019-06-19T18:06:35Z

@Eideren Yes, the issue is that data is loaded to GPU memory right away.
Technically one could hook deserialization of buffer, but at the point it's called, there is not enough context info to know if the data needs to be set aside or not for something else.

Command list stuff would help in that you could get data from buffer without a command list, so you could just read back the GPU memory just like reading CPU memory.

Also, if everything is somewhat usable without editor-side mesh collider, we should definitely split this in another PR and merge the other part. Sorry that I didn't realize it was OK without!

Eideren · 2019-06-21T00:27:53Z

@xen2 There you go, I'll create the mesh collider PR from the master branch as soon as this one is merged in.

xen2 · 2019-06-21T22:27:56Z

@Eideren thanks, code looks good to me!
I will run a quick test to see if everything works good in practice and then merge it.

xen2 · 2019-06-22T17:03:10Z

I noticed a bug when trying the ThirdPersonPlatformer.
It happens only in specific area of the map.
Probably due to the raycast.
Here's a video of it:
https://youtu.be/DOEGVDvNUkI

Eideren · 2019-06-22T19:42:29Z

@xen2 I'll take a look at it

Eideren · 2019-06-25T13:45:53Z

Solved, the issue laid within ShapeSweep, that's what the TPP template uses by default for its camera collision, works fine now.

xen2 · 2019-06-26T14:43:24Z

Thanks.
I did some additional testing and everything looks good, LGTM for merge!

Maybe one last thing before doing so: could you add a short markdown file at deps\BulletPhysics\README.md that contains:

link to repo for BulletSharp.NetStandard.dll and libbulletc.dll (https://github.com/Eideren/BulletSharpPInvoke if I understood correctly)
which version is currently compiled (git commit or tag)
any extra info that might be useful to remember when trying to build it again (I suppose most of it is already in your BulletSharpPInvoke repo, in which case it's all good, but if you think of anything else...)

Maybe you could also either update or delete the batch files (esp. build.bat & checkout.bat -- sign.bat looks OK already) so that people are not confused trying to fetch different repo.

Eideren · 2019-06-26T15:25:27Z

@xen2 Sure I meant to deal with those but kind of forgot, thanks for reminding me.
So, sign.bat is as up to date as I can do on my end, not sure how all of that works but the paths should all be mapped correctly ( paradox.snk was swapped to xenko.public.snk but I don't know if the functionality, if any, is retained ).

I think you should probably fork/copy my repo to xenko3d's account and then modify checkout.bat and the new README.md to point to that fork instead.

xen2 · 2019-06-27T10:23:41Z

Merged, thanks again and sorry it was delayed (esp. I didn't realize this could be split in two to take care of the more delicate part later)

Eideren · 2019-06-27T10:30:35Z

@xen2 Nah, it's my fault for not mentioning it sooner :)

xen2 · 2019-06-27T10:44:32Z

@Eideren no worries, the delicate part should have been taken care of sooner too!
Anyway, I have made a fork of BulletSharpPInvoke in xenko org: https://github.com/xenko3d/BulletSharpPInvoke
I will give you full permissions on it.

Eideren added 4 commits December 10, 2018 17:56

Include nuget to proj

6d4eb5a

Straightforward changes

adf5366

Potentially harmful changes

fde0242

Minor proj update

f813753

xen2 reviewed Dec 11, 2018

View reviewed changes

Matrix conversion fix

222b702

xen2 mentioned this pull request Dec 11, 2018

Xenko matrix memory layout discussion #291

Open

Main conversion done, next up the rest of the CharacterComponent

f23ed46

Eideren added 3 commits December 18, 2018 14:18

Character component, waiting on bullet wrapper for the rest

0964856

Static mesh base

cbcc27d

Static Mesh: Update shape factories

5c11f1f

[Physics] Expose Capsule's readonly data

a6c8e90

Eideren added 2 commits June 21, 2019 02:23

[Physics] Provide read access to collider shape's data

0dda055

[Physics] Split static mesh collider into new PR

05421ae

[Physics] Renamed NewShapeFromDesc to CreateShape

d65a7f2

[Physics] Fix ShapeSweeps output

c527b17

Eideren changed the title ~~[WIP] Update BulletSharp to latest version~~ Update BulletSharp to latest version Jun 25, 2019

[Physics] Provide public access to callbacks' recycle and buffer

ae1f4b4

Eideren force-pushed the bulletsharp_nuget branch from 8f6cb37 to ae1f4b4 Compare June 26, 2019 07:49

[Physics] Clarify bullet lib's usage and sources

a09fa7e

[Physics] Update README.md with new BulletSharpPInvoke repo location

58e4e12

xen2 merged commit 5cee86c into stride3d:master Jun 27, 2019

Eideren added a commit to Eideren/xenko that referenced this pull request Jun 27, 2019

[Physics] Restore mesh implementation state from PR stride3d#289

26a0e5d

Eideren mentioned this pull request Jun 27, 2019

[WIP] [Physics] Restore mesh implementation state from PR #289 #479

Closed

7 tasks

Eideren mentioned this pull request Oct 24, 2020

Static Mesh (Trimesh) Collider For Stride #890

Closed

Eideren mentioned this pull request Dec 18, 2021

[Physics] Static mesh, asset and editor support #1251

Merged

7 tasks

Eideren deleted the bulletsharp_nuget branch September 30, 2023 13:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update BulletSharp to latest version #289

Update BulletSharp to latest version #289

Eideren commented Dec 10, 2018 •

edited

Kryptos-FR commented Dec 10, 2018 •

edited

xen2 Dec 11, 2018 •

edited

sinkingsugar commented Dec 11, 2018 •

edited

Eideren commented Dec 11, 2018 •

edited

xen2 commented Dec 11, 2018

xen2 commented Dec 11, 2018

xen2 commented Dec 11, 2018

tebjan commented Dec 11, 2018

xen2 commented Dec 11, 2018 •

edited

tebjan commented Dec 11, 2018

Eideren commented Dec 11, 2018

xen2 commented Dec 11, 2018

Eideren commented Dec 14, 2018 •

edited

xen2 commented Dec 17, 2018

GutterLab commented Jan 8, 2019

tebjan commented Jan 21, 2019 •

edited

Eideren commented Jan 31, 2019

Flavelius commented Feb 1, 2019

Kryptos-FR commented Feb 1, 2019

Flavelius commented Feb 1, 2019

xen2 commented Jun 17, 2019 •

edited

xen2 commented Jun 18, 2019

Eideren commented Jun 19, 2019

xen2 commented Jun 19, 2019

Eideren commented Jun 21, 2019

xen2 commented Jun 21, 2019

xen2 commented Jun 22, 2019

Eideren commented Jun 22, 2019

Eideren commented Jun 25, 2019

xen2 commented Jun 26, 2019 •

edited

Eideren commented Jun 26, 2019

xen2 commented Jun 27, 2019

Eideren commented Jun 27, 2019

xen2 commented Jun 27, 2019

Update BulletSharp to latest version #289

Update BulletSharp to latest version #289

Conversation

Eideren commented Dec 10, 2018 • edited

Kryptos-FR commented Dec 10, 2018 • edited

xen2 Dec 11, 2018 • edited

Choose a reason for hiding this comment

sinkingsugar commented Dec 11, 2018 • edited

Eideren commented Dec 11, 2018 • edited

xen2 commented Dec 11, 2018

xen2 commented Dec 11, 2018

xen2 commented Dec 11, 2018

tebjan commented Dec 11, 2018

xen2 commented Dec 11, 2018 • edited

tebjan commented Dec 11, 2018

Eideren commented Dec 11, 2018

xen2 commented Dec 11, 2018

Eideren commented Dec 14, 2018 • edited

xen2 commented Dec 17, 2018

GutterLab commented Jan 8, 2019

tebjan commented Jan 21, 2019 • edited

Eideren commented Jan 31, 2019

Flavelius commented Feb 1, 2019

Kryptos-FR commented Feb 1, 2019

Flavelius commented Feb 1, 2019

xen2 commented Jun 17, 2019 • edited

xen2 commented Jun 18, 2019

Eideren commented Jun 19, 2019

xen2 commented Jun 19, 2019

Eideren commented Jun 21, 2019

xen2 commented Jun 21, 2019

xen2 commented Jun 22, 2019

Eideren commented Jun 22, 2019

Eideren commented Jun 25, 2019

xen2 commented Jun 26, 2019 • edited

Eideren commented Jun 26, 2019

xen2 commented Jun 27, 2019

Eideren commented Jun 27, 2019

xen2 commented Jun 27, 2019

Eideren commented Dec 10, 2018 •

edited

Kryptos-FR commented Dec 10, 2018 •

edited

xen2 Dec 11, 2018 •

edited

sinkingsugar commented Dec 11, 2018 •

edited

Eideren commented Dec 11, 2018 •

edited

xen2 commented Dec 11, 2018 •

edited

Eideren commented Dec 14, 2018 •

edited

tebjan commented Jan 21, 2019 •

edited

xen2 commented Jun 17, 2019 •

edited

xen2 commented Jun 26, 2019 •

edited