Define publicly available BSDF data file for comparative tests of builds

[mkae]

A publicly available BSDF dataset for crosschecks of different builds (OSX, Linux, Windows, 32 bits, 64 bits, etc.) should be defined. This would help figuring out possible bottlenecks in BSDFProcessor (e.g. file loading or delays after changing of parameters as observed for OSX 10.10, 32 bit Vista and 64 W7) and allow unit testing of libbsdf.

[mkae]

If I load MERL's "green-fabric.binary" I get this on my MSYS2-based Windows Vista 32 bit console running on a Laptop powered by a T5900 Intel Core Duo CPU:

[ReflectanceCalculator::computeReflectances] 42.7656(s)
[scene_util::updateWavelength] 23.323(s)
[MainWindow::openFile] fileName: C:/msys32/home/Marko_2/MERL/green-fabric.binary (79.972(s))

Which times are you seeing on your end, @KimuraRyo?

[mkae]

Adding -O3 to CMAKE_CXX_FLAGS for BSDFProcessor already results in a speedup of the two latter computations by almost a factor of 2:

[ReflectanceCalculator::computeReflectances] 37.1265(s)
[scene_util::updateWavelength] 11.4038(s)
[MainWindow::openFile] fileName: C:/msys32/home/Marko_2/BSDF/MERL/green-fabric.binary (44.642(s))

Another drastic increase of speed by factors of 3 up to 10 one gets with setting the -O3 for CMAKE_C_FLAGS and CMAKE_CXX_FLAGS for libbsdf and BSDFProcessor:

[ReflectanceCalculator::computeReflectances] 4.62197(s)
[scene_util::updateWavelength] 3.30603(s)
[MainWindow::openFile] fileName: C:/msys32/home/Marko_2/BSDF/MERL/green-fabric.binary (11.623(s))

A noticeably faster calculation (3-4s) when e.g. toggling the Log plot checkbox or selecting another Display mode is also observed.

Perhaps there's even more possibility for increasing performance?

I guess the cause for this is, that MSVC already switches on a variety of optimisations which aren't active by default on MSYS2...

[aphirst]

Does -O3 provide noticeable performance improvements over -O2 ? I only ask as, as much as I love -O3, it can sometimes cause problems which can be awkward to debug if you forget you have it set.

[mkae]

-O2 is as good as -O3:

[ReflectanceCalculator::computeReflectances] 4.63651(s)
[scene_util::updateWavelength] 3.51107(s)
[MainWindow::openFile] fileName: C:/msys32/home/Marko_2/BSDF/MERL/green-fabric.binary (12.639(s))

:)

[mkae]

So, this is on a late-2013 iMac i7:

[ReflectanceCalculator::computeReflectances] 1.09656(s)
[scene_util::updateWavelength] 0.828403(s)
[MainWindow::openFile] fileName: /Users/marko/green-fabric.binary (1.986(s))

being another 4-6 times faster. 👍 (Parallelization is doing its job nicely.)

[aphirst]

@mkae Mmm, well, if -O2 is as good as, or almost as good as, -O3, it's probably a good idea to use that for builds.

[mkae]

As pointed out in PR #32 it might be a good idea to make use of CMAKE_BUILD_TYPE=Release and CMAKE_C(XX)_FLAGS_RELEASE="-O2"...

[mkae]

This is on W7 on an i5 650 @ 3.2 GHz:

[ReflectanceCalculator::computeReflectances] 0.587973(s)
[scene_util::updateWavelength] 0.689205(s)
[MainWindow::openFile] fileName: H:/green-fabric.binary (2.842(s))

Toggling log: 6s

being even noticeably faster than on the i7 running OSX above.

[mkae]

This is a Xeon X5560 @ 2.8 GHz:

[ReflectanceCalculator::computeReflectances] 0.149092(s)
[scene_util::updateWavelength] 0.193181(s)
[MainWindow::openFile] fileName: C:/Temp/green-fabric.binary (0.841(s))

Toggling log: 1s

[paraphoton]

I loaded the "green-fabric.binary" to the version 1.1.4 and found nothing special. What I only found is editing takes a bit moment to complete, which may be sufficiently long (5 sec or more?) for users start to think that the program stopped running.

I checked it:

• WIndows 10 Pro x64
• BSDFProcessor 1.1.4
• CPU: Core i7-5600U

Sorry but I don't know how to see the numerical timing as you have presented in this issue. Hope this info helps.

[mkae]

Thanks for testing, @paraphoton, the debug output can be seen on the console, if the application was started from one that is. At least on OSX, WVista and W7.

But from #33 I figure that Windows 10 doesn't show the requested information to you at all... So, I am afraid there is nothing you can do, if I understood correctly, @KimuraRyo!

[paraphoton]

Actually, I launched the program from within the console on Windows 10. But I see no message there.

[mkae]

Yeah, that's what @KimuraRyo wrote in #33. :-(
I don't have a W10 to test on.

[mkae]

Finally I tested the 1.1.4 binaries supplied by you on GitHub on my T5900 Intel Core Duo CPU:

[ReflectanceCalculator::computeReflectances] 2.11996(s)
[scene_util::updateWavelength] 2.31179(s)
[MainWindow::openFile] fileName: C:/msys32/home/Marko_2/BSDF/MERL/green-fabric.binary (6.245(s))

Compared to the above for this machine (see 3rd post of this thread) your binary is another 2 times faster than the one built using MSYS2 with only -O3 set.

So, I guess I still need to find the right switches to make MSYS2's gcc build faster executables, or try to use clang instead...

[mkae]

This is a Xeon X5560 @ 2.8 GHz now with your 1.1.4 binaries:

[ReflectanceCalculator::computeReflectances] 0.271621(s)
[scene_util::updateWavelength] 0.240701(s)
[MainWindow::openFile] fileName: C:/Temp/green-fabric.binary (1.144(s))

Toggling log: 1s

which - now surprisingly - seems a bit slower than the measurement above for the code built on the X5560...

So, it seems like it is beneficial to compile the executable on that hardware which it will be used on later!

Update: @KimuraRyo, on this 64-bit machine I do have spurious hangs of your 32-bit application when loading the test file.

[mkae]

BTW, @paraphoton, if you install a basic MSYS2 on your Windows 10 and start @KimuraRyo's executable from its MSYS2 shell you will get the desired output on the console.

[mkae]

@aphirst, can you perhaps also run a speed test on your system(s)?

[aphirst]

@mkae

Windows 7 Pro x64, MSYS2 x64, Intel i5-3320M, Intel HD 4000

[ReflectanceCalculator::computeReflectances] 0.746866(s)
[scene_util::updateWavelength] 0.809133(s)
[MainWindow::openFile] fileName: C:/Users/[redacted]/Downloads/green-fabric.binary (2.756(s))

Time required to toggle the "log" setting ~ 1s

[mkae]

Windows 10 Home, MSYS2 x64, Pentium Dual-Core T4500 @ 2.3GHz

[ReflectanceCalculator::computeReflectances] 4.10584(s)
[scene_util::updateWavelength] 3.43917(s)
[MainWindow::openFile] fileName: C:/Users/Marko_2/Documents/green-fabric.binary (15.529(s))

Time required to toggle the "log" setting ~ 5s

[mkae]

This is on a Intel Core i5-3320M @ 2.60 GHz

[ReflectanceCalculator::computeReflectances] 0.60644(s)
[MainWindow::updateWavelength] 0.867574(s)
[MainWindow::openFile] fileName: C:/DRIVES/P/BSDFs/green-fabric.binary (3.263(s))

Time required to toggle the "log" setting ~ 1s

as installed from MSYS2's binary repo, i.e. not built on this specific hardware.

[github-actions]

Stale issue message

[aphirst]

@KimuraRyo I am not sure whether you were informed of this at the time - perhaps I did via email, perhaps someone else did, or perhaps not - but it is with great regret that I must inform you of @mkae 's passing in February of 2017. Although I no longer work at the same company (or in the same field), he continues to be very sorely missed.

[KimuraRyo]

I did not know it. Thank you for letting me know. The last time I communicated with him was in January of 2017. I regret that I cannot thank him directly for his support. May he rest in peace.