Allow bfloat16 computations on compatible CPUs with Intel Extension for PyTorch

[AngryLoki]

Modern CPUs have native AVX512 BF16 instructions, which significantly improves matmul and conv2d operations.

With Bfloat16 instructions UNET steps are 40-50% faster on both AMD and Intel CPUs.
There are minor visible changes with bf16, but no avalanche effects, so this feature is enabled by default with new --use-cpu-bf16=auto option.
It can be disabled with --use-cpu-bf16=no.

With the following command (note: ComfyUI never mention this, but setting correct environment variables is highly important, see this page), KSampler node is almost 2 times faster (also memory usage is proportionally smaller):

LD_PRELOAD=libtrick.so:/src/oneapi/compiler/2024.0/lib/libiomp5.so:/usr/lib64/libtcmalloc.so \
KMP_AFFINITY=granularity=fine,compact,1,0 KMP_BLOCKTIME=1 OMP_NUM_THREADS=16 \
numactl -C 0-15 -m 0 python main.py --cpu --bf16-vae --bf16-unet

--use-cpu-bf16=no - 1.68s/it	--use-cpu-bf16=auto - 1.22it/s

[simonlui]

Going to chime in here since I did significant work on the XPU side of IPEX for ComfyUI. This patch basically turns on CPU mode for IPEX, doesn't it? I have been meaning to write a patch for something like this for a while so thanks for doing the work to enable this. Had a few comments and nudges on things that could be improved but nothing else looks terribly wrong and I think this will improve everyone's experience with running the project although I am not sure if the bar to get that speed is enough to make it a default option for people to try, IPEX does have a minimum requirement of AVX2 needed on the CPU in order to even work. I would also suggest changing the README too to note this is available. Hopefully, when @comfyanonymous is less busy with things, he can take a look at the PR.

[AngryLoki]

While testing with Flux, I discovered few interesting things:

1. ipex allocates extra memory (even with weight_prepack=False) so that with ipex Flux + OS does not fit 64GB.
2. ipex focuses on models with forward() method; for other models most of optimizations are not available
3. new pytorch builds can perform the most heavy bf16 ops on CPU (read: matmul) without ipex.

So I reworked patch so that there is no requirement for ipex-for-cpu anymore.

After checking with flux-schnell (which is already distributed in bf16-format):

• Without patch: 54GB ram, prompt executed in 242.54 seconds
• With patch: 35GB ram, prompt executed in 118.42 seconds

[AngryLoki]

Rebased, removed section from Readme... (not too much information about CPU in Readme anyways).

@comfyanonymous , I understand that CPU is low priority, but the idea is still there and here are some results with SDXL:

	Original (fp32)[1]	Original (fp32, libtrick)[2]	Patched, bf16[3]
Memory	16G	16G	9.1G
s/it	24.59	19.81	9.84
Total time	00:12:23	00:10:05	00:05:05

I. e. more than 2 times faster.

[1] Without patch, fp32, launched with LD_PRELOAD=/src/oneapi/compiler/2024.0/lib/libiomp5.so:/usr/lib64/libtcmalloc.so KMP_AFFINITY=granularity=fine,compact,1,0 KMP_BLOCKTIME=1 OMP_NUM_THREADS=16 numactl -C 0-15 -m 0 python main.py --cpu - "Optimization notice"-d, mkl_blas_def_sgemm_kernel_0_zen dominates due to MKL issues

[2] Without patch, fp32, launched with LD_PRELOAD=$(pwd)/libtrick.so:/src/oneapi/compiler/2024.0/lib/libiomp5.so:/usr/lib64/libtcmalloc.so KMP_AFFINITY=granularity=fine,compact,1,0 KMP_BLOCKTIME=1 OMP_NUM_THREADS=16 numactl -C 0-15 -m 0 python main.py --cpu, added for fair comparison

[3] With patch, bf16 on AMD Ryzen 9 7950X3D, launched with LD_PRELOAD=/src/oneapi/compiler/2024.0/lib/libiomp5.so:/usr/lib64/libtcmalloc.so KMP_AFFINITY=granularity=fine,compact,1,0 KMP_BLOCKTIME=1 OMP_NUM_THREADS=16 numactl -C 0-15 -m 0 python main.py --cpu --bf16-vae --bf16-unet. MKL has no impact on performance (it uses OneDNN JIT kernels).

For SDXL there are some differences in results (bigger than SD1.5), but probably not critical:

FP32	BF16