[MPS] Add fmax fmin op

[qqaatw]

Fixes #ISSUE_NUMBER

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/95191

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Long queue on MacOS M1 runners

❌ 3 Failures

As of commit 11c289b:

NEW FAILURES - The following jobs have failed:

• win-vs2019-cpu-py3 / test (default, 1, 2, windows.4xlarge) (gh)
• win-vs2019-cpu-py3 / test (default, 2, 2, windows.4xlarge) (gh)
• win-vs2019-cpu-py3 / test (functorch, 1, 1, windows.4xlarge) (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[qqaatw]

Guards possible duplicate includes in the future.

[kulinseth]

Looks good.

[qqaatw]

@pytorchbot label "accept2run"

[kulinseth]

@pytorchbot merge -f "MPS tests are green."

[pytorchmergebot]

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[zejun-chen]

Hi,
May i know why assert double datatype here? Is it because the Apple GPU doesn't support double precision?
BTW, why doesn't unary op have such assertion?
Thank you.

[qqaatw]

Yes it doesn't support double precision AFAIK. As for why unary ops don't have such assertion, I think they're just missed. But in general, you're not allowed to create an fp64 MPS tensor nor allowed to type cast / move a fp64 tensor to MPS.

[zejun-chen]

Thank you for help. I find the code below,

pytorch/aten/src/ATen/native/mps/OperationUtils.mm

Lines 83 to 85 in 71d0202

	TORCH_CHECK_TYPE(false,
	"Cannot convert a float64 Tensor to MPS as the MPS framework doesn't support float64. "
	"Please use float32 instead.")

and

pytorch/aten/src/ATen/native/mps/operations/BitwiseOps.mm

Lines 95 to 108 in 71d0202

	static const std::string& getMetalType(const c10::ScalarType& t) {
	// Mapping from c10::ScalarType to integral type that can be used for bitwise ops
	// As bitwise ops sign-agnostic map signed/unsigned char and boolean to the same type
	static std::unordered_map<c10::ScalarType, std::string> scalar_to_metal_type = {
	{c10::ScalarType::Long, "long"},
	{c10::ScalarType::Int, "int"},
	{c10::ScalarType::Short, "short"},
	{c10::ScalarType::Byte, "char"},
	{c10::ScalarType::Char, "char"},
	{c10::ScalarType::Bool, "char"},
	};
	auto it = scalar_to_metal_type.find(t);
	TORCH_CHECK(it != scalar_to_metal_type.end(), "Unsupported type ", t);

Any FP64 data type will throw error in these MPS utilities.

BTW, here code is used to register the supported datatype for the fmax kernel, right? Does it mean only the half and the float32 kernels will be precompiled when AOT compiling.

pytorch/aten/src/ATen/native/mps/operations/BinaryKernel.mm

Lines 122 to 123 in 71d0202

	REGISTER_FMAX_OP(float);
	REGISTER_FMAX_OP(half);

Thank you.