spirv-opt: debugprintf instrumentation reports "Mixed stage shader module not supported"

[natevm]

This is a continuation of the issue over on the Vulkan Validation Layers repo here: KhronosGroup/Vulkan-ValidationLayers#4350

A quick summary is that I'd like to support printf with multiple entry points. This should be doable so long as no entry point calls a common function (which is true if --inline-entry-points-exhaustive is used, which is included by default as part of the -o0 flag).

I believe the code that needs changing is here:

SPIRV-Tools/source/opt/instrument_pass.cpp

Lines 968 to 991 in 0a43a84

	bool InstrumentPass::InstProcessEntryPointCallTree(InstProcessFunction& pfn) {
	// Make sure all entry points have the same execution model. Do not
	// instrument if they do not.
	// TODO(greg-lunarg): Handle mixed stages. Technically, a shader module
	// can contain entry points with different execution models, although
	// such modules will likely be rare as GLSL and HLSL are geared toward
	// one model per module. In such cases we will need
	// to clone any functions which are in the call trees of entrypoints
	// with differing execution models.
	uint32_t ecnt = 0;
	uint32_t stage = SpvExecutionModelMax;
	for (auto& e : get_module()->entry_points()) {
	if (ecnt == 0)
	stage = e.GetSingleWordInOperand(kEntryPointExecutionModelInIdx);
	else if (e.GetSingleWordInOperand(kEntryPointExecutionModelInIdx) !=
	stage) {
	if (consumer()) {
	std::string message = "Mixed stage shader module not supported";
	consumer()(SPV_MSG_ERROR, 0, {0, 0, 0}, message.c_str());
	}
	return false;
	}
	++ecnt;
	}

So, I believe the logic there should instead ask "do any entry points call a common function? if so, disable printf instrumentation."

@greg-lunarg does that seem right?

[greg-lunarg]

@greg-lunarg does that seem right?

Yes.

[natevm]

Okay, cool. Then I might poke around here and see if I can detect common function calls. I'm a little unfamiliar with the code here, but I figure I should be able to work it out. If I can get that working, I can open a PR here. :)

[greg-lunarg]

There is a function that traverses the call tree of an entry point function. From that you can build a set of ids of functions that are called. From that, you can do an intersection on the function sets of two entry points.