Pathfinder warning message is misleading

[jgabry]

Description:

This warning message

stan/src/stan/services/pathfinder/psis.hpp

Lines 263 to 268 in 6d35a76

	if (smoothed.second > 0.7) {
	logger.warn(std::string("Pareto k value (") +
	std::to_string(smoothed.second) + ") is greater than 0.7 which often"
	" indicates model" " misspecification.");
	}
	}

is not quite right. Misspecification is a possibility, but the model may often be specified just fine and the Pareto k diagnostic could be high because Pathfinder + PSIS might still be a poor approximation to the posterior.

Reproducible Steps:

Look at the code I linked to above to see the message, you don't need to run a model.

Current Output:

The warning in the code linked above.

Expected Output:

A more accurate warning message indicating that it might be misspecification but could also likely be that the approximation is poor.

Additional Information:

Provide any additional information here.

Current Version:

v2.32.2

[ssp3nc3r]

maybe something like,

"which may indicate poor approximation of the target distribution; re-parameterization may help."

And if re-parameterization may help — hope springs eternal — are there other pathfinder diagnostics or hints on where to start?

[jgabry]

@avehtari What do you think?

[jgabry]

maybe something like,

"which may indicate poor approximation of the target distribution; re-parameterization may help."

I think it may still be a good idea to mention misspecification because that's still a possibility. Maybe:

Pareto k value is greater than 0.7 which may indicate a poor approximation 
to the target distribution or model misspecification. 

Or we could say more:

Pareto k value is greater than 0.7 which indicates infinite variance 
of the importance ratios used for resampling. 
This may be due to a poor approximation to the target distribution or model misspecification. 

That's a bit verbose but much more informative. Maybe @avehtari will have a better idea.

[avehtari]

The model misspecification is not relevant here, and probably was mistakenly carried over from LOO-CV diagnostic.

I would write
"Pareto k diagnostic value is greater than 0.7 which may indicate a poor approximation and indicates that importance resampling is not able to improve the approximation."