Yes, this is a limitation of our API currently. One option is to use a parameter grid that only allows valid combinations, by using a list of parameter dicts wherein each setting is valid. A more robust forward-thinking solution might consider n_components and k optionally being functions of X and y.
@jnothman Thank you, can you please elaborate a little bit more on the function-based approach? I am also considering to use the RandomizedSearchCV, but will probably face the same problem there, right?
With randomized search you can't encode constraints between multiple
parameters, but you can use error_score=0.
The function-based approach would require someone changing the n_components
interface to allow it to be a function. It's not something we've generally
done, so it's probably on the scale of requiring an Enhancement Proposal to
suggest that this approach be used across the board.
You could potentially do something hacky like inherit from TruncatedSVD and
overwrite fit(X, y), such that it changes n_components depending on X.shape.