Regression error = 0.0 - looking for suggestions

[jchiang4]

Describe the bug
After a long spark job, regression error comes back with 0 error and have double checked that response is not in the predictor. How to trouble shoot this issue?

Here's the log printout of the last section -

"BestModelName" : "OpGBTRegressor_000000000029_0",
"HoldoutEvaluation" : [ "com.salesforce.op.evaluators.MultiMetrics", "{\"metrics\":{\"regression evaluation metrics\":{\"RootMeanSquaredError\":0.0,\"MeanSquaredError\":0.0,\"R2\":\"NaN\",\"MeanAbsoluteError\":0.0}}}" ],
"ValidationParameters" : {
"parallelism" : 8,
"seed" : 112233,
"evaluator" : "root mean square error",
"stratify" : false,
"numFolds" : 3
},
"ProblemType" : "Regression",
"EvaluationMetric" : "RootMeanSquaredError",
"TrainEvaluation" : [ "com.salesforce.op.evaluators.MultiMetrics", "{\"metrics\":{\"regression evaluation metrics\":{\"RootMeanSquaredError\":0.0,\"MeanSquaredError\":0.0,\"R2\":\"NaN\",\"MeanAbsoluteError\":0.0}}}" ],
"DataPrepResults" : {
"className" : "com.salesforce.op.stages.impl.tuning.DataSplitterSummary"
},
"BestModelUID" : "OpGBTRegressor_000000000029"
}
}
18/12/13 21:43:39 INFO OpWorkflowRunner: Total run time: 3h21m42.079s

Does anyone have any suggestions on how best to troubleshoot?

Thanks,

Jim

[tovbinm]

This is abnormal. How large is the datasize?

@kinfaikan @Jauntbox ideas?

[jchiang4]

Not too big. 1.8M rows, 32 columns. Approx size = 0.3 GB. I've also tried to reduce to only 200K rows and it gives me the same issue. Any ideas?

[jchiang4]

Also, reduced the predictor features as well. Still same issue.

[Jauntbox]

Hey Jim, just got back from vacation so am taking a look at this. It's pretty hard to tell what could be going wrong without some more info.

For example, if you look at the feature contributions in the ModelInsights case class, what does the model think the most important features are?
Is something going wrong with setting the label perhaps? Eg. I could see this happening if the labels all got set to the same value somehow
Do other regression algorithms work on this dataset, or is this a problem with the GBTRegressor only?

[jchiang4]

Thanks. Will continue investigating next week. Will update this thread with more info.

[jchiang4]

Thanks for your suggestions, Jauntbox. You were correct. The labels got set to the same value in a data cleansing routine, hence the 0.0 error. Closing the issue.

[jchiang4]

Closing issue. Thanks, Jim