What’s a loss function? At its core, a loss function is incredibly simple: It’s a method of evaluating how well your algorithm models your dataset. If your predictions are totally off, your loss function will output a higher number. If they’re pretty good, it’ll output a lower number. As you change pieces of your algorithm to try and improve your model, your loss function will tell you if you’re getting anywhere.
Loss functions are related to model accuracy, a key component of AI/ML governance.
Most people confuse loss function and cost function.
A loss function/error function is for a single training example/input.
A cost function, on the other hand, is the average loss over the entire training dataset.