Unfortunately JB has a bug with setError() discussed on StackOverflow and Google Code.
Fortunately it can be addressed using reflection as a dirty workaround, using EditTextErrorFixed in place of EditText. Tested to work with both 4.2 and 4.2.1 and an updated Galaxy Nexus.
The screenshot shows that the icon on the bottom EditTextErrorFixed persists even if the focus changes. In addition it incorporates another fix where if the user presses Delete on an already empty EditText the error disappears (another bug?).