[py3] Add extra details to the AvroTypeException

[sireliah]

Hey, I decided to prepare a small improvement to the AvroTypeException and the schema validation in python3.

My use case is following:

• I use very complex and nested schemas, mostly with record data types.
• If datum happens to be incorrect (for instance int instead of string), the AvroTypeException is raisen and schema is printed to stdout.
• Since the schema is long, it's nearly impossible to compare data with schema.

Solution:

• Introduce aggregate for the exceptions (it may be more than one error).
• Output the diff between the datum and the schema.
• Color the output.

What do you think about the idea? Feedback/review would be really appreciated.

[sireliah]

Hej, can I have any feedback from people interested in maintaining the Python version?

[kant111]

Can we have this for Java as well?

[sireliah]

Ha, it would be nice, but unfortunately I have very limited experience with Java. After briefly looking at the code, I think it would be possible to introduce some error aggregator, but I'd need to spend some time understanding how the validation works in Java version of this lib.

[soxofaan]

To whom it might concern: I have comparable pull request for the PHP library (forked by wikimedia):
https://github.com/wikimedia/avro-php/pull/4

[kojiromike]

Is there any way we can disable these term colors when stdout is not a tty, or when the termcap doesn't support it?

[sireliah]

Absolutely, I think it's enough to check sys.stdout.isatty(). I'll do it.

[sireliah]

@kojiromike I considered using colorama, but finally I decided not to add another dependency and just check if the terminal is tty.

[kojiromike]

Have you considered using an extras_require on colorama?

[sireliah]

That would be good idea, but the purpose of colorama would be to clean the stdout from all color strings, so the code could work for ex. on Windows. Then the lib should be mandatory requirement, otherwise Windows users would get some weird strings in output.

Another problem I discovered is that colorama makes the code untestable, because the lib injects different ANSI escape character sequences depending on the OS. I experimented with this solution and found that there is no easy way to unit-test such code.

[kojiromike]

I love it. I totally want to improve these error messages. I have one comment, that we should offer a way to turn term colors off when we need to.

[soxofaan]

I agree with @kojiromike : I don't think terminal color codes belong in exception messages.
The terminal is nowadays not the only place where exceptions are displayed, you have IDE views, web interfaces, notification systems, etc and these color codes will not play nice there

[sireliah]

Hey @kojiromike, do you think it's safe to merge this PR? Should I improve something else?

[kojiromike]

@sireliah it's ok with me (if you still feel like fixing the merge conflict in io.py) but I don't have commit privileges. You'd have to get someone like @Fokko or @cutting on board.

[Fokko]

@sireliah Can you resolve the conflicts?

[sireliah]

@Fokko Sure, I will do!

[sireliah]

Ok, conflicts resolved and all tests are passing. However I see a potential threat of conflicts with @kojiromike pull request here: #516

[Fokko]

@sireliah Can you rebase?

[sireliah]

@Fokko In #516 we discussed with @kojiromike possibility to refactor the validation and have the simplified version. I'm working currently on rewriting the validation in such fashion that it is compatible with his changes.

Also I noticed that with the ANSI term colors it would be highly adviced to follow standards: https://no-color.org/ https://bixense.com/clicolors/ and check for NO_COLOR vars, so I need to modify part of the code anyway.

Ah, also - after a year of merges in the master, rebasing my branch is extremely difficult. : ]