unorderable types: str() > int()

[simonm3]

I get the above error message. Works fine if I exclude the object columns.

[simonm3]

Some columns contained errors e.g. a numeric column had NaN values as " NA".
Would be useful if such failure reported the offending columns rather than failing.

[JosPolfliet]

Good point, I never thought about column types that are not numeric, character or dates. They should indeed just be ignored with a warning message that the type is not supported for analysis.

[simonm3]

As well as reporting them as ignored would be useful to show frequency
counts if possible. For example if you have values "P", "1", "1.0" then it
is clear what the problem is.

On 4 November 2016 at 09:24, Jos Polfliet notifications@github.com wrote:

Good point, I never thougt about column types that are not numeric,
character or dates. They should indeed just be ignored with a warning
message that the type is not supported for analysis.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#29 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABJN6RGNI1wgYOvDx79TcG-B-qhr_cIbks5q6vnDgaJpZM4Klcs0
.

[arsenyinfo]

It would be also useful if one could convert these columns into same type: e.g. if I have both str and int within one column, it's probably a good idea to treat it as str while making the report.

I'd suggest to add a param to ProfileReport for this case with three options: exclude weird columns, cast to str or just raise an exception.

@JosPolfliet would you like to get a PR on this?

[JosPolfliet]

Yes @arsenyinfo, if you have time feel free to send a PR and I will review! Thanks.

[arsenyinfo]

@JosPolfliet PTAL at PR above. Thanks!

[JosPolfliet]

My friends, I haven't forgotten this, I am just travelling and haven't made the time. I'm back on Saturday and will take a look then. Happy holidays!

…

On Wed, 28 Dec 2016 at 10:50, Arseny Kravchenko ***@***.***> wrote: @JosPolfliet <https://github.com/JosPolfliet> PTAL at PR above. Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#29 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALznoftdolHE9VDohyBF6tnVQwdC8c5Dks5rMjDHgaJpZM4Klcs0> .

[simonm3]

In addition to the mixed type fields there is a list type which also crashes the whole report currently. As a short term fix would be good to flag it up as an unsupported type. In the longer term would be useful to see most common values and distribution.

[JosPolfliet]

I thought this was fixed in the last version. It was related to a Pandas bug. PR was closed but this issue wasn't (mea culpa).

Can you share a working example of when it fails?

[simonm3]

from pandas_profiling import ProfileReport
a=pd.DataFrame(dict(a=[1,2,3], b=[4,5,6], mylist=[["item1", "item2"], ["item3", "item2", "item3"], ["item2", "item2"]]))
ProfileReport(a)

unhashable type list

[simonm3]

BTW this is a really great package but can't you change the name to something without underscores, hyphens and capital letters?....e.g.

from pandaseda import eda
eda()

[conradoqg]

Hey,

With the above PR #82 merged, you can have dicts, lists and other object types in your dataframe. The profiler will mark those fields as unsupported (since there is not much analysis that we can do).

If in the future we find a nice way to report something related to that data type we can open a feature request.

About the package naming, I think it's a good suggestion and we should open a specific issue for this.
@romainx can you do that?

I think we can close this issue after those changes.

Best

[romainx]

Ok I've just created a new issue of the name (#87).