ValueError: index must be monotonic increasing or decreasing #10

phillip1029 · 2020-06-09T04:02:26Z

I am able to generate the same report on Titanic data as in the Medium articles. However, when I try to test the Boston housing data, I get the errors as below:

ValueError Traceback (most recent call last)
~\AppData\Local\Continuum\anaconda3\envs\envSDS\lib\site-packages\pandas\core\indexes\base.py in get_slice_bound(self, label, side, kind)
5166 try:
-> 5167 return self._searchsorted_monotonic(label, side)
5168 except ValueError:

~\AppData\Local\Continuum\anaconda3\envs\envSDS\lib\site-packages\pandas\core\indexes\base.py in _searchsorted_monotonic(self, label, side)
5127
-> 5128 raise ValueError("index must be monotonic increasing or decreasing")
5129

ValueError: index must be monotonic increasing or decreasing

During handling of the above exception, another exception occurred:

KeyError Traceback (most recent call last)
in
----> 1 my_report = sv.analyze(dfx)

Any ideas on the error?

Thanks.

fbdesignpro · 2020-06-09T12:32:40Z

Hi! Thank you for the report. I initially thought the data set was only on Kaggle. Finding an alternate source and will report here on what I find.

fbdesignpro · 2020-06-09T13:18:40Z

I was able to reproduce the issue, will look for a fix!

roytaas · 2020-06-09T15:06:23Z

I also get the same error: ValueError("index must be monotonic increasing or decreasing").
Also with older versions of sweetviz.

ValueError Traceback (most recent call last)
~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/indexes/base.py in get_slice_bound(self, label, side, kind)
4844 try:
-> 4845 return self._searchsorted_monotonic(label, side)
4846 except ValueError:

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/indexes/base.py in _searchsorted_monotonic(self, label, side)
4805
-> 4806 raise ValueError("index must be monotonic increasing or decreasing")
4807

ValueError: index must be monotonic increasing or decreasing

During handling of the above exception, another exception occurred:

KeyError Traceback (most recent call last)
in ()
4 #my_report1.show_html('Report.html')
5 #feature_config = sv.FeatureConfig(skip="Id", force_cat=["Resultaat betaalgedrag"])
----> 6 my_report1 = sv.compare_intra(tm1, tm1["Resultaat betaalgedrag"] == 0, [0, 1])
7 my_report1.show_html('Report.html')

~/anaconda3/envs/python3/lib/python3.6/site-packages/sweetviz/sv_public.py in compare_intra(source_df, condition_series, names, target_feat, feat_cfg, pairwise_analysis)
42 report = sweetviz.DataframeReport([data_true, names[0]], target_feat,
43 [data_false, names[1]],
---> 44 pairwise_analysis, feat_cfg)
45 return report
46

~/anaconda3/envs/python3/lib/python3.6/site-packages/sweetviz/dataframe_report.py in init(self, source, target_feature_name, compare, pairwise_analysis, fc)
214 # start = time.perf_counter()
215 self.progress_bar.set_description(':' + f.source.name + '')
--> 216 self._features[f.source.name] = sa.analyze_feature_to_dictionary(f)
217 self.progress_bar.update(1)
218 # print(f"DONE FEATURE------> {f.source.name}"

~/anaconda3/envs/python3/lib/python3.6/site-packages/sweetviz/series_analyzer.py in analyze_feature_to_dictionary(to_process)
98 sweetviz.series_analyzer_numeric.analyze(to_process, returned_feature_dict)
99 elif returned_feature_dict["type"] == FeatureType.TYPE_CAT:
--> 100 sweetviz.series_analyzer_cat.analyze(to_process, returned_feature_dict)
101 elif returned_feature_dict["type"] == FeatureType.TYPE_BOOL:
102 sweetviz.series_analyzer_cat.analyze(to_process, returned_feature_dict)

~/anaconda3/envs/python3/lib/python3.6/site-packages/sweetviz/series_analyzer_cat.py in analyze(to_process, feature_dict)
121 compare_dict["stats"] = dict()
122
--> 123 do_detail_categorical(to_process, feature_dict)
124
125 feature_dict["minigraph"] = GraphCat("mini", to_process)

~/anaconda3/envs/python3/lib/python3.6/site-packages/sweetviz/series_analyzer_cat.py in do_detail_categorical(to_process, updated_dict)
21
22 category_counts = utils.get_clamped_value_counts(to_process.source_counts["value_counts_without_nan"],
---> 23 config["Graphs"].getint("detail_graph_max_categories"))
24
25 # Iterate through ALL VALUES and get stats

~/anaconda3/envs/python3/lib/python3.6/site-packages/sweetviz/utils.py in get_clamped_value_counts(value_counts, max_categories_incl_other)
12 else:
13 categories_shown_as_is = max_categories_incl_other - 1
---> 14 clamped_series = pd.Series(value_counts[0:categories_shown_as_is])
15
16 categories_in_other = value_counts[categories_shown_as_is:]

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/series.py in getitem(self, key)
908 key = check_bool_indexer(self.index, key)
909
--> 910 return self._get_with(key)
911
912 def _get_with(self, key):

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/series.py in _get_with(self, key)
913 # other: fancy integer or otherwise
914 if isinstance(key, slice):
--> 915 return self._slice(key)
916 elif isinstance(key, ABCDataFrame):
917 raise TypeError(

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/series.py in _slice(self, slobj, axis, kind)
863
864 def _slice(self, slobj: slice, axis: int = 0, kind=None):
--> 865 slobj = self.index._convert_slice_indexer(slobj, kind=kind or "getitem")
866 return self._get_values(slobj)
867

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/indexes/numeric.py in _convert_slice_indexer(self, key, kind)
422
423 # translate to locations
--> 424 return self.slice_indexer(key.start, key.stop, key.step, kind=kind)
425
426 def _format_native_types(

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/indexes/base.py in slice_indexer(self, start, end, step, kind)
4711 slice(1, 3)
4712 """
-> 4713 start_slice, end_slice = self.slice_locs(start, end, step=step, kind=kind)
4714
4715 # return a slice

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/indexes/base.py in slice_locs(self, start, end, step, kind)
4924 start_slice = None
4925 if start is not None:
-> 4926 start_slice = self.get_slice_bound(start, "left", kind)
4927 if start_slice is None:
4928 start_slice = 0

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/indexes/base.py in get_slice_bound(self, label, side, kind)
4846 except ValueError:
4847 # raise the original KeyError
-> 4848 raise err
4849
4850 if isinstance(slc, np.ndarray):

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/indexes/base.py in get_slice_bound(self, label, side, kind)
4840 # we need to look up the label
4841 try:
-> 4842 slc = self.get_loc(label)
4843 except KeyError as err:
4844 try:

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/indexes/numeric.py in get_loc(self, key, method, tolerance)
506 except (TypeError, NotImplementedError):
507 pass
--> 508 return super().get_loc(key, method=method, tolerance=tolerance)
509
510 @cache_readonly

~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
2646 return self._engine.get_loc(key)
2647 except KeyError:
-> 2648 return self._engine.get_loc(self._maybe_cast_indexer(key))
2649 indexer = self.get_indexer([key], method=method, tolerance=tolerance)
2650 if indexer.ndim > 1 or indexer.size > 1:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Float64HashTable.get_item()

KeyError: 0.0

fbdesignpro · 2020-06-09T15:16:37Z

Fix submitted with the latest commit (1.0a7).

@phillip1029 @roytaas let me know if that fixed it!

Thank you again for the reports!

Francois

roytaas · 2020-06-09T15:32:32Z

I still get the same error

fbdesignpro · 2020-06-09T15:33:56Z

@roytaas darn; is that with the same Boston dataset?

roytaas · 2020-06-09T15:52:03Z

It is the same dataset, but i didn't use the boston dataset, but my own dataset :)
If you want, I can also try the boston housing data from Kaggle..

roytaas · 2020-06-09T16:02:37Z

Wel it works on the boston dataset. In my own dataset I have string variables, which I don't see in the boston and titanic set, is that a problem?
My target variable is numerical...

fbdesignpro · 2020-06-09T16:49:46Z

@roytaas string data should be fine and show up as "text". There are all sorts of combinations though so there is something I didn't cover in that particular case. Is this a dataset you would be able to share with me? If so e-mail me at fb at fbdesignpro.com. If not I will look at the call stack again.

fbdesignpro · 2020-06-10T00:52:17Z

Actually, looking at the code, it has to be a different error (but it could be with the same data); @roytaas could you send the call stack you are getting again? Because you should not be getting the same error on version 1.0a7.

phillip1029 · 2020-06-10T18:54:02Z

@fbdesignpro Thank you so much for fixing this. After uninstalling and reinstalling the package, I am getting the right results now. Great job.

fbdesignpro · 2020-06-11T13:45:31Z

@roytaas let me know if you're still getting this in version 1.0a7. To make sure you are using the latest version you can use pip install sweetviz --upgrade. Looking forward to hearing if that worked, or if there is a different issue!

ikunal95 · 2020-06-13T10:02:48Z

@fbdesignpro Hi, I am facing the same error even after upgrading it 1.0a7.

ValueError Traceback (most recent call last)
~/.local/lib/python3.8/site-packages/pandas/core/indexes/base.py in get_slice_bound(self, label, side, kind)
4843 try:
-> 4844 return self._searchsorted_monotonic(label, side)
4845 except ValueError:

~/.local/lib/python3.8/site-packages/pandas/core/indexes/base.py in _searchsorted_monotonic(self, label, side)
4804
-> 4805 raise ValueError("index must be monotonic increasing or decreasing")
4806

ValueError: index must be monotonic increasing or decreasing

During handling of the above exception, another exception occurred:

KeyError Traceback (most recent call last)
in
1 import sweetviz
----> 2 my_report = sweetviz.compare([Individual_Loan_Train, "Training Data"], [Individual_Loan_Test, "Test Data"],target_feat= "default")

~/.local/lib/python3.8/site-packages/sweetviz/sv_public.py in compare(source, compare, target_feat, feat_cfg, pairwise_analysis)
20 feat_cfg: FeatureConfig = None,
21 pairwise_analysis: str = 'auto'):
---> 22 report = sweetviz.DataframeReport(source, target_feat, compare,
23 pairwise_analysis, feat_cfg)
24 return report

~/.local/lib/python3.8/site-packages/sweetviz/dataframe_report.py in init(self, source, target_feature_name, compare, pairwise_analysis, fc)
214 # start = time.perf_counter()
215 self.progress_bar.set_description(':' + f.source.name + '')
--> 216 self._features[f.source.name] = sa.analyze_feature_to_dictionary(f)
217 self.progress_bar.update(1)
218 # print(f"DONE FEATURE------> {f.source.name}"

~/.local/lib/python3.8/site-packages/sweetviz/series_analyzer.py in analyze_feature_to_dictionary(to_process)
98 sweetviz.series_analyzer_numeric.analyze(to_process, returned_feature_dict)
99 elif returned_feature_dict["type"] == FeatureType.TYPE_CAT:
--> 100 sweetviz.series_analyzer_cat.analyze(to_process, returned_feature_dict)
101 elif returned_feature_dict["type"] == FeatureType.TYPE_BOOL:
102 sweetviz.series_analyzer_cat.analyze(to_process, returned_feature_dict)

~/.local/lib/python3.8/site-packages/sweetviz/series_analyzer_cat.py in analyze(to_process, feature_dict)
121 compare_dict["stats"] = dict()
122
--> 123 do_detail_categorical(to_process, feature_dict)
124
125 feature_dict["minigraph"] = GraphCat("mini", to_process)

~/.local/lib/python3.8/site-packages/sweetviz/series_analyzer_cat.py in do_detail_categorical(to_process, updated_dict)
20 num_values_compare = updated_dict["compare"]["base_stats"]["num_values"].number
21
---> 22 category_counts = utils.get_clamped_value_counts(to_process.source_counts["value_counts_without_nan"],
23 config["Graphs"].getint("detail_graph_max_categories"))
24

~/.local/lib/python3.8/site-packages/sweetviz/utils.py in get_clamped_value_counts(value_counts, max_categories_incl_other)
12 else:
13 categories_shown_as_is = max_categories_incl_other - 1
---> 14
15 # Fix for #10
16 # clamped_series = pd.Series(value_counts[0:categories_shown_as_is])

~/.local/lib/python3.8/site-packages/pandas/core/series.py in getitem(self, key)
908 key = check_bool_indexer(self.index, key)
909
--> 910 return self._get_with(key)
911
912 def _get_with(self, key):

~/.local/lib/python3.8/site-packages/pandas/core/series.py in _get_with(self, key)
913 # other: fancy integer or otherwise
914 if isinstance(key, slice):
--> 915 return self._slice(key)
916 elif isinstance(key, ABCDataFrame):
917 raise TypeError(

~/.local/lib/python3.8/site-packages/pandas/core/series.py in _slice(self, slobj, axis, kind)
863
864 def _slice(self, slobj: slice, axis: int = 0, kind=None):
--> 865 slobj = self.index._convert_slice_indexer(slobj, kind=kind or "getitem")
866 return self._get_values(slobj)
867

~/.local/lib/python3.8/site-packages/pandas/core/indexes/numeric.py in _convert_slice_indexer(self, key, kind)
422
423 # translate to locations
--> 424 return self.slice_indexer(key.start, key.stop, key.step, kind=kind)
425
426 def _format_native_types(

~/.local/lib/python3.8/site-packages/pandas/core/indexes/base.py in slice_indexer(self, start, end, step, kind)
4710 slice(1, 3)
4711 """
-> 4712 start_slice, end_slice = self.slice_locs(start, end, step=step, kind=kind)
4713
4714 # return a slice

~/.local/lib/python3.8/site-packages/pandas/core/indexes/base.py in slice_locs(self, start, end, step, kind)
4923 start_slice = None
4924 if start is not None:
-> 4925 start_slice = self.get_slice_bound(start, "left", kind)
4926 if start_slice is None:
4927 start_slice = 0

~/.local/lib/python3.8/site-packages/pandas/core/indexes/base.py in get_slice_bound(self, label, side, kind)
4845 except ValueError:
4846 # raise the original KeyError
-> 4847 raise err
4848
4849 if isinstance(slc, np.ndarray):

~/.local/lib/python3.8/site-packages/pandas/core/indexes/base.py in get_slice_bound(self, label, side, kind)
4839 # we need to look up the label
4840 try:
-> 4841 slc = self.get_loc(label)
4842 except KeyError as err:
4843 try:

~/.local/lib/python3.8/site-packages/pandas/core/indexes/numeric.py in get_loc(self, key, method, tolerance)
506 except (TypeError, NotImplementedError):
507 pass
--> 508 return super().get_loc(key, method=method, tolerance=tolerance)
509
510 @cache_readonly

~/.local/lib/python3.8/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
2646 return self._engine.get_loc(key)
2647 except KeyError:
-> 2648 return self._engine.get_loc(self._maybe_cast_indexer(key))
2649 indexer = self.get_indexer([key], method=method, tolerance=tolerance)
2650 if indexer.ndim > 1 or indexer.size > 1:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Float64HashTable.get_item()

KeyError: 0.0

fbdesignpro · 2020-06-15T12:53:00Z

@ikunal95 @phillip1029 am I on crazy pills? It looks like my updates are not taking effect on the pyc compiled code or something! (but it seems to update the source for you guys) I wonder if it has to do with how I am uploading builds through pip!? There is something strange going on here. It's like it's using an OLD compiled version because it's crashing on a line that low longer exists in 1.0a7 (?!). i.e.

~/.local/lib/python3.8/site-packages/sweetviz/utils.py in get_clamped_value_counts(value_counts, max_categories_incl_other)
12 else:
13 categories_shown_as_is = max_categories_incl_other - 1
---> 14
15 # Fix for #10
16 # clamped_series = pd.Series(value_counts[0:categories_shown_as_is])

Line 14 has NO code in it, and the call stack you're giving me is consistent with the old version of source. e.g. PREVIOUSLY:

(Notice how line 14 no longer exists and that is what I had fixed previously...)

Can you try uninstalling then reinstalling the package? Thanks!

WillArevalo · 2020-06-16T14:55:42Z

Excelent, you fix the problem qwith the last update, thanks 👍

fbdesignpro · 2020-06-16T15:00:55Z

Great, thank you! @phillip1029 @ikunal95 does reinstalling fix it? Looks like "just upgrading" isn't working for some reason... :/

phillip1029 · 2020-06-17T02:34:06Z

Reinstalling fixed the issue for me. Thanks.

…

On Tue, Jun 16, 2020, 11:01 AM Francois Bertrand ***@***.***> wrote: Great, thank you! @phillip1029 <https://github.com/phillip1029> @ikunal95 <https://github.com/ikunal95> does reinstalling fix it? Looks like "just upgrading" isn't working for some reason... :/ — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABXGCSIRETELOJHKHQJ55KLRW6CLPANCNFSM4NZAOIBA> .

fbdesignpro · 2020-06-17T02:40:03Z

@phillip1029 great! Thank you that makes some sense! I will update the instructions for updating as it seems reinstalling is the way to go.

fbdesignpro added the bug Something isn't working label Jun 9, 2020

fbdesignpro closed this as completed Jun 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ValueError: index must be monotonic increasing or decreasing #10

ValueError: index must be monotonic increasing or decreasing #10

phillip1029 commented Jun 9, 2020

fbdesignpro commented Jun 9, 2020 •

edited

fbdesignpro commented Jun 9, 2020

roytaas commented Jun 9, 2020

fbdesignpro commented Jun 9, 2020

roytaas commented Jun 9, 2020

fbdesignpro commented Jun 9, 2020

roytaas commented Jun 9, 2020

roytaas commented Jun 9, 2020

fbdesignpro commented Jun 9, 2020

fbdesignpro commented Jun 10, 2020

phillip1029 commented Jun 10, 2020

fbdesignpro commented Jun 11, 2020

ikunal95 commented Jun 13, 2020

fbdesignpro commented Jun 15, 2020

WillArevalo commented Jun 16, 2020

fbdesignpro commented Jun 16, 2020

phillip1029 commented Jun 17, 2020 via email

fbdesignpro commented Jun 17, 2020

ValueError: index must be monotonic increasing or decreasing #10

ValueError: index must be monotonic increasing or decreasing #10

Comments

phillip1029 commented Jun 9, 2020

fbdesignpro commented Jun 9, 2020 • edited

fbdesignpro commented Jun 9, 2020

roytaas commented Jun 9, 2020

fbdesignpro commented Jun 9, 2020

roytaas commented Jun 9, 2020

fbdesignpro commented Jun 9, 2020

roytaas commented Jun 9, 2020

roytaas commented Jun 9, 2020

fbdesignpro commented Jun 9, 2020

fbdesignpro commented Jun 10, 2020

phillip1029 commented Jun 10, 2020

fbdesignpro commented Jun 11, 2020

ikunal95 commented Jun 13, 2020

fbdesignpro commented Jun 15, 2020

WillArevalo commented Jun 16, 2020

fbdesignpro commented Jun 16, 2020

phillip1029 commented Jun 17, 2020 via email

fbdesignpro commented Jun 17, 2020

fbdesignpro commented Jun 9, 2020 •

edited