Add column order validation

[jeffzi]

This PR adds an ordered argument to DataFrameSchema and to model.BaseConfig for the model api. It closes #342.

A side effect is that 2 schema errors will be raised if there is a permutation:

import pandas as pd
import pandera as pa

schema = pa.DataFrameSchema({"a": pa.Column(int), "b": pa.Column(int)}, ordered=True)
df = pd.DataFrame([[1, 2]], columns=["b", "a"])
schema.validate(df, lazy=True)
#> Traceback (most recent call last):
#> <ipython-input-1-f6ae10d7ea6a> in <module>
#> ----> 1 schema.validate(df, lazy=True)
#> ~/Projects/development/pandera/pandera/schemas.py in validate(self, check_obj, head, tail, sample, random_state, lazy, inplace)
#>     582         if lazy and error_handler.collected_errors:
#>     583             raise errors.SchemaErrors(
#> --> 584                 error_handler.collected_errors, check_obj
#>     585             )
#>     586 
#> SchemaErrors: A total of 2 schema errors were found.
#> Error Counts
#> - column_not_ordered: 2
#> Schema Error Summary
#>                                       failure_cases  n_failure_cases
#> schema_context  column check                                        
#> DataFrameSchema <NA>   column_ordered        [a, b]                2
#> Usage Tip
#> Directly inspect all errors by catching the exception:
#> ```
#> try:
#>     schema.validate(dataframe, lazy=True)
#> except SchemaErrors as err:
#>     err.failure_cases  # dataframe of schema errors
#>     err.data  # invalid dataframe
#> ```

^{Created on 2020-12-11 by the reprexpy package}

Note: The PR targets the dev branch.

[cosmicBboy]

thanks @jeffzi! I'm currently working on all the documentation for the 0.6.0 release #350, I can add column order validation to that task too

[codecov]

Codecov Report

Merging #352 (776b82d) into dev (4d2a205) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##              dev     #352      +/-   ##
==========================================
+ Coverage   99.02%   99.03%   +0.01%     
==========================================
  Files          21       21              
  Lines        2351     2383      +32     
==========================================
+ Hits         2328     2360      +32     
  Misses         23       23              

Impacted Files	Coverage Δ
pandera/model.py	100.00% <100.00%> (ø)
pandera/schema_components.py	98.91% <100.00%> (+0.08%)	⬆️
pandera/schemas.py	98.70% <100.00%> (+0.04%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4d2a205...776b82d. Read the comment docs.

[jeffzi]

I initially forgot to look into MultiIndex order. Now I've tested adding a ordered argument to pandera.MultiIndex and ran into issues:

1. ordered does not make sense if some index levels are not named. I'm raising an error if ordered=True and at least one index is not named.
2. Duplicates (:imp:): I found a bug where duplicate indexes in a DataFrame.MultiIndex are not passed to DataFrameSchema.validate() (base class of pandera.MultiIndex). It's caused by https://github.com/pandera-dev/pandera/blob/5e35569bf49101a22a52a55a057edff3319e64d1/pandera/schema_components.py#L574

to_frame drops duplicates. The documentation says it implicitly:

Column ordering is determined by the DataFrame constructor with data as a dict.

import pandas as pd

df = pd.DataFrame(
    index=pd.MultiIndex.from_arrays([[1], [2], [3]], names=["b", "a", "a"]),
)
print(df.index.to_frame()) 
#>        b  a
#> b a a      
#> 1 2 3  1  3

df = pd.DataFrame(
    index=pd.MultiIndex.from_arrays([[1], [2], [3], [4]], names=["a", "b", "c", "a"]),
)
print(df.index.to_frame())
#>          a  b  c
#> a b c a         
#> 1 2 3 4  4  2  3

^{Created on 2020-12-11 by the reprexpy package}

@cosmicBboy That bug should be addressed before merging this PR (make one test fail). I can take a look over the weekend.

Pinging @ktroutman since he seemed interested in that discussion :)

[cosmicBboy]

I just touched this part of the codebase so I took a crack at the multiindex -> df conversion, preserving duplicate index names... tried to emulate the behavior of pandas as much as possible

[jeffzi]

Awesome @cosmicBboy 🔥 I fixed a bug with optional columns and added documentation.
On my side it's good to go, let me know if you see anything else.

[cosmicBboy]

failing test doesn't have to do with the PR, the issues should be fixed by #351, merging this now