feat(StoNEZD): Implement StoNEZD by classes

[Fangop]

This pr provides a tiny refactor of CNLSZ and an implementation of StoNEZD.
First, thanks for the major refactoring of CNLSZ class, this pr used inheritance to reduce the duplicated codes for consistency of our package.
Second, the StoNEZD model has been implemented. It is now simple to implement this kind o advanced model since we have so many basic models as complements.

The testing codes are provided below:

CNLSZ

from pystoned import CNLSZ

import pandas as pd
import numpy as np

# import Finnish electricity distribution firms data
url = 'https://raw.githubusercontent.com/ds2010/pyStoNED-Tutorials/master/Data/firms.csv'
df = pd.read_csv(url, error_bad_lines=False)
df.head(5)

# output (total cost)
y  = df['TOTEX']

# inputs 
x1  = df['Energy']
x1  = np.asmatrix(x1).T
x2  = df['Length']
x2  = np.asmatrix(x2).T
x3  = df['Customers']
x3  = np.asmatrix(x3).T
x   = np.concatenate((x1, x2, x3), axis=1)

# Z variables
z = df['PerUndGr']

# import the CNLSZ module
cet = "mult"
fun = "cost"
rts = "crs"

model = CNLSZ.CNLSZ(y, x, z, cet, fun, rts)
model.optimize()

model.display_residual()

StoNEDZ

# import package pystoned
from pystoned import StoNEDZ

import pandas as pd
import numpy as np

# import Finnish electricity distribution firms data
url = 'https://raw.githubusercontent.com/ds2010/pyStoNED-Tutorials/master/Data/firms.csv'
df = pd.read_csv(url, error_bad_lines=False)
df.head(5)

# output (total cost)
y  = df['TOTEX']

# inputs 
x1  = df['Energy']
x1  = np.asmatrix(x1).T
x2  = df['Length']
x2  = np.asmatrix(x2).T
x3  = df['Customers']
x3  = np.asmatrix(x3).T
x   = np.concatenate((x1, x2, x3), axis=1)

# Z variables
z = df['PerUndGr']

# import the CNLSZ module
cet = "mult"
fun = "cost"
rts = "crs"

model = StoNEDZ.StoNEDZ(y, x, z, cet, fun, rts)
model.optimize()

model.display_residual()

print(model.get_technical_inefficiency("MOM"))

The models based on CNLS/StoNED are now available!

• The implementation of get_frontier is in the next pr.
• CNLSG and CNLSZG looks nice, maybe just a little modification to structure the files.
• Maybe implement some well-know models for users, like StoNED+DDF.
• Some other user functions can be provided, like marginal productivity.

The models have not been refactored are below:

• Free disposal hull
• DEA/DEADDF

[ds2010]

Thanks for your new pr and suggestions. After the refactoring work, we could schedule a meeting to discuss the next step work.

Regarding this pr, I am thinking about whether we should integrate the CNLSZ to CNLS class similar to the CNLSDDF class. The user can free to choose whether they need the Z variables (or undesirable variables). In some R packages, they usually include the z variables in the main routine. What do you think?

[Fangop]

@ds2010 , great idea.
I thought merging Z variable into CNLS class as a condition may be an elegant way to implement this.
The codes will be refactored again, and pushed to this pr!

[ds2010]

OK, Good. Sorry, forgot to say the StoNEZD class. It also could be integrated into StoNED.

[Fangop]

@ds2010 , The merge has been done.
Please use codes below to check the models.
Note that for CNLS/ICNLS/StoNED, we should call with keyword arguments for the cet, fun, and rts.
For instance:

model = CNLS.CNLS(y, x, cet=cet, fun=fun, rts=rts)

Rather than:

model = CNLS.CNLS(y, x, cet, fun, rts)

Since the arguments are much more complicated, calling functions with keyword arguments can help precisely using models.

CNLS prod

import pandas as pd
import numpy as np

# import the package pystoned
from pystoned import CNLS

# import Finnish electricity distribution firms data
url = 'https://raw.githubusercontent.com/ds2010/pyStoNED-Tutorials/master/Data/firms.csv'
df = pd.read_csv(url, error_bad_lines=False)

# output
y = df['Energy']

# inputs
x1 = df['OPEX']
x1 = np.asmatrix(x1).T
x2 = df['CAPEX']
x2 = np.asmatrix(x2).T
x = np.concatenate((x1, x2), axis=1)

# define and solve the CNLS model

cet = "addi"
fun = "prod"
rts = "vrs"

model = CNLS.CNLS(y, x, cet=cet, fun=fun, rts=rts)
model.optimize(False)
model.display_residual()

CNLS cost

import pandas as pd
import numpy as np

# import the package pystoned
from pystoned import CNLS

# import Finnish electricity distribution firms data
url = 'https://raw.githubusercontent.com/ds2010/pyStoNED-Tutorials/master/Data/firms.csv'
df = pd.read_csv(url, error_bad_lines=False)
df.head(5)

# output
y = df['TOTEX']

# inputs
x1 = df['Energy']
x1 = np.asmatrix(x1).T
x2 = df['Length']
x2 = np.asmatrix(x2).T
x3 = df['Customers']
x3 = np.asmatrix(x3).T
x = np.concatenate((x1, x2, x3), axis=1)

# define and solve the CNLS model

cet = "mult"
fun = "cost"
rts = "crs"

model = CNLS.CNLS(y, x, cet=cet, fun=fun, rts=rts)
model.optimize()

model.display_residual()

StoNEZD

# import package pystoned
from pystoned import *

import pandas as pd
import numpy as np

# import Finnish electricity distribution firms data
url = 'https://raw.githubusercontent.com/ds2010/pyStoNED-Tutorials/master/Data/firms.csv'
df = pd.read_csv(url, error_bad_lines=False)

# output (total cost)
y  = df['TOTEX']

# inputs 
x1  = df['Energy']
x1  = np.asmatrix(x1).T
x2  = df['Length']
x2  = np.asmatrix(x2).T
x3  = df['Customers']
x3  = np.asmatrix(x3).T
x   = np.concatenate((x1, x2, x3), axis=1)

# Z variables
z = df['PerUndGr']

# import the CNLSZ module
cet = "mult"
fun = "cost"
rts = "crs"

model = StoNED.StoNED(y, x, z, cet, fun, rts)
model.optimize()

model.display_residual()

print(model.get_technical_inefficiency("MOM"))

[ds2010]

@JulianaTa Yes. Good suggestion.
we can further remove the following command lines:
cet = "addi"
fun = "prod"
rts = "vrs"
and directly use the model = CNLS.CNLS(y, x, cet="addi", fun="prod", rts="vrs"), which is a more common one.

One more comment:

• In our pyStoNED package, can we call the class using model = CNLS(y, x, cet="addi", fun="prod", rts="vrs") instead of the current model = CNLS.CNLS(y, x, cet="addi", fun="prod", rts="vrs") ?

[Fangop]

Here is some information about the comment.
In the CNLS.CNLS(..., the first CNLS refer to the module(file) name CNLS and the second CNLS refer to the class name.
For using single CNLS for calling the class, we can modify the import command:
Using from pystoned.CNLS import CNLS.

For a further information, here is a part of architecture of the current pyStoNED:

pystoned  --- CNLS --- CNLS
         |--- StoNED --- StoNED
         |--- CQER --- CQR
                   --- CER

The first stage is the main module pystoned.

Sharing a same for modules and classes is common in python.
But having a more categorized second level may improve the user experience.

[ds2010]

OK I see. Thanks!