CLA fails with TypeError: '<' not supported between instances of 'NoneType' and 'float'

[anarchy89]

When I do the following, I can get the target weights.

mu = returns_series
S = risk_models.sample_cov(df)

ef = EfficientFrontier(mu, S)
raw_weights = ef.max_sharpe(risk_free_rate=0.0004)
max_sharpe_pf = dict(ef.clean_weights())
print(cleaned_weights)
ef.portfolio_performance(verbose=True)

I get this,

OrderedDict([('asset1', 0.01464), ('asset2', 0.00269), ('asset3', 0.00481), ('asset4', 0.00314), ('asset5', 0.01349), ('asset6', 0.01292), ('asset7', 0.00397), ('asset8', 0.0014), ('asset9', 0.005), ('asset10', 0.00197), ('asset11', 0.00021), ('asset12', 0.00548), ('asset13', 0.00808), ('asset14', 0.00071), ('asset15', 0.00569), ('asset16', 0.00128), ('asset17', 0.00046), ('asset18', 0.06866), ('asset19', 0.0007), ('asset20', 0.00213), ('asset21', 0.00172), ('asset22', 0.00522), ('asset23', 0.33997), ('asset24', 0.00078), ('asset25', 0.00079), ('asset26', 0.00106), ('asset27', 0.0055), ('asset28', 0.25442), ('asset29', 0.00689), ('asset30', 0.00132), ('asset31', 0.21176), ('asset32', 0.00518), ('asset33', 0.00796)])
Expected annual return: 0.3%
Annual volatility: 0.0%
Sharpe Ratio: 4901822.92
/Users/xxx/miniconda3/envs/ai/lib/python3.8/site-packages/pypfopt/efficient_frontier/efficient_frontier.py:404: UserWarning: The risk_free_rate provided to portfolio_performance is different to the one used by max_sharpe. Using the previous value.
  warnings.warn(
(0.0025311989428367086, 4.3477681211645237e-10, 4901822.920275426)

However when I try,

cla = CLA(mu, S)
print(cla.max_sharpe())
cla.portfolio_performance(verbose=True)

I get the following error

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-731-4dfc5af627ff> in <module>
      1 cla = CLA(mu, S)
----> 2 print(cla.max_sharpe())
      3 cla.portfolio_performance(verbose=True)

~/miniconda3/envs/ai/lib/python3.8/site-packages/pypfopt/cla.py in max_sharpe(self)
    378         """
    379         if not self.w:
--> 380             self._solve()
    381         # 1) Compute the local max SR portfolio between any two neighbor turning points
    382         w_sr, sr = [], []

~/miniconda3/envs/ai/lib/python3.8/site-packages/pypfopt/cla.py in _solve(self)
    336                         self.w[-1][i],
    337                     )
--> 338                     if (self.ls[-1] is None or l < self.ls[-1]) and l > CLA._infnone(
    339                         l_out
    340                     ):

TypeError: '<' not supported between instances of 'NoneType' and 'float'

Any Idea why this happens?

Here is a sample of my mu and S

mu is

asset1      0.001109
asset2    0.001651
asset3       0.006850
asset4      0.008584
asset5      0.014223
asset6     0.000375
asset7      0.009315
asset8      0.006099
asset9      0.000802
asset10      0.000526
asset11      0.004706
asset12      0.012302
asset13    0.004301
asset14      0.001811
asset15     0.004071
asset16      0.006880
asset17       0.001191
asset18      0.000108
asset19       0.004835
asset20      0.000477
asset21     0.001538
asset22     0.003291
asset23       0.003846
asset24      0.003557
asset25     0.005364
asset26     0.000073
asset27       0.011400
asset28      0.001145
asset29      0.000955
asset30        0.003946
asset31       0.000931
asset32      0.011024
asset33      0.012595

Here is a preview of S, i cant paste the whole thing because its 33 by 33

0.025999 | 0.062508 | -0.058524 | -0.032703 | -0.009928 | 0.009853 | -0.016384 | 0.005907 | -0.010904 | 0.007083 | ... | -0.008924 | 0.002112 | 0.068362 | -0.016891 | 0.034906 | -0.000746 | 0.003678 | -0.031238 | -0.054474 | -0.041703
0.062508 | 0.159056 | -0.184588 | -0.089246 | -0.052268 | 0.020778 | -0.055927 | 0.002663 | -0.028131 | 0.009429 | ... | -0.031285 | -0.041631 | 0.134849 | -0.068001 | 0.077892 | -0.011617 | -0.003193 | -0.068701 | -0.139866 | -0.122891
-0.058524 | -0.184588 | 0.527175 | 0.156472 | 0.170847 | -0.004327 | 0.156618 | 0.090105 | 0.067120 | 0.036571 | ... | 0.144244 | 0.238333 | 0.068252 | 0.237680 | -0.082467 | 0.051147 | 0.016966 | 0.031354 | 0.218510 | 0.236453
-0.032703 | -0.089246 | 0.156472 | 0.063347 | 0.055401 | -0.004898 | 0.051392 | 0.015272 | 0.030596 | 0.004461 | ... | 0.043299 | 0.052218 | -0.035347 | 0.064125 | -0.048398 | 0.012993 | 0.004915 | 0.039782 | 0.095891 | 0.088705
-0.009928 | -0.052268 | 0.170847 | 0.055401 | 0.108775 | 0.011654 | 0.068900 | 0.038511 | 0.027025 | 0.025390 | ... | 0.050873 | 0.145137 | 0.076153 | 0.095925 | -0.005434 | 0.032286 | 0.037794 | 0.007141 | 0.065100 | 0.096941
0.009853 | 0.020778 | -0.004327 | -0.004898 | 0.011654 | 0.007455 | 0.003526 | 0.007720 | 0.004208 | 0.006834 | ... | 0.007199 | 0.013899 | 0.038925 | 0.003175 | 0.009831 | 0.003068 | 0.005427 | -0.006667 | -0.010487 | -0.004679
-0.016384 | -0.055927 | 0.156618 | 0.051392 | 0.068900 | 0.003526 | 0.053981 | 0.028695 | 0.026781 | 0.014705 | ... | 0.047741 | 0.085304 | 0.030648 | 0.074555 | -0.024080 | 0.019169 | 0.013931 | 0.015959 | 0.070154 | 0.079168
0.005907 | 0.002663 | 0.090105 | 0.015272 | 0.038511 | 0.007720 | 0.028695 | 0.028618 | 0.010712 | 0.015772 | ... | 0.032128 | 0.063601 | 0.074239 | 0.048253 | 0.005605 | 0.012862 | 0.007793 | -0.015116 | 0.014521 | 0.028828
-0.010904 | -0.028131 | 0.067120 | 0.030596 | 0.027025 | 0.004208 | 0.026781 | 0.010712 | 0.029829 | 0.004976 | ... | 0.035519 | 0.004307 | -0.003429 | 0.023013 | -0.032236 | 0.002741 | -0.003645 | 0.029937 | 0.050853 | 0.036014
0.007083 | 0.009429 | 0.036571 | 0.004461 | 0.025390 | 0.006834 | 0.014705 | 0.015772 | 0.004976 | 0.010393 | ... | 0.015242 | 0.040526 | 0.051256 | 0.023947 | 0.009532 | 0.008378 | 0.008900 | -0.010890 | 0.000245 | 0.012209
-0.018393 | -0.069133 | 0.227598 | 0.067385 | 0.096261 | 0.005353 | 0.075344 | 0.045440 | 0.032347 | 0.023110 | ... | 0.066856 | 0.131673 | 0.063500 | 0.110922 | -0.024627 | 0.028631 | 0.020088 | 0.009501 | 0.088525 | 0.107356
0.057139 | 0.073907 | 0.274531 | 0.011136 | 0.172500 | 0.037764 | 0.093012 | 0.116761 | -0.011275 | 0.074317 | ... | 0.073633 | 0.353416 | 0.395560 | 0.193212 | 0.115323 | 0.069389 | 0.076442 | -0.136390 | -0.045501 | 0.078905
0.038337 | 0.081170 | -0.103536 | -0.049892 | 0.013561 | 0.014638 | -0.021515 | 0.003763 | -0.032985 | 0.012921 | ... | -0.036439 | 0.059895 | 0.105703 | -0.015737 | 0.076885 | 0.011017 | 0.034312 | -0.057262 | -0.095340 | -0.047683
-0.012954 | -0.042376 | 0.141476 | 0.050583 | 0.062633 | 0.008706 | 0.052814 | 0.029027 | 0.041668 | 0.015277 | ... | 0.060369 | 0.053314 | 0.033513 | 0.061228 | -0.036157 | 0.013331 | 0.004701 | 0.029514 | 0.075413 | 0.069192
0.026521 | 0.067901 | -0.110517 | -0.045554 | -0.028227 | 0.006462 | -0.033031 | -0.007728 | -0.022021 | 0.000327 | ... | -0.030205 | -0.019917 | 0.041933 | -0.040456 | 0.041511 | -0.005503 | 0.003633 | -0.032209 | -0.072389 | -0.060127
-0.024764 | -0.069288 | 0.136418 | 0.053271 | 0.050663 | -0.001666 | 0.045828 | 0.016635 | 0.028722 | 0.006287 | ... | 0.041557 | 0.047903 | -0.016156 | 0.056905 | -0.039587 | 0.011811 | 0.004537 | 0.032090 | 0.080098 | 0.074627
-0.007158 | -0.009839 | -0.038815 | 0.005426 | -0.007343 | 0.001128 | -0.004438 | -0.014045 | 0.015434 | -0.006488 | ... | 0.001048 | -0.047118 | -0.046753 | -0.026473 | -0.022883 | -0.007873 | -0.005826 | 0.032945 | 0.017598 | -0.003002
0.002297 | 0.009139 | -0.006640 | 0.004161 | 0.002845 | 0.006780 | 0.004207 | 0.002098 | 0.019252 | 0.002285 | ... | 0.017175 | -0.024676 | 0.005541 | -0.008593 | -0.014634 | -0.003872 | -0.006188 | 0.017960 | 0.012130 | -0.002494
-0.000514 | -0.013022 | 0.057128 | 0.018267 | 0.044876 | 0.006702 | 0.025929 | 0.015512 | 0.010235 | 0.011358 | ... | 0.018793 | 0.059641 | 0.039099 | 0.035298 | 0.002611 | 0.013303 | 0.017474 | 0.000522 | 0.019212 | 0.034190
0.006165 | 0.011249 | 0.055896 | 0.019584 | 0.035611 | 0.015727 | 0.028602 | 0.022910 | 0.037483 | 0.014709 | ... | 0.048679 | 0.011585 | 0.057865 | 0.022782 | -0.019161 | 0.004276 | -0.001639 | 0.018895 | 0.031898 | 0.020793
0.056202 | 0.126471 | -0.078586 | -0.059277 | 0.006989 | 0.024380 | -0.017966 | 0.025391 | -0.020483 | 0.023254 | ... | -0.007343 | 0.050759 | 0.178932 | -0.007877 | 0.080281 | 0.008089 | 0.018929 | -0.073678 | -0.107334 | -0.066915
-0.034814 | -0.055969 | -0.021147 | 0.033343 | -0.046674 | -0.008647 | -0.004908 | -0.030342 | 0.050138 | -0.023730 | ... | 0.026168 | -0.160807 | -0.159354 | -0.053312 | -0.096384 | -0.029575 | -0.047406 | 0.096377 | 0.086423 | 0.005683
-0.007345 | -0.013142 | 0.012920 | 0.002835 | -0.018253 | -0.007195 | -0.004208 | -0.003650 | -0.002945 | -0.005997 | ... | -0.001627 | -0.020508 | -0.028693 | -0.001744 | -0.010866 | -0.004916 | -0.011852 | 0.002364 | 0.008777 | -0.000421
-0.008924 | -0.031285 | 0.144244 | 0.043299 | 0.050873 | 0.007199 | 0.047741 | 0.032128 | 0.035519 | 0.015242 | ... | 0.059033 | 0.049980 | 0.045391 | 0.061687 | -0.030197 | 0.011589 | -0.001153 | 0.016897 | 0.064281 | 0.058934
0.002112 | -0.041631 | 0.238333 | 0.052218 | 0.145137 | 0.013899 | 0.085304 | 0.063601 | 0.004307 | 0.040526 | ... | 0.049980 | 0.251829 | 0.164916 | 0.148482 | 0.037661 | 0.052183 | 0.061840 | -0.044085 | 0.039742 | 0.115733
0.068362 | 0.134849 | 0.068252 | -0.035347 | 0.076153 | 0.038925 | 0.030648 | 0.074239 | -0.003429 | 0.051256 | ... | 0.045391 | 0.164916 | 0.311828 | 0.073794 | 0.094396 | 0.031281 | 0.035819 | -0.101553 | -0.087288 | -0.019001
-0.016891 | -0.068001 | 0.237680 | 0.064125 | 0.095925 | 0.003175 | 0.074555 | 0.048253 | 0.023013 | 0.023947 | ... | 0.061687 | 0.148482 | 0.073794 | 0.118977 | -0.014769 | 0.031254 | 0.022565 | -0.004048 | 0.079883 | 0.107430
0.034906 | 0.077892 | -0.082467 | -0.048398 | -0.005434 | 0.009831 | -0.024080 | 0.005605 | -0.032236 | 0.009532 | ... | -0.030197 | 0.037661 | 0.094396 | -0.014769 | 0.066013 | 0.005539 | 0.018751 | -0.058114 | -0.087998 | -0.051547
-0.000746 | -0.011617 | 0.051147 | 0.012993 | 0.032286 | 0.003068 | 0.019169 | 0.012862 | 0.002741 | 0.008378 | ... | 0.011589 | 0.052183 | 0.031281 | 0.031254 | 0.005539 | 0.011027 | 0.013330 | -0.006052 | 0.011819 | 0.026687
0.003678 | -0.003193 | 0.016966 | 0.004915 | 0.037794 | 0.005427 | 0.013931 | 0.007793 | -0.003645 | 0.008900 | ... | -0.001153 | 0.061840 | 0.035819 | 0.022565 | 0.018751 | 0.013330 | 0.024158 | -0.009907 | -0.004099 | 0.020094
-0.031238 | -0.068701 | 0.031354 | 0.039782 | 0.007141 | -0.006667 | 0.015959 | -0.015116 | 0.029937 | -0.010890 | ... | 0.016897 | -0.044085 | -0.101553 | -0.004048 | -0.058114 | -0.006052 | -0.009907 | 0.062937 | 0.074137 | 0.040891
-0.054474 | -0.139866 | 0.218510 | 0.095891 | 0.065100 | -0.010487 | 0.070154 | 0.014521 | 0.050853 | 0.000245 | ... | 0.064281 | 0.039742 | -0.087288 | 0.079883 | -0.087998 | 0.011819 | -0.004099 | 0.074137 | 0.152636 | 0.126022
-0.041703 | -0.122891 | 0.236453 | 0.088705 | 0.096941 | -0.004679 | 0.079168 | 0.028828 | 0.036014 | 0.012209 | ... | 0.058934 | 0.115733 | -0.019001 | 0.107430 | -0.051547 | 0.026687 | 0.020094 | 0.040891 | 0.126022 | 0.133659

Any idea why the CLA fails?

[robertmartin8]

Hi @anarchy89, thanks for the detailed bug report!

There seems to be something strange going on with your expected returns and covariance matrix. The expected returns are very small compared to the volatilities – perhaps the returns haven't been annualised?

This might explain why even the EfficientFrontier code gives a weird result:

Expected annual return: 0.3%
Annual volatility: 0.0%
Sharpe Ratio: 4901822.92

[anarchy89]

the expect returns are actually daily, is the covariance matrix annualised or in percentage terms?

[robertmartin8]

They both need to have the same frequency (ideally annualised). The sample cov matrix calculated from PyPortfolioOpt is indeed annualised, so you should probably multiply your expected returns by 252.

…

On Wed, 17 Mar 2021, 15:39 anarchy89, ***@***.***> wrote: the expect returns are actually daily, is the covariance matrix annualised? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#313 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACZ7ECIM2PUFXYLD5IAFYPDTEBMDFANCNFSM4ZJISJ2Q> .

[anarchy89]

is there a way to un-annualise the covariance matrix? can i divide it by 252?

[robertmartin8]

@anarchy89 you can divide it by 252 to convert it from annualised to daily

[anarchy89]

ok thanks! let me give it a try

[anarchy89]

by the way, to annualise daily returns, i thought the formula was (1+dailyreturn)^252– 1

[anarchy89]

@robertmartin8 hey, annualised my daily returns by multiplying it by 252, it still doesnt work. i get the same TypeError: '<' not supported between instances of 'NoneType' and 'float'

[robertmartin8]

@anarchy89

i thought the formula was (1+dailyreturn)^252– 1

This is indeed more precise.

Can you paste the output of the EfficientFrontier code?

If there are no problems with the input data, I'm surprised that CLA isn't working (as it's functionality is covered by realistic unit tests).

Is there a reason why you need CLA over EfficientFrontier.max_sharpe?

[anarchy89]

i am just trying different methods to do a sanity check because im getting crazy huge numbers, after annualising the daily returns, i get this

Expected annual return: 90.0%
Annual volatility: 0.0%
Sharpe Ratio: 51202989.32

I dont understand how volatility can be 0

[robertmartin8]

@anarchy89 I suspect there is something irregular about the covariance matrix, e.g there is a zero volatility asset.

That, or you have linearly dependent columns that allow you to form "perfect hedges", e.g if one column is LONG AAPL and another is SHORT AAPL, their covariances will be -1 so you can get zero volatility that way.

Could you post the diagonals of your covariance matrix? e.g np.diag(S).

[anarchy89]

Hey @robertmartin8 you were right. I increased the length of the historical data. It has no problems now. Thanks a lot! By the way, on a separate note, is it possible to use gekko optimiser in your system ?

[robertmartin8]

Great, glad to hear it's been fixed.

on a separate note, is it possible to use gekko optimiser in your system?

PyPortfolioOpt's supported backends are any of the CVXPY backends (see here) or scipy. Is there a particular feature of Gekko that you find useful (just so I know)?

[robertmartin8]

Closing for now – feel free to open a separate issue to request features re Gekko

[anarchy89]

@robertmartin8 hey sorry for the delay was busy. I was just reading about the library in a few research papers and it looked pretty decent thats why I was asking. what i meant was, is it possible to attach it as an optimizer directly into your code, or does it have to be programmed by you specifically.