Positions of removed securities are always liquidated during rebalance even with active insights

[fishstoryyy]

Expected Behavior

In the PortfolioConstructionModel class, the CreateTargets method should only create a flatten target for securities that are removed from the universe when we want to liquidate removed securities, i.g., their insights expired.

Actual Behavior

If rebalance is required (for whatever reason), CreateTargets always creates a flatten target for securities that are removed from the universe (even if the insights are still active on those securities).

The issue is that inside the CreateTargets method, once IsRebalanceDue[source] returns true for whatever reason, rebalance will set in and these lines [source] will always be executed to liquidate removed securities, even if RebalanceOnSecurityChanges is False.

Potential Solution

My solution is to add RebalanceOnSecurityChanges as a condition to this line. Please refer to my fork here:

# if RebalanceOnSecurityChanges is False, we can continue rebalance and let the insight 
# period on those removed securities to determine whether they should be liquidated or not   
if (_removedSymbols != null && RebalanceOnSecurityChanges)        
{
var universeDeselectionTargets = _removedSymbols.Select(symbol => new PortfolioTarget(symbol, 0));  
targets.AddRange(universeDeselectionTargets);                
_removedSymbols = null;
}

RebalanceOnSecurityChanges is True by default, so a position will be liquidated by default if the security is removed from the universe. With this solution, if RebalanceOnSecurityChanges is set to False, the rebalance process is no longer liquidating a position just because the security is removed from the universe (liquidation might still happen later during the rebalance process, if insight expires).

Reproducing the Problem

The following algorithm reproduces the issue:

class badSell(QCAlgorithm):

    def Initialize(self):
        self.SetStartDate(2021, 5, 20)
        self.SetEndDate(2021, 5, 22)
        self.SetCash(100000) 
        self.Settings.RebalancePortfolioOnSecurityChanges = False
        
        self.AddUniverse(self.coarse)
        self.UniverseSettings.Resolution = Resolution.Daily
        
        self.SetAlpha(ConstantAlphaModel(InsightType.Price, InsightDirection.Up, timedelta(days=2))
        )
        
        
        self.SetPortfolioConstruction( EqualWeightingPortfolioConstructionModel(lambda time: None) )
        
        self.SetExecution(ImmediateExecutionModel() )
        
        tickers = ['SPY', 'AMZN']
        self.symbols = [Symbol.Create(ticker, SecurityType.Equity, Market.USA) for ticker in tickers]
        
        self.calls = 0

    def coarse(self, coarse):
        if self.calls == 0:
            self.calls += 1
            # return SPY
            return [self.symbols[0]]
        elif self.calls == 1:
            self.calls -= 1
            # return amazon and SPY is liquidated
            return [self.symbols[1]]

In the algorithm,

1. SPY and AMZN is taking turns to be included in the universe, so securities are being added and removed.
2. insight lasts for 2 days
3. RebalancePortfolioOnSecurityChanges is set to False and daily automatic rebalance is turned off by lambda time: None to make sure we are rebalancing only because there is a new insight on every other day and NOT because one of the securities is removed from the universe.
4. Nevertheless, the arrival of a new insight for AMZN triggers rebalance on the second day, so SPY was liquidated [source] even if its 2-day insight is still active.

System Information

Checklist

• I have completely filled out this template
• I have confirmed that this issue exists on the current master branch
• I have confirmed that this is not a duplicate issue by searching issues

• I have provided detailed steps to reproduce the issue

[fishstoryyy]

Moreover, the comment in this thread suggests that we need to do the following to make sure positions are not liquidates on removed securities:

self.Settings.RebalancePortfolioOnSecurityChanges = False
self.SetPortfolioConstruction( EqualWeightingPortfolioConstructionModel(lambda time: None) ) # not good

The fact that we need to turn off rebalance (second line) is not desirable because we may still want to rebalance daily to achieve an ideal portfolio weight.

With this issue fixed, only self.Settings.RebalancePortfolioOnSecurityChanges = False is needed and we can still rebalance our portfolio however we want.

[jaredbroad]

Behavior adjusted by custom PCM, overriding the settings as you've shown.

Not sure its really a bug if its removed from the universe could argue it shouldn't be traded.