Fixings are thrown out, breaking valuation of several instruments

[gbfredrik]

Hi,
A bug seems to have been introduced with ORE 13 which is causing fixings to be disregarded because they are not registered as required. This means it is no longer possible to value a portfolio of e.g. FX average forwards . I believe the behavior stems from this commit, specifically in MarketDataLoader::populateFixings: d0e0c00.

The cause
After the changes, the default behavior is no longer to load all fixings provided. Instead, it:

1. asks the portfolio for its required fixings. (See lines 170-175 below). This always returns an empty map because the trades have not been built yet, consistent with the warning in the docstring of Trade::fixings.

2. asks some market data objects (specifically IndexForwardingCurves, Zero/YoYInflationCurves, and CommodityCurves) if they require fixings (see lines 178-180 below). If you have, say, USD-SOFR under your IndexForwardingCurves, then your SOFR fixings will be marked as necessary. On the other hand, if your market only has discounting and generic yield curves, no fixings are flagged for usage.

Then, only if steps 1 and/or 2 yielded any required fixings, the condition on line 191 will evaluate to true. If so, MarketDataCsvLoaderImpl::retrieveFixings will trigger and by default load any other fixings provided to ORE in the CSV.

Thus, if I run a portfolio of only FX average forwards (assuming no index, inflation, or commodity curves are used), it will not keep any of the FX fixings provided and then they cannot be priced. This can happen for any portfolio not using the curve types mentioned.

Engine/OREAnalytics/orea/app/marketdataloader.cpp

Lines 162 to 194 in df9d768

	void MarketDataLoader::populateFixings(
	const std::vector<QuantLib::ext::shared_ptr<ore::data::TodaysMarketParameters>>& todaysMarketParameters,
	const std::set<QuantLib::Date>& loaderDates) {
	LOG("Asking portfolio for its required fixings");
	FixingMap portfolioFixings;
	std::map<std::pair<std::string, QuantLib::Date>, std::set<QuantLib::Date>> lastAvailableFixingLookupMap;
	// portfolio fixings will warn if missing
	if (inputs_->portfolio()) {
	portfolioFixings = inputs_->portfolio()->fixings();
	LOG("The portfolio depends on fixings from " << portfolioFixings.size() << " indices");
	for (const auto& it : portfolioFixings)
	addRelevantFixings(it, lastAvailableFixingLookupMap);
	}
	LOG("Add fixings possibly required for bootstrapping TodaysMarket");
	for (const auto& tmp : todaysMarketParameters) {
	for (const auto d : loaderDates)
	addMarketFixingDates(d, fixings_, *tmp);
	LOG("Add fixing possibly required for equity index delta risk decomposition")
	additional_equity_fixings(fixings_, *tmp, inputs_->refDataManager(),
	inputs_->curveConfigs().get());
	}
	if (inputs_->eomInflationFixings()) {
	LOG("Adjust inflation fixing dates to the end of the month before the request");
	amendInflationFixingDates(fixings_);
	}
	if (fixings_.size() > 0 && impl_)
	impl()->retrieveFixings(loader_, fixings_, lastAvailableFixingLookupMap);
	applyFixings(loader_->loadFixings());

The solution
Hoping for your input here! I imagine step 1 should be able to retrieve the trades' required fixings, then the condition would correctly evaluate to true and load them in. I can naively avoid the issue like this for the time being by forcing fixings to be retrieved (but that's likely not the way to go for a proper solution):

Any feedback on this would be much appreciated. I can provide a minimal input data set to replicate the issue, if you're interested.

I also want to offer a major thank you for the work that went into the 13th release; it is very valuable to us!

Kind regards,
Fredrik

[pcaspers]

Hi @gbfredrik, sorry for the very late response. In case we load all fixings, which is the default when running ore app, we should always call impl()->retrieveFixings(), i.e.:

    if ((inputs_->allFixings() || fixings_.size() > 0) && impl_)
        impl()->retrieveFixings(loader_, fixings_, lastAvailableFixingLookupMap);

In case allFixings() is false we need to build the portfolio against a dummy market before asking it for its required fixings. I will make the related code changes now and we will release them in version 14 which is due at the beginning of November.

Many thanks for the heads up!

[gbfredrik]

Hi @pcaspers,
Thanks for getting back on this issue! We're looking forward to the release.

/Fredrik