standard deviation with scope

[HuifangYeo]

Hi,

I'm trying to take the difference between two measures and apply standard deviation across the historical dates:

How do I view the differences at the historical dates level and the std at the portfolio level?

[Skyscimitar]

Hi Huifang,
Could you share more details on how your cube is setup please?
With your current screenshot I'm not sure I understand what your question is.
From what I can see on the screenshot, the Daily PnL and Benchmark measures are not defined on the portfolio level so it's normal that the Difference measure has no value there either. The Error tracking measure which you aggregate with a scope on l["Historical Dates"] is correctly displayed on the portfolio level.
If you want to see the Daily PnL, Benchmarks and Differences measures at the portfolio level then you most likely need to use an aggrgegation function on them with a scope on the l["Historical Dates" level.

[HuifangYeo]

Hi Daniel,
below is my cube:

What I meant is that I have values for the Daily PnL at the Historical Date level, and Difference also showed up at the Historical Date level. However, for Error tracking, the values at the Historical Date is all 0.0000 instead of the value m["Daily PnL"] - m["Benchmark"] which I have defined:

m["Error tracking"] = tt.agg.std(
    m["Daily PnL"] - m["Benchmark"], mode="population", scope=tt.OriginScope(l["Historical Dates"])
)

My understanding is, at the scope level or below, it should apply m["Daily PnL"] - m["Benchmark"].
Above the scope (which means the portfolio level), it should apply std to the underlying data.

[Skyscimitar]

Thank you for the more detailed explanation, I understand the issue now.
The behavior you described is correct, the original measure should keep it's values at scope level or below. I'll let you know when we have a fix for this.
In the meantime you can use the tt.where function to force the correct value on the HistoricalDate level:

m["PnL vs Benchmark"] = m["Daily Pnl"] - m["Benchmark"]
m["Std"] = tt.agg.std(m["PnL vs Benchmark"], scope=tt.OriginScope(l["Historical Dates"]))
m["Error tracking"] = tt.where(l["Historical Dates"].isnull(), m["Std"], m["PnL vs Benchmark"])

[HuifangYeo]

Thanks Daniel!

[alexis-bvq]

Coming back on this ticket, isn't it normal that we have 0 under the scoped level?
Std is the difference between the initial value and the mean of it, down to the scoped levels the mean is the value therefore the diff is 0.
Therefore the post pro is just computing the std function down to the scoped level that return 0, it is therefore expected to specifically use a where clause if you want a different result under this scoped level.

Maybe I missunderstand the std function but just wanted to make sure we are not creating an issue out of a normal behaviour.

[Skyscimitar]

Technically yes, the value of std at scope level is indeed 0.

However since this is an aggregation function the implied contract for my point of view is that the underlying measure remain unchanged below the scope level.

Both behaviors seem acceptable, but I think for functions like std you don't expect to get the std on the values you are computing it for since it's 0 everywhere, it's rather useless.