Local testing broken, seems related to market consistency issue

[reumle]

Hello, thank you for this package!

test results.

some what long session log, can skip to the bottom.

PS C:\proj\PS1908\julia\ESG_24T2\EconomicScenarioGenerators.jl\test> julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.10.2 (2024-03-01)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.10) pkg> activate .
  Activating project at `C:\proj\PS1908\julia\ESG_24T2\EconomicScenarioGenerators.jl\test`

(test) pkg> st
Status `C:\proj\PS1908\julia\ESG_24T2\EconomicScenarioGenerators.jl\test\Project.toml`
  [ae264745] Copulas v0.1.23
  [31c24e10] Distributions v0.25.108
  [1d18cbdc] EconomicScenarioGenerators v0.6.0 `C:\proj\PS1908\julia\ESG_24T2\EconomicScenarioGenerators.jl#main`
  [b9b1ffdd] FinanceCore v2.1.1
  [77f2ae65] FinanceModels v4.9.1
  [09f84164] HypothesisTests v0.11.0
  [860ef19b] StableRNGs v1.0.2
  [2913bbd2] StatsBase v0.34.3
  [28d57a85] Transducers v0.4.81
  [8dfed614] Test

(test) pkg> test EconomicScenarioGenerators
     Testing EconomicScenarioGenerators
      Status `C:\Users\elm\AppData\Local\Temp\jl_Bka5oe\Project.toml`
  [ae264745] Copulas v0.1.23
  [31c24e10] Distributions v0.25.108
  [1d18cbdc] EconomicScenarioGenerators v0.6.0 `C:\proj\PS1908\julia\ESG_24T2\EconomicScenarioGenerators.jl#main`
  [b9b1ffdd] FinanceCore v2.1.1
  [77f2ae65] FinanceModels v4.9.1
  [09f84164] HypothesisTests v0.11.0
  [860ef19b] StableRNGs v1.0.2
  [2913bbd2] StatsBase v0.34.3
  [28d57a85] Transducers v0.4.81
  [8dfed614] Test
      Status `C:\Users\elm\AppData\Local\Temp\jl_Bka5oe\Manifest.toml`
⌅ [47edcb42] ADTypes v0.2.7
  [1520ce14] AbstractTrees v0.4.5
  [d88a00a0] AccessibleOptimization v0.1.1
  [7d9f7c33] Accessors v0.1.36
  [33016aad] AccessorsExtra v0.1.68
  [79e6a3ab] Adapt v4.0.4
  [66dad0bd] AliasTables v1.0.0
  [dce04be8] ArgCheck v2.3.0
  [4fba245c] ArrayInterface v7.10.0
  [198e06fe] BangBang v0.4.1
  [9718e550] Baselet v0.1.1
  [62783981] BitTwiddlingConvenienceFunctions v0.1.5
  [2a0fbf3d] CPUSummary v0.2.4
  [49dc2e85] Calculus v0.5.1
  [d360d2e6] ChainRulesCore v1.23.0
  [fb6a15b2] CloseOpenIntervals v0.1.12
  [861a8166] Combinatorics v1.0.2
  [38540f10] CommonSolve v0.2.4
  [bbf7d656] CommonSubexpressions v0.3.0
  [34da2185] Compat v4.14.0
  [a33af91c] CompositionsBase v0.1.2
  [88cd18e8] ConsoleProgressMonitor v0.1.2
  [187b0558] ConstructionBase v1.5.5
  [ae264745] Copulas v0.1.23
  [adafc99b] CpuId v0.3.1
  [a8cc5b0e] Crayons v4.1.1
  [667455a9] Cubature v1.5.1
  [9a962f9c] DataAPI v1.16.0
  [82cc6244] DataInterpolations v4.7.2
  [02685ad9] DataPipes v0.3.14
  [864edb3b] DataStructures v0.18.20
  [e2d170a0] DataValueInterfaces v1.0.0
  [244e2a9f] DefineSingletons v0.1.2
  [163ba53b] DiffResults v1.1.0
  [b552c78f] DiffRules v1.15.1
  [b4f34e82] Distances v0.10.11
  [31c24e10] Distributions v0.25.108
  [ffbed154] DocStringExtensions v0.9.3
  [fa6b7ba4] DualNumbers v0.6.8
  [1d18cbdc] EconomicScenarioGenerators v0.6.0 `C:\proj\PS1908\julia\ESG_24T2\EconomicScenarioGenerators.jl#main`
  [4e289a0a] EnumX v1.0.4
  [e2ba6199] ExprTools v0.1.10
  [1a297f60] FillArrays v1.10.0
  [b9b1ffdd] FinanceCore v2.1.1
  [77f2ae65] FinanceModels v4.9.1
  [64ca27bc] FindFirstFunctions v1.2.0
  [f6369f11] ForwardDiff v0.10.36
  [069b7b12] FunctionWrappers v1.1.3
  [77dc65aa] FunctionWrappersWrappers v0.1.3
  [46192b85] GPUArraysCore v0.1.6
  [3e5b6fbb] HostCPUFeatures v0.1.16
  [34004b35] HypergeometricFunctions v0.3.23
  [09f84164] HypothesisTests v0.11.0
  [615f187c] IfElse v0.1.1
  [22cec73e] InitialValues v0.3.1
  [18e54dd8] IntegerMathUtils v0.1.2
  [8197267c] IntervalSets v0.7.10
  [3587e190] InverseFunctions v0.1.13
  [92d709cd] IrrationalConstants v0.2.2
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.5.0
  [358108f5] JMcDM v0.7.15
  [b964fa9f] LaTeXStrings v1.3.1
  [10f19ff3] LayoutPointers v0.1.15
  [1d6d02ad] LeftChildRightSiblingTrees v0.2.0
  [2ab3a3ac] LogExpFunctions v0.3.27
  [e6f89c97] LoggingExtras v1.0.3
  [bdcacae8] LoopVectorization v0.12.169
  [1914dd2f] MacroTools v0.5.13
  [d125e4d3] ManualMemory v0.1.8
  [bcdb8e00] Metaheuristics v3.3.5
  [128add7d] MicroCollections v0.2.0
  [e1d29d7a] Missings v1.2.0
  [37188c8d] MvNormalCDF v0.3.1
  [77ba4419] NaNMath v1.0.2
  [6fe1bfb0] OffsetArrays v1.14.0
  [7f7a1694] Optimization v3.24.3
  [bca83a33] OptimizationBase v0.0.5
  [3aafef2f] OptimizationMetaheuristics v0.2.0
  [bac558e1] OrderedCollections v1.6.3
  [90014a1f] PDMats v0.11.31
  [1d0040c9] PolyesterWeave v0.2.1
  [aea7be01] PrecompileTools v1.2.1
  [21216c6a] Preferences v1.4.3
  [08abe8d2] PrettyTables v2.3.1
  [27ebfcd6] Primes v0.5.6
  [33c8b6b6] ProgressLogging v0.1.4
  [92933f4c] ProgressMeter v1.10.0
  [1fd47b50] QuadGK v2.9.4
  [3cdcf5f2] RecipesBase v1.3.4
  [731186ca] RecursiveArrayTools v3.13.0
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.0
  [79098fc4] Rmath v0.7.1
  [f2b01f46] Roots v2.1.5
  [7e49a35a] RuntimeGeneratedFunctions v0.5.13
  [94e857df] SIMDTypes v0.1.0
  [476501e8] SLEEFPirates v0.6.42
  [0bca4576] SciMLBase v2.34.0
  [c0aeaf25] SciMLOperators v0.3.8
  [53ae85a6] SciMLStructures v1.1.0
  [eb7571c6] SearchSpaces v0.2.0
  [efcf1570] Setfield v1.1.1
  [66db9d55] SnoopPrecompile v1.0.3
  [a2af1166] SortingAlgorithms v1.2.1
  [276daf66] SpecialFunctions v2.3.1
  [171d559e] SplittablesBase v0.1.15
  [860ef19b] StableRNGs v1.0.2
  [aedffcd0] Static v0.8.10
  [0d7ed370] StaticArrayInterface v1.5.0
  [90137ffa] StaticArrays v1.9.3
  [1e83bf80] StaticArraysCore v1.4.2
  [82ae8749] StatsAPI v1.7.0
  [2913bbd2] StatsBase v0.34.3
  [4c63d2b9] StatsFuns v1.3.1
  [892a3eda] StringManipulation v0.3.4
  [2efcf032] SymbolicIndexingInterface v0.3.16
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.11.1
⌅ [6aa5eb33] TaylorSeries v0.16.0
  [5d786b92] TerminalLoggers v0.1.7
  [8290d209] ThreadingUtilities v0.5.2
  [28d57a85] Transducers v0.4.81
  [3a884ed6] UnPack v1.0.2
  [3d5dd08c] VectorizationBase v0.21.66
  [48feb556] WilliamsonTransforms v0.1.4
  [7bc98958] Cubature_jll v1.0.5+0
  [efe28fd5] OpenSpecFun_jll v0.5.5+0
  [f50d1b31] Rmath_jll v0.4.0+0
  [0dad84c5] ArgTools v1.1.1
  [56f22d72] Artifacts
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8ba89e20] Distributed
  [f43a241f] Downloads v1.6.0
  [7b1f6079] FileWatching
  [9fa8497b] Future
  [b77e0a4c] InteractiveUtils
  [b27032c2] LibCURL v0.6.4
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [ca575930] NetworkOptions v1.2.0
  [44cfe95a] Pkg v1.10.0
  [de0858da] Printf
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization
  [6462fe0b] Sockets
  [2f01184e] SparseArrays v1.10.0
  [10745b16] Statistics v1.10.0
  [4607b0f0] SuiteSparse
  [fa267f1f] TOML v1.0.3
  [a4e569a6] Tar v1.10.0
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
  [e66e0078] CompilerSupportLibraries_jll v1.1.0+0
  [deac9b47] LibCURL_jll v8.4.0+0
  [e37daf67] LibGit2_jll v1.6.4+0
  [29816b5a] LibSSH2_jll v1.11.0+1
  [c8ffd9c3] MbedTLS_jll v2.28.2+1
  [14a3606d] MozillaCACerts_jll v2023.1.10
  [4536629a] OpenBLAS_jll v0.3.23+4
  [05823500] OpenLibm_jll v0.8.1+2
  [bea87d4a] SuiteSparse_jll v7.2.1+1
  [83775a58] Zlib_jll v1.2.13+1
  [8e850b90] libblastrampoline_jll v5.8.0+1
  [8e850ede] nghttp2_jll v1.52.0+1
  [3f19e933] p7zip_jll v17.4.0+2
        Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading.Precompiling project...
  2 dependencies successfully precompiled in 15 seconds. 184 already precompiled.
     Testing Running tests...
Market Consistency: Test Failed at C:\Users\elm\.julia\packages\EconomicScenarioGenerators\QS4Ec\test\interest.jl:116
  Expression: ≈(mean(samples), market_price, rtol = 0.01)
   Evaluated: 155.73565310766418 ≈ 165.10176108340627 (rtol=0.01)

Stacktrace:
  [1] macro expansion
    @ C:\desktop\Julia-1.10.2\share\julia\stdlib\v1.10\Test\src\Test.jl:672 [inlined]
  [2] macro expansion
    @ C:\Users\elm\.julia\packages\EconomicScenarioGenerators\QS4Ec\test\interest.jl:116 [inlined]
  [3] macro expansion
    @ C:\desktop\Julia-1.10.2\share\julia\stdlib\v1.10\Test\src\Test.jl:1577 [inlined]
  [4] macro expansion
    @ C:\Users\elm\.julia\packages\EconomicScenarioGenerators\QS4Ec\test\interest.jl:112 [inlined]
  [5] macro expansion
    @ C:\desktop\Julia-1.10.2\share\julia\stdlib\v1.10\Test\src\Test.jl:1577 [inlined]
  [6] macro expansion
    @ C:\Users\elm\.julia\packages\EconomicScenarioGenerators\QS4Ec\test\interest.jl:77 [inlined]
  [7] macro expansion
    @ C:\desktop\Julia-1.10.2\share\julia\stdlib\v1.10\Test\src\Test.jl:1577 [inlined]
  [8] macro expansion
    @ C:\Users\elm\.julia\packages\EconomicScenarioGenerators\QS4Ec\test\interest.jl:76 [inlined]
  [9] macro expansion
    @ C:\desktop\Julia-1.10.2\share\julia\stdlib\v1.10\Test\src\Test.jl:1577 [inlined]
 [10] top-level scope
    @ C:\Users\elm\.julia\packages\EconomicScenarioGenerators\QS4Ec\test\interest.jl:4
Test Summary:            | Pass  Fail  Total  Time
InterestRateModel        |   19     1     20  7.7s
  Vasicek                |    6            6  0.9s
  CoxIngersollRoss       |    6            6  0.3s
  Hull White             |    7     1      8  6.5s
    with AbstractYield   |    3     1      4  5.5s
      Market Consistency |          1      1  2.1s
    with Rate            |    4            4  1.0s
ERROR: LoadError: Some tests did not pass: 19 passed, 1 failed, 0 errored, 0 broken.
in expression starting at C:\Users\elm\.julia\packages\EconomicScenarioGenerators\QS4Ec\test\interest.jl:2
in expression starting at C:\Users\elm\.julia\packages\EconomicScenarioGenerators\QS4Ec\test\runtests.jl:13
ERROR: Package EconomicScenarioGenerators errored during testing

some context

I ran tests because i had somewhat strange results running the instructions form the help pages

using FinanceModels, EconomicScenarioGenerators
rates =[0.01, 0.01, 0.03, 0.05, 0.07, 0.16, 0.35, 0.92, 1.40, 1.74, 2.31, 2.41] ./ 100
mats = [1/12, 2/12, 3/12, 6/12, 1, 2, 3, 5, 7, 10, 20, 30]


curve = FinanceModels.fit(
    Spline.Cubic(),
    CMTYield.(rates,mats),
    Fit.Bootstrap()
)


m = HullWhite(.1,.002,curve) # a, σ, curve

s = ScenarioGenerator(
    # 1/12,  # timestep
    1,  # timestep
        30., # projection horizon
        m,  # model
    )

n = 1000
curves = [YieldCurve(s) for i in 1:n];

using Plots

times = 1:30
p=plot(title="EconomicScenarioGenerators.jl Hull White Model")

# plot the zero rates
for d in curves
    plot!(p,times,rate.(zero.(d,times)),alpha=0.2,label="")
end

plot!(times,rate.(zero.(curve,times)),line=(:black, 5), label="Given Yield Curve")
p
    

I systematically got charts like here, where simulated curves were shifted upwards as compared to the initial curve, which does not look right : The scenarios should average to the input market data, so as to be market consistent. I am not completely sure if this applies in the price space or yield space, but anyway the chart seems off.

See attached file (chart saved from vscode into an SVG file, need to load an svg extension in VSCode in order to display it there)

• It looks related to issue Hull White Market Consistency #28 and Resolve Hull White issues (market consistency) #32 ?

Happy to help!

[alecloudenback]

Thanks for the report, I can try to take a look this week. One suspicion is that the default rate from some types of Yield curves (in the Yields.jl package, the predecessor to the current FinanceModels package) returned rate types of Periodic instead of Continuous. All of the yield models from Finance Models now return a continuous type. Just a theory but one of the places I’d start looking.

[alecloudenback]

A bit more color: I recently migrate one of the dependencies in FinanceModels from BSplineKit to DataInterpolations (which is more lightweight and loads faster). This is related to the functionality that provides the interpolations for the curves. Something in that isn't working, so as a temporary fix, I have limited the compat of FinanceModels to use the version prior to the upstream change. Updating/resolving your environment should fix this (after the new version gets merged into General) and I've created a couple of issues to track the follow up. LMK if you still need help or encounter other issues.

[reumle]

Thank you Alec, tests pass!