Update futures margins #4479

Jay-Jay-D · 2020-06-02T20:43:10Z

Kudos to @gsalaz98 for this awesome work!

Description

Update future margins data.

Related Issue

N/A

Motivation and Context

Better and updated margins improves futures backtesting modelling and behavior.

Requires Documentation Change

N/A

How Has This Been Tested?

N/A

Types of changes

Bug fix (non-breaking change which fixes an issue)
Refactor (non-breaking change which improves implementation)
Performance (non-breaking change which improves performance. Please add associated performance test and results)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Non-functional change (xml comments/documentation/etc)

Checklist:

My code follows the code style of this project.
I have read the CONTRIBUTING document.
I have added tests to cover my changes.
All new and existing tests passed.
My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>

gsalaz98

We'll need to do a bit more work to filter out and isolate some of these issues. Overall, this is pretty good 😃

Data/future/cme/margins/CJY.csv

Data/future/cbot/margins/AW.csv

Data/future/cbot/margins/BCF.csv

gsalaz98 · 2020-06-02T22:19:45Z

Data/future/cbot/margins/EH.csv

-# https://www.cmegroup.com/tools-information/lookups/advisories/clearing/Chadv13-594.html
+20130410,4180,3800
+20131202,2970,2700
+20131204,4180,3800


Seems like some of these old values' initial margins are not 110% of the maintenance.

Data/future/cbot/margins/ZF.csv

gsalaz98 · 2020-06-02T22:33:04Z

Data/future/cme/margins/GE.csv

 20140919,715,650
-20141017,632.5,575
+20140922,176,160


Pretty sporadic, though it seems standard for this contract. If I remember correctly, the Eurodollar contract's margins apply to tiers of contracts. Maybe the changes in tiers causes the value to be very volatile 🤔

gsalaz98 · 2020-06-02T22:35:29Z

Data/future/nymex/margins/1S.csv

+20150616,6025,5477
+20150617,5834,5303
+20150618,5636,5123
+20150619,5431,4937


Just some thoughts, no changes required:

These margins change every day. Maybe this is the percentage of a single contract that is required as initial margin rather than fixed portfolio values. Either way, this is going to be great for backtesting now that we have all of these margins.

gsalaz98 · 2020-06-02T22:36:30Z

Data/future/nymex/margins/A0D.csv

 20170310,1375,1250
-# https://www.cmegroup.com/notices/clearing/2017/05/Chadv17-167.html
+20170407,13750,12500


This is 10x?

Good catch!

Data/future/nymex/margins/A9N.csv

gsalaz98 · 2020-06-02T22:40:23Z

Data/future/nymex/margins/EWN.csv

 20131015,4950,4500
-# https://www.cmegroup.com/notices/clearing/2017/03/Chadv17-108.html
+20131016,16500,15000
+20131029,27500,25000


Wrong order of magnitude.

jaredbroad

Suspect margins

Data/future/cbot/margins/EH.csv

* Fixes previously incorrect margins

jaredbroad

Please sanity check these last bits and then should be good to go.

jaredbroad · 2020-06-10T04:01:04Z

Data/future/cme/margins/DC.csv

+20130320,550,500
+20130404,1430,1300
+20130417,550,500
+20130502,1430,1300


This one still seems not quite correct. Perhaps can just skip all the mid-month ones.

Despite the back and forth, this is one of the test cases I focused on early to ensure we had proper margins. I took a look closely, and this one is correct.

The margins change frequently because when the front-month contract expires, the new front-month contract still retains its old margin value, but it shifts to a lower margin value once that contract is closer to expiry.

jaredbroad · 2020-06-10T04:06:43Z

Data/future/comex/margins/AUP.csv

 20170303,385,350
-# https://www.cmegroup.com/notices/clearing/2017/05/Chadv17-183.html
-20170519,363,330
-# https://www.cmegroup.com/notices/clearing/2017/11/Chadv17-467.html
+20170306,348,316
+20170307,336,305
+20170308,324,294
+20170309,311,282


Wonder if we should increase this to every month minimum. Every 3 days is a little much for this data

jaredbroad · 2020-06-10T04:11:32Z

Data/future/nymex/margins/A0D.csv

-# https://www.cmegroup.com/tools-information/lookups/advisories/clearing/Chadv13-268.html
-20130611,2475,2250
-# https://www.cmegroup.com/tools-information/lookups/advisories/clearing/Chadv13-304.html
+20130102,85,77


This seems abnormally low

jaredbroad · 2020-06-10T04:13:32Z

Data/future/nymex/margins/A0F.csv

+20130206,72,65
+20130207,2200,2000


Front-month roll over?

jaredbroad · 2020-06-10T04:19:47Z

Data/future/nymex/margins/A9N.csv

+20130905,8517,7742
+20130906,6387,5806
+20130909,61600,56000
+20130910,66000,60000
 20140421,52800,48000
-# https://www.cmegroup.com/tools-information/lookups/advisories/clearing/Chadv14-279.html
 20140714,45100,41000
-# https://www.cmegroup.com/tools-information/lookups/advisories/clearing/Chadv14-469.html
 20141205,55000,50000


Why do you think this shifts from every 3 days to every few months?

My guess is that CME decided to switch from percentage values to a fixed dollar value around the time of the 2013-09-09 entry, but I have no known way to confirm this. CME currently shows fixed margin values on their margins page, but those that have percentage margins tended to behave like this contract did early on.

* Only the first margin of a percentage margin will be used from now onwards

jaredbroad · 2020-06-24T20:53:04Z

Known issues for posterity:

1 - These are the margins of the first contract on-chain - at least a month out. The real margins approaching expiration are much lower. Including every single day's update is a lot of unnecessary data that doesn't improve the platform much.

2 - LEAN margin modeling for these contracts is a large approximation and should actually be per contract not a fixed number for all contracts. Rewrite needed to move these into an algorithm/approximation based on the SPAN risk formula.

That said -- its still a large improvement with many new, previously unmodelled contracts added so will merge as is and plan for rewrite of futures margins at a later date.

Review addressed

Update futures margins

a335690

Jay-Jay-D requested a review from gsalaz98 June 2, 2020 20:43

Jay-Jay-D self-assigned this Jun 2, 2020

Jay-Jay-D added the data-integrity Tickets relating to keeping the LEAN data accurate. label Jun 2, 2020

gsalaz98 previously requested changes Jun 2, 2020

View reviewed changes

gsalaz98 added 2 commits June 3, 2020 15:20

Address review - removes duplicates in margins and skip non-USD margins

b0f787a

Removes bad margin files and fixes unit tests

b638c45

jaredbroad reviewed Jun 5, 2020

View reviewed changes

Data/future/cbot/margins/EH.csv Outdated Show resolved Hide resolved

Data/future/cbot/margins/EH.csv Outdated Show resolved Hide resolved

gsalaz98 added 2 commits June 8, 2020 17:55

Updates margins using new contract selection strategy

81f2c43

* Fixes previously incorrect margins

Address self-review: Fixes FuturesMarginBuyingPowerModelTests

89015db

jaredbroad reviewed Jun 10, 2020

View reviewed changes

jaredbroad assigned gsalaz98 Jun 12, 2020

gsalaz98 added 2 commits June 15, 2020 16:00

Address review: Removes changing percentage margins

3baf8c3

* Only the first margin of a percentage margin will be used from now onwards

Address review: Cleans up percentage margins and adds missing margins

8d0d272

Martin-Molinero approved these changes Jun 24, 2020

View reviewed changes

Martin-Molinero merged commit c3062e4 into master Jun 24, 2020

Martin-Molinero deleted the update-margins branch August 18, 2020 18:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update futures margins #4479

Update futures margins #4479

Jay-Jay-D commented Jun 2, 2020 •

edited

Loading

gsalaz98 left a comment

gsalaz98 Jun 2, 2020

gsalaz98 Jun 2, 2020

gsalaz98 Jun 2, 2020

gsalaz98 Jun 2, 2020

Jay-Jay-D Jun 3, 2020

gsalaz98 Jun 2, 2020

jaredbroad left a comment

jaredbroad left a comment

jaredbroad Jun 10, 2020

gsalaz98 Jun 10, 2020 •

edited

Loading

jaredbroad Jun 10, 2020

jaredbroad Jun 10, 2020

jaredbroad Jun 10, 2020

jaredbroad Jun 10, 2020

gsalaz98 Jun 10, 2020

jaredbroad commented Jun 24, 2020

		20130206,72,65
		20130207,2200,2000

Update futures margins #4479

Update futures margins #4479

Conversation

Jay-Jay-D commented Jun 2, 2020 • edited Loading

Kudos to @gsalaz98 for this awesome work!

Description

Related Issue

Motivation and Context

Requires Documentation Change

How Has This Been Tested?

Types of changes

Checklist:

gsalaz98 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jaredbroad left a comment

Choose a reason for hiding this comment

jaredbroad left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gsalaz98 Jun 10, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jaredbroad commented Jun 24, 2020

Jay-Jay-D commented Jun 2, 2020 •

edited

Loading

gsalaz98 Jun 10, 2020 •

edited

Loading