tsfresh associate feature with different date (ID)

[popohoma999]

Hi Team,

I am new to tsfresh and I am excited about it's power for extracting features based on statistics. I have extract features using time series on a stock across a year time while each date has around 400 slices/rows having 1 minute internal in between.

I am attaching my sample data here

I have extracted features and they display in Date (ID) level. Is there any factors that can also associate features across multiple Dates/ID ?

As now it looks to me each row is independent and different dates/ID don't talk to each other.

Thanks,

PoPo

[nils-braun]

Hi PoPo,

that is correct - IDs are treated independently. Depending on your use-case, you could however extract features in two iterations: first for each date independently (by using the Date as the ID) and then maybe per week/month/whatever-makes-sense-to-you. You can do the latter by adding a new ID column which contains the month number/week number etc. Later for your use-case you could use all features together.

Hope this helps!

[popohoma999]

Thank you for your reply Nils. Another way I can think of is referring to your jupyter notebook sample (https://github.com/blue-yonder/tsfresh/blob/main/notebooks/05%20Timeseries%20Forecasting.ipynb), using your roll_time_series function

So the new "id" combined minimum of 5 dates of time series, and will roll up to 365 days for the data evolved to 1 year time, so that I can extract features based on the new "id"

Do you think that works? Last line should extract the feature for me based on 1 year (defined as max 365 days) time series and it should have memory for the price movement in 1 year.

Another question is, I am focusing on your given feature "linear trend slope". For my case, given I undersetand correctly, it should calculate the daily slope from 1 year ago, accumulate for 1 year and averaging the result out right? What exactly is this "linear trend slope chunk len 5 mean" or "linear trend slope chunk len 6 mean" ? I thought it will look at latest 5 or 10 days window but seems not.

Please share some insight as your library is great but need to ensure we know how to use it.

Thanks,

PoPo

[popohoma999]

Hi Nils - Could you shed some light on this?

[nils-braun]

Hi @popohoma999 ! Sorry, I missed your question.
I assume you are referring to the agg_linear_trend feature. The feature calculator does a linear least-squares regression on a lower-sampled version of the time series values. The chunk len defines how many values are aggregated. For example, a chunk len of 5 means, chunks of 5 time series values are aggregated (e.g. with a mean) and then a linear regression is computed on the aggregated values. So I think your understanding is basically the other way round: it will first aggregated in chunks and then calculate the linear regression.

To make it clear on an example, the feature agg_linear_trend__attr_"slope"__chunk_len_5__f_agg_"max" will first take chunks of 5 data points and take the maximum of them, producing a time series which is downsampled by a factor of 5. Then, it will calculate the slope of these samples.

Do you think that works?

Sure, that will work! I can not tell you if it makes sense for your use-case (you know best about this :)) but technically it sounds correct!

[popohoma]

Thank you Nils. Is there a plan for tsFresh to continue times serious pattern as linear upward, seasonal, exponential decay, constabt steady, etf so we can easily label it out?

[nils-braun]

I currently do not have the bandwidth for new feature development, but I am happy for any contribution! If someone want to build these features I am happy to help and review :-)

[popohoma999]

Got it Nils

…

On Sat, Apr 8, 2023, 04:19 Nils Braun ***@***.***> wrote: I currently do not have the bandwidth for new feature development, but I am happy for any contribution! If someone want to build these features I am happy to help and review :-) — Reply to this email directly, view it on GitHub <#1008 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A6IPOI7AYOVGVNHDRARZV23XABZD3ANCNFSM6AAAAAAVPLU6LI> . You are receiving this because you were mentioned.Message ID: ***@***.***>