Outlier detection failed: issue with reading timestamps?

[statsguy1995]

I have a pandas dataframe dat with two columns, the first one having timestamps, named 'time', and the second one containing the data values, named 'value'. The timestamps have a frequency of 5 min i.e. the values there are of the type "2018-01-01 00:00:00", "2018-01-01 00:05:00", "2018-01-01 00:10:00", etc.

I invoked kats.TimeSeriesData() on this dataset, then apply outlier detection.
Code run by me:
dat['time'] = pd.to_datetime(dat['time']) #converting timestamp column to pandas datetime
dat_ts = TimeSeriesData(dat)
from kats.detectors.outlier import OutlierDetector
ts_outlierDetection = OutlierDetector(dat_ts, 'additive') # call OutlierDetector
ts_outlierDetection.detector()

I get the following error on running the last line.

ERROR:root:!! Traceback (most recent call last):
!! File ".../kats/detectors/outlier.py", line 114, in detector
outlier = self.__clean_ts__(ts)
!! File ".../kats/detectors/outlier.py", line 88, in __clean_ts__
result = seasonal_decompose(original, model=self.decomp)
!! File ".../pandas/util/_decorators.py", line 207, in wrapper
return func(*args, **kwargs)
!! File ".../statsmodels/tsa/seasonal.py", line 140, in seasonal_decompose
pfreq = freq_to_period(pfreq)
!! File ".../statsmodels/tsa/tsatools.py", line 829, in freq_to_period
raise ValueError("freq {} not understood. Please report if you "
!! ValueError: freq T not understood. Please report if you think this is in error.
ERROR:root:Outlier Detection Failed

I looked into line 829 of tsatools.py and it seems it can only support few timestamp formats and hence not recognizing the 5 min frequency in my timeseries data? Should I be setting something like freq='5min' somewhere in the codes?

I also ran the outlier detection code on daily data, i.e. freq = 'D', and there seems to be no issues (this is essentially the example in the tutorial).

[Axemen]

Hey @statsguy1995,

At the moment anything in KATS that use seasonal decomposition (Such as the OutlierDetector) all rely on the statsmodels library for the decomposition. And at the moment they do not support intra-day frequencies for decomposition.

[mturk24]

Hi, I wanted to follow up to see if intraday frequencies will be available at all, and if so is there an estimated time in future?

[milton-logothetis]

@Axemen Are you familiar of any workarounds for supplying custom frequency to the underlying seasonal_decompose so that it doesn't fail? That is without manually changing the source code.

[Axemen]

Hey @milton-logothetis,

At the moment there is no workaround without changing the underlying code.

It is possible to supply custom frequencies to the seasonal_decompose function through the period argument, However, Kats has no way of passing it through at the moment. But this is something that can be added in the future.