Timeseries data #260
Replies: 4 comments
-
Hello developers, does this PostgresML handles the time series data? |
Beta Was this translation helpful? Give feedback.
-
In a word, yes. If your dataset scale is:
|
Beta Was this translation helpful? Give feedback.
-
I would like to forecast the time series data, How PostgresML handle the
time series?
is it possible for multivariate time series prediction with PostgresML?
…On Wed, Aug 24, 2022 at 9:18 PM Montana Low ***@***.***> wrote:
In a word, yes. If your dataset scale is:
- *Sub millions*: everything will be sub second, including training
and especially inference (can be sub millisecond), even without using best
practices like indexes.
- *Millions*: Vanilla Postgres will be able to train models in seconds
at this scale with $100 laptop hardware. You probably don't need to do
anything special other than have indexes for inference. As you approach
billions you may want to look into query parrallelism
<https://www.postgresql.org/docs/current/parallel-query.html>. You'll
also want to get familiar with EXPLAIN to analyze why queries take
longer than you expect.
- *Billions*: consider partitioning
<https://www.postgresql.org/docs/current/ddl-partitioning.html> your
tables to enable more parallelism, or using the Timescale extension
<https://github.com/timescale/timescaledb> that optimizes for
timeseries explicitly to keep blazing fast performance.
- *Trillions*: you may want to generate summary or rollup tables for
individual statistics that will reduce the overall training time
- *Quadrillions*: there are probably going to be special
considerations at this scale, but generally they can be solved by sharding
across multiple physical machines. Consider using the Citus extension
<https://github.com/citusdata/citus> to make this easier to manage.
—
Reply to this email directly, view it on GitHub
<#260 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AJWFP4YVTFE2E5VESFY4PJTV2Y73RANCNFSM57OEUBQQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
--
Regards,
Premkumar Thirumalaisamy
|
Beta Was this translation helpful? Give feedback.
-
There are many ways to perform multivariate time series prediction with PostgresML. The following algorithms are supported: https://postgresml.org/user_guides/training/algorithm_selection/ For example, here is a blog post detailing how you might use XGBoost to formulate the problem, although this usage of XGBoost is from Python, the steps can be adapted to PostgresML: https://cprosenjit.medium.com/multivariate-time-series-forecasting-using-xgboost-1728762a9eeb There is also support for fine tuning deep learning models that have been published to HuggingFace. For example: https://huggingface.co/spaces/keras-io/timeseries-classification-from-scratch It might help give more specific answers if you described your objective/dataset/domain in depth. |
Beta Was this translation helpful? Give feedback.
-
Timeseries data
Beta Was this translation helpful? Give feedback.
All reactions