You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently the ETL is using pandas to pre-process the data in Glue job. Those code will have OOM problem to process the large dataset, let’s refactor the code to using Glue extensions and transforms to process the data in distributed system.
Use Case
Proposed Solution
Other
👋 I may be able to implement this feature request
This is a 🚀 Feature Request
The text was updated successfully, but these errors were encountered:
Currently the ETL is using pandas to pre-process the data in Glue job. Those code will have OOM problem to process the large dataset, let’s refactor the code to using Glue extensions and transforms to process the data in distributed system.
Use Case
Proposed Solution
Other
This is a 🚀 Feature Request
The text was updated successfully, but these errors were encountered: