Skip to content

0.9.68 | Invincible 🦸‍♂️
Compare
Choose a tag to compare
@wangxiaoyou1993 wangxiaoyou1993 released this 28 Mar 15:35
0.9.68
8ddf586
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

What's Changed

🎉 Exciting New Features

🐍 Custom python source in Streaming pipeline

This feature allows users to use custom Python source code in their Streaming pipelines, which provides more flexibility and control for advanced users who need to incorporate custom logic or functionality into their streaming pipelines. Here is the code template:

from mage_ai.streaming.sources.base_python import BasePythonSource
from typing import Callable

if 'streaming_source' not in globals():
    from mage_ai.data_preparation.decorators import streaming_source


@streaming_source
class CustomSource(BasePythonSource):
    def init_client(self):
        """
        Implement the logic of initializing the client.
        """

    def batch_read(self, handler: Callable):
        """
        Batch read the messages from the source and use handler to process the messages.
        """
        while True:
            records = []
            # Implement the logic of fetching the records
            if len(records) > 0:
                handler(records)

by @wangxiaoyou1993 in #4763

📦 OracleDB exporter

Add OracleDB exporter template to standard batch pipeline.
image

by @matrixstone in #4785

🔧 Server logging format template overriding

This feature enables users to override the default server logging format template in Mage. This provides more flexibility in customizing the logging output, which can be useful for troubleshooting, monitoring, or integrating with external logging systems.
Here is an example setting:

export SERVER_LOGGING_TEMPLATE=$'%(asctime)s\t[%(name)25.25s]\t%(levelname)5s: %(message)s'

by @afal007 in #4772

🔄 Update pipeline name before pipeline creation

Allow user to edit the pipeline name and add a description/tags before creating a new pipeline. Previously, a randomized name was always used for new pipelines, and then users had to go to the pipeline's settings to update the name.
image

by @johnson-mage in #4790

🐛 Bug Fixes

💅 Enhancements & Polish

✨ New Contributors

Full Changelog: 0.9.66...0.9.68

Contributors

matrixstone, afal007, and 4 other contributors
Assets 2