All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog.
** Sub-task
- STREAMPIPES-427 - Remove username from REST API paths
- STREAMPIPES-434 - Use auto-setup over manual setup dialog
- STREAMPIPES-435 - Add service account authentication
- STREAMPIPES-436 - Prepare StreamPipes client for pipeline elements and adapters
- STREAMPIPES-437 - Add initial authorization model
- STREAMPIPES-439 - Add UI to manage users
- STREAMPIPES-441 - Add email sending capability
- STREAMPIPES-453 - Add object-level permission management
- STREAMPIPES-455 - Add configuration option to provide external hostname
- STREAMPIPES-457 - Add self-registration and password reset
- STREAMPIPES-458 - Secure adapter endpoints
- STREAMPIPES-469 - Merge LICENSE and NOTICE files
- STREAMPIPES-470 - Fix Maven setup
** Bug
- STREAMPIPES-226 - Check usage of running adapter instances before deletion
- STREAMPIPES-237 - Domain properties of imported connect adapter templates are not used
- STREAMPIPES-259 - Use suitable descriptions in pipeline element config
- STREAMPIPES-396 - Data Set Adapter does not work with JSON Array
- STREAMPIPES-407 - Unable to register custom sink
- STREAMPIPES-424 - ISS Adapter is broken
- STREAMPIPES-425 - Time-Range in Data Explorer is not updated
- STREAMPIPES-429 - Error on sinks.notifications.jvm.email
- STREAMPIPES-440 - Access „Http Server“ adapter endpoint over ui container
- STREAMPIPES-444 - Fix asset dashboard
- STREAMPIPES-460 - Runtime options resolver does not support service discovery
- STREAMPIPES-479 - Custom streampipes extensions missing Spring method
- STREAMPIPES-484 - Replace SDK methods to receive files with StreamPipes Client requests
- STREAMPIPES-495 - Icons in documentation are missing
- STREAMPIPES-496 - streampipes-backend/api/v2/users/profile returning status 405 Method Not Allowed
** New Feature
- STREAMPIPES-263 - Add bundled extensions module for standalone jvm adapter and pipeline elements
- STREAMPIPES-295 - HTTP Server adapter
- STREAMPIPES-335 - Toggle slider as static property
- STREAMPIPES-398 - Provide custom theme settings
- STREAMPIPES-408 - Provide time settings in data explorer as query parameters
- STREAMPIPES-409 - Support count query in data explorer
- STREAMPIPES-410 - Support auto-aggregation in Data Explorer API v4
- STREAMPIPES-412 - Add histogram widget to data explorer
- STREAMPIPES-413 - Add density widget to data explorer
- STREAMPIPES-414 - Support filtering in Data Explorer API
- STREAMPIPES-415 - Add pie chart widget to data explorer
- STREAMPIPES-417 - Provide dashboard id as url parameter in live dashboard
- STREAMPIPES-421 - Configure collection static properties CSV file
- STREAMPIPES-423 - Support other languages in CodeInput static property
- STREAMPIPES-428 - Add map widget to data explorer
- STREAMPIPES-501 - Add throughput monitoring processor
- STREAMPIPES-504 - Support tree-based static property configuration
** Improvement
- STREAMPIPES-204 - Loading time of install pipeline element page takes long
- STREAMPIPES-229 - Change location of save button in pipeline editor
- STREAMPIPES-281 - Update consul version
- STREAMPIPES-376 - Store pipeline element descriptions in CouchDB storage
- STREAMPIPES-380 - Refactor connect UI
- STREAMPIPES-383 - Harmonize web servers used by StreamPipes services
- STREAMPIPES-384 - Show pipeline element icons in installation view
- STREAMPIPES-386 - Harmonize StreamPipes Connect Service
- STREAMPIPES-387 - Split StreamPipes Connect worker into two different services
- STREAMPIPES-397 - Remove obsolete module configs from UI
- STREAMPIPES-399 - Use new StreamPipes logo in UI
- STREAMPIPES-401 - Persist data explorer settings
- STREAMPIPES-402 - Improve data explorer widget config
- STREAMPIPES-406 - Bump Flink Version
- STREAMPIPES-418 - Datalake sink should use event schema to detect type of property
- STREAMPIPES-422 - Word cloud dashboard widget not updating to new events
- STREAMPIPES-438 - Harmonize Model Submitter
- STREAMPIPES-443 - Remove duplicate code from the adapter modules
- STREAMPIPES-447 - Processing Element: Detect Value Change
- STREAMPIPES-448 - Processing Element: Boolean Filter
- STREAMPIPES-449 - Update Processing Element API in module streampipes-processors-filters-jvm
- STREAMPIPES-472 - Reduce dependency overhead of extensions modules
- STREAMPIPES-474 - Update Maven archetypes
- STREAMPIPES-477 - CLI logs service name optional
- STREAMPIPES-480 - Update Spring version
- STREAMPIPES-486 - Update Flink and add log4j to dependency management
- STREAMPIPES-487 - Refactor Flink modules to use new service builder pattern
- STREAMPIPES-489 - Properly stop Flink jobs when running in debug mode
- STREAMPIPES-490 - Links on wiki page incorrect, they include .../docs/docs.. in url
- STREAMPIPES-499 - Update Jsplumb dependency
- STREAMPIPES-502 - Replace reserved keywords of events stored in data lake storage
- STREAMPIPES-503 - Add slide toggle static property
- STREAMPIPES-505 - Improve node discovery of OPC-UA adapter
- STREAMPIPES-509 - Use data lake APIs in live dashboard
- STREAMPIPES-510 - Provide better overview of pipeline elements in editor
- STREAMPIPES-511 - Support change of username and password in profile view
** Test
- STREAMPIPES-476 - Implementation of Pipeline Tests
- STREAMPIPES-483 - Provide E2E tests for preprocessing rules within the adapter
** Task
- STREAMPIPES-442 - Upgrade Angular to v12
- STREAMPIPES-273 - Remove DataSource concept
- STREAMPIPES-274 - Add new modules for streampipes-client and API
- STREAMPIPES-275 - Pipeline Element Templates
- STREAMPIPES-15 - Let backend provide Kafka messaging properties
- STREAMPIPES-74 - Automatically restart pipelines on system startup
- STREAMPIPES-245 - Add pipeline monitoring feature
- STREAMPIPES-248 - Auto-restart pipelines that are stopped on system restart
- STREAMPIPES-249 - More compact data processor/sink definition
- STREAMPIPES-250 - Support lists in Siddhi wrapper
- STREAMPIPES-251 - Add stop feature to cli
- STREAMPIPES-252 - Support MQTT transport protocol
- STREAMPIPES-288 - Provide count aggregation in default version
- STREAMPIPES-294 - Wordcloud visualization in dashboard
- STREAMPIPES-296 - Status visualization in dashboard
- STREAMPIPES-303 - Add autocomplete feature for semantic type selection in Connect
- STREAMPIPES-362 - Live preview in pipeline editor
- STREAMPIPES-372 - Pipeline Health Check
- STREAMPIPES-173 - Backend should be run without any dependency on IntelliJ
- STREAMPIPES-197 - Improve Documentation
- STREAMPIPES-200 - Too many bold text parts in UI
- STREAMPIPES-220 - Add requiredFloatParameter builder method with ValueSpecification
- STREAMPIPES-246 - Remove module app-file-export
- STREAMPIPES-254 - Add correction value rule to preprocessing pipeline in connect adapters
- STREAMPIPES-256 - PLC4X S7 adapter should use PooledPlcDriverManager
- STREAMPIPES-257 - Add the connect master REST API to backend
- STREAMPIPES-260 - Add connection retry option for consul on startup
- STREAMPIPES-269 - Improve container-core serialization performance
- STREAMPIPES-270 - Update Angular version
- STREAMPIPES-280 - Refactor data explorer management
- STREAMPIPES-287 - Support GroupBy clause in Siddhi wrapper
- STREAMPIPES-293 - Extend Siddhi wrapper
- STREAMPIPES-306 - [OPC UA] Implement pull adapter
- STREAMPIPES-307 - Extend SDK to add default code block
- STREAMPIPES-308 - Add String mappings to BooleanToState Processor
- STREAMPIPES-310 - Improve export speed of data download
- STREAMPIPES-334 - Customize Label Property in NumberLabeler
- STREAMPIPES-352 - Add restart policy to compose files
- STREAMPIPES-360 - Improve Adapter Management
- STREAMPIPES-369 - Improve user guidance in live dashboard
- STREAMPIPES-370 - Change widget type in live dashboard
- STREAMPIPES-374 - Improve management of secret configs
- STREAMPIPES-375 - Restore pipeline element positions in pipeline canvas
- STREAMPIPES-126 - [Postgres Sink] Tries to Creating Table for each incomming event
- STREAMPIPES-127 - [Postgres Sink] Using Wrong datatype while cerating table
- STREAMPIPES-162 - Postgres Column Names are case sentitiv and save leads to error
- STREAMPIPES-163 - Exception during save in MYSQl Sink with Timestamp value
- STREAMPIPES-243 - Configuration of And Processor is broken
- STREAMPIPES-253 - Missing timestamp property scope (header) in connect
- STREAMPIPES-264 - Cannot login after k8s restart
- STREAMPIPES-267 - NumberLabeler does not work with doubles
- STREAMPIPES-271 - Event property dialog breaks global dialog layout
- STREAMPIPES-282 - Fix empty route warning in backend
- STREAMPIPES-284 - NullPointerException at org.apache.streampipes.manager.execution.http.GraphSubmitter.detachGraphs(GraphSubmitter.java:94)
- STREAMPIPES-285 - Connect worker - Error after PLC turned off
- STREAMPIPES-302 - File Set Adapter does not start on pipeline invokation
- STREAMPIPES-312 - Duplicate activemq svc definition in helm chart
- STREAMPIPES-313 - org.apache.streampipes.connect.adapter.exception.AdapterException: Could not resolve runtime configurations from http://connect-worker-main:8098
- STREAMPIPES-315 - "==" operator does not work in Number Labeler
- STREAMPIPES-317 - [Postgres Sink] Connection via SSL not possible
- STREAMPIPES-321 - Fail to stop pipeline if one of the PE is stopped manually/not running
- STREAMPIPES-336 - Remove connect-master service build section from docker-compose.yml
- STREAMPIPES-342 - Maven build failure due to missing Apache license header
- STREAMPIPES-358 - pod restart freezes a pipeline (k8s)
- STREAMPIPES-359 - Documentation of pipeline elements does not provide accurate details
- STREAMPIPES-361 - Prevent running pipelines to be modified
- STREAMPIPES-365 - Pipeline verification not working
- STREAMPIPES-366 - Unexpected error when modifying pipelines
- STREAMPIPES-367 - Avoid orphaned pipelines
- STREAMPIPES-368 - Avoid duplicated visualizable pipelines
- STREAMPIPES-371 - Wrong pipeline is started after modification
- STREAMPIPES-373 - Updating a pipeline breaks running visualizations
- STREAMPIPES-378 - Mapping properties are not correctly extracted
- STREAMPIPES-379 - Runtime name is not shown in PLC4X S7 adapter
- STREAMPIPES-268 - Update Maven dependencies
- STREAMPIPES-364 - Improve robustness after pipeline updates
- STREAMPIPES-377 - Remove obsolete code from knowledge module
- STREAMPIPES-133 - Add StaticProperty for entering code
- STREAMPIPES-134 - Add UserDefinedOutputStrategy
- STREAMPIPES-238 - Harmonize file upload for Connect & pipeline elements
- STREAMPIPES-8 - Static Property Group not working when only one child is present
- STREAMPIPES-99 - Upload Adapter Template
- STREAMPIPES-119 - Redirect to pipeline overview when modifying pipeline not working
- STREAMPIPES-123 - using specific runtame name leads to exception during extraction
- STREAMPIPES-142 - Handle non-primitive and primitive event data returned from Siddhi
- STREAMPIPES-143 - Numerical Filter in Siddhi not working
- STREAMPIPES-152 - Error when decoding secret properties
- STREAMPIPES-153 - DateTime is not supported in MySQL adapter
- STREAMPIPES-154 - Dashboard does not show data
- STREAMPIPES-155 - SplitArray controller produces wrong output schema
- STREAMPIPES-158 - AppId is empty in Connect adapters
- STREAMPIPES-165 - S7 adpater does not work
- STREAMPIPES-170 - NullPointer in JS Evaluator if no event is returned
- STREAMPIPES-172 - Default colors of line chart are not readable
- STREAMPIPES-186 - Remove environment variable to Kafka Rest
- STREAMPIPES-188 - Cannot import pre-exported pipeline description
- STREAMPIPES-189 - System error while guess the schema of the data
- STREAMPIPES-195 - "Do not show again" button in StreamPipes tour not working
- STREAMPIPES-201 - Initial Installation Setup renders elements besides each other
- STREAMPIPES-202 - Docs toggle and slide out in PE configuration is behaving oddly
- STREAMPIPES-205 - Search function in pipeline editor not working properly
- STREAMPIPES-207 - Fix image widget in data explorer
- STREAMPIPES-210 - Map visualization in Dashboard is not working anymore
- STREAMPIPES-214 - CSV Metadata Enricher does not appear in lite version
- STREAMPIPES-215 - Data Set adapters are shown Data Stream Tab
- STREAMPIPES-218 - Pipeline element configuration dialog closes when clicking on editor canvas
- STREAMPIPES-219 - Refresh button in Data Explorer is not working
- STREAMPIPES-227 - Missing label in Boolean Counter configuration
- STREAMPIPES-230 - Large images are not transmitted
- STREAMPIPES-231 - Images are not shown in Data Explorer
- STREAMPIPES-233 - Modifying a pipeline breaks existing visualizations
- STREAMPIPES-240 - NullPointer Exception in processor image enricher
- STREAMPIPES-241 - CustomTransformOutputStrategy of Processor SplitArray not working
- STREAMPIPES-242 - Mqtt adapter next button can not be clicked when using authentication
- STREAMPIPES-244 - JS Evalutor is not working in docker
- STREAMPIPES-114 - New sink to support writing data to MQTT.
- STREAMPIPES-115 - New sink to support writing data to MySQL
- STREAMPIPES-132 - Add data processor to evaluate JavaScript
- STREAMPIPES-149 - Processor: State Buffer
- STREAMPIPES-159 - Processor: Detect Signal Edge
- STREAMPIPES-160 - Adapter: Flic Button
- STREAMPIPES-166 - Add ColorPicker static property to SDK
- STREAMPIPES-208 - Use mapping properties in collections
- STREAMPIPES-213 - Enable user to delete data in data lake
- STREAMPIPES-217 - File Management View
- STREAMPIPES-11 - Cleanup POM files
- STREAMPIPES-117 - Share IntelliJ run configuration in version control
- STREAMPIPES-118 - Add configuration file to S7 adapter
- STREAMPIPES-128 - Upload Excel Option for PLC4x (S7) Adapter
- STREAMPIPES-129 - Upload Excel Option for PLC4x (S7) Adapter
- STREAMPIPES-130 - Image upload adapter
- STREAMPIPES-131 - Increase Accuracy of the Geo Distance Calculation
- STREAMPIPES-136 - Limit container size in cli and installer
- STREAMPIPES-141 - Refactor all adapter implementations to use assets
- STREAMPIPES-145 - Migrate pipeline editor module from AngularJS to Angular
- STREAMPIPES-146 - Support CustomOutputStrategy in SiddhiEventEngine
- STREAMPIPES-156 - Processor: State Labeler
- STREAMPIPES-167 - IntelliJ Configurations for Extension as Project Files
- STREAMPIPES-177 - Migrate pipeline details view from AngularJS to Angular
- STREAMPIPES-178 - Migrate pipeline view to Angular
- STREAMPIPES-190 - Migrate pipeline element installation from AngularJS to Angular
- STREAMPIPES-191 - Remove MyElements view
- STREAMPIPES-193 - Remove AngularJS dependencies from core modules
- STREAMPIPES-194 - Remove icon description from widgets description in dashboard
- STREAMPIPES-196 - Refactor existing installer
- STREAMPIPES-198 - Fix progress bar background on startup
- STREAMPIPES-199 - Remove advanced settings from setup
- STREAMPIPES-203 - Add more options to install pipeline elements page
- STREAMPIPES-206 - Remove tson-ld and deprecated UI models
- STREAMPIPES-211 - Add Polling interval for PLC4X S7 adaptrer
- STREAMPIPES-216 - Move watertank simulator source to StreamPipes Connect
- STREAMPIPES-221 - Update maven archetypes for pipeline elements
- STREAMPIPES-222 - Upgrade Spring version
- STREAMPIPES-225 - Bundle all Flink pipeline elements
- STREAMPIPES-232 - Trigger Github Actions to push to Dockerhub on release preparation branches
- STREAMPIPES-235 - Add html font format only option as freetext static property
- STREAMPIPES-236 - Fix Telegram sink using html font format option in request
- STREAMPIPES-239 - Multi-arch docker image for Flink
- STREAMPIPES-140 - Add streampipes-maven-plugin to core
- New live dashboard
- New notification view
- Adapter icons canbe provided by container
- Multiple properties are supported in PLC4X adapter
- New data processor to merge data streams by timestamp
- New data processor to enrich streams
- Multi-arch docker images
- New adapter for NETIO power sockets
- New data sink for MQTT
- New data processor for numerical and text values
- Improvements to the Kafka Publisher Sink
- Improvements to the Notification Sink
- Upgrade to latest Angular version, improved UI build process for smaller files
- Consider domain property for live preview
- Support mapping properties in TSON-LD
- Update RDF4J and Empire dependencies
- Upgrade Siddhi version
- Password field in postgres sink is now marked as password
- Fix a bug in the REST pull adapter
- Fix broken links in UI
- Fix bug that caused pipelines not to be properly saved
- Many more minor bug fixes
- Added a welcome screen to the UI during startup
- Added an umbrella pipeline element to reduce memory requirements of lite version
- Bumped Flink to latest version 1.9.1
- Added CSV enrichment processor
- Added event counter processor
- Support FileStaticProperty in Pipeline Editor
- Do not trigger cache refresh of pipeline elements during initial installation
- Websocket URL in live dashboard does not depend anymore on hostname
- Optimize Dockerfiles
- Installer now works without providing a hostname
- Optimize caching of pipeline element descriptions
- Fixed a bug in the OPC-UA Adapter
- Fixed a bug that prevented error messages to be shown in the pipeline view
- Fixed a bug that cause the pipeline modification to fail
- Added a new StreamPipes Connect adapter for Apache PLC4X
- Added a new StreamPipes Connect adapter for Apache Pulsar
- Added a new data sink to send events to Apache Pulsar
- All StreamPipes services use a new Docker Image based on OpenJ9 which drastically reduces memory consumption
- Added a new static property that handles secrets such as passwords
- Added a new static property to specify property groups
- Added a new external event processor in preparation for the upcoming python wrapper
- Added configuration options to FileStreamProtocol
- Pipeline Elements provide their own documentation and icons
- Added support for binary message formats: SMILE, FST and CBOR
- Added a new processor boolean inverter
- Added an OPC-UA adapter to Connect
- Added a new random data generator to Connect
- Added a new IoTDB sink
- Added a new OPC-UA sink
- Added a new settings page to select preferred message formats
- Added support for runtime-resolvable static properties in Connect
- Added a new static property StaticPropertyAlternatives that handles alternatives
- Extracted Connect adapters from Backend to new worker-based architecture
- Added support for multiple dashboards within one pipeline
- Extracted RDF4J HTTP repository to own service
- Added a feature to gracefully stop pipelines when containers are stopped
- Added support for Alternatives/Group static properties to Connect
- Added a feedback button
- Added authentication to MQTT adapter
- Added improved asset support for data streams
- Uninstallation of data sinks not working
- Duplicated events in pipelines with two dashboard sinks
- Trend detection fires too often
- Rules in ROS adapter are not working
- MQTT Protocol NullPointer in Guess Schema
- Unit conversion is not stored in connect
- Error when reading domain properties
- Shared usage of transport protocols and formats breaks pipeline element installation
- Modifying links between pipeline elements
- Validation of alternative configuration at MQTT adapter not working as expected
- Dashboard does not work after editing pipeline
- Dots in keys within StreamPipes Connect are currently not working
- Error in Dashboard with ROS messages with header field
- CSV Format does not guess numbers correctly
- AppendOutputStrategy: Renaming does not work correctly
- Wrong extractor for Runtime-resolvable selections in documentation
- ProtocolMatch adds wrong error message
- Uninstalling pipeline elements not working
- Customize event output in Siddhi wrapper
- Connect not showing Error Messages
- Improve edit schema dialog in Connect
- Directly mark event property as timestamp
- Avoid using function calls in Connect UI
- Make UI more browser-independent
- Let DeclarersSingleton declare supported protocols
- Improve support for runtime resolvable static properties
- Package StreamPipes backend as jar instead of war file
- Migrate pipeline element containers to Spring Boot
- Pipeline Tutorial still has Field Hasher in last step
- Migrate Connect containers to Spring Boot
- Enable gzip compression in UI
- In Installation Dialogue Enter not working as expected
- Extended and improved documentation
- Update Kafka
- Update Flink
- Update Maven plugin dependencies
- Update Powermock Version
- Update jetty version
- Update Checkstyle version
- Add method to SDK to get type of Mapping Property
- Update Jackson libraries
- Update documentation
- Update Maven archetypes
- Always show consistency check messages of pipeline elements
- StreamPipes Connect: Sort adapters by name
- Add categories to StreamPipes Connect
- Add Wikipedia adapter
- Add Coindesk adapter
- Add IEXCloud adapter
- Extract labels and descriptions to external resources file
- Add extractor for Static Properties in all Connect adapters
- Implement Interfaces for historic data access
- Add description for Connect configuration properties
- StreamPipes Connect: Ensure correct ordering of static properties
- Adapter Event-Property Order
- Fix bug in data set support of Connect
- Upgrade UI to latest Angular version 7
- Refactor authentication/configuration checks in UI
- Ensure correct ordering of stream requirements
- Fix link to documentation in StreamPipes UI
- Improve behaviour of customizing window
- Improved validation of pipelines in the pipeline editor
- Case-insensitive pipeline element search
- Customize dialog is hidden if no static properties are present
- Export and import of Adapter descriptions
- Migration guide for SIP-08
- New map visualization
- Improved support for complex data types
- Introduction of new event model
- New Maven archetype: pe-source
- SSL support
- Bug Fix: Tutorial not shown
- Bug Fix: Transform Output Strategy not working
- Refactor Maven archetypes to support new event model and wrapper classes
- Upgrade to latest AngularJS version
- Ensure correct ordering of static properties
- Bug Fix: AnyStaticProperty not working
- Bug Fix: Pipeline can be stored without a Sink
- Bug Fix: Pipeline modelling: duplicated runtimeNames
- Use of the UI without Internet
- Bug Fix: CollectionStaticProperty not working
- OpenSenseMap adapter due to API changes
- Maven archetypes to easily create new pipeline elements using the SDK
- UI improvements
- Several bug fixes
- Beta release of StreamPipes Connect Library
- Tutorials for better user guidance
- New Wrapper for the Siddhi CEP engine
- New Project streampipes-pipeline-elements contains more than 40 new pipeline elements
- Various bug fixes and stability improvements
- Many UX improvements (e.g., harmonized styles)
- Dashboard does not reload after new visualization type has been created
- Improved test coverage
- The installer makes it easy to install StreamPipes on Linux, MacOS and Windows
- Live data preview for data streams in the pipeline editor
- Initial support for data sets
- Default for configurations can now be provided as environment variable, with the same name
- Pipeline elements can be directly installed at installation time
- Extended the SDK to create new pipeline elements
- Several UI improvements to make the definition of pipelines more intuitive
- Several bug fixes and code improvements