SAMZA-1407 upgrade junit version to 4.12#373
Closed
fredji97 wants to merge 37 commits intoapache:masterfrom
Closed
SAMZA-1407 upgrade junit version to 4.12#373fredji97 wants to merge 37 commits intoapache:masterfrom
fredji97 wants to merge 37 commits intoapache:masterfrom
Conversation
Author: Jacob Maes <jmaes@linkedin.com> Reviewers: Jagadish <jvenkatr@linkedin.com>, Prateek Maheshwari <pmaheshw@linkedin.com> Closes apache#372 from jmakes/samza-1518
Adding a versioned page for release/upgrade notes. We can start this process from the next major version release, aka 0.14.0. Please update this page as and when you add new features/configs/API or deprecate features/configs/API. Basically, anything that can be useful for Samza users trying to upgrade. Note: `site.version` is not necessarily same as samza release version. For now, I am using it as a placeholder. Hopefully, with the next generation of our website, it will be better defined. Author: navina <navina@apache.org> Reviewers: Jacob Maes <jmaes@linkedin.com> Closes apache#301 from navina/versioning
The Fix includes the following changes: - Catch all exceptions inside the heartbeat thread and not just IOException. - A time based force kill when the heartbeat is invalid, this makes the monitor immune to threads that may keep the container stuck in the shutdown sequence. When the timeout occurs, a System.exit(1) is called. - Increasing number of retries for failed heartbeats from 3 to 6. This prevents short intermittent network failurs from causing the containers to be invalidated. Author: Abhishek Shivanna <abhisheks91@gmail.com> Reviewers: Jacob Maes <jmaes@linkedin.com> Closes apache#375 from abhishekshivanna/container-heartbeat
… API Updated and clarified the documentation and error messages related to Serdes for Input/Output/PartitionBy streams. Author: Prateek Maheshwari <pmaheshw@linkedin.com> Reviewers: Jagadish Venkatraman <vjagadish1989@gmail.com> Closes apache#376 from prateekm/documentation-cleanup
…eck all Author: Bharath Kumarasubramanian <bkumaras@linkedin.com> Reviewers: Xinyu Liu <xinyuliu.us@gmail.com> Closes apache#378 from bharathkk/master
Initial version of table API, it includes - Core table API (Table, TableDescriptor, TableSpec) - Local table implementation for in-memory and RocksDb - The writeTo() and stream-table join operators nickpan47 xinyuiscool prateekm could you help review? Author: Wei Song <wsong@linkedin.com> Reviewers: Yi Pan <nickpan47@gmail.com>, Christopher Pettitt <cpettitt@linkedin.com> Closes apache#349 from weisong44/table-api-14
Key fixes vjagadish1989 lhaiesp srinipunuru - Switched Producer source vs destination assumptions in `send`, `register` - Check `OME.key` if `OME.partitionId` is null for to get partitionId - Upcoming offset changed the `END_OF_STREAM` rather than `newestOffset` + 1, eventHub returns an error if the offset does not exist in the system - Made the NewestOffset+1 as upcoming offset, consumer checks if the offset is valid on startup - Differentiated between streamNames and streamIds in configs, consumer, producer - Checkpoint table named after job name - Checkpoint prints better message for invalid key on write QOL - How to ignore integration tests - Improved logging EDIT: - Also added Round Robin producer partitioning Author: Daniel Chen <29577458+dxichen@users.noreply.github.com> Reviewers: Jagadish<jagadish@apache.org> Closes apache#377 from dxichen/eventhub-connector-fix
**Samza tools** : Contains the following tools that can be used for playing with Samza sql or any other samza job. 1. Generate kafka events : Tool used to generate avro serialized kafka events 2. Event hub consumer : Tool used to consume events from event hubs topic. This can be used if the samza job writes events to event hubs. 3. Samza sql console : Tool used to execute SQL using samza sql. Adds documentation on how to use Samza SQL on a local machine and on a yarn environment and their associated Samza tooling. https://issues.apache.org/jira/browse/SAMZA-1526 Author: Srinivasulu Punuru <spunuru@linkedin.com> Reviewers: Yi Pan<nickpan47@gmail.com>, Jagadish<jagadish@apachehe.org> Closes apache#374 from srinipunuru/docs.1
…By Op Seems the stream and the partitionBy op has the same id. So in rendering I added the stream as the id for the node. Also resolved the run.id collision issue. Author: xiliu <xiliu@xiliu-ld1.linkedin.biz> Reviewers: Jagadish V <vjagadish1989@gmail.com> Closes apache#385 from xinyuiscool/SAMZA-1534
Add overview documentation to explain how partitionBy(), checkpoint and state works in batch. Also organized the existing hdfs consumer/producer docs into the same hadoop folder under documentation. Author: xinyuiscool <xinyuliu.us@gmail.com> Reviewers: Jake Maes <jmakes@gmail.com> Closes apache#381 from xinyuiscool/SAMZA-1512
…oordinator Sending a fresh review as I lost the earlier diffs. This is the new approach that we discussed by adding the processor list in the config and passing it to grouper. Author: Aditya Toomula <atoomula@atoomula-ld1.linkedin.biz> Reviewers: Yi Pan <nickpan47@gmail.com>, Shanthoosh V <svenkataraman@linkedin.com> Closes apache#383 from atoomula/samza
Still need to add tutorials, and configs to configurations table vjagadish1989 for review Author: Daniel Chen <29577458+dxichen@users.noreply.github.com> Reviewers: Jagadish <jagadish@apache.org> Closes apache#382 from dxichen/eventhub-docs
Added document for table API to feature preview - Brief description of table - sendTo() operator for table - join() operator for stream-table-join Author: Wei Song <wsong@linkedin.com> Reviewers: Prateek Maheshwari <pmaheshwari@apache.org> Closes apache#387 from weisong44/table-api-14
Author: Aditya Toomula <atoomula@atoomula-ld1.linkedin.biz> Reviewers: Jagadish <jagadish@apache.org> Closes apache#384 from atoomula/kinesis-docs
Author: Prateek Maheshwari <pmaheshw@linkedin.com> Reviewers: Shanthoosh Venkataraman <svenkata@linkedin.com> Closes apache#389 from prateekm/disable-flaky-test
Author: Jacob Maes <jmakes@apache.org> Author: Jacob Maes <jmaes@linkedin.com> Reviewers: Jagadish <jvenkatr@linkedin.com> Closes apache#241 from jmakes/samza-1356
Author: Jacob Maes <jmakes@apache.org> Reviewers: Xinyu Liu <xiliu@linkedin.com>,Jagadish <jvenkatr@linkedin.com>,Boris Shkolnik <boryas@apache.org> Closes apache#391 from jmakes/merge-script-improvements
…d Log4j Author: Jacob Maes <jmakes@apache.org> Reviewers: Jagadish <jvenkatr@linkedin.com>,Yi Pan (Data Infrastructure) <nickpan47@gmail.com> Closes apache#388 from jmakes/async-stream-appender
…oducer.errors==true Author: Jacob Maes <jmakes@apache.org> Reviewers: Boris Shkolnik <boryas@apache.org> Closes apache#390 from jmakes/samza-1539
Author: Jacob Maes <jmakes@apache.org> Reviewers: Jagadish <jvenkatr@linkedin.com> Closes apache#393 from jmakes/streamappender-sonarcloud
…ointMgr - Additionally, updated all unit-tests. Author: Jagadish <jvenkatraman@linkedin.com> Reviewers: Prateek M <prmaheshw@linkedin.com> Closes apache#394 from vjagadish1989/kcm-fix
- Rewrite container handling to be asynchronous - Verified various failure scenarios using Unit tests, and deployments of a local Samza job. Author: Jagadish <jvenkatraman@linkedin.com> Author: Fred Ji <haifeng.ji@gmail.com> Author: Srinivasulu Punuru <spunuru@linkedin.com> Reviewers: Jacob Maes<jmakes@linkedin.com>, Xinyu Liu<xinyuiscool@gmail.com> Closes apache#380 from vjagadish1989/cluster-mgr-refactor1
Docs update for both master and 0.14.0 branch. Author: xiliu <xiliu@xiliu-ld1.linkedin.biz> Reviewers: Yi Pan <nickpan47@gmail.com> Closes apache#396 from xinyuiscool/SAMZA-1550
Update master to use 0.14.1-SNAPSHOT version. Author: xiliu <xiliu@xiliu-ld1.linkedin.biz> Reviewers: Yi Pan <nickpan47@gmail.com> Closes apache#400 from xinyuiscool/SAMZA-1550-2
Add it so Samza compiles with the latest kafka. Author: xiliu <xiliu@xiliu-ld1.linkedin.biz> Reviewers: Boris Shkolnik <sborya@gmail.com> Closes apache#402 from xinyuiscool/SAMZA-1553
This reverts commit 5238aaa.
Author: Dong Lin <lindong28@gmail.com> Reviewers: Xinyu Liu <xinyuliu.us@gmail.com> Closes apache#395 from lindong28/SAMZA-1530
The existing version of the udf implementation doesn't seem to support udfs in the where clauses because the Type of the object returned is "ANY" and when you do a
`select * from kafka.topic where regexMatch('.*foo', Name)` it fails in the query validation, because calcite doesn't know the type of regexMatch.
To solve the problem, We made the scalarUdf generic with a strongly typed return type.
https://issues.apache.org/jira/browse/SAMZA-1535
This PR can be merged into trunk not the 0.14.
Author: Srinivasulu Punuru <spunuru@linkedin.com>
Reviewers: Xinyu Liu <xinyuliu.us@gmail.com>
Closes apache#386 from srinipunuru/udf-where.1
Currently testConfigInit checks for a hardcoded number for udfs. Whenever a new UDF is added, This test is going to fail if it is not updated. Changed the test to validate the number of udfs based on the config that is passed. Author: Srinivasulu Punuru <spunuru@linkedin.com> Reviewers: Prateek Maheshwari <pmaheshwari@apache.org> Closes apache#403 from srinipunuru/testfix.1
Approximate RocksDB memory usage = Configured Block Cache size + MemTable size + Indexes and Bloom Filters size = rocksdb.block-cache-size + rocksdb.size-all-mem-tables + rocksdb.estimate-table-readers-mem Author: Prateek Maheshwari <pmaheshw@linkedin.com> Reviewers: Jagadish <jagadish@apache.org> Closes apache#404 from prateekm/rocksdb-memory
Right now Samza SQL supports queries with just two levels i.e. `select * from foo.bar`. But there can be sources that are identified though multiple levels. for e.g. `select * from kafka.clusterName.topicName`. This change adds the support for sql queries with sources that have more than two levels. Author: Srinivasulu Punuru <spunuru@linkedin.com> Reviewers: Miguel S<misanchez@linkedin.com>, Aditya T<atoomula@linkedin.com> Closes apache#405 from srinipunuru/multi-level.1
Modified the `TableUtils#deleteProcessorEntity` to provide an option to disable optimistic locking during a call to Azure Table Storage service. sborya PawasChhokra nickpan47 Review please? Author: navina <navina@apache.org> Reviewers: Shanthoosh V<svenkata@linkedin.com>, Boris S<bshkolni@linkedin.com> Closes apache#379 from navina/azure-etag-fix
Contributor
|
@fredji97 : Please can you resolve the conflict and we should be good to merge this. |
Author
|
all history change were pulled in when resolving the conflict. I am going to open another PR and close this one. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
"./gradlew clean check" passed