Welcome to TasCreed!
TasCreed is a framework library of job executor, to guarantee the jobs executed thoroughly.
If we want to execute a batch job, we would expect:
- this job must be executed
- if the job execution failed by environment issue, it can retry itself
- if the executing node crashes, another node can take the job
This is the basic aim of TasCreed, and it can provide the at least once job execution guarantee, even node crash encountered.
Furthermore, TasCreed can provide you some more convenience:
- execute tens of tasks in parallel, with simple configuration
- split a batch of data into partitions, and execute them parallelly
- create long run job to execute mini-batches continuously
These features are really useful in some special scenarios, TasCreed can help on this in a simple way.
Please be aware that, TasCreed task is executed at least once, so the side effect of your task should be idempotent, this is also a common requirement for the retryable batch jobs.
Overview introduces the basic concepts of TasCreed, you can take a cup of coffee to have a glance.
- Features introduces the features of historical TasCreed versions.
- Planned Features introduces the planned features of TasCreed in future versions.
- Application Config introduces how to config the TasCreed application.
- Application Detailed Config introduces the detailed config of TasCreed application.
- Job Define Config introduces how to config a job define file.
- Job Request introduces how to submit job request to trigger a job instance.
- Params introduces the params usage.
- Advanced Usage introduces some advanced features you can leverage in TasCreed job.
- Update Param Config introduces how to update params and configs for the later tasks.
- Ban Jobs introduces how to pause and resume a job in different levels.
- Progression introduces the visualization of progression of a job.
- States introduces the state of task, step and job, as well as the state machine.
- Phase introduces the phase feature of step, as a weaker dependency relationship.
- Archive introduces the archive feature of done jobs, users can choose their own combinations of implementations, such as etcd, es.
- Trait introduces the trait feature of TasCreed data objects, users can configure the optional characters of the job, step and task in different level.
- Routine introduces a new kind of executor named routine, the long run routine job across the whole TasCreed cluster can be defined as a routine. It is an important complement of the task executor.
- Annotation introduces the TasCreed defined annotations, to help register the task executors and routine executors in code.
- Node Duty introduces the duty rules of the TasCreed application nodes, to help users control which nodes can work as expected.
- Schedule introduces the schedule feature, users can schedule trigger time of job defines in advance, which means TasCreed can work as a time scheduler.
Special thanks to people who give your support on this project.
Copyright 2023-2024 eBay Inc.
Authors/Developers: Lionel Liu
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
- Lionel Liu
Please see here for all contributors.