From 380c68488d15d6806f1060680eed8c096ae80898 Mon Sep 17 00:00:00 2001 From: Bowen Lan <62091230+bowenlan-amzn@users.noreply.github.com> Date: Thu, 1 Jul 2021 16:29:48 -0700 Subject: [PATCH] Release 1.0.0.0 (#40) Signed-off-by: bowenlan-amzn --- .github/workflows/links.yml | 21 +++ ADMINS.md | 7 + CONTRIBUTING.md | 141 +----------------- DEVELOPER_GUIDE.md | 79 ++++++++++ MAINTAINERS.md | 6 +- README.md | 68 ++++----- RELEASING.md | 1 + SECURITY.md | 3 + ...rch.job-scheduler.release-notes-1.0.0.0.md | 18 +++ 9 files changed, 166 insertions(+), 178 deletions(-) create mode 100644 .github/workflows/links.yml create mode 100644 ADMINS.md create mode 100644 DEVELOPER_GUIDE.md create mode 100644 RELEASING.md create mode 100644 SECURITY.md create mode 100644 release-notes/opensearch.job-scheduler.release-notes-1.0.0.0.md diff --git a/.github/workflows/links.yml b/.github/workflows/links.yml new file mode 100644 index 00000000..6169780a --- /dev/null +++ b/.github/workflows/links.yml @@ -0,0 +1,21 @@ +name: Link Checker +on: + push: + branches: [ main ] + pull_request: + branches: [ main ] + +jobs: + linkchecker: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v2 + - name: lychee Link Checker + id: lychee + uses: lycheeverse/lychee-action@master + with: + args: --accept=200,403,429 "**/*.html" "**/*.md" "**/*.txt" "**/*.json" + env: + GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}} + - name: Fail if there were link errors + run: exit ${{ steps.lychee.outputs.exit_code }} \ No newline at end of file diff --git a/ADMINS.md b/ADMINS.md new file mode 100644 index 00000000..534cc1be --- /dev/null +++ b/ADMINS.md @@ -0,0 +1,7 @@ +## Admins + +| Admin | GitHub ID | Affiliation | +| --------------- | --------------------------------------- | ----------- | +| Henri Yandell | [hyandell](https://github.com/hyandell) | Amazon | + +[This document](https://github.com/opensearch-project/.github/blob/main/ADMINS.md) explains what admins do in this repo. and how they should be doing it. If you're interested in becoming a maintainer, see [MAINTAINERS](MAINTAINERS.md). If you're interested in contributing, see [CONTRIBUTING](CONTRIBUTING.md). \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a5807448..24338dda 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,139 +1,4 @@ -# Contributing Guidelines +# Contributing to this project -Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional -documentation, we greatly value feedback and contributions from our community. - -Please read through this document before submitting any issues or pull requests to ensure we have all the necessary -information to effectively respond to your bug report or contribution. - - -## Reporting Bugs/Feature Requests - -We welcome you to use the GitHub issue tracker to report bugs or suggest features. - -When filing an issue, please check [existing open](https://github.com/opensearch-project/job-scheduler/issues), or [recently closed](https://github.com/opensearch-project/job-scheduler/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already -reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: - -* A reproducible test case or series of steps -* The version of our code being used -* Any modifications you've made relevant to the bug -* Anything unusual about your environment or deployment - - -## Contributing via Pull Requests -Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: - -1. You are working against the latest source on the *main* branch. -2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. -3. You open an issue to discuss any significant work - we would hate for your time to be wasted. - -To send us a pull request, please: - -1. Fork the repository. -2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. -3. Ensure local tests pass. -4. Commit to your fork using clear commit messages. -5. Send us a pull request, answering any default questions in the pull request interface. -6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. - -GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and -[creating a pull request](https://help.github.com/articles/creating-a-pull-request/). - - -## Finding contributions to work on -Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/opensearch-project/job-scheduler/labels/help%20wanted) issues is a great place to start. - - -## Code of Conduct -This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). -For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact -opensource-codeofconduct@amazon.com with any additional questions or comments. - -## Developer Certificate of Origin - -OpenSearch is an open source product released under the Apache 2.0 license (see either [the Apache site](https://www.apache.org/licenses/LICENSE-2.0) or the [LICENSE.txt file](./LICENSE.txt)). The Apache 2.0 license allows you to freely use, modify, distribute, and sell your own products that include Apache 2.0 licensed software. - -We respect intellectual property rights of others and we want to make sure all incoming contributions are correctly attributed and licensed. A Developer Certificate of Origin (DCO) is a lightweight mechanism to do that. - -The DCO is a declaration attached to every contribution made by every developer. In the commit message of the contribution, the developer simply adds a `Signed-off-by` statement and thereby agrees to the DCO, which you can find below or at [DeveloperCertificate.org](http://developercertificate.org/). - -``` -Developer's Certificate of Origin 1.1 -By making a contribution to this project, I certify that: -(a) The contribution was created in whole or in part by me and I - have the right to submit it under the open source license - indicated in the file; or -(b) The contribution is based upon previous work that, to the - best of my knowledge, is covered under an appropriate open - source license and I have the right under that license to - submit that work with modifications, whether created in whole - or in part by me, under the same open source license (unless - I am permitted to submit under a different license), as - Indicated in the file; or -(c) The contribution was provided directly to me by some other - person who certified (a), (b) or (c) and I have not modified - it. -(d) I understand and agree that this project and the contribution - are public and that a record of the contribution (including - all personal information I submit with it, including my - sign-off) is maintained indefinitely and may be redistributed - consistent with this project or the open source license(s) - involved. - ``` -We require that every contribution to OpenSearch is signed with a Developer Certificate of Origin. Additionally, please use your real name. We do not accept anonymous contributors nor those utilizing pseudonyms. - -Each commit must include a DCO which looks like this - -``` -Signed-off-by: Jane Smith -``` -You may type this line on your own when writing your commit messages. However, if your user.name and user.email are set in your git configs, you can use `-s` or `– – signoff` to add the `Signed-off-by` line to the end of the commit message. - -## Security issue notifications -If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. - -## Developer Certificate of Origin - -OpenSearch Common Utils is an open source product released under the Apache 2.0 license (see either [the Apache site](https://www.apache.org/licenses/LICENSE-2.0) or the [LICENSE file](./LICENSE)). The Apache 2.0 license allows you to freely use, modify, distribute, and sell your own products that include Apache 2.0 licensed software. - -We respect intellectual property rights of others and we want to make sure all incoming contributions are correctly attributed and licensed. A Developer Certificate of Origin (DCO) is a lightweight mechanism to do that. - -The DCO is a declaration attached to every contribution made by every developer. In the commit message of the contribution, the developer simply adds a `Signed-off-by` statement and thereby agrees to the DCO, which you can find below or at [DeveloperCertificate.org](http://developercertificate.org/). - -``` -Developer's Certificate of Origin 1.1 -By making a contribution to this project, I certify that: -(a) The contribution was created in whole or in part by me and I - have the right to submit it under the open source license - indicated in the file; or -(b) The contribution is based upon previous work that, to the - best of my knowledge, is covered under an appropriate open - source license and I have the right under that license to - submit that work with modifications, whether created in whole - or in part by me, under the same open source license (unless - I am permitted to submit under a different license), as - Indicated in the file; or -(c) The contribution was provided directly to me by some other - person who certified (a), (b) or (c) and I have not modified - it. -(d) I understand and agree that this project and the contribution - are public and that a record of the contribution (including - all personal information I submit with it, including my - sign-off) is maintained indefinitely and may be redistributed - consistent with this project or the open source license(s) - involved. - ``` -We require that every contribution to OpenSearch Common Utils is signed with a Developer Certificate of Origin. Additionally, please use your real name. We do not accept anonymous contributors nor those utilizing pseudonyms. - -Each commit must include a DCO which looks like this - -``` -Signed-off-by: Jane Smith -``` -You may type this line on your own when writing your commit messages. However, if your user.name and user.email are set in your git configs, you can use `-s` or `– – signoff` to add the `Signed-off-by` line to the end of the commit message. - -## Licensing - -See the [LICENSE](https://github.com/opensearch-project/job-scheduler/blob/main/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. - -We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes. +OpenSearch is a community project that is built and maintained by people just like **you**. +[This document](https://github.com/opensearch-project/.github/blob/main/CONTRIBUTING.md) explains how you can contribute to this and related projects. \ No newline at end of file diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md new file mode 100644 index 00000000..c76675af --- /dev/null +++ b/DEVELOPER_GUIDE.md @@ -0,0 +1,79 @@ +- [Developer Guide](#developer-guide) + - [Forking and Cloning](#forking-and-cloning) + - [Install Prerequisites](#install-prerequisites) + - [JDK 14](#jdk-14) + - [Setup](#setup) + - [Build](#build) + - [Building from the command line](#building-from-the-command-line) + - [Debugging](#debugging) + - [Using IntelliJ IDEA](#using-intellij-idea) + - [Submitting Changes](#submitting-changes) + +## Developer Guide + +So you want to contribute code to this project? Excellent! We're glad you're here. Here's what you need to do. + +### Forking and Cloning + +Fork this repository on GitHub, and clone locally with `git clone`. + +### Install Prerequisites + +#### JDK 14 + +OpenSearch components build using Java 14 at a minimum. This means you must have a JDK 14 installed with the environment variable `JAVA_HOME` referencing the path to Java home for your JDK 14 installation, e.g. `JAVA_HOME=/usr/lib/jvm/jdk-14`. + +## Setup + +1. Check out this package from version control. +2. Launch Intellij IDEA, choose **Import Project**, and select the `settings.gradle` file in the root of this package. +3. To build from the command line, set `JAVA_HOME` to point to a JDK >= 14 before running `./gradlew`. +- Unix System + 1. `export JAVA_HOME=jdk-install-dir`: Replace `jdk-install-dir` with the JAVA_HOME directory of your system. + 2. `export PATH=$JAVA_HOME/bin:$PATH` + +- Windows System + 1. Find **My Computers** from file directory, right click and select **properties**. + 2. Select the **Advanced** tab, select **Environment variables**. + 3. Edit **JAVA_HOME** to path of where JDK software is installed. + +## Build +The JobScheduler plugin uses the [Gradle](https://docs.gradle.org/4.10.2/userguide/userguide.html) +build system. +1. Checkout this package from version control. +1. To build from command line set `JAVA_HOME` to point to a JDK >=14 +1. Run `./gradlew build` + +Then you will find the built artifact located at `build/distributions` directory + +## Install +Once you have built the plugin from source code, run +```bash +opensearch-plugin install file://${PLUGIN_ZIP_FILE_PATH} +``` +to install the JobScheduler plugin to your OpenSearch. + +## Develop a plugin that extends JobScheduler +JobScheduler plugin provides a SPI for other plugins to implement. Essentially, you need to +1. Define your *JobParameter* type by implementing `ScheduledJobParameter` interface +1. Implement your JobParameter parser function that can deserialize your JobParameter from XContent +1. Create your *JobRunner* implementation by implementing `ScheduledJobRunner` interface +1. Create your own plugin which implements `JobSchedulerExtension` interface + - don't forget to create the service provider configuration file in your resources folder and + bundle it into your plugin artifact + +Please refer to the `sample-extension-plugin` subproject in this project, which provides a complete +example of using JobScheduler to run periodical jobs. + +The sample extension plugin takes an index name as input and logs the index shards to opensearch +logs according to the specified Schedule. And it also exposes a REST endpoint for end users to +create/delete jobs. + + +### Using IntelliJ IDEA + +Launch Intellij IDEA, choose **Import Project**, and select the `settings.gradle` file in the root of this package. + +### Submitting Changes + +See [CONTRIBUTING](CONTRIBUTING.md). \ No newline at end of file diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 998cdc4b..2160506c 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -1,10 +1,10 @@ -# OpenSearch Job Scheduler Maintainers - ## Maintainers | Maintainer | GitHub ID | Affiliation | +| --------------- | --------- | ----------- | | Ashish Agrawal | [lezzago](https://github.com/lezzago) | Amazon | | Bowen Lan | [bowenlan-amzn](https://github.com/bowenlan-amzn) | Amazon | | Drew Baugher | [dbbaughe](https://github.com/dbbaughe) | Amazon | | Mohammad Qureshi | [qreshi](https://github.com/qreshi) | Amazon | | Sriram Kosuri | [skkosuri-amzn](https://github.com/skkosuri-amzn) | Amazon | -| --------------- | --------- | ----------- | + +[This document](https://github.com/opensearch-project/.github/blob/main/MAINTAINERS.md) explains what maintainers do in this repo, and how they should be doing it. If you're interested in contributing, see [CONTRIBUTING](CONTRIBUTING.md). diff --git a/README.md b/README.md index bb2eb62c..01212ca2 100644 --- a/README.md +++ b/README.md @@ -1,56 +1,50 @@ +[![Test and Build Workflow](https://github.com/opensearch-project/job-scheduler/workflows/Build%20and%20Test/badge.svg)](https://github.com/opensearch-project/job-scheduler/actions) +[![codecov](https://codecov.io/gh/opensearch-project/job-scheduler/branch/main/graph/badge.svg)](https://codecov.io/gh/opensearch-project/job-scheduler) +![PRs welcome!](https://img.shields.io/badge/PRs-welcome!-success) + + + +- [OpenSearch Job Scheduler](#opensearch-job-scheduler) +- [Contributing](#contributing) +- [Getting Help](#getting-help) +- [Code of Conduct](#code-of-conduct) +- [Security](#security) +- [License](#license) +- [Copyright](#copyright) + # OpenSearch Job Scheduler OpenSearch JobScheduler plugin provides a framework for OpenSearch plugin developers to schedule periodical jobs running within OpenSearch nodes. You can schedule jobs by specify an interval, or using Unix Cron expression to define more flexible schedule to execute -your job. +your job. OpenSearch plugin developers can easily extend JobScheduler plugin to schedule jobs like running aggregation query against raw data and save the aggregated data into a new index every hour, or keep monitoring the shard allocation by calling OpenSearch API and post the output to a Webhook. -## Build -The JobScheduler plugin uses the [Gradle](https://docs.gradle.org/4.10.2/userguide/userguide.html) -build system. -1. Checkout this package from version control. -1. To build from command line set `JAVA_HOME` to point to a JDK >=14 -1. Run `./gradlew build` - -Then you will find the built artifact located at `build/distributions` directory - -## Install -Once you have built the plugin from source code, run -```bash -opensearch-plugin install file://${PLUGIN_ZIP_FILE_PATH} -``` -to install the JobScheduler plugin to your OpenSearch. - -## Develop a plugin that extends JobScheduler -JobScheduler plugin provides a SPI for other plugins to implement. Essentially, you need to -1. Define your *JobParameter* type by implementing `ScheduledJobParameter` interface -1. Implement your JobParameter parser function that can deserialize your JobParameter from XContent -1. Create your *JobRunner* implementation by implementing `ScheduledJobRunner` interface -1. Create your own plugin which implements `JobSchedulerExtension` interface - - don't forget to create the service provider configuration file in your resources folder and - bundle it into your plugin artifact - -Please refer to the `sample-extension-plugin` subproject in this project, which provides a complete -example of using JobScheduler to run periodical jobs. - -The sample extension plugin takes an index name as input and logs the index shards to opensearch -logs according to the specified Schedule. And it also exposes a REST endpoint for end users to -create/delete jobs. +## Contributing -## Code of Conduct +See [developer guide](DEVELOPER_GUIDE.md) and [how to contribute to this project](CONTRIBUTING.md). -This project has adopted an [Open Source Code of Conduct](https://www.opensearch.org/codeofconduct.html). +## Getting Help +If you find a bug, or have a feature request, please don't hesitate to open an issue in this repository. -## Security issue notifications +For more information, see [project website](https://opensearch.org/) and [documentation](https://docs-beta.opensearch.org/). If you need help and are unsure where to open an issue, try [forums](https://discuss.opendistrocommunity.dev/). + +## Code of Conduct + +This project has adopted the [Amazon Open Source Code of Conduct](CODE_OF_CONDUCT.md). For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq), or contact [opensource-codeofconduct@amazon.com](mailto:opensource-codeofconduct@amazon.com) with any additional questions or comments. + +## Security If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public GitHub issue. +## License + +This project is licensed under the [Apache v2.0 License](./LICENSE) -## Licensing +## Copyright -See the [LICENSE](./LICENSE.txt) file for our project's licensing. We will ask you to confirm the licensing of your contribution. +Copyright 2020-2021 Amazon.com, Inc. or its affiliates. All Rights Reserved. \ No newline at end of file diff --git a/RELEASING.md b/RELEASING.md new file mode 100644 index 00000000..6903e716 --- /dev/null +++ b/RELEASING.md @@ -0,0 +1 @@ +This project follows the [OpenSearch release process](https://github.com/opensearch-project/.github/blob/main/RELEASING.md). \ No newline at end of file diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 00000000..e0ecd902 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,3 @@ +## Reporting a Vulnerability + +If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/) or directly via email to aws-security@amazon.com. Please do **not** create a public GitHub issue. \ No newline at end of file diff --git a/release-notes/opensearch.job-scheduler.release-notes-1.0.0.0.md b/release-notes/opensearch.job-scheduler.release-notes-1.0.0.0.md new file mode 100644 index 00000000..be726ced --- /dev/null +++ b/release-notes/opensearch.job-scheduler.release-notes-1.0.0.0.md @@ -0,0 +1,18 @@ +## Version 1.0.0.0-rc1 2021-05-25 + +Compatible with OpenSearch 1.0.0-rc1 + +### Maintenance + * Rename settings from OpenDistro and OpenSearch, with backwards compatibility, using fallback ([#20](https://github.com/opensearch-project/job-scheduler/pull/20)) + +### Bug Fixes +* Correcting failure logging ([#35](https://github.com/opensearch-project/job-scheduler/pull/35)) + +### Refactoring + * Change plugin setting name from 'opensearch.' to 'pluigns.' ([#27](https://github.com/opensearch-project/job-scheduler/pull/27)) + * Rename namespaces from odfe to opensearch ([#24](https://github.com/opensearch-project/job-scheduler/pull/24)) + * Change path of REST APIs for 'Sample Extension Plugin' and naming convension in filename and comments ([#25](https://github.com/opensearch-project/job-scheduler/pull/25)) + * Rename other identifiers from opendistro or elasticsearch to OpenSearch ([#28](https://github.com/opensearch-project/job-scheduler/pull/28)) + +### Documentation + * Update issue template with multiple labels ([#21](https://github.com/opensearch-project/job-scheduler/pull/21))