This repository was archived by the owner on Aug 7, 2025. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 128
Added github action to update feature catalog MD file #1714
Merged
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
0854c60
Added github action to update feature catalog file
k-a-il 2d2a84d
Added gh action to generate feature catalog page and create PR
k-a-il 967cb9e
Script to generate md file from ext and community feature files
k-a-il 9f2226f
Removed unnecessary repo name
k-a-il b76f2a5
Added steps to detect GH action run-id
k-a-il f0f7d43
Fixed comments from pr
k-a-il 3cd0cb4
Upper-cased abbr for cron job definition
k-a-il File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| name: Update feature catalog page | ||
| on: | ||
| schedule: | ||
| - cron: 0 10 * * TUE | ||
| workflow_dispatch: | ||
| jobs: | ||
| generate-feature-catalog-file: | ||
| name: Generate feature catalog page | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout docs repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Latest run-id from community repository | ||
| run: | | ||
| latest_workflow_id=$(curl -s https://api.github.com/repos/localstack/localstack/actions/workflows \ | ||
| | jq '.workflows[] | select(.name=="AWS / Archive feature files").id') | ||
| latest_run_id=$(curl -s \ | ||
| https://api.github.com/repos/localstack/localstack/actions/workflows/$latest_workflow_id/runs | jq '.workflow_runs[0].id') | ||
| echo "Latest run-id: ${latest_run_id}" | ||
| echo "FEATURES_ARTIFACTS_COMMUNITY_RUN_ID=${latest_run_id}" >> $GITHUB_ENV | ||
|
|
||
| - name: Download features files from Collect feature files (GitHub) | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| path: features-files-community | ||
| name: features-files | ||
| github-token: ${{ secrets.GH_PAT_FEATURE_CATALOG_PAGE }} # PAT with access to artifacts from GH Actions | ||
| repository: localstack/localstack | ||
| run-id: ${{ env.FEATURES_ARTIFACTS_COMMUNITY_RUN_ID }} | ||
|
|
||
| - name: Latest run-id from ext repository | ||
| run: | | ||
| latest_workflow_id=$(curl -s https://api.github.com/repos/localstack/localstack-ext/actions/workflows \ | ||
| | jq '.workflows[] | select(.name=="AWS / Archive feature files").id') | ||
| latest_run_id=$(curl -s \ | ||
| https://api.github.com/repos/localstack/localstack-ext/actions/workflows/$latest_workflow_id/runs | jq '.workflow_runs[0].id') | ||
| echo "Latest run-id: ${latest_run_id}" | ||
| echo "FEATURES_ARTIFACTS_EXT_RUN_ID=${latest_run_id}" >> $GITHUB_ENV | ||
|
|
||
| - name: Download features files from Collect feature files from PRO (GitHub) | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| path: features-files-ext | ||
| name: features-files-ext | ||
| repository: localstack/localstack | ||
| github-token: ${{ secrets.GH_PAT_FEATURE_CATALOG_PAGE_PRO }} # PAT with access to artifacts from GH Actions | ||
| run-id: ${{ env.FEATURES_ARTIFACTS_EXT_RUN_ID }} | ||
|
|
||
| - name: Generate feature catalog page | ||
| run: python3 scripts/generate_feature_catalog_page.py | ||
| env: | ||
| PATH_FEATURE_FILES_COMMUNITY: 'features-files-community' | ||
| PATH_FEATURE_FILES_EXT: 'features-files-ext' | ||
| PATH_FEATURE_CATALOG_MD: 'content/en/user-guide/aws/feature-coverage.md' | ||
|
|
||
| - name: Create PR | ||
| uses: peter-evans/create-pull-request@v7 | ||
| with: | ||
| title: "Update Feature catalog page" | ||
| body: "This PR updates Feature catalog page based on feature catalog YAML files" | ||
| branch: "update-feature-catalog" | ||
| add-paths: "content/en/user-guide/aws/feature-coverage.md" | ||
| author: "LocalStack Bot <localstack-bot@users.noreply.github.com>" | ||
| committer: "LocalStack Bot <localstack-bot@users.noreply.github.com>" | ||
| commit-message: "Upgrade feature catalog" | ||
| labels: "documentation" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| import os | ||
| import sys | ||
| from pathlib import Path | ||
|
|
||
| import yaml | ||
|
|
||
| DEFAULT_STATUS = 'unsupported' | ||
| DEFAULT_EMULATION_LEVEL = 'CRUD' | ||
| FEATURES_FILE_NAME='features.yml' | ||
|
|
||
| MD_FILE_HEADER = """--- | ||
| title: "AWS Service Feature Coverage" | ||
| linkTitle: "⭐ Feature Coverage" | ||
| weight: 1 | ||
| description: > | ||
| Overview of the implemented AWS APIs and their level of parity with the AWS cloud | ||
| aliases: | ||
| - /localstack/coverage/ | ||
| - /aws/feature-coverage/ | ||
| hide_readingtime: true | ||
| --- | ||
|
|
||
|
|
||
| ## Emulation Levels | ||
|
|
||
| * CRUD: The service accepts requests and returns proper (potentially static) responses. | ||
| No additional business logic besides storing entities. | ||
| * Emulated: The service imitates the functionality, including synchronous and asynchronous business logic operating on service entities. | ||
|
|
||
| | Service / Feature | Implementation status | Emulation Level | Limitations | | ||
| |-------------------|----------------|-----------------|--------------------------|""" | ||
|
|
||
| class FeatureCatalogMarkdownGenerator: | ||
| md_content = [MD_FILE_HEADER] | ||
|
|
||
| def __init__(self, file_path: str): | ||
| self.file_path = file_path | ||
| pass | ||
|
|
||
| def add_service_section(self, feature_file_content: str): | ||
| service_name = feature_file_content.get('name') | ||
| emulation_level = feature_file_content.get('emulation_level', DEFAULT_EMULATION_LEVEL) | ||
| self.md_content.append(f"| **{service_name}** | [Details 🔍] | {emulation_level} | |") | ||
|
|
||
| def add_features_rows(self, feature_file_content: str): | ||
| for feature in feature_file_content.get('features', []): | ||
| feature_name = feature.get('name', '') | ||
| documentation_page = feature.get('documentation_page') | ||
| if documentation_page: | ||
| feature_name = f'[{feature_name}]({documentation_page})' | ||
| status = feature.get('status', DEFAULT_STATUS) | ||
|
|
||
| limitations = feature.get('limitations', []) | ||
| limitations_md = '\n '.join(limitations) if limitations else '' | ||
|
|
||
| self.md_content.append(f"| {feature_name} | {status} | | {limitations_md} |") | ||
|
|
||
| def generate_file(self): | ||
| try: | ||
| with open(self.file_path, "w") as feature_coverage_md_file: | ||
| feature_coverage_md_file.writelines(s + '\n' for s in self.md_content) | ||
| except Exception as e: | ||
| print(f"Error writing to file: {e}") | ||
| sys.exit(1) | ||
|
|
||
| def load_yaml_file(file_path: str): | ||
| try: | ||
| with open(file_path, 'r') as file: | ||
| return yaml.safe_load(file) | ||
| except yaml.YAMLError as e: | ||
| sys.stdout.write(f"::error title=Failed to parse features file::An error occurred while parsing {file_path}: {e}") | ||
| sys.exit(1) | ||
| except FileNotFoundError: | ||
| sys.stdout.write(f"::error title=Missing features file::No features file found at {file_path}") | ||
| sys.exit(1) | ||
|
|
||
| def get_service_path_to_abs_community_ext_paths(community_files_path: str, ext_files_path: str) -> dict[str, (str, str)]: | ||
| relative_to_abs_paths = {} | ||
| for community_abs_path in Path(community_files_path).rglob(FEATURES_FILE_NAME): | ||
| rel_path = str(community_abs_path.relative_to(community_files_path)) | ||
| relative_to_abs_paths[rel_path] = (community_abs_path, None) | ||
|
|
||
| for abs_path_ext in Path(ext_files_path).rglob(FEATURES_FILE_NAME): | ||
| rel_path = str(abs_path_ext.relative_to(ext_files_path)) | ||
| if rel_path in relative_to_abs_paths: | ||
| community_abs_path, _ = relative_to_abs_paths[rel_path] | ||
| relative_to_abs_paths[rel_path] = (community_abs_path, abs_path_ext) | ||
| else: | ||
| relative_to_abs_paths[rel_path] = (None, abs_path_ext) | ||
| return relative_to_abs_paths | ||
|
|
||
| def main(): | ||
| community_feature_files_path = os.getenv('PATH_FEATURE_FILES_COMMUNITY') | ||
| ext_feature_files_path = os.getenv('PATH_FEATURE_FILES_EXT') | ||
| feature_catalog_md_file_path = os.getenv('PATH_FEATURE_CATALOG_MD') | ||
|
|
||
| service_path_to_abs_paths = get_service_path_to_abs_community_ext_paths(community_feature_files_path, ext_feature_files_path) | ||
| md_generator = FeatureCatalogMarkdownGenerator(feature_catalog_md_file_path) | ||
|
|
||
| for service_name in sorted(service_path_to_abs_paths): | ||
| abs_path_community, abs_path_ext = service_path_to_abs_paths.get(service_name) | ||
| service_definition_created = False | ||
| if abs_path_community: | ||
| feature_file_community = load_yaml_file(abs_path_community) | ||
| md_generator.add_service_section(feature_file_community) | ||
| service_definition_created = True | ||
| md_generator.add_features_rows(feature_file_community) | ||
| if abs_path_ext: | ||
| feature_file_ext = load_yaml_file(abs_path_ext) | ||
| if not service_definition_created: | ||
| md_generator.add_service_section(feature_file_community) | ||
| md_generator.add_features_rows(feature_file_ext) | ||
| md_generator.generate_file() | ||
|
|
||
| if __name__ == "__main__": | ||
| main() |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.