Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] (RADICAL) Improve Job documentation structure #3511

Merged
merged 3 commits into from
May 2, 2024

Conversation

landlord-matt
Copy link
Contributor

@landlord-matt landlord-matt commented Apr 25, 2024

Changes

It seems the number of parameters has outgrown the current structure of the job documentation. This PR ensures that

  • all documented fields are listed in the first index section
  • the parameter hierarchy of the task parameters is reflected and the task parameters are grouped

Tests

  • make test run locally
  • relevant change in docs/ folder
  • covered with integration tests in internal/acceptance
  • relevant acceptance tests are passing
  • using Go SDK

@landlord-matt landlord-matt requested review from a team as code owners April 25, 2024 14:43
@landlord-matt landlord-matt requested review from hectorcast-db and removed request for a team April 25, 2024 14:43
@landlord-matt
Copy link
Contributor Author

Any thoughts on any of these two suggestions (repurposing the top of the argument reference as index and grouping the task parameters)? I guess it is not trivial to structure given that some of the arguments can be placed on both job and task level.

Perhaps main commiter @alexott?

@nkvuong nkvuong requested a review from alexott May 1, 2024 18:40
@alexott alexott enabled auto-merge May 2, 2024 11:02
@alexott alexott disabled auto-merge May 2, 2024 12:11
@alexott alexott enabled auto-merge May 2, 2024 12:11
@alexott alexott added this pull request to the merge queue May 2, 2024
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.70%. Comparing base (5d255ad) to head (381bdfc).
Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3511      +/-   ##
==========================================
- Coverage   82.71%   82.70%   -0.02%     
==========================================
  Files         183      183              
  Lines       18827    18827              
==========================================
- Hits        15573    15571       -2     
- Misses       2339     2340       +1     
- Partials      915      916       +1     

see 1 file with indirect coverage changes

Merged via the queue into databricks:main with commit fb762de May 2, 2024
5 checks passed
@alexott
Copy link
Contributor

alexott commented May 2, 2024

Thank you @landlord-matt !

mgyucht added a commit that referenced this pull request May 3, 2024
### New Features and Improvements
* Allow changes to `display_name` for `databricks_service_principal` ([#3523](#3523)).
* Modify `databricks_repo` to support Git folders in the workspace ([#3447](#3447)).
* Retry requests in the Databricks Go SDK until context timeout ([#3537](#3537)).
* Set ForceSendFields for boolean/integer values explicitly set to false/0 ([#3385](#3385)).

### Bug fixes
* Added `ConflictsWith` to legacy tasks in `databricks_job` to avoid errors when job parameters are used ([#3500](#3500)).
* Fixed spn secret permanent diff ([#3525](#3525)).
* Introduced `common.NoClientData` helper function and use it in `databricks_aws_crossaccount_policy` data source to allow its use with workspace- or account-level providers ([#3508](#3508)).
* Set `id` attribute of `databricks_metastore` data source ([#3503](#3503)).

### Documentation Changes
* Added documentation for compute options for tasks in `databricks_job` resource ([#3504](#3504)).
* Added warning about depends_on order ([#3489](#3489)).
* Fixed various doc typos ([#3512](#3512)).
* Fixed `databricks_current_metastore` data source names in docs examples ([#3515](#3515)).
* Improved Job documentation structure ([#3511](#3511)).
* Removed confusing mention of cloud types for `databricks_clusters` and `databricks_sql_warehouses` data sources ([#3518](#3518)).

### Exporter
* Added UC support for DLT pipelines ([#3494](#3494)).
* Ignored instance pools with the empty name ([#3522](#3522)).

### Internal Changes
* Added DATABRICKS_USER_AGENT_EXTRA ([#3520](#3520)).
* Adjusted list of supported platforms in `godownloader-databricks-provider.sh` ([#3527](#3527)).
* Switched to Go SDK for `databricks_directory` resource and data source ([#3509](#3509)).
* Fixed codecov for repository ([#3530](#3530)).

Dependency updates:

 * Bump databricks-sdk-go to v0.40.0 ([#3534](#3534)).
@mgyucht mgyucht mentioned this pull request May 3, 2024
github-merge-queue bot pushed a commit that referenced this pull request May 3, 2024
* Release v1.42.0

### New Features and Improvements
* Allow changes to `display_name` for `databricks_service_principal` ([#3523](#3523)).
* Modify `databricks_repo` to support Git folders in the workspace ([#3447](#3447)).
* Retry requests in the Databricks Go SDK until context timeout ([#3537](#3537)).
* Set ForceSendFields for boolean/integer values explicitly set to false/0 ([#3385](#3385)).

### Bug fixes
* Added `ConflictsWith` to legacy tasks in `databricks_job` to avoid errors when job parameters are used ([#3500](#3500)).
* Fixed spn secret permanent diff ([#3525](#3525)).
* Introduced `common.NoClientData` helper function and use it in `databricks_aws_crossaccount_policy` data source to allow its use with workspace- or account-level providers ([#3508](#3508)).
* Set `id` attribute of `databricks_metastore` data source ([#3503](#3503)).

### Documentation Changes
* Added documentation for compute options for tasks in `databricks_job` resource ([#3504](#3504)).
* Added warning about depends_on order ([#3489](#3489)).
* Fixed various doc typos ([#3512](#3512)).
* Fixed `databricks_current_metastore` data source names in docs examples ([#3515](#3515)).
* Improved Job documentation structure ([#3511](#3511)).
* Removed confusing mention of cloud types for `databricks_clusters` and `databricks_sql_warehouses` data sources ([#3518](#3518)).

### Exporter
* Added UC support for DLT pipelines ([#3494](#3494)).
* Ignored instance pools with the empty name ([#3522](#3522)).

### Internal Changes
* Added DATABRICKS_USER_AGENT_EXTRA ([#3520](#3520)).
* Adjusted list of supported platforms in `godownloader-databricks-provider.sh` ([#3527](#3527)).
* Switched to Go SDK for `databricks_directory` resource and data source ([#3509](#3509)).
* Fixed codecov for repository ([#3530](#3530)).

Dependency updates:

 * Bump databricks-sdk-go to v0.40.0 ([#3534](#3534)).

* Changelog
@landlord-matt landlord-matt deleted the patch-4 branch May 7, 2024 07:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants