Skip to content

Commit

Permalink
addressed pr comments, updated strings to scalar type
Browse files Browse the repository at this point in the history
  • Loading branch information
ShourieG committed Apr 29, 2024
1 parent e2c4c5c commit 3a96b7f
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 7 deletions.
2 changes: 1 addition & 1 deletion packages/google_cloud_storage/docs/README.md
Expand Up @@ -14,7 +14,7 @@ even though it can get expensive with dealing with a very large number of files.
3. If any major error occurs that stops the main thread, the logs will be appropriately generated, describing said error.


NOTE: Currently only `JSON` is supported with respect to object/file formats. We also support gzipped JSON objects/files. As for authentication types, we currently have support for `JSON credential keys` and `credential files`. If a download for a file/object fails or gets interrupted, the download is retried two times. This is currently not user-configurable.
NOTE: Currently only `JSON` is supported with respect to object/file formats. We also support gzipped JSON objects/files. As for authentication types, we currently have support for `JSON credential keys` and `credential files`. If a download for a file/object fails or gets interrupted, the download is retried three times. This is currently not user-configurable.

## The GCS credentials key file:
This is a one-time download JSON key file that you get after adding a key to a GCP service account.
Expand Down
3 changes: 1 addition & 2 deletions packages/google_cloud_storage/fields/agent.yml
Expand Up @@ -31,6 +31,5 @@
- name: os.codename
type: keyword
example: "stretch"
description: >
description: >-
OS codename, if any.
12 changes: 8 additions & 4 deletions packages/google_cloud_storage/manifest.yml
Expand Up @@ -103,14 +103,16 @@ policy_templates:
- name: buckets
type: yaml
title: Buckets
description: "This attribute contains the details about a specific bucket like, name, number_of_workers, poll, poll_interval and bucket_timeout. The attribute 'name' is specific to a bucket as it describes the bucket name, while the fields number_of_workers, poll, poll_interval and bucket_timeout can exist both at the bucket level and at the global level. If you have already defined the attributes globally, then you can only specify the name in this yaml config. If you want to override any specific attribute for a specific bucket, then, you can define it here. Any attribute defined in the yaml will override the global definitions. Please see the relevant [Documentation](https://www.elastic.co/guide/en/beats/filebeat/8.5/filebeat-input-gcs.html#attrib-buckets) for further information.\n"
description: >
This attribute contains the details about a specific bucket like, name, number_of_workers, poll, poll_interval and bucket_timeout. The attribute 'name' is specific to a bucket as it describes the bucket name, while the fields number_of_workers, poll, poll_interval and bucket_timeout can exist both at the bucket level and at the global level. If you have already defined the attributes globally, then you can only specify the name in this yaml config. If you want to override any specific attribute for a specific bucket, then, you can define it here. Any attribute defined in the yaml will override the global definitions. Please see the relevant [Documentation](https://www.elastic.co/guide/en/beats/filebeat/8.5/filebeat-input-gcs.html#attrib-buckets) for further information.
required: true
show_user: true
default: |
# You can define as many buckets as you want here.
- name: logs
- name: logs_2
# - name: logs
# - name: logs_2
# The config below is an example of how to override the global config.
Expand All @@ -127,7 +129,9 @@ policy_templates:
show_user: false
default: |
# - regex: "event/"
description: "If the GCS bucket will have events that correspond to files that this integration shouldn’t process, file_selectors can be used to limit the files that are downloaded. This is a list of selectors which is made up of regex patters. \nThe regex should match the GCS bucket filepath. Regexes use [RE2 syntax](https://pkg.go.dev/regexp/syntax). Files that don’t match one of the regexes will not be processed.\n"
description: >
If the GCS bucket will have events that correspond to files that this integration shouldn’t process, file_selectors can be used to limit the files that are downloaded. This is a list of selectors which is made up of regex patters. The regex should match the GCS bucket filepath. Regexes use [RE2 syntax](https://pkg.go.dev/regexp/syntax). Files that don’t match one of the regexes will not be processed.
- name: timestamp_epoch
type: integer
title: Timestamp Epoch
Expand Down

0 comments on commit 3a96b7f

Please sign in to comment.