Copying logs on kokoro artifacts incase of integration test failure. #1380
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Tulsishah
requested review from
sethiay,
vadlakondaswetha and
raj-prince
as code owners
Tulsishah
changed the title
Tulsishah
changed the title
gargnitingoogle
previously approved these changes
Sep 19, 2023
Tulsishah
changed the title
gargnitingoogle
pushed a commit
that referenced
this pull request
Oct 4, 2023
…1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments
gargnitingoogle
pushed a commit
that referenced
this pull request
Oct 10, 2023
…1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments
gargnitingoogle
pushed a commit
that referenced
this pull request
Oct 25, 2023
…1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments
gargnitingoogle
pushed a commit
that referenced
this pull request
Oct 26, 2023
…1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments
Tulsishah
added a commit
that referenced
this pull request
Oct 27, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changing description and flag names * fixing comment * updating flag help description * Formating changes * Update run_load_test_and_fetch_metrics.sh * undo unnecessay changes * empty commit --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
pushed a commit
that referenced
this pull request
Oct 30, 2023
…1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments
gargnitingoogle
added a commit
that referenced
this pull request
Oct 30, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changing description and flag names * fixing comment * updating flag help description * Formating changes * Update run_load_test_and_fetch_metrics.sh * undo unnecessay changes * empty commit --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
pushed a commit
that referenced
this pull request
Oct 31, 2023
…1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments
gargnitingoogle
added a commit
that referenced
this pull request
Oct 31, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changing description and flag names * fixing comment * updating flag help description * Formating changes * Update run_load_test_and_fetch_metrics.sh * undo unnecessay changes * empty commit --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
Tulsishah
added a commit
that referenced
this pull request
Nov 1, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * adding big query table setup * changes for small pr * adding requirements and setup scripts * indentation fix * removing unnecessary changes * adding start build time * adding requirements and setup scripts * python file changes * small fix * testing * testing gsheet upload * perfmetrics/scripts/ls_metrics/listing_benchmark.py * testing * removing unnecessary changes * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changes for small pr * indentation fix * removing unnecessary changes * adding requirements and setup scripts * merge with parent * testing * testing gsheet upload * merge with parent * removing unnecessary changes * fixing requirements.in * Creating big query tables for perf tests - 2 (#1445) * adding big query table setup * changes for small pr * adding requirements and setup scripts * fixing requirements.in * merge * merge * small fixes * small fixes * small fix * undo testing changes * formating * testing changes * undo testing changes * remove unnecessary functions * fixing indentation * small fix * adding testing changes * small fix * undo testing changes * formating * fixing unit tests * formatting * fixing commets and adding test changes * undo testing changes --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
pushed a commit
that referenced
this pull request
Nov 1, 2023
…1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments
gargnitingoogle
added a commit
that referenced
this pull request
Nov 1, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changing description and flag names * fixing comment * updating flag help description * Formating changes * Update run_load_test_and_fetch_metrics.sh * undo unnecessay changes * empty commit --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
added a commit
that referenced
this pull request
Nov 1, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * adding big query table setup * changes for small pr * adding requirements and setup scripts * indentation fix * removing unnecessary changes * adding start build time * adding requirements and setup scripts * python file changes * small fix * testing * testing gsheet upload * perfmetrics/scripts/ls_metrics/listing_benchmark.py * testing * removing unnecessary changes * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changes for small pr * indentation fix * removing unnecessary changes * adding requirements and setup scripts * merge with parent * testing * testing gsheet upload * merge with parent * removing unnecessary changes * fixing requirements.in * Creating big query tables for perf tests - 2 (#1445) * adding big query table setup * changes for small pr * adding requirements and setup scripts * fixing requirements.in * merge * merge * small fixes * small fixes * small fix * undo testing changes * formating * testing changes * undo testing changes * remove unnecessary functions * fixing indentation * small fix * adding testing changes * small fix * undo testing changes * formating * fixing unit tests * formatting * fixing commets and adding test changes * undo testing changes --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
Tulsishah
added a commit
that referenced
this pull request
Nov 7, 2023
…ches (#1455) * Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * adding big query table setup * changes for small pr * adding requirements and setup scripts * python file changes * small fix * indentation fix * removing unnecessary changes * local changes * adding start build time * adding requirements and setup scripts * python file changes * small fix * testing * testing gsheet upload * perfmetrics/scripts/ls_metrics/listing_benchmark.py * testing * removing unnecessary changes * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changes for small pr * indentation fix * removing unnecessary changes * adding requirements and setup scripts * merge with parent * testing * testing gsheet upload * merge with parent * removing unnecessary changes * fixing requirements.in * adding build scripts * updating json file * small fix * testing * testing * testing * testing * Creating big query tables for perf tests - 2 (#1445) * adding big query table setup * changes for small pr * adding requirements and setup scripts * fixing requirements.in * merge * merge * merge * testing * small fixes * small fixes * small fix * Merge * undo testing changes * formating * testing changes * undo testing changes * remove unnecessary functions * fixing indentation * removing env files * adding testing statements * small fix * adding testing changes * small fix * undo testing changes * formating * fixing unit tests * formatting * small fix * small fix * small fix * small fix * fixing commets and adding test changes * undo testing changes * removing unnecessary changes * formating Readme * changing branch name --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
Tulsishah
added a commit
that referenced
this pull request
Nov 10, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * adding big query table setup * changes for small pr * adding requirements and setup scripts * python file changes * small fix * indentation fix * removing unnecessary changes * local changes * adding start build time * adding requirements and setup scripts * python file changes * small fix * testing * testing gsheet upload * perfmetrics/scripts/ls_metrics/listing_benchmark.py * testing * removing unnecessary changes * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changes for small pr * indentation fix * removing unnecessary changes * adding requirements and setup scripts * merge with parent * testing * testing gsheet upload * merge with parent * removing unnecessary changes * fixing requirements.in * adding build scripts * updating json file * small fix * testing * testing * testing * testing * Creating big query tables for perf tests - 2 (#1445) * adding big query table setup * changes for small pr * adding requirements and setup scripts * fixing requirements.in * merge * merge * merge * testing * small fixes * small fixes * small fix * Merge * undo testing changes * formating * testing changes * undo testing changes * remove unnecessary functions * fixing indentation * removing env files * adding testing statements * small fix * adding testing changes * small fix * undo testing changes * formating * fixing unit tests * formatting * small fix * small fix * small fix * small fix * fixing commets and adding test changes * undo testing changes * removing unnecessary changes * formating Readme * testing * fix gaps * fix gaps * undo test changes * test changes for config file setup * test changes for config file setup * adding column in table * fixing python script to add config file flag * json to string for big query table * adding testing changes * fixing branch name * Undo test changes * Adding config file flag in readme * small fix * small fix * updating Readme * small fix * fixing if condition * undo testing changes * undo testing changes * adding unit tests * fixing comment * small fix * fix comments * small fix * compress the json content --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
pushed a commit
that referenced
this pull request
Nov 28, 2023
…1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments
gargnitingoogle
added a commit
that referenced
this pull request
Nov 28, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changing description and flag names * fixing comment * updating flag help description * Formating changes * Update run_load_test_and_fetch_metrics.sh * undo unnecessay changes * empty commit --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
added a commit
that referenced
this pull request
Nov 28, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * adding big query table setup * changes for small pr * adding requirements and setup scripts * indentation fix * removing unnecessary changes * adding start build time * adding requirements and setup scripts * python file changes * small fix * testing * testing gsheet upload * perfmetrics/scripts/ls_metrics/listing_benchmark.py * testing * removing unnecessary changes * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changes for small pr * indentation fix * removing unnecessary changes * adding requirements and setup scripts * merge with parent * testing * testing gsheet upload * merge with parent * removing unnecessary changes * fixing requirements.in * Creating big query tables for perf tests - 2 (#1445) * adding big query table setup * changes for small pr * adding requirements and setup scripts * fixing requirements.in * merge * merge * small fixes * small fixes * small fix * undo testing changes * formating * testing changes * undo testing changes * remove unnecessary functions * fixing indentation * small fix * adding testing changes * small fix * undo testing changes * formating * fixing unit tests * formatting * fixing commets and adding test changes * undo testing changes --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
added a commit
that referenced
this pull request
Nov 28, 2023
…ches (#1455) * Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * adding big query table setup * changes for small pr * adding requirements and setup scripts * python file changes * small fix * indentation fix * removing unnecessary changes * local changes * adding start build time * adding requirements and setup scripts * python file changes * small fix * testing * testing gsheet upload * perfmetrics/scripts/ls_metrics/listing_benchmark.py * testing * removing unnecessary changes * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changes for small pr * indentation fix * removing unnecessary changes * adding requirements and setup scripts * merge with parent * testing * testing gsheet upload * merge with parent * removing unnecessary changes * fixing requirements.in * adding build scripts * updating json file * small fix * testing * testing * testing * testing * Creating big query tables for perf tests - 2 (#1445) * adding big query table setup * changes for small pr * adding requirements and setup scripts * fixing requirements.in * merge * merge * merge * testing * small fixes * small fixes * small fix * Merge * undo testing changes * formating * testing changes * undo testing changes * remove unnecessary functions * fixing indentation * removing env files * adding testing statements * small fix * adding testing changes * small fix * undo testing changes * formating * fixing unit tests * formatting * small fix * small fix * small fix * small fix * fixing commets and adding test changes * undo testing changes * removing unnecessary changes * formating Readme * changing branch name --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
added a commit
that referenced
this pull request
Nov 28, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * adding big query table setup * changes for small pr * adding requirements and setup scripts * python file changes * small fix * indentation fix * removing unnecessary changes * local changes * adding start build time * adding requirements and setup scripts * python file changes * small fix * testing * testing gsheet upload * perfmetrics/scripts/ls_metrics/listing_benchmark.py * testing * removing unnecessary changes * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changes for small pr * indentation fix * removing unnecessary changes * adding requirements and setup scripts * merge with parent * testing * testing gsheet upload * merge with parent * removing unnecessary changes * fixing requirements.in * adding build scripts * updating json file * small fix * testing * testing * testing * testing * Creating big query tables for perf tests - 2 (#1445) * adding big query table setup * changes for small pr * adding requirements and setup scripts * fixing requirements.in * merge * merge * merge * testing * small fixes * small fixes * small fix * Merge * undo testing changes * formating * testing changes * undo testing changes * remove unnecessary functions * fixing indentation * removing env files * adding testing statements * small fix * adding testing changes * small fix * undo testing changes * formating * fixing unit tests * formatting * small fix * small fix * small fix * small fix * fixing commets and adding test changes * undo testing changes * removing unnecessary changes * formating Readme * testing * fix gaps * fix gaps * undo test changes * test changes for config file setup * test changes for config file setup * adding column in table * fixing python script to add config file flag * json to string for big query table * adding testing changes * fixing branch name * Undo test changes * Adding config file flag in readme * small fix * small fix * updating Readme * small fix * fixing if condition * undo testing changes * undo testing changes * adding unit tests * fixing comment * small fix * fix comments * small fix * compress the json content --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
pushed a commit
that referenced
this pull request
Nov 29, 2023
…1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments
gargnitingoogle
added a commit
that referenced
this pull request
Nov 29, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changing description and flag names * fixing comment * updating flag help description * Formating changes * Update run_load_test_and_fetch_metrics.sh * undo unnecessay changes * empty commit --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
added a commit
that referenced
this pull request
Nov 29, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * adding big query table setup * changes for small pr * adding requirements and setup scripts * indentation fix * removing unnecessary changes * adding start build time * adding requirements and setup scripts * python file changes * small fix * testing * testing gsheet upload * perfmetrics/scripts/ls_metrics/listing_benchmark.py * testing * removing unnecessary changes * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changes for small pr * indentation fix * removing unnecessary changes * adding requirements and setup scripts * merge with parent * testing * testing gsheet upload * merge with parent * removing unnecessary changes * fixing requirements.in * Creating big query tables for perf tests - 2 (#1445) * adding big query table setup * changes for small pr * adding requirements and setup scripts * fixing requirements.in * merge * merge * small fixes * small fixes * small fix * undo testing changes * formating * testing changes * undo testing changes * remove unnecessary functions * fixing indentation * small fix * adding testing changes * small fix * undo testing changes * formating * fixing unit tests * formatting * fixing commets and adding test changes * undo testing changes --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
added a commit
that referenced
this pull request
Nov 29, 2023
…ches (#1455) * Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * adding big query table setup * changes for small pr * adding requirements and setup scripts * python file changes * small fix * indentation fix * removing unnecessary changes * local changes * adding start build time * adding requirements and setup scripts * python file changes * small fix * testing * testing gsheet upload * perfmetrics/scripts/ls_metrics/listing_benchmark.py * testing * removing unnecessary changes * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changes for small pr * indentation fix * removing unnecessary changes * adding requirements and setup scripts * merge with parent * testing * testing gsheet upload * merge with parent * removing unnecessary changes * fixing requirements.in * adding build scripts * updating json file * small fix * testing * testing * testing * testing * Creating big query tables for perf tests - 2 (#1445) * adding big query table setup * changes for small pr * adding requirements and setup scripts * fixing requirements.in * merge * merge * merge * testing * small fixes * small fixes * small fix * Merge * undo testing changes * formating * testing changes * undo testing changes * remove unnecessary functions * fixing indentation * removing env files * adding testing statements * small fix * adding testing changes * small fix * undo testing changes * formating * fixing unit tests * formatting * small fix * small fix * small fix * small fix * fixing commets and adding test changes * undo testing changes * removing unnecessary changes * formating Readme * changing branch name --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
added a commit
that referenced
this pull request
Nov 29, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * adding big query table setup * changes for small pr * adding requirements and setup scripts * python file changes * small fix * indentation fix * removing unnecessary changes * local changes * adding start build time * adding requirements and setup scripts * python file changes * small fix * testing * testing gsheet upload * perfmetrics/scripts/ls_metrics/listing_benchmark.py * testing * removing unnecessary changes * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changes for small pr * indentation fix * removing unnecessary changes * adding requirements and setup scripts * merge with parent * testing * testing gsheet upload * merge with parent * removing unnecessary changes * fixing requirements.in * adding build scripts * updating json file * small fix * testing * testing * testing * testing * Creating big query tables for perf tests - 2 (#1445) * adding big query table setup * changes for small pr * adding requirements and setup scripts * fixing requirements.in * merge * merge * merge * testing * small fixes * small fixes * small fix * Merge * undo testing changes * formating * testing changes * undo testing changes * remove unnecessary functions * fixing indentation * removing env files * adding testing statements * small fix * adding testing changes * small fix * undo testing changes * formating * fixing unit tests * formatting * small fix * small fix * small fix * small fix * fixing commets and adding test changes * undo testing changes * removing unnecessary changes * formating Readme * testing * fix gaps * fix gaps * undo test changes * test changes for config file setup * test changes for config file setup * adding column in table * fixing python script to add config file flag * json to string for big query table * adding testing changes * fixing branch name * Undo test changes * Adding config file flag in readme * small fix * small fix * updating Readme * small fix * fixing if condition * undo testing changes * undo testing changes * adding unit tests * fixing comment * small fix * fix comments * small fix * compress the json content --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
pushed a commit
that referenced
this pull request
Nov 29, 2023
…1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments
gargnitingoogle
added a commit
that referenced
this pull request
Nov 29, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changing description and flag names * fixing comment * updating flag help description * Formating changes * Update run_load_test_and_fetch_metrics.sh * undo unnecessay changes * empty commit --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
added a commit
that referenced
this pull request
Nov 29, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * adding big query table setup * changes for small pr * adding requirements and setup scripts * indentation fix * removing unnecessary changes * adding start build time * adding requirements and setup scripts * python file changes * small fix * testing * testing gsheet upload * perfmetrics/scripts/ls_metrics/listing_benchmark.py * testing * removing unnecessary changes * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changes for small pr * indentation fix * removing unnecessary changes * adding requirements and setup scripts * merge with parent * testing * testing gsheet upload * merge with parent * removing unnecessary changes * fixing requirements.in * Creating big query tables for perf tests - 2 (#1445) * adding big query table setup * changes for small pr * adding requirements and setup scripts * fixing requirements.in * merge * merge * small fixes * small fixes * small fix * undo testing changes * formating * testing changes * undo testing changes * remove unnecessary functions * fixing indentation * small fix * adding testing changes * small fix * undo testing changes * formating * fixing unit tests * formatting * fixing commets and adding test changes * undo testing changes --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
added a commit
that referenced
this pull request
Nov 29, 2023
…ches (#1455) * Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * adding big query table setup * changes for small pr * adding requirements and setup scripts * python file changes * small fix * indentation fix * removing unnecessary changes * local changes * adding start build time * adding requirements and setup scripts * python file changes * small fix * testing * testing gsheet upload * perfmetrics/scripts/ls_metrics/listing_benchmark.py * testing * removing unnecessary changes * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changes for small pr * indentation fix * removing unnecessary changes * adding requirements and setup scripts * merge with parent * testing * testing gsheet upload * merge with parent * removing unnecessary changes * fixing requirements.in * adding build scripts * updating json file * small fix * testing * testing * testing * testing * Creating big query tables for perf tests - 2 (#1445) * adding big query table setup * changes for small pr * adding requirements and setup scripts * fixing requirements.in * merge * merge * merge * testing * small fixes * small fixes * small fix * Merge * undo testing changes * formating * testing changes * undo testing changes * remove unnecessary functions * fixing indentation * removing env files * adding testing statements * small fix * adding testing changes * small fix * undo testing changes * formating * fixing unit tests * formatting * small fix * small fix * small fix * small fix * fixing commets and adding test changes * undo testing changes * removing unnecessary changes * formating Readme * changing branch name --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
gargnitingoogle
added a commit
that referenced
this pull request
Nov 29, 2023
* Fix DiffFiles utility function 1. It chunkifies DiffFiles utility function. This rationalizes the memory requirement of file-op of this utility. This utility earlier used to the read the complete files in memory, but now only a chunk is read and compared at a time. 2. Rename DiffFiles to AreFilesIdentical Renames file-op utility DiffFiles to AreFilesIdentical and changes it to return bool instead of int. * Copying logs on kokoro artifacts incase of integration test failure. (#1380) * adding print log file statement * failure test * testing error * undo testing changes * formating * fix comment * adding test error * small fix * small fix * testing * adding action artifacts * small fix * small fix * removing error statement * testing * testing * testing * undo changes * undo testing changes * fixing comments * Upgrading go version from 1.21.0 to 1.21.1 (#1401) * Upgrading go version to 1.21.1 * Temp changes to run perf/integration test * Revert "Temp changes to run perf/integration test" This reverts commit 18dd533. * Updating semantics docs (#1407) * Print stack-trace on crash For now, it just prints stack-trace whenever anyone calls logger.Fatalf(...). This needs to be enhanced further to include more scenarios (i.e. more sources of crashes). * Throw error in case of unexpected fields (#1416) Also, improved the doc for config-file. * Upgraded the fuse library (#1419) * add "create-empty-file: true" tests to run_tests_mounted_directory.sh (#1413) * Upgrade golang from 1.21.1 to 1.21.2 (#1431) Upgrade from golang 1.21.1 to 1.21.2 * Update yaml package version from v2 to v3 in integration tests (#1434) * update go yaml package version from v2 to v3 in integration tests * Empty-Commit * Passing gcsfuse flags from build.sh to mount the test bucket for perf… (#1430) * Passing gcsfuse flags from build.sh to mount the test bucket for performing list benchmarking * removing __pycache * small fix * small fix * unmount after fio tests * small fix * empty commit * testing kokoro perf tests * undo testing changes * adding upload flag * small fix * small fix * upload to upload_gs * unnecessary change * adding big query table setup * changes for small pr * adding requirements and setup scripts * python file changes * small fix * indentation fix * removing unnecessary changes * local changes * adding start build time * adding requirements and setup scripts * python file changes * small fix * testing * testing gsheet upload * perfmetrics/scripts/ls_metrics/listing_benchmark.py * testing * removing unnecessary changes * Creating big query tables for perf tests - 1 (#1444) * adding big query table setup * changes for small pr * indentation fix * removing unnecessary changes * adding start build time * changes for small pr * indentation fix * removing unnecessary changes * adding requirements and setup scripts * merge with parent * testing * testing gsheet upload * merge with parent * removing unnecessary changes * fixing requirements.in * adding build scripts * updating json file * small fix * testing * testing * testing * testing * Creating big query tables for perf tests - 2 (#1445) * adding big query table setup * changes for small pr * adding requirements and setup scripts * fixing requirements.in * merge * merge * merge * testing * small fixes * small fixes * small fix * Merge * undo testing changes * formating * testing changes * undo testing changes * remove unnecessary functions * fixing indentation * removing env files * adding testing statements * small fix * adding testing changes * small fix * undo testing changes * formating * fixing unit tests * formatting * small fix * small fix * small fix * small fix * fixing commets and adding test changes * undo testing changes * removing unnecessary changes * formating Readme * testing * fix gaps * fix gaps * undo test changes * test changes for config file setup * test changes for config file setup * adding column in table * fixing python script to add config file flag * json to string for big query table * adding testing changes * fixing branch name * Undo test changes * Adding config file flag in readme * small fix * small fix * updating Readme * small fix * fixing if condition * undo testing changes * undo testing changes * adding unit tests * fixing comment * small fix * fix comments * small fix * compress the json content --------- Co-authored-by: Nitin Garg <gargnitin@google.com> Co-authored-by: Prince Kumar <princer@google.com> Co-authored-by: Ayush Sethi <ayushsethi@google.com> Co-authored-by: Ashmeen Kaur <57195160+ashmeenkaur@users.noreply.github.com> Co-authored-by: Nitin Garg <113666283+gargnitingoogle@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Copying logs on kokoro artifacts incase of integration test failure.
O_DIRECT flag was giving error of invalid argument while io.Copy.
Link to the issue in case of a bug fix.
NA
Testing details