{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":96542958,"defaultBranch":"master","name":"YDBTest","ownerLogin":"YottaDB","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2017-07-07T13:50:05.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/29870125?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1708713500.0","currentOid":""},"activityList":{"items":[{"before":"a38250b9529bb70136f1b4f4b3792783d5dd28ec","after":"59dadb4c00535ae7f645d62e791dc3b0106284f6","ref":"refs/heads/master","pushedAt":"2024-06-26T13:02:33.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[#628] [#630] [#631] Audit Listener utility for testing audit logging\n\nThis utility is a test/reference implementation server\nfor the audit logging facility. See audit_listener.md\nfor features, compilation instructions and usage with\nexamples.\n\nFor help on audit logging and get the original version\nof this utility, visit:\nhttp://tinco.pair.com/bhaskar/gtm/doc/books/ao/UNIX_manual/ch03s06.html#audit_principal_device","shortMessageHtmlLink":"[#628] [#630] [#631] Audit Listener utility for testing audit logging"}},{"before":"2363abd78cf1046123b67483a102c0d227b53856","after":"a38250b9529bb70136f1b4f4b3792783d5dd28ec","ref":"refs/heads/master","pushedAt":"2024-06-25T09:00:11.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Fix rare sync issue in v70001/gtm8843_17 subtest\n\nRare cases, v70001/gtm8843_17 subtest failed,\nbecause of syncing issues.\n\nAs the log shows, two client requests (\"foo?\" and \"zeex\")\ngot merged. After fixing this issue, the server was also\nmissing the second request, due to timing issues.\n\n```\n< 4\tserver-recv-data: foo?\n---\n> 4\tserver-recv-data: foo?zeex\n\n< 6\tserver: non-blocking \"READWRITE\" query: READWRITE\n---\n> 6\tserver: non-blocking \"READWRITE\" query: WRITE\n\n< 3\tclient: non-blocking \"READWRITE\" query: READWRITE\n---\n> 3\tclient: non-blocking \"READWRITE\" query: WRITE\n```\n\nFix: added locking mechanism, which synchronizes server-client\nprogram flow, e.g. the first checkpoint is \"server is ready\",\nso the client does not sends a request until the server is up.","shortMessageHtmlLink":"Fix rare sync issue in v70001/gtm8843_17 subtest"}},{"before":"55b689ec3048cc46dc5620b1190b4f3b3c38ba27","after":"2363abd78cf1046123b67483a102c0d227b53856","ref":"refs/heads/master","pushedAt":"2024-06-25T02:15:13.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Fix framework copy_ydb_dist_dir.csh script to copy subdirectories\n\nThe comment in the script describes its purpose:\n\n> Since the test framework cannot write files to the $ydb_dist script, this function\n> creates a pseudo $ydb_dist directory under the caller specified directory ($1) that\n> contains all relevent files\n\nThe option \"-r\" has been added to the \"cp\" command, which now allows that sub-directories, such \"plugin/\", to be copied as well.","shortMessageHtmlLink":"Fix framework copy_ydb_dist_dir.csh script to copy subdirectories"}},{"before":"9dc098016c685eb302fb1bccc142229e8bfbbe2c","after":"55b689ec3048cc46dc5620b1190b4f3b3c38ba27","ref":"refs/heads/master","pushedAt":"2024-06-21T04:02:14.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[#632] New v70005/trigger_stats-gtmf135406 tests trigger statistics for ZSHOW and ^%YGBLSTAT\n\nThe GT.M [release note](http://tinco.pair.com/bhaskar/gtm/doc/articles/GTM_V7.0-005_Release_Notes.html#GTM-F135406) says:\n\n> GT.M reports the STG, KTG and ZTG statistics as part of the ZSHOW and YGBLSTAT outputs. These accumulators provide\nadditional details on invoked SET KILL and ZTRIGGER and triggers. Due to the nature of some statistics, their order\nof appearance in ZSHOW output may change. Previously, GT.M did not record these statistics in ZSHOW and YGBLSTAT. (GTM-F135406)\n\nNote that the release note says \"Due to the nature of some statistics, their order of appearance in ZSHOW output may change.\"\nTherefore this test sorts the order of the stats to ensure they are always the same in the test output.","shortMessageHtmlLink":"[#632] New v70005/trigger_stats-gtmf135406 tests trigger statistics f…"}},{"before":"2cb49ad0423d4d4d2d00ca9ce34cd4d0bc7c8b81","after":"9dc098016c685eb302fb1bccc142229e8bfbbe2c","ref":"refs/heads/master","pushedAt":"2024-06-19T02:49:31.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[!2019] Add instructions for setting up `perf` to YDBTest README.md\n\nThese should have been added to !2019.","shortMessageHtmlLink":"[!2019] Add instructions for setting up perf to YDBTest README.md"}},{"before":"4731006ee4a6d117e610286d433bdb2e2a59cc73","after":"2cb49ad0423d4d4d2d00ca9ce34cd4d0bc7c8b81","ref":"refs/heads/master","pushedAt":"2024-06-18T07:08:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[#731] Ensure test output directory name is unique when multiple gtmtest invocations occur in the same second.\n\nImplement this using mktemp to append a 3-character random code onto the end of the directory name.","shortMessageHtmlLink":"[#731] Ensure test output directory name is unique when multiple gtmt…"}},{"before":"0c8941e2b262e244dd130fc21d867ca96b83d8e2","after":"4731006ee4a6d117e610286d433bdb2e2a59cc73","ref":"refs/heads/master","pushedAt":"2024-06-14T07:02:45.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Fix v70003/sub_level_map_hide-gtmde270421 subtest by restoring a fix which was reverted unintentionally\n\nNars [have discovered](https://gitlab.com/YottaDB/DB/YDBTest/-/merge_requests/1941#note_1945640005) that a fix was reverted:\n\n> The `v70003/view_device-gtmf157495` subtest failed with a `/dev/tty: No such device or address` symptom. This was fixed by 34b1b776 so I was puzzled as to why it still fails. Turns out that fix was reverted unintentionally by https://gitlab.com/YottaDB/DB/YDBTest/-/merge_requests/1941","shortMessageHtmlLink":"Fix v70003/sub_level_map_hide-gtmde270421 subtest by restoring a fix …"}},{"before":"c095843e928171767c9a316922422bebe999ada5","after":"0c8941e2b262e244dd130fc21d867ca96b83d8e2","ref":"refs/heads/master","pushedAt":"2024-06-12T00:07:01.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Fix two old HPUX tests that use `which` in a way that stops the test.\n\nIssue: If `which` returns false then the test will quit early since tests\nare run with `tcsh -e`.\n\nThis test is only run on obsolete HPUX Linux, so this fix will never be run.\nHowever, it could be copied into new code since it is an example of using `which`\nto check for the existence of a command, which would cause bugs in future tests.\nSo best to fix it.","shortMessageHtmlLink":"Fix two old HPUX tests that use which in a way that stops the test."}},{"before":"a6229a7fc947847185011a59b19fc65d9d9c2018","after":"c095843e928171767c9a316922422bebe999ada5","ref":"refs/heads/master","pushedAt":"2024-06-10T08:57:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Fix rare sync issue in v70001/gtm8843_16 subtest\n\nRare cases, v70001/gtm8843_16 subtest failed,\nbecause the opposite side is sending data too quickly,\nthe actual side is missing the incoming data.\n\n```\n< 6 server: non-blocking \"READ\" query: READ\n---\n> 6 server: non-blocking \"READ\" query:\n```\n\nThe fix is trivial: instead of waiting (HANG) some, added\ntimeout to WRITE /WAIT instruction.\n\nHopefully, no test machine will be so slow that sender side\nsends data only after receiving side timeouts.\nIt was almost always working with hang=0.5, now timeout=2s.\n\n(If so, adding some locking might help, keeping the threads\nbetter synchronized, but it's also not a 100% guarantee to\nnot to miss data when the machine is extremly slow.)","shortMessageHtmlLink":"Fix rare sync issue in v70001/gtm8843_16 subtest"}},{"before":"39149cc4374f3e1490d70e91ad7ad786041af518","after":"a6229a7fc947847185011a59b19fc65d9d9c2018","ref":"refs/heads/master","pushedAt":"2024-06-07T13:51:16.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Fix regression in !1999 where specifying subtest using -t would no longer setup test properly","shortMessageHtmlLink":"Fix regression in !1999 where specifying subtest using -t would no lo…"}},{"before":"7688607d11122cd0bd02f08f9f02ccaded33ef93","after":"39149cc4374f3e1490d70e91ad7ad786041af518","ref":"refs/heads/master","pushedAt":"2024-06-04T14:12:12.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Further reduce disk space usage of rollback_A/D9D12002408 subtest\n\nBackground\n----------\n* A recent commit 719d52ff reduced the disk space usage of the `rollback_A/D9D12002408` subtest.\n\n* Inspite of that, this subtest filled up the disk on a few systems during in-house testing.\n\nIssue\n-----\n2 causes were identified.\n\n* `com/mupip_rollback.csh` (invoked by `rollback_A/u_inref/D9D12002408.csh` did a backward\n and forward rollback and stored the intermediate `.dat` and `.mjl` files at various stages\n in the test output directory taking up a lot of space.\n\n* `rollback_A/u_inref/D9D12002408.csh` does a `com/backup_dbjnl.csh` call to save the `.dat`\n and `.mjl` files before the rollback call.\n\nFix\n---\n* Just like is done in `rollback_B/u_inref/repeat_rollback_after_crash.csh`, the env var\n `gtm_test_forward_rollback` is set to 0 in `rollback_A/u_inref/D9D12002408.csh`. This ensured\n that `com/mupip_rollback.csh` returns after doing a backward rollback and does not spend time\n doing a forward rollback (and comparing the extracts and or taking backups etc.).\n\n* The `com/backup_dbjnl.csh` calls are commented out with a comment that they can be re-enabled\n as and when needed (i.e. in case of a test failure).","shortMessageHtmlLink":"Further reduce disk space usage of rollback_A/D9D12002408 subtest"}},{"before":"cf7d95fd6de8c04d26abe19f8b7dc268de4e6d1c","after":"7688607d11122cd0bd02f08f9f02ccaded33ef93","ref":"refs/heads/master","pushedAt":"2024-06-04T13:56:32.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[#395] Fix rundown_argless/gtm7706 subtest failure (shmid/semid ipc id match)\n\nBackground\n----------\n* During in-house testing, the `rundown_argless/gtm7706` subtest failed in one rare test run\n with the following diff.\n\n ```diff\n 17a18\n > s 884747 0x00000000 777 user 3\n ```\n\nIssue\n-----\n* See https://gitlab.com/YottaDB/DB/YDBTest/-/issues/395 for a description of the issue.\n\n* Below are the actual semids/shmids from the above failure.\n\n ```sh\n $ cat ipcs-to-be-removed.out\n 884747 1245190 884752 1245204 884754 1245207 884748 1245192 884753 1245206\n\n $ cat ipcs.out\n m 426004 0x00000000 666 user 4624384 0\n s 884745 0x00000000 777 user 3\n s 884746 0x00000000 777 user 3\n s 884747 0x00000000 777 user 3\n ```\n\n* From the flow of the test script, the first id `884747` above corresponds to the source side\n journal pool shmid.\n\n* But we find it matching in the `ipcs` output against a line starting with a `s`.\n\n* This means we matched the shmid `884747` against a semid `884747` line and incorrectly\n concluded the shmid did not get removed when actually it did get removed.\n\nFix\n---\n* The fix is easy and is to separate the semids/shmids in `ipcs-to-be-removed.out` into 2 separate\n lists and check them individually against the `ipcs.out` lines starting with `m` and `s`\n respectively.","shortMessageHtmlLink":"[#395] Fix rundown_argless/gtm7706 subtest failure (shmid/semid ipc i…"}},{"before":"719d52ff47801e55c254c25c5f6d0d2e64918b8b","after":"cf7d95fd6de8c04d26abe19f8b7dc268de4e6d1c","ref":"refs/heads/master","pushedAt":"2024-06-04T08:45:55.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[#720] Fix !1861 regression which prevented `gtmtest.csh` from resetting `$gtm_tst`\n\nIf the `$gtm_tst` was set, it prevented `gtmtest -s Txxx` from working.\nThe introduction of this in !1861 was an unnecessary think'o.\n\nNarayanan encountered this issue as documented here:\nhttps://gitlab.com/YottaDB/DB/YDBTest/-/merge_requests/1861#note_1933119662","shortMessageHtmlLink":"[#720] Fix !1861 regression which prevented gtmtest.csh from resett…"}},{"before":"03f22bf6e1fe9b35134183e96d13eabaab2002e1","after":"719d52ff47801e55c254c25c5f6d0d2e64918b8b","ref":"refs/heads/master","pushedAt":"2024-06-03T09:48:01.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Reduce runtime/diskspace of rollback_A/D9D12002408 and rollback_B/repeat_rollback_after_crash subtests\n\n* During an in-house test run, the recently re-enabled `rollback_A/D9D12002408` subtest\n filled up the source side test output directory as it used up `73.2Gb` of space.\n\n The space usage was split up was follows.\n\n ```\n 5.379Gb INSTANCE1_0_backward_extract.out\n 15.745Gb bak1\n 15.745Gb dir_INSTANCE1_0_mupip_rollback_1\n 25.666Gb dir_INSTANCE1_0_mupip_rollback_2\n ```\n\n All of these are artifacts created by the `com/mupip_rollback.csh` script invoked in\n `rollback_A/u_inref/D9D12002408.csh`.\n\n These are temporary files created and when the test eventually passes, these would be\n removed.\n\n Therefore, the disk space usage by this test is reduced by cutting down on the time that\n the updates happen in this test. This is done by reducing various `sleep` commands in the\n test by `6x`.\n\n That is, a sleep of `90` seconds is reduced to `15` seconds, `30` to `5`.\n\n Along with this, the epoch interval is also reduced from `100` to `18` to ensure this test\n generates EPOCH records even with the reduced sleep.\n\n With this, it is expected that the `.dat` and `.mjl` file sizes that are created will be\n 6 times smaller than usual and so the total space needed for this test would be around 10Gb\n (which is a more reasonable expectation of available space) instead of 70Gb.\n\n* A similar issue was seen with the `rollback_B/repeat_rollback_after_crash` subtest where the\n space used was close to 95Gb. It was not clear why only some rare test runs end up using a\n lot of space whereas other test runs of the same subtest do not. In any case, here too the\n test runtime was reduced by cutting down the sleeps by 6x (e.g. `120` seconds to `15` seconds).","shortMessageHtmlLink":"Reduce runtime/diskspace of rollback_A/D9D12002408 and rollback_B/rep…"}},{"before":"053ad8e3fd99d1a42a134f06457fe9b5fe8e92e5","after":"03f22bf6e1fe9b35134183e96d13eabaab2002e1","ref":"refs/heads/master","pushedAt":"2024-06-03T08:31:10.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[YDB#1083] New r202/ydb1083 subtest to test relinkctl file crash recovery takes 1 second (not 30 seconds)\n\n* This new subtest is enabled only for DBG/Debug builds and not for PRO/Release builds. This is because\n on some systems (Debian 12 x86_64 for example), it was seen that the breakpoint of `rel_latch()` was\n never reached in `gdb`. But if one tries to set a breakpoint of say `shmget()` and once that is\n reached, one sets the `rel_latch()` breakpoint, it gets reached. Not sure why it does not work. It\n most likely has to do with Release build symbol resolution in gdb. So I disabled this test for\n Release builds.\n\n* An additional cosmetic change was to rework the following files so each subtest shows up in a\n separate line (new convention, thanks to @ern0).\n - r200/instream.csh\n - r202/instream.csh","shortMessageHtmlLink":"[YDB#1083] New r202/ydb1083 subtest to test relinkctl file crash reco…"}},{"before":"e1dd5249b622d1e04d379bb6382cea15ba18d4dd","after":"053ad8e3fd99d1a42a134f06457fe9b5fe8e92e5","ref":"refs/heads/master","pushedAt":"2024-05-30T10:04:04.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[YottaDB/Util/YDBEncrypt#6] Add new stage 8f to socket/tsocerrors subtest (gtmtls_passwd_* env var not set)\n\n* This is a test of YottaDB/Util/YDBEncrypt@a65a93f","shortMessageHtmlLink":"[YottaDB/Util/YDBEncrypt#6] Add new stage 8f to socket/tsocerrors sub…"}},{"before":"09d831419ce16778a37a6ae21b52de0559ba3ab4","after":"e1dd5249b622d1e04d379bb6382cea15ba18d4dd","ref":"refs/heads/master","pushedAt":"2024-05-29T03:46:40.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[Docker] Add changes to fix the `sudo` tests when run inside of a container\n\n- Update documentation to explain the requirement of using the `SYS_TIME`\n capability when running the `sudo` tests\n- Add apt packages required by the `plugins` subtests to the container\n build file\n- Update the container build file to add the `gtmtest` user to the `sudo` group\n in order to allow the user to run the `sudo` tests\n- Update the container build file to add a `gtmtest-sudo` sudoers configuration\n file to allow the `gtmtest` user to run the sudo command without a password\n- Fix the `gtm7759` test output to allow any timestamp format for system logs","shortMessageHtmlLink":"[Docker] Add changes to fix the sudo tests when run inside of a con…"}},{"before":"a70b72b321a8a935ee0531502469aae3a236e7ec","after":"09d831419ce16778a37a6ae21b52de0559ba3ab4","ref":"refs/heads/master","pushedAt":"2024-05-28T17:43:56.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Handle non-zero exit status from set_gtmroutines.csh in com/do_random_settings.csh\n\n* See https://gitlab.com/YottaDB/DB/YDBTest/-/merge_requests/2008#note_1921643820 for more details.","shortMessageHtmlLink":"Handle non-zero exit status from set_gtmroutines.csh in com/do_random…"}},{"before":"b2873145675b9a4c910fe8f2b89cdb537bd88b46","after":"a70b72b321a8a935ee0531502469aae3a236e7ec","ref":"refs/heads/master","pushedAt":"2024-05-24T16:47:02.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Re-enable secshr/gtm7617 subtest (disabled temporarily in 37dfe3a7)\n\n* This subtest was disabled a long time ago in the hope that it will be re-enabled\n soon after. But it was long forgotten. Finally re-enabled in this commit.\n\n This commit reverts commit 37dfe3a7.\n\n The changes for this bullet are in\n - secshr/instream.csh\n - secshr/outref/outref.txt\n\n* In addition, while running this subtest after re-enabling, I noticed that it hung.\n This is because the test tampers with `gtm_dist` env var (to hold strings with control\n characters in them) and expects to see `GTMSECSHRINIT` syslog messages with the control\n characters replaced by `*`s (as described by the below GT.M V6.0-002 release note).\n\n http://tinco.pair.com/bhaskar/gtm/doc/articles/GTM_V6.0-002_Release_Notes.html#GTM-7617\n\n _The gtmsecshr wrapper program replaces control characters with asterisks (*) before sending messages\n to syslog. Previously control characters in $gtm_dist were sent unaltered to syslog, and if the\n syslog service did not suppress control characters, the log file's appearance could be altered by\n embedded control characters such as newlines, carriage returns and vertical tabs. [UNIX] (GTM-7617)_\n\n The hang was because no `GTMSECSHRINIT` messages showed up and that is because `ydb_dist`\n env var was still set and even if `gtm_dist` is changed by the test, the former would prevail.\n\n Therefore, the fix is to also make sure `ydb_dist` env var gets tampered with in the same\n fashion and/or unset. One of these two scenarios is randomly chosen now by this test\n as both should result in the same behavior.\n\n The changes for this bullet are in\n - secshr/inref/gtm7617.m","shortMessageHtmlLink":"Re-enable secshr/gtm7617 subtest (disabled temporarily in 37dfe3a)"}},{"before":"6e86a4bb46ad4ae71d58776550fcf43426592655","after":"b2873145675b9a4c910fe8f2b89cdb537bd88b46","ref":"refs/heads/master","pushedAt":"2024-05-24T12:59:53.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Fix gtmtest to behave when an old build of GTM is not found.\n\nPrior to this fix you could not tell what was wrong at all, and many tests would PASS and then produce this error during cleanup:\n```\nsetenv: Too many arguments.\nsetenv: Too many arguments.\n```\n\n**Background:** `gtmtest` looks for an old version 6 build of GTM and randomly selects one to create databases.\nThis is to ensure that old databases are still compatible with new GTM/YDB builds. However, if the system\ndoes not contain old builds then instead of the above error message it should just default to using the current build `$verno`.\n\nIt now does so by respecting `$gtm_test_nopriorgtmver`: it defaults `$gtm_test_v6_dbcreate_rand_ver` to `$verno` if\n`$gtm_test_nopriorgtmver` is set, and otherwise exits with the following error message if it still can't find an old GTM version:\n\n```\nRANDOMVER-E-CANNOTRUN : Could not determine previous version matching the given criteria - -gte V63003 -lt V70000. Exiting...\n```","shortMessageHtmlLink":"Fix gtmtest to behave when an old build of GTM is not found."}},{"before":"03a1d9e94344dc801cc7906a7efa57811f703d79","after":"6e86a4bb46ad4ae71d58776550fcf43426592655","ref":"refs/heads/master","pushedAt":"2024-05-24T12:37:15.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Fix !1861 regression where `gtmtest -st` could no longer select specific tests and workaround bug in tcsh v6.22.03\n- `-st` could not use uppercase test names previously specified by `-t`\n- `-st` would select multiple tests when only one was specified, if that one was a substring of another test name or file\n- `gtmtest -t basic -st basic` would fail on RHEL 9 which bundles tcsh v6.22.03 which has a bug where `$var:h` expands to\nempty string instead of `$var` if there is no `/` in the string. The tcsh bug and fix is listed\n[here](https://github.com/tcsh-org/tcsh/blob/master/Fixes#L71-L75).","shortMessageHtmlLink":"Fix !1861 regression where gtmtest -st could no longer select speci…"}},{"before":"376209d13a88c69eccf2854076e3140c953cb8ec","after":"03a1d9e94344dc801cc7906a7efa57811f703d79","ref":"refs/heads/master","pushedAt":"2024-05-24T10:41:00.000Z","pushType":"push","commitsCount":4,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[COSMETIC] Remove dead code related to too-old-libconfig in socket/u_inref/tsocerrors.csh\n\n* While addressing various issues in the `socket/tsocerrors` subtest, I noticed that it had\n lots of code to handle very old systems (RRHEL 6 and Ubuntu 12.04 which only had\n `libconfig8` whereas the test expected at least `libconfig9` or higher). The variable\n `newconfig` was set to 0 to identify such an old system.\n\n* All systems where YottaDB is currently supported on are guaranteed `libconfig9` or higher\n and so all code that was for `newconfig == 0` is removed as part of this commit thereby\n simplifying the script even if it is only a cosmetic change.","shortMessageHtmlLink":"[COSMETIC] Remove dead code related to too-old-libconfig in socket/u_…"}},{"before":"349edeb7230fdb4d91021dddc97a01d9f0ce4ddf","after":"376209d13a88c69eccf2854076e3140c953cb8ec","ref":"refs/heads/master","pushedAt":"2024-05-23T07:25:49.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[#598] New v70003/socket_opts-gtmf135169 Verify additional options for SOCKET devices\n\nRelease Note says:\n\n> The OPEN and USE commands for SOCKET devices support\n> assigning characteristics maintained with the POSIX\n> setsockopt() service using the OPTIONS deviceparameter\n> for the newly created socket or the current socket.\n>\n> OPTIONS=expr Applies to: SOC\n>\n> The argument (expr) is a string which contains a comma\n> separated list of setsockopt options. If the option takes\n> a value, it is given after an equal sign (=) following\n> the option. The supported options are:\n>\n> - KEEPALIVE - a positive value enables SO_KEEPALIVE.\n> A zero value disables SO_KEEPALIVE.\n> - KEEPIDLE - sets the TCP_KEEPIDLE socket value.\n> - KEEPCNT - sets the TCP_KEEPCNT socket value.\n> - KEEPINTVL - sets the TCP_KEEPINTVL socket value.\n> - SNDBUF - sets the size of the socket's network send\n> buffer (SO_SNDBUF) in bytes.\n>\n> Examples:\n>\n> USE dev:OPTIONS=\"KEEPALIVE=1,KEEPIDLE=50\"\n> This enables SO_KEEPALIVE and set TCP_KEEPIDLE to 50 seconds.\n>\n> USE dev:OPTIONS=\"KEEPALIVE=0\"\n> This disables SO_KEEPALIVE.\n>\n> _Note_\n> _For more information on the use of these options, please review_\n> _the man page for setsockopt . On Linux, \"man 7 socket\" and _\n> _\"man 7 tcp\" provide additional information._\n>\n> The $ZSOCKET() function supports an \"OPTIONS\" keyword which takes\n> an index argument and returns a string of the OPTIONS previously\n> specified for the selected socket. The string may not exactly\n> match the string originally specified but has the same meanings.\n>\n> The new keywords \"KEEPALIVE\", \"KEEPCNT\", \"KEEPIDLE\", \"KEEPINTVL\",\n> and \"SNDBUF\" return the individual items. If the system's current\n> value for the item doesn't match the value previously specified\n> with the OPTIONS device parameter, both values are returned\n> separated by a semicolon (\";\"): \"uservalue;systemvalue\".\n>\n> The \"ZIBFSIZE\" keyword may return the system value for SO_RCVBUF\n> in addition to the value from the ZIBFSIZE device parameter. Note\n> that the operating system may modify the values specified for\n> SO_RCVBUF and SO_SNDBUF so the returned values for those options\n> obtained with POSIX getsockopt() service may be different than\n> those specified using setsockopt(). (GTM-F135169)\n\nSee http://tinco.pair.com/bhaskar/gtm/doc/articles/GTM_V7.0-003_Release_Notes.html#GTM-F135169","shortMessageHtmlLink":"[#598] New v70003/socket_opts-gtmf135169 Verify additional options fo…"}},{"before":"3be732fbe8509702bcc6270b4cc8e7532cb88ddc","after":"349edeb7230fdb4d91021dddc97a01d9f0ce4ddf","ref":"refs/heads/master","pushedAt":"2024-05-21T12:54:50.000Z","pushType":"push","commitsCount":4,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Re-enable v60000/gtm7413 subtest (disabled temporarily in 37dfe3a7)\n\n* This subtest was disabled a long time ago in the hope that it will be re-enabled\n soon after. But it was long forgotten. Finally re-enabled in this commit.\n\n* This commit reverts commit 37dfe3a7.","shortMessageHtmlLink":"Re-enable v60000/gtm7413 subtest (disabled temporarily in 37dfe3a)"}},{"before":"f2573cbd3ece25f0b010fe58d497d27c1a1df361","after":"3be732fbe8509702bcc6270b4cc8e7532cb88ddc","ref":"refs/heads/master","pushedAt":"2024-05-21T12:49:18.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Fix v43001b_1/D9C05002121 subtest failure (TEST-E-TIMEOUT) if $ydb_test_4g_db_blks > 0\n\nBackground\n----------\n* The above subtest failed in in-house testing with the following symptom once in a while.\n\n ```diff\n 283a284,285\n > TEST-E-TIMEOUT, waited for 180 secs, but at least one of the regions were not updated\n > ^a at start:0 now:0 ^b at start:0 now:0\n ```\n\nIssue\n-----\n* After some analysis, it turned out that this failure happens only when `HUGEDB:1` random\n option is set.\n\n* After some more analysis, the connection was finally established.\n\n* The `TEST-E-TIMEOUT` was issued by `v43001b/inref/wait.m` when it ran on the secondary side\n as it found the number of `SETS` to always be `0`.\n\n* But I did verify that the update process actually processed a lot of updates from the source\n side in the 180 seconds (3 minutes) that it complained as no udpates having happened.\n\n* In the primary side of the failure directory, I tried the following command (just like what\n `wait.m` would do) and found the following output.\n\n ```sh\n $ mumps -run %XCMD 'write $view(\"gvstats\",\"AREG\"),!'\n SET:4324858,KIL:0,GET:0,DTA:0,ORD:0,ZPR:0,QRY:0,LKS:0,LKF:0,CTN:4325005,DRD:40,DWT:14758,NTW:4324858,...\n ```\n\n* I tried the same command on the secondary side and found the following output.\n\n ```sh\n mumps -run %XCMD 'write $view(\"gvstats\",\"AREG\"),!'\n SET:0,KIL:0,GET:0,DTA:0,ORD:0,ZPR:0,QRY:0,LKS:0,LKF:0,CTN:4325005,DRD:0,DWT:980,NTW:0,...\n ```\n\n* Notice that the `SET` statistic had a value of 0 on the secondary side.\n\n* The reason turned out to be because `dbcreate.gde` on the secondary side had the following\n lines added to it.\n\n ```\n template -region -nostats\n change -region DEFAULT -nostats\n ```\n\n* Whereas `dbcreate.gde` on the primary side did not have the above lines added to it.\n\n* Because of this, no gvstat collection was happening for key gvstats like `SET` only on the\n secondary side.\n\n* This change to add `-nostats` happened because of a recent commit 8e60e41c. But the reason\n this change happened only on the secondary side is because this subtest handles its own\n replication by doing a `unsetenv test_replic` (in `v43001b/u_inref/D9C05002121.csh`) and\n this caused the `init_gld.csh` call for the secondary side to happen with `test_replic` env\n var set whereas it was unset for the same call on the primary side.\n\n* This `test_replic` env var mismatch between the 2 calls to `init_gld.csh` is the cause of the\n failure.\n\n* In any case, the reason why `-nostats` was added for the secondary side is relevant for the\n primary side too. And that is if `ydb_test_4g_db_blks` env var is set to a random high value,\n we would end up with huge statsdb files and other related issues as described in the commit\n messages of 8e60e41c and f8c8d64c.\n\nFix\n---\n* The fix is simple and is to disable `ydb_test_4g_db_blks` for this subtest. That way we won't\n ever end up adding `-nostats` to the dbcreate calls. That way `wait.m` will always get the\n correct number of `SETS` and the subtest flow will continue the same way as it used to.","shortMessageHtmlLink":"Fix v43001b_1/D9C05002121 subtest failure (TEST-E-TIMEOUT) if $ydb_te…"}},{"before":"153dd5f6e9eb6e6e3ae2c30ebfdaff816aec915a","after":"f2573cbd3ece25f0b010fe58d497d27c1a1df361","ref":"refs/heads/master","pushedAt":"2024-05-21T02:48:36.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[YDB#1007] Test r136/ydb565 is now fixed to account for behaviour introduced in YDB!1517\nYDB now produces YDB-E-XCRETNULLREF instead of empty string, so take this into account in the test.","shortMessageHtmlLink":"[YDB#1007] Test r136/ydb565 is now fixed to account for behaviour int…"}},{"before":"0906da51fa873b153822e1fcf48dbef9a05ce7e5","after":"153dd5f6e9eb6e6e3ae2c30ebfdaff816aec915a","ref":"refs/heads/master","pushedAt":"2024-05-17T13:57:58.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[YDB#936] [V7.0-004] [GTM-DE308470] Preserve appropriate left-to-right evaluation within $SELECT() when using FULL_BOOLEAN compilation\n\n>Expratoms in a boolean expression are evaluated in left-to-right order in FULL_BOOLEAN compilation mode. Previously, side effects in a $SELECT() could affect earlier expratoms in a boolean expression. (GTM-DE308470)\n\nFor example, `GTM>view \"FULL_BOOLEAN\" xecute \"set x=0 w (x=1)!$select($increment(x)=0:0,1:0)\"` should return `0` but prior to v70004 returns `1`.","shortMessageHtmlLink":"[YDB#936] [V7.0-004] [GTM-DE308470] Preserve appropriate left-to-righ…"}},{"before":"86dab7d90c2dd821211f776e163ba2ba7c6bd428","after":"0906da51fa873b153822e1fcf48dbef9a05ce7e5","ref":"refs/heads/master","pushedAt":"2024-05-17T12:48:53.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"Re-enable msreplic_A/updateproc_nohang subtest (temporarily disabled in f9b4a2b9)\n\n* This subtest was disabled because it hung in some cases. When I ran it now, I noticed\n that it still hung once in a while. But in those cases, the random prior version that\n was picked was V6.3-005 or lower and the hang was because the update process became a\n `defunct`/`zombie` process causing the receiver server to never terminate and result in\n the test hang.\n\n Found the below relevant GT.M release note.\n\n http://tinco.pair.com/bhaskar/gtm/doc/articles/GTM_V6.3-005_Release_Notes.html#GTM-8945\n\n _The receiver process recovers after its update process was terminated with a signal while\n idle. Previously, on Linux systems with glibc 2.25 or newer, the receiver process could hang\n indefinitely, requiring manual cleanup of the process and shared memory. This issue was only observed\n in the GT.M development environment, and was never reported by a user. (GTM-8945)_\n\n Therefore, changed `msreplic_A/u_inref/updateproc_nohang.csh` to choose a prior version\n that was GREATER THAN `V63005`.\n\n* While testing, I noticed that this subtest should disable `gtm_test_use_V6_DBs` env var\n as we don't want the `dbcreate.csh` call done using the random prior version to create\n the database using a random V6 version that is a relatively future version. This caused\n problems when the random prior version chosen by the subtest then tries to access the\n database file.\n\n But instead of adding a `setenv gtm_test_use_V6_DBs 0` to `msreplic_A/u_inref/updateproc_nohang.csh`\n I looked into adding it in the framework script `com/disable_settings_msr_priorver.csh`\n as it is used by various tests and can benefit from this centralized change.\n\n Turns out a lot of the callers of `com/disable_settings_msr_priorver.csh` were indeed\n doing a `setenv gtm_test_use_V6_DBs 0` in the caller script and so all such usages could\n now be removed.\n\n* While looking into this, noticed `replic_compress/u_inref/mix_connections.csh` had a copyright\n block that was duplicated so removed that.\n\n* And finally was able to re-enable the subtest after all the above changes. This was done in the\n following files\n - msreplic_A/instream.csh\n - msreplic_A/outref/outref.txt","shortMessageHtmlLink":"Re-enable msreplic_A/updateproc_nohang subtest (temporarily disabled in "}},{"before":"b2ceec743ec70e92fa6c235cadcf123ee07494e0","after":"86dab7d90c2dd821211f776e163ba2ba7c6bd428","ref":"refs/heads/master","pushedAt":"2024-05-17T07:15:34.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[YDB#1007] Ensure that external calls that return invalid values produce errors.\n- XCRETNULLREF/ZCCONVERT messages are now errors, not mere log messages.\n- Also update test 63009/gtm9149 to reflect the new behaviour.","shortMessageHtmlLink":"[YDB#1007] Ensure that external calls that return invalid values prod…"}},{"before":"3eb72eaba713c764f96bd5d143edce6deff9fff4","after":"b2ceec743ec70e92fa6c235cadcf123ee07494e0","ref":"refs/heads/master","pushedAt":"2024-05-17T06:40:48.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nars1","name":"Narayanan Iyer","path":"/nars1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/28790583?s=80&v=4"},"commit":{"message":"[#720] Developer friendly test environment -- Stage 1\n\nMake YottaDB tests easier to debug by bypassing the 8 nested script layers run by `gtmtest` before running the subtest script itself:\n* `gtmtest.csh` =\\> `submit.csh` =\\> `submit.awk` =\\> `$submit_tests` (built by submit.awk) =\\> `submit_test.csh` =\\> \\`.csh` =\\> `submit_subtest.csh` =\\> \\`.csh`\n\nInstead of this heavy nesting, this MR sets up environment variables so that allows the developer to invoke these scripts directly, jumping in at any desired level. Further rationale and future features are documented in #720. This MR implements Stage 1 of #720 as follows.\n\n**`README.md` overhaul:**\n* Provide clean, tested, instructions to install the test system, including:\n * Document passwordless ssh setup.\n * Instructions to set up rsyslog logging.\n * Document the YDB developers' de-facto standard for subtest names and MR titles.\n* Document new features in README section Modular Testing.\n* Add troubleshooting section to README.\n\n**Basic features:**\n* Provide environment setup scripts adequate to support gtmtest and development debugging. Specifically:\n * a template `.cshrc` for user-specific settings.\n * `test_env.csh` for non-user tester settings (called by `.cshrc`).\n* Plus scripts intended to be run by the developer:\n * `settest` alias takes the same arguments as `gtmtest`, e.g. `-t and -st `\n * \\`.csh` scripts can be run directly, though it leaves its artifacts in the current directory\n * `runscript ` alias invokes any specified test script directly, except collect its artifacts into `$r`\n * `runtest` alias invokes submit_test.csh to run the test/subtest selected by `settest`, leaving artifacts in `$r`\n * `runsubtest` similar to `runtest` but invokes fewer tester layers and can only run a subtest, not a test suite.\n * `YDBTest/com/*.csh` scripts can also be run directly (or with `runscript` to collate artifacts)\n * `gtmtest` run a set of tests (see gtmtest -h), e.g. `gtmtest -t and -st `\n* Maintains a handy symlink to the latest test results ('artifacts') -- accessible to the command line via the `$r` variable.\n* Continue to allow the running of `gtmtest.csh`, as previously.\n* Tested on Ubuntu/Debian.\n* Option to prevent randomized tests by uncommenting a .cshrc` template line. Recommended for developers as randomization makes fault reproduction difficult.\n* Add tracing debugger `cshdebug.csh`: `-h` for help.\n* Not require encryption setup by default.\n* Auto-sync the test source to its run-location (previously had to remember before every test run). It is still possible to do so manually using `tsync` (or the deprecated `rst`) alias. This syncs your `$cwd`'s git repository if it is a YDBTest checkout, otherwise syncs `$work_dir/YDBTest`.\n* `gtmtest` improvements:\n * `gtmtest -t` now also accepts format `[test/]subtest_substring`\n * added tab completion for `gtmtest -t` and `settest -t`\n * `gtmtest -help` now outputs using `less` rather than `more`\n * `gtmtest -st` option now accepts subtest name prefix as an alternate to subtest name\n* Add `difftest` alias to easily see a graphical diff of the most recent test. Uses the diff tool selected in the user's git config. This alias also serves as an example to the user on easy access to test results using `$r`.\n\n**More detailed changes:**\n* Make java library finder use sort to correctly find the latest rather than earliest version on the system.\n* Add `.gitignore` to ignore `*.o` files which are often created when running test scripts directly\n* All test aliases now run from the directory specified by `$gtm_test_com_individual`, if available, falling back to `$work_dir/YDBTest`. This allows the user to develop test scripts in `$gtm_test_com_individual` and run actual tests from another git clone of YDBTest. Furthermore, setting `$force_gtm_test_com_individual` prevents gtmtest from doing its default redirect to gtmtest in $tst_src. See comment in `.cshrc` for details.\n* `defaults_csh` no longer overwrites `$gtm_tst` if it is already set\n* `gtmtest.csh` now uses existing variable `$gtm_tst` instead of `$gtm_test/$tst_src`. By default they are the same, but this change allows the user to specify a `tst_src` directory outside the `$gtm_test` directory, if desired.\n* `lasttest.csh` now runs after every `gtmtest` invocation and sets `$subtests` to hold complete list of subtests and the very last subtest in `$subtest`. This variable may be used used by the user to access the latest test results.\n* In anticipation of making build_and_install_yottadb.csh a more general-purpose script, make some tweaks:\n * Make `set_java_paths.csh` work properly if `JAVA_HOME` column or current `$HOST` are not found in `serverconf.txt`. This also lets us remove a workaround from `build_and_install_yottadb.csh`\n * Add `-p` to `mkdir` commands to avoid errors if directories already exist.\n * Add `-f` to `rm` commands to avoid errors if it doesn't exist.\n * Simplify some quoting so that my editor comprehends it and things display properly.\n\n**Also address various pre-existing bugs:**\n* Remove gtmtest errors in submit_test.csh when 'test_norandomsettings' is set.\n* Make 'gtmtest -help' output using 'less' rather than 'more'.\n* Make java library finder use sort to correctly find the latest rather than earliest version on the system.\n* Fix gtmtest when `-norandomsettings` is used:\n * Remove incorrect error 'do_random_settings: Command not found.' when random off. Anyway, it is not a useful message as it only occurs when the developer specifies it on the command line and already knows.\n * Ensure gtm_test_fake_enospc is defined even when random off. Makes java tests and probably others work.\n* Fix gtmtest to prevent it from producing errors in certain unusual circumstances, specifically:\n * Fail properly when $backslash_quote is not defined.\n * Remove superfluous double backslash to avoid a gawk warning (the warning is usually hidden, so hasn't been noticed).\n* Prevent gtm7952.csh from causing a hard-to-debug hang if pid file was not created by process under test. (If the pid file didn't exist then `$ourpid` was blank which reduced the grep arguments to 1 and made grep search stdin (i.e. wait forever) instead of searching the pid file.)\n* Allow developer to run submit_subtest.csh directly (not through gtmtest.csh) without error.\n* Fix switch_chset.csh to stop bombing out of entire shell whenever there are no *.o files (prevented me from running java test via gtmtest on localhost -- quite apart from settest).\n* Fix subtest_helperlogs.csh so that it doesn't bomb out of the test if env vars contain $ or \" (e.g. gtm_etrap).\n\n**Example usage:**\n```\nsettest -t xcall -st ydbtypes\nruntest\nmeldtest # alias that uses `$r` to diff test output against expected output\n```","shortMessageHtmlLink":"[#720] Developer friendly test environment -- Stage 1"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEb70Y8gA","startCursor":null,"endCursor":null}},"title":"Activity · YottaDB/YDBTest"}