Source members are not listed in VS Code

[GuriCat]

Type: Bug

Describe the bug

When source member text contains DBCS string, members are not listed.

To Reproduce

1. Create source file and add members with DBCS text.

> CRTSRCPF FILE(QGPL/TEST) IGCDTA(*YES) TEXT('Test source PF for VSCode')
  ライブラリーQGPLにファイルTESTが作成された。                     
> ADDPFM FILE(QGPL/TEST) MBR(MBR1) TEXT('送信定義一覧／追加・修正')    
  メンバーMBR1がQGPLのファイルTESTに追加された。                 
> ADDPFM FILE(QGPL/TEST) MBR(MBR2) TEXT('送信定義一覧／追加・修正')    
  メンバーMBR2がQGPLのファイルTESTに追加された。                 

2. Goto VS Code -> IBM i extension -> Object browser. Add new filter to confirm that the new members are not listed.

4. Change member text to SBCS string, then reflesh the Object browser to see the members are listed.

> CHGPFM FILE(QGPL/TEST) MBR(MBR1) TEXT('Member 1')    
  QGPLのファイルTESTのメンバーMBR1が変更された。 
> CHGPFM FILE(QGPL/TEST) MBR(MBR2) TEXT('Member 2')    
  QGPLのファイルTESTのメンバーMBR2が変更された。 

Thank you very much for your support.

Extension version: 2.9.0
VS Code version: Code 1.87.2 (863d2581ecda6849923a2118d93a088b0745d9d6, 2024-03-08T15:20:17.278Z)
OS version: Windows_NT x64 10.0.22621
Modes:

[worksofliam]

Please share the contents of your Code for IBM i output channel after the list doesn't appear.

[GuriCat]

Thank you very much for your quick reply.

Here's Code for IBM i output channel.
Source file is SNDLIB/QRPGLESRC (same members as before).

/home/PGMR40: /QOpenSys/usr/bin/qsh
system  "CHKOBJ OBJ(SNDLIB/QRPGLESRC) OBJTYPE(*FILE) AUT(*UPD)"
{
    "code": 0,
    "signal": null,
    "stdout": "",
    "stderr": ""
}

/home/PGMR40: LC_ALL=EN_US.UTF-8 system "call QSYS/QZDFMDB2 PARM('-d' '-i' '-t')"
With MEMBERS As (
        SELECT
          rtrim(cast(a.system_table_schema as char(10) for bit data)) as LIBRARY,
          b.avgrowsize as RECORD_LENGTH,
          a.iasp_number as ASP,
          rtrim(cast(a.system_table_name as char(10) for bit data)) AS SOURCE_FILE,
          rtrim(cast(b.system_table_member as char(10) for bit data)) as NAME,
          coalesce(rtrim(cast(b.source_type as varchar(10) for bit data)), '') as TYPE,
          coalesce(rtrim(varchar(b.partition_text)), '') as TEXT,
          b.NUMBER_ROWS as LINES,
          extract(epoch from (b.CREATE_TIMESTAMP))*1000 as CREATED,
          extract(epoch from (b.LAST_SOURCE_UPDATE_TIMESTAMP))*1000 as CHANGED
        FROM qsys2.systables AS a
          JOIN qsys2.syspartitionstat AS b
            ON b.table_schema = a.table_schema AND
              b.table_name = a.table_name
      )
      Select * From MEMBERS
      Where LIBRARY = 'SNDLIB'
        And SOURCE_FILE = 'QRPGLESRC'
        And NAME Like '%'
        And TYPE Like '%'
      Order By NAME ASC
{
    "code": 0,
    "signal": null,
    "stdout": "DB2>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n  ?>\n\nLIBRARY    RECORD_LENGTH        ASP    SOURCE_FILE  NAME       TYPE       TEXT                                                                                                                           LINES                CREATED              CHANGED             \n---------- -------------------- ------ ------------ ---------- ---------- ------------------------------------------------------------------------------------------------------------------------------ -------------------- -------------------- --------------------\nSNDLIB                      112      0 QRPGLESRC    SNDDEFD    DSPF       送信定義一覧／追加・修正                                                                                                                      124        1712670631000        1712683676000\nSNDLIB                      112      0 QRPGLESRC    SNDDEFR    RPGLE      送信定義一覧／追加・修正                                                                                                                      386        1712683661000        1712683692000\n\n  2 RECORD(S) SELECTED.",
    "stderr": ""
}

[worksofliam]

That is very odd, but I suspect I know why.

Here is the formatted output, which shows rows are coming back:

LIBRARY    RECORD_LENGTH        ASP    SOURCE_FILE  NAME       TYPE       TEXT                                                                                                                           LINES                CREATED              CHANGED             
---------- -------------------- ------ ------------ ---------- ---------- ------------------------------------------------------------------------------------------------------------------------------ -------------------- -------------------- --------------------
SNDLIB                      112      0 QRPGLESRC    SNDDEFD    DSPF       送信定義一覧／追加・修正                                                                                                                      124        1712670631000        1712683676000
SNDLIB                      112      0 QRPGLESRC    SNDDEFR    RPGLE      送信定義一覧／追加・修正                                                                                                                      386        1712683661000        1712683692000

  2 RECORD(S) SELECTED.",
    

Are you also able to see if there is any errors in the Developer Tools console after the list is not shown in the UI?

[GuriCat]

Hello,

Are you also able to see if there is any errors in the Developer Tools console after the list is not shown in the UI?

I see "Invalid time value" on the console.

Below is the formatted output and you can see that columns after "TEXT" ("LINES", "CREATED" and "CHANGED") are not aligned because Shift-codes are eliminated..
Could this be causing the problem?

LIBRARY    RECORD_LENGTH        ASP    SOURCE_FILE  NAME       TYPE       TEXT                                                                                                                           LINES                CREATED              CHANGED
--------- -------------------- ------ ------------ ---------- ---------- ------------------------------------------------------------------------------------------------------------------------------ -------------------- -------------------- --------------------
SNDLIB                      112      0 QRPGLESRC    SNDDEFD    DSPF       送信定義一覧／追加・修正                                                                                                                      470        1712670631000        1713451402000
SNDLIB                      112      0 QRPGLESRC    SNDDEFR    SQLRPGLE   送信定義一覧／追加・修正                                                                                                                      399        1712683661000        1713288588000
SNDLIB                      112      0 QRPGLESRC    SNDLOGD    DSPF       送信ログ                                                                                                                                      470        1713448238000        1713450929000
SNDLIB                      112      0 QRPGLESRC    TESTEDTW   RPGLE      日付と時刻を先行０付きで表示-> 8桁では無理？                                                                                                  9        1713451802000        1713453741000
SNDLIB                      112      0 QRPGLESRC    XXCONDTEST RPGLE      論理演算には関係式または標識が必要                                                                                                              9        1713279932000        1713280201000
SNDLIB                      112      0 QRPGLESRC    XXXSELTEST DSPF       送信定義一覧／追加・修正                                                                                                                       88        1712936853000        1712942716000

[worksofliam]

Hi @GuriCat that is exactly right! But the good news is that you were kind enough to provide me examples, in which I have fixed and written test cases for.

This shows what the issue is:

header
{name: 'TEXT', from: 73, length: 126}

line.substring(fromPos, fromPos + header.length).length
126

line.substring(fromPos, fromPos + header.length)
'日付と時刻を先行０付きで表示-> 8桁では無理？                                                                                                  9   '

But, I have added some logic to handle of the offset (guessing because of DBCS characters).

We are working on a number of fixes for this. You are welcome to build the branch yourselves to test our work:

• Component API #1985
• Improved support for DBCS output #1993

[GuriCat]

I switched to v2.9.2 (pre-release) and confirm that the problem was solved!
I appreciate your quick support.

During the test, I noticed that source file (not member) text is "0" when TEXT is blank.

Will this be fixed in the official release?
Or should I report it separately as a new bug?

[worksofliam]

@GuriCat i saw this as well so I will look into it.

Thanks!

edit: made a PR for the fix #2007