Need to enter data in Thai language in wc3270 emulator

[FWDQK]

Hi Team,
We need to enter values in Thai in wc3270 emulator.
Tried using charset and utf8, but same did not worked.
Please help to suggest how we can achieve this.

[robinmatz]

Hey @FWDQK ,

I was able to locate the root cause of your problem.

In this block of code in x3270.py the string that is sent to wc3270 is encoded in unicode_escape

def _write(
        self,
        txt: Any,
        ypos: Optional[int] = None,
        xpos: Optional[int] = None,
        enter: int = 0,
    ) -> None:
        txt = txt.encode("unicode_escape")
        if ypos is not None and xpos is not None:
            self._check_limits(ypos, xpos)
            self.mf.send_string(txt, ypos, xpos)
        else:
            self.mf.send_string(txt)
        time.sleep(self.wait_write)
        for i in range(enter):
            self.mf.send_enter()
            time.sleep(self.wait)

If I replace txt = txt.encode("unicode_escape") with txt = txt.encode("utf8") the string is send in thai to wc3270. However, changing this causes issues with other characters like ëçá, among others.

I am not quite sure what to do about this one.

[FWDQK]

Hey @FWDQK ,

I was able to locate the root cause of your problem.

In this block of code in x3270.py the string that is sent to wc3270 is encoded in unicode_escape

def _write(
        self,
        txt: Any,
        ypos: Optional[int] = None,
        xpos: Optional[int] = None,
        enter: int = 0,
    ) -> None:
        txt = txt.encode("unicode_escape")
        if ypos is not None and xpos is not None:
            self._check_limits(ypos, xpos)
            self.mf.send_string(txt, ypos, xpos)
        else:
            self.mf.send_string(txt)
        time.sleep(self.wait_write)
        for i in range(enter):
            self.mf.send_enter()
            time.sleep(self.wait)

If I replace txt = txt.encode("unicode_escape") with txt = txt.encode("utf8") the string is send in thai to wc3270. However, changing this causes issues with other characters like ëçá, among others.

I am not quite sure what to do about this one.

Ok.
How we can pass code to set utf8 via script?

[FWDQK]

Hey @FWDQK ,
I was able to locate the root cause of your problem.
In this block of code in x3270.py the string that is sent to wc3270 is encoded in unicode_escape

def _write(
        self,
        txt: Any,
        ypos: Optional[int] = None,
        xpos: Optional[int] = None,
        enter: int = 0,
    ) -> None:
        txt = txt.encode("unicode_escape")
        if ypos is not None and xpos is not None:
            self._check_limits(ypos, xpos)
            self.mf.send_string(txt, ypos, xpos)
        else:
            self.mf.send_string(txt)
        time.sleep(self.wait_write)
        for i in range(enter):
            self.mf.send_enter()
            time.sleep(self.wait)

If I replace txt = txt.encode("unicode_escape") with txt = txt.encode("utf8") the string is send in thai to wc3270. However, changing this causes issues with other characters like ëçá, among others.
I am not quite sure what to do about this one.

Ok. How we can pass code to set utf8 via script?

Hi @robinmatz ,
Please help to confirm on how we can handle to pass utf8 as true via script to wc3270 using robotframework.
Manually, if we launch wc3270, we are able to input data in Thai but via script, it fails to enter.

[robinmatz]

as for now, as an alternative to Write, Write Bare, etc. you can use Execute Command String(<thai characters>).

Here is a small example, I tried with the pub400.com host:

*** Settings ***
Library    Mainframe3270/


*** Test Cases ***
Bug
    ${extra_args}=    Create List   -utf8    -codepage    thai
    Open Connection    pub400.com     extra_args=${extra_args}
    Wait Field Detected
    Execute Command    String(พยัญชนะ)
    Take Screenshot
    Sleep    5 s
    [Teardown]    Close Connection

[FWDQK]

as for now, as an alternative to Write, Write Bare, etc. you can use Execute Command String(<thai characters>).

Here is a small example, I tried with the pub400.com host:

*** Settings ***
Library    Mainframe3270/


*** Test Cases ***
Bug
    ${extra_args}=    Create List   -utf8    -codepage    thai
    Open Connection    pub400.com     extra_args=${extra_args}
    Wait Field Detected
    Execute Command    String(พยัญชนะ)
    Take Screenshot
    Sleep    5 s
    [Teardown]    Close Connection

Hi @robinmatz ,
I tried to apply the same logic, but it is still not writing in Thai.

[FWDQK]

as for now, as an alternative to Write, Write Bare, etc. you can use Execute Command String(<thai characters>).
Here is a small example, I tried with the pub400.com host:

*** Settings ***
Library    Mainframe3270/


*** Test Cases ***
Bug
    ${extra_args}=    Create List   -utf8    -codepage    thai
    Open Connection    pub400.com     extra_args=${extra_args}
    Wait Field Detected
    Execute Command    String(พยัญชนะ)
    Take Screenshot
    Sleep    5 s
    [Teardown]    Close Connection

Hi @robinmatz , I tried to apply the same logic, but it is still not writing in Thai.
Tried by keeping only ${extra_args}= Create List -utf8 -codepage thai, still same behavior.
System language and region also set to Thai.

[robinmatz]

Hey there,

terriblly sorry, but I really don't know the solution to this issue. @samuelpcabral , can you think of anything?

In any case, keep in mind Robot -Framework-Mainframe3270-Library is basically proxying x3270 commands to a shell process. Therefore, I highly suggest you try to find the appropriate x3270 Actions on in their official wiki: https://x3270.miraheze.org/wiki/Main_Page

This advice also goes for the other issue you raised.

[samuelpcabral]

Hello @FWDQK as you can see in the main file of the library "py3270.py" there is a comment in the beggining:

    Python 3+ note: unicode strings should be used when communicating with the Emulator methods.
    utf-8 is used internally when reading from or writing to the 3270 emulator (this includes
    reading lines, constructing data to write, reading statuses).

So.. all of the commands are using encode or decode with utf-8, and are probably overwriting the connection settings.

I already thought that this would cause a problem on a mainframe that doesn't work in utf-8, but it needs to be tested...

unfortunately at the moment I don't have time for such a big change.. if you know how to program in python and get a solution, we will be more than happy to receive a merge request.

[FWDQK]

Hello @FWDQK as you can see in the main file of the library "py3270.py" there is a comment in the beggining:

    Python 3+ note: unicode strings should be used when communicating with the Emulator methods.
    utf-8 is used internally when reading from or writing to the 3270 emulator (this includes
    reading lines, constructing data to write, reading statuses).

So.. all of the commands are using encode or decode with utf-8, and are probably overwriting the connection settings.

I already thought that this would cause a problem on a mainframe that doesn't work in utf-8, but it needs to be tested...

unfortunately at the moment I don't have time for such a big change.. if you know how to program in python and get a solution, we will be more than happy to receive a merge request.

Hi @samuelpcabral ,
Please help to explain what the requirement is and where the changes has to be done.
So accordingly, I will check internally for some python developer to help.

[samuelpcabral]

@FWDQK You need to search in the two main files: x3270.py and py3270.py all occurrences of hardcoded "utf-8".

there are two in the file x3270.py and sixteen in the file py3270.py

To work correctly utf-8 must remain the default encoder, and to change it it would have to be an argument in the keyword "Open Connection"

And unfortunately I don't have access to a mainframe in a different encoding to test this.

[FWDQK]

Ok.
Got this.
We can also test it in our environment, where we need to deploy it.
We can do it if you can help with one function reference, as this change may have huge impact.
Maybe we can connect on call (would give better clarity) or via this chat.

[samuelpcabral]

We have a lot of tests in a public mainframe.. you just need to run this one: atest/mainframe.robot or when do a merge request the github actions will run all the tests.
https://github.com/Altran-PT-GDC/Robot-Framework-Mainframe-3270-Library/actions

[FWDQK]

We have a lot of tests in a public mainframe.. you just need to run this one: atest/mainframe.robot or when do a merge request the github actions will run all the tests. https://github.com/Altran-PT-GDC/Robot-Framework-Mainframe-3270-Library/actions

Hi,
Can you help with some guidance for the changes required.
So we can try to help develop the same.

[samuelpcabral]

Hello @FWDQK any update about this one?

[doppio-wacharin]

as for now, as an alternative to Write, Write Bare, etc. you can use Execute Command String(<thai characters>).
Here is a small example, I tried with the pub400.com host:

*** Settings ***
Library    Mainframe3270/


*** Test Cases ***
Bug
    ${extra_args}=    Create List   -utf8    -codepage    thai
    Open Connection    pub400.com     extra_args=${extra_args}
    Wait Field Detected
    Execute Command    String(พยัญชนะ)
    Take Screenshot
    Sleep    5 s
    [Teardown]    Close Connection

Hi @robinmatz , I tried to apply the same logic, but it is still not writing in Thai.

Let me share my experience from this point. For someone facing the same prob.

Initially, I experienced the same thing as @FWDQK did. But now this is working for me. Let me share my finding.

I've tried executing the command x3270 -script -codepage thai directly in the terminal and use the command string("พยัญชนะ") , and I see the same thing as it shown in the robotframework-mainframe3270. So, it is not the library but it should be something relating to the machine env.

I installed the "font" to the machine running x3270. then Execute Command String(พยัญชนะ) command is working find.
This would be a great workaround for this.

May be a simple solution can be creating a new keyword name "Write Unicode" which is calling Execute command String(${txt})