Not able to succeed run data-driven xlsx file in parallel (pabot)

[hrnaltnts]

Hello,
I have tried to execute my data-driven(xlsx) test case (25 processes) using this guidence. When I try to run pabot --testlevelsplit --pabotlib (...) open the same test case 12 times, I hoped it would divide the 25 processes in the xlsx file into 12.
I have made mistake somewhere or no way to run data-driven(xlsx) cases in parallel.
Thanks in advance.

[Snooz82]

It should execute them in parallel.
which versions of Pabot and DataDriver are you using?

can you share an example to reproduce?

[hrnaltnts]

Pabot version 2.1.0, DataDriver version 1.5.0

More explanation about case;

• Library DataDriver optimize_pabot=Binary ../TestData/MostSearched.xlsx sheet_name=Lapas1

• Library pabot.SharedLibrary

• pabot --testlevelsplit --pabotlib -t 'Verify Most Searched Term' TestCases/MostSearchedTerms.robot

• MostSearched.xlsx this file has 25 keywords which most searched internally on my project

Currently, I have been using data-driven for searching each 25 keywords step by step, and it works fine. When I try to execute in parallel (pabot) with data-driven, opens 12 different browsers but all of them execute only the first keyword at the same time.

[hrnaltnts]

Hello again,
If you need I can porvide more information about my case. I have been trying a lot conbination but not able to solve this problem. Always, all bwowsers execute first variable of xlsx file.

[pianofab]

I am seeing the same (or similar) phenomenon. The test cases from my csv are not executed in parallel (even though pabot is being used) so the overall execution time does not improve at all.

Same versions:

robotframework            4.1.3                    pypi_0    pypi
robotframework-datadriver 1.5.0              pyhd8ed1ab_0    conda-forge
robotframework-pabot      2.1.0                    pypi_0    pypi
robotframework-stacktrace 0.4.1                    pypi_0    pypi

My total execution time (sequential) is about 30 seconds. I already know that pabot works correctly in my environment because it has successfully parallelized tasks otherwise. It is only in conjunction with datadriver that I see this issue.

I have 12 items in my csv file. I thought that this failure to parallelize was due to not enough processes starting in parallel (my machine does not have 12 cores) so I even tried as follows:

pabot --testlevelsplit --pabotlib --processes 12 datadriven.robot

No dice. It still takes about 30 seconds.

I also tried setting optimize_pabot=Atomic and Equal but besides a change in logging behavior the execution time does not improve.

[hrnaltnts]

Hello @Snooz82,

Coud you please check this issue if you need more details i can provide?
Thanks in advance.

[Snooz82]

Hi @hrnaltnts
Hi @pianofab

Can you give me a reproducible example?

like xls or csv file and robot file?

[Snooz82]

@pianofab

You said that you have 12 rows. And you have 12 processes. Therefore each process gets one test case.

then the optimize_pabot would not have any affect.

are you sure that the test are not running in parallel?
Can you post you console log, with --loglevel DEBUG as cli option?

[pianofab]

@pianofab

You said that you have 12 rows. And you have 12 processes. Therefore each process gets one test case.

then the optimize_pabot would not have any affect.

are you sure that the test are not running in parallel? Can you post you console log, with --loglevel DEBUG as cli option?

Quick explanation as to what I am doing (I cannot share the source code for confidentiality reasons).

UDP-based client <=> UDP-based server 1
:
UDP-based client <=> UDP-based server 12

I have a hand-coded test where I start these processes manually with Robot commands. With a test that hand-codes the parallelization, I am getting about 11 seconds execution time.

If I use the data-driven approach (so the 12 instances are listed in the CSV file) the execution time is always about 30 seconds; it does not matter if I run under robot or pabot with any setting.

[pianofab]

noparallel.log

This is the output of
pabot --testlevelsplit --pabotlib --verbose datadriven.robot

After filtering out a bunch of known warnings from my tests that you do not care about. Please let me know if this is what you were looking for @Snooz82 .

[pianofab]

The Robot source for datadriven.robot looks like this (I have deleted some nonessential parts).
Note that the test itself performs multiple steps. The first step starts what I called the UDP Server; the second step starts what I called the UDP Client and runs the actual test.
If the bug is what the other reporter surmised ("but all of them execute only the first keyword at the same time") this would explain why I am seeing the same issue.

*** Settings ***
Documentation	Runs TEC tests with Robot Framework.
...		Data-driven via CSV file. [graph.html|Suite Details]
Resource	keywords.robot
Suite Setup	Log    Suite Setup
Suite Teardown	Terminate All Processes
Task Setup	Log	Per Task Setup
Task Teardown	Log	Per Task Teardown
Library	Process
Library    DataDriver    file=${EXECDIR}${/}DataFiles${/}HWTypesOne_File.csv    optimize_pabot=Atomic
Task Timeout	2 minutes
Variables	globalvariables.py
Test Template    TEC Tests One-To-One DataDriven

*** Variables ***
${number}	10
${UDPportone}	46000
${UDPportmany}	47000

*** Tasks ***
TEC Tests DataDriven with ${HW_endpoint} and ${UDP_port}    0    49000

*** Keywords ***
TEC Tests One-To-One DataDriven
  [Arguments]    ${HW_endpoint}    ${UDP_port}
  Start TEC HW Emulators    ${HW_endpoint}    ${UDP_port}
  Test TEC Clients One-To-One	${UDP_port}    1

[pianofab]

HWTypesOne_File.csv

[pianofab]

I should also add that you could perhaps emulate my programs via netcat, if you wanted to reproduce a similar scenario: https://help.ubidots.com/en/articles/937233-sending-tcp-udp-packets-using-netcat

[hrnaltnts]

Hi @hrnaltnts Hi @pianofab

Can you give me a reproducible example?

like xls or csv file and robot file?

Hello @Snooz82

I attached xlsx file and robot file. (please look next comment)

I would like to summarize the situation again, I am able to execute this file as data-driven but when I try to execute it in parallel with pabot, opened one browser and run suit setup then closed itself. After that, opens 12 different browsers but each of them runs only the first item of xlsx file, otherwise, I have expected to run other different next items in a row of the excel file on each different browser like when executing as data-driven.

pabot --testlevelsplit --pabotlib TestCase.robot
If you need more detail please let me know. If you connect me from linkedIn I can provide you screenshots or video records about execution.

Thanks again for your interest!

[hrnaltnts]

@Snooz82

This is a simple sample for execution.

robotfile.zip
MostSearched.xlsx

pabot --testlevelsplit --pabotlib MostSearchedTerms.robot

Each different browser should search different keywords in a row.

[Snooz82]

i try to look at it soonish
Sorry for that delay

[hrnaltnts]

@Snooz82 No worries. Thank you so much :))

[Snooz82]

i reduced it so that SeleniumLibrary is not longer involved.

*** Settings ***
Library    DataDriver    MostSearched.xlsx   sheet_name=Lapas1
Test Template     Verify Most Searched Term from Excel


*** Test Cases ***
Verify Most Searched Term ${items}    ${items}
    [Tags]  SmokeDaily

*** Keywords ***
Verify Most Searched Term from Excel
        [Arguments]   ${items}
        Log To Console    ${items}
        Sleep    2s

and with this call:
❯ pabot --testlevelsplit --pabotlib atest/TestCases/DEBUG/MostSearchedTerms.robot

It worked well:

❯ pabot --testlevelsplit --pabotlib atest/TestCases/DEBUG/MostSearchedTerms.robot
Robot Framework remote server at 127.0.0.1:8270 started.
2022-02-20 17:03:54.158097 [PID:41009] [0] [ID:0] EXECUTING MostSearchedTerms.Verify Most Searched Term ${items}
2022-02-20 17:03:54.789723 [PID:41009] [0] [ID:0] IGNORED MostSearchedTerms.Verify Most Searched Term ${items}
2022-02-20 17:03:54.792975 [PID:41010] [2] [ID:2] EXECUTING MostSearchedTerms
2022-02-20 17:03:54.793081 [PID:41011] [0] [ID:1] EXECUTING MostSearchedTerms
2022-02-20 17:03:54.794744 [PID:41012] [4] [ID:7] EXECUTING MostSearchedTerms
2022-02-20 17:03:54.794984 [PID:41013] [1] [ID:3] EXECUTING MostSearchedTerms
2022-02-20 17:03:54.795382 [PID:41014] [3] [ID:6] EXECUTING MostSearchedTerms
2022-02-20 17:03:54.795402 [PID:41015] [5] [ID:4] EXECUTING MostSearchedTerms
2022-02-20 17:03:54.796403 [PID:41016] [6] [ID:5] EXECUTING MostSearchedTerms
2022-02-20 17:03:54.799286 [PID:41017] [7] [ID:9] EXECUTING MostSearchedTerms
2022-02-20 17:03:54.801228 [PID:41018] [8] [ID:10] EXECUTING MostSearchedTerms
2022-02-20 17:03:54.802888 [PID:41019] [9] [ID:8] EXECUTING MostSearchedTerms
2022-02-20 17:04:00.220102 [PID:41017] [7] [ID:9] PASSED MostSearchedTerms in 5.2 seconds
2022-02-20 17:04:00.260934 [PID:41012] [4] [ID:7] PASSED MostSearchedTerms in 5.3 seconds
2022-02-20 17:04:00.280685 [PID:41014] [3] [ID:6] PASSED MostSearchedTerms in 5.3 seconds
2022-02-20 17:04:00.294396 [PID:41018] [8] [ID:10] PASSED MostSearchedTerms in 5.3 seconds
2022-02-20 17:04:00.314187 [PID:41019] [9] [ID:8] PASSED MostSearchedTerms in 5.3 seconds
2022-02-20 17:04:02.233756 [PID:41010] [2] [ID:2] PASSED MostSearchedTerms in 7.2 seconds
2022-02-20 17:04:02.234373 [PID:41011] [0] [ID:1] PASSED MostSearchedTerms in 7.2 seconds
2022-02-20 17:04:02.239987 [PID:41013] [1] [ID:3] PASSED MostSearchedTerms in 7.2 seconds
2022-02-20 17:04:02.241076 [PID:41015] [5] [ID:4] PASSED MostSearchedTerms in 7.2 seconds
2022-02-20 17:04:02.338540 [PID:41016] [6] [ID:5] PASSED MostSearchedTerms in 7.3 seconds
25 tests, 25 passed, 0 failed, 0 skipped.
===================================================
Output:  /Users/snooz/Source/Snooz82/robotframework-datadriver/output.xml
Log:     /Users/snooz/Source/Snooz82/robotframework-datadriver/log.html
Report:  /Users/snooz/Source/Snooz82/robotframework-datadriver/report.html
Stopping PabotLib process
Robot Framework remote server at 127.0.0.1:8270 stopped.
PabotLib process stopped
Total testing: 1 minute 3.10 seconds
Elapsed time:  8.52 seconds

[Snooz82]

Where do you get the impression that it is not parallel?
@pianofab You log also seems to work perfectly in parallel...

[hrnaltnts]

@Snooz82 Could you check each of them search different keywords? Because I already got the same result but all execution runs the same keyword.

[Snooz82]

Then it is because all you test cases are called the same.

[Snooz82]

When all your tests have the same names, pabot & DataDriver can not different them.

[Snooz82]

Look at my example. There the variable is part of the name.
i could also implement a feature to automatically index TestCases, if there are duplicates.

[hrnaltnts]

@Snooz82 I noticed now. I am able to execute successfully
Have a great week, Thank you so much again :))

[dovJos]

Hello,
I have tried to execute my data-driven(xlsx) test case but I faced this issue

My Excel File inside the Data Folder
Tests_Excel_Request_Generator_V0_all.xlsx

And my script

How I run it
pabot --testlevelsplit --pabotlib Tests/Request_Tools/Production/New_Publication/poc.robot

Did I made mistake somewhere?

robotframework-datadriver==1.6.1
robotframework-pabot==2.5.3

Thanks in advance.

[Snooz82]

@dovJos

can you please run it with --loglevel TRACE and console output?

[dovJos]

@Snooz82 I do have this output

[Snooz82]

Sorry to say so, but i am not able to reproduce that error.
With Pabot it is needed to run in with --verbose to see the console outputs from DataDriver.

I will close it but please open a new issue, if there is still that problem.