Get help with testing, discuss unit testing strategies etc.


Post by jayaprakasha »

i understand we should use --max-workers to run tests in parallel in browser stack. Normally i goes like this..

webdriver https://localhost --filter="Workflow>" --rerun-failed --browserstack=user,key --cap browserName=chrome --cap platform=windows

But how to mention the browser capabilities if i need to execute 3 instances in parallel?

Post by nickolay »

You don't mention that in capabilities, only in "normal" command line options:
webdriver https://localhost --filter="Workflow>" --max-workers 3 --rerun-failed --browserstack=user,key --cap browserName=chrome --cap platform=windows

Post by jayaprakasha »

But when i run the above command, in browser-stack i cannot see parallel tests running. It shows that parallel tests running 1/5.

Post by nickolay »

This may mean your test suite is quite small. It divides the suite in separate "chunks", 20 tests per "chunk" and runs those chunks in parallel. If you have <20 tests in suite there will be only 1 chunk. The size of the chunk can be specified with the "--chunk-size" option.

Post by jayaprakasha »

Great! i can see the tests in parallel in browser stack as i gave the "--chunk-size 5" option. But, the test cases pass only in one thread/instance and the other thread always fails .

For example,
Test cases executed = 11
maxworkers =2
chunk size = 5
threads run in browser stack= 2
i can see 5 browser stack instances have run totally and 2 in parallel in which one thread always fails. Is it because one fails another one starts to try again and it tries for 5 attempts?(See attachment-only the 5th instance is running some of the test cases, all others gets stopped)

My command prompt throws this error for the failed threads,

<Exception from launcher>
While running big script: return (function (args) {
var me = Siesta.my.activeHarness

return me ? me[ 'launchAutomatedTests' ].apply(me, args)
Exception: TypeError: Cannot set property 'automationElementId' of undefined

TypeError: Cannot set property 'automationElementId' of undefined
at Object.<anonymous> (https://elsevier.uat.ordersetsmanager.com/application/
resources/siesta/siesta-all.js:46986:45)
at Object.map (https://elsevier.uat.ordersetsmanager.co ... n/resource
s/siesta/siesta-all.js:67:28)
at Object.launchAutomatedTests (https://elsevier.uat.ordersetsmanager.com/app
lication/resources/siesta/siesta-all.js:46983:43)
at Object.override [as launchAutomatedTests] (https://elsevier.uat.ordersetsm
anager.com/application/resources/siesta/siesta-all.js:817:32)
at __EXECUTE_SCRIPT_RESULT__.automationElementId (eval at execute (https://el
sevier.uat.ordersetsmanager.com/application/resources/siesta/siesta-all.js:46363
:17), <anonymous>:4:58)
at eval (eval at execute (https://elsevier.uat.ordersetsmanager.com/applicati
on/resources/siesta/siesta-all.js:46363:17), <anonymous>:5:15)
at eval (eval at execute (https://elsevier.uat.ordersetsmanager.com/applicati
on/resources/siesta/siesta-all.js:46363:17), <anonymous>:5:234)
at eval (<anonymous>)
at Object.execute (https://elsevier.uat.ordersetsmanager.co ... ation/reso
urces/siesta/siesta-all.js:46363:17)
at Object.override [as execute] (https://elsevier.uat.ordersetsmanager.com/ap
plication/resources/siesta/siesta-all.js:817:32)
</Exception from launcher>
Attachments
0001.JPG
0001.JPG (369.44 KiB) Viewed 8254 times

Post by nickolay »

Hm.. Hard to say. Yes, it will re-start the test, if it has failed because of some selenium error. Any chance you can provide a reproducible test case?

Perhaps this is something related to the test code itself. Can you try lets say keeping only 2 test in the suite and running it using --chunk-size=1 --max-workers=2

Post by jayaprakasha »

This is as i said before, one test case runs in one thread successfully. The other one doesn't run as the thread gets closed, a new thread opens and again fails. Finally that test case is failed with the below mentioned exception as shown in the cmd in the previous post.

Post by nickolay »

Sorry, its not enough information to be able to understand/reproduce the problem.. In our setup (test suites of Bryntum products) running several chunks in parallel in BrowserStack works fine.

Perhaps the tests in other chunks somehow depends on the tests in the 1st chunk. This can be for example if 1st test logs into the application, the authentication cookie is received, saved in the browser, and then the following tests do not perform the log in, relying on being launched _after_ the 1st tests.

When running several tests in parallel the order of tests execution is not defined, so every test should be standalone (check that log in is required, allocate exclusive set of data in DB if necessary etc).

Post by jayaprakasha »

Thanks much..That was right! My test cases depends on the same login function. So i tried to execute two test cases that do no depend on each other(no login at all), but still i am facing the same exception and the thread gets closed.
My concern is that the exception is on siesta-all.js and not on my scripts, please take a look...

While running big script: return (function (args) {
var me = Siesta.my.activeHarness

return me ? me[ 'launchAutomatedTests' ].apply(me, args)
Exception: TypeError: Cannot set property 'automationElementId' of undefined

TypeError: Cannot set property 'automationElementId' of undefined
at Object.<anonymous> (https://elsevier.remote/application/resources/siesta/s
iesta-all.js:46986:45)
at Object.map (https://elsevier.remote/application/reso ... /siesta-al
l.js:67:28)
at Object.launchAutomatedTests (https://elsevier.remote/application/resources
/siesta/siesta-all.js:46983:43)
at Object.override [as launchAutomatedTests] (https://elsevier.remote/applica
tion/resources/siesta/siesta-all.js:817:32)
at __EXECUTE_SCRIPT_RESULT__.automationElementId (eval at execute (https://el
sevier.remote/application/resources/siesta/siesta-all.js:46363:17), <anonymous>:
4:58)
at eval (eval at execute (https://elsevier.remote/application/resources/siest
a/siesta-all.js:46363:17), <anonymous>:5:15)
at eval (eval at execute (https://elsevier.remote/application/resources/siest
a/siesta-all.js:46363:17), <anonymous>:5:213)
at eval (<anonymous>)
at Object.execute (https://elsevier.remote/application/reso ... esta/siest
a-all.js:46363:17)
at Object.override [as execute] (https://elsevier.remote/application/resource
s/siesta/siesta-all.js:817:32)
</Exception from launcher>

Post by nickolay »

Hm.. What is your Siesta version? Try with the latest version? Everything definitely just works in our setup.

Post Reply