Get help with testing, discuss unit testing strategies etc.


Post by chauncey-garrett »

Hello,

I'm seeing two issues with the webdriver wrapper in the bin folder:

1) The first is that the $DIR variable is not quoted at the end of the file. It should be quoted to prevent word splitting:
"$DIR/binary/nodejs/$NODE_BIN_PATH/node" "$DIR/webdriver-launcher.js" "$DIR" "NOT_USED" "$@" 2>"$STDERRLOG"
2) The second error I'm running into is that the webdriver command is defaulting to the Linux OS when running tests in SauceLabs when using the Chrome and Firefox browsers (IE is determined to be under Windows). This bug occurs when passing in either "Windows 7" or "Windows 2008" as the OS capability using the "–cap platform=Windows 7" option for the webdriver wrapper script.

The specific error that is presented is below:
[WARN] Requested resolution 1280×1024 is not supported for this OS/browser/version/device combo. Retrying w/o requesting any specific resolution.
Launching test suite, OS: Linux, browser: Chrome 48.0.2564.97
I've confirmed that SauceLabs supports this resolution for Windows 7 / Chrome and Windows 7 / Firefox.

I hope this helps and that we can see a bugfix for these issues. Please let me know if you have any other questions.

Post by nickolay »

1) Fixed in our sources, thanks
2) Seems to work fine for me:
nickolay@outpost:~/workspace/JavaScript/siesta$ bin/webdriver lh/siesta/examples --include 010 --saucelabs $SL_USER,$SL_KEY,lh --browser chrome --cap platform="Windows 7"
Launching local tunnel to SauceLabs: bryntumdev
Launching test suite, OS: Windows, browser: Chrome 54.0.2840.59
[PASS]  5.code_coverage/010_range.t.js
Need to quote the "Windows 7" as it contains a space may be?

Post by chauncey-garrett »

This is how we launch webdriver:
# Launch Siesta WebDriver.
  webdriver "${URL}" \
    --build "${SL_BUILD_ID}" \
    --cap browserName="${SELENIUM_BROWSER}" \
    --cap name="Automated Testing" \
    --cap platform="${SELENIUM_PLATFORM}" \
    --cap tags="Jenkins" \
    --cap version="${SELENIUM_VERSION}" \
    --coverage-report-format="${COVERAGE_FORMAT}" \
    --coverage-report-dir="${COVERAGE_REPORT_DIR}" \
    --jenkins \
    --previous-coverage-report="${PREVIOUS_COVERAGE_REPORT}" \
    --port "${SELENIUM_PORT}" \
    --report-file="${REPORT_FILE}" \
    --report-format="${REPORT_FORMAT}" \
    --saucelabs-enable-screenshots \
    --saucelabs-enable-video \
    --saucelabs-key="${SL_KEY}" \
    --saucelabs-user="${SL_USER}" \
    --saucelabs-tunnel-identifier "${SL_TUNNEL}"
The browser variable is quoted there. Perhaps there is something in the additional options that is causing the issue? Would the output from debug mode be helpful to you?

Post by nickolay »

Can you echo the value of ${SELENIUM_PLATFORM} and check the logs what value it actually receives? My guess is - its empty.
echo PLATFORM=${SELENIUM_PLATFORM} 

Post by chauncey-garrett »

This is a portion of the output from our Jenkins job:
+webdriver:8> '/hostonly/jenkins/workspace/redacted/Windows_2008chromelatest/label/redacted/bin/../src/vendor/siesta/bin/webdriver' https://redacted/index.html --build 'redacted Testing stage2 #51' --cap 'browserName=chrome' --cap 'name=Automated Testing' --cap 'platform=Windows 2008' --cap 'tags=redacted' --cap 'version=latest' '--coverage-report-format=html+raw' '--coverage-report-dir=/hostonly/jenkins/workspace/redacted/SELENIUM_DRIVER/Windows_2008chromelatest/label/redacted/bin/../report/code-coverage' --jenkins '--previous-coverage-report=' --port 50221 '--report-file=/hostonly/jenkins/workspace/redacted/SELENIUM_DRIVER/Windows_2008chromelatest/label/redacted/bin/../report/jenkins/report.xml' '--report-format=JUnit' --saucelabs-enable-screenshots --saucelabs-enable-video '--saucelabs-key=redacted' '--saucelabs-user=redacted' --saucelabs-tunnel-identifier SELENIUM_DRIVER_Windows_2008chromelatest_label_redacted-1477924296499
[WARN] Requested resolution 1280x1024 is not supported for this OS/browser/version/device combo. Retrying w/o requesting any specific resolution.
Launching test suite, OS: Linux, browser: Chrome 48.0.2564.97
[PASS]  /stuf/app/vms/tests/login/valid.t.js
You can see that the platform is set as Windows 2008 in quotes but that the OS is changed to Linux.

Post by nickolay »

Really can't see how it can happen. Please do the following:

Find this line in the "bin/siesta-launcher-all.js":
me.debug("Dispatcher start")
and replace it to:
me.debug("Dispatcher start")
console.log(JSON.stringify(options.cap))
And update this thread with the output from the command.

Post by chauncey-garrett »

It's been a while but I was finally able to come back to this issue.

Here's a boiled down reproducible test case:
# Does NOT use the correct browser
$ SELENIUM_BROWSER=chrome SELENIUM_PLATFORM="Windows 7" SELENIUM_VERSION=52 \
./src/vendor/siesta/bin/webdriver "redacted" \
    --cap browserName="${SELENIUM_BROWSER}" \
    --cap platform="${SELENIUM_PLATFORM}" \
    --cap version="${SELENIUM_VERSION}" \
    --saucelabs="redacted"
Launching local tunnel to SauceLabs: automation_jenkins
[WARN] Requested resolution 1280x1024 is not supported for this OS/browser/version/device combo. Retrying w/o requesting any specific resolution.
Launching test suite, OS: Linux, browser: Chrome 48.0.2564.97
...

# Works as expected
$ ./src/vendor/siesta/bin/webdriver "https://redacted" \
    --cap browserName="chrome" \
    --cap platform="Windows 7" \
    --cap version="52" \
    --saucelabs="redacted"
Launching local tunnel to SauceLabs: automation_jenkins
Launching test suite, OS: Windows, browser: Chrome 52.0.2743.82
...
It seems that having the SELENIUM environmental variables causes the wrong browser to be set. These variables are automatically set by the Jenkins Sauce OnDemand Plugin so I can't just rename them. I'm able to reproduce this outside of Jenkins in my dev environment. I'm using Siesta v4.2.2.

Post by nickolay »

Reproduced. Turns out this is a behavior of nodejs bindings for Selenium: https://www.npmjs.com/package/selenium-webdriver (search for "SELENIUM_BROWSER").

Ticket created: https://app.assembla.com/spaces/bryntum ... gs/details#

Right now you can unset the SELENIUM_BROWSER environment variable, before launching the bin/webdriver.

Please let me know if that helps.

Post Reply