Get help with testing, discuss unit testing strategies etc.


Post by bperel »

Hi,

I am setting up a Windows scheduled task to run our Siesta tests on a regular schedule.
The Windows machine is accessed through RDP.

Setting up a scheduled task with the option "Run only when user is logged on" worked : the tests are run as expected, but (as the option suggests) only if a person is connected to the server through RDP at the time where the task is supposed to be run.
Since we don't want to have this restriction, I switched to selecting "Run whether user is logged in or not".

Image

This leads to the Siesta tests failing with the following error :
[ERROR] Setup failed: Error: Can't create first page, runner: a Siesta.Launcher.Runner.WebDriverNodeJS, exception: WebDriverError: Process unexpectedly closed with status: 0
The command that is run through the Scheduled task is the following:
C:/siesta/bin/webdriver.bat 'https://user:password@mysite.com/tests' --headless --cap browser=firefox --report-format HTML --report-file tests_reports/2018-01-25_084001/html --report-format Junit --report-file tests_reports/2018-01-25_084001/junit.xml
The return code of the command is 3.

I suppose that it is a Selenium-related error. Is there a way to fix it?

Note: Not sure if it is relevant, but if "Run whether user is logged in or not" is selected, then the error above always occur, whether I'm connected to the server through RDP or not.
Bruno

Post by nickolay »

I'm afraid I'm not an Windows expert, so can't provide much help here. But AFAIK, Windows RDP has 2 modes - with desktop interaction and pure console. It can be that the error is caused because the tests are running in console mode.

We are actually very interested in figuring out the robust way of launching Siesta tests remotely on Windows machines (to integrate in Siesta directly), please post here any findings you'll have.

Post by bperel »

Yes, I think the Webdriver process having to deal with console mode is the problem here, indeed. Strangely enough though, running the tests with Chrome (still with "user logged in or not" option) through a scheduled task (same settings than the other browsers apart from the capabilities param) seems to work. So here's the sum-up:
  • Firefox:
    C:/siesta/bin/webdriver.bat 'https://user:password@mysite.com/tests' --headless --cap browser=firefox
    =>
    [ERROR] Setup failed: Error: Can't create first page, runner: a Siesta.Launcher.Runner.WebDriverNodeJS, exception: WebDriverError: Process unexpectedly closed with status: 0
  • IE 11 :
    C:/siesta/bin/webdriver.bat 'https://user:password@mysite.com/tests' --headless --cap browser=IE --cap browser_version=11.0
    =>
    [ERROR] Setup failed: Error: Error while opening harness page: 5
  • Chrome:
    C:/siesta/bin/webdriver.bat 'https://user:password@mysite.com/tests' --headless --cap browser=chrome --cap browser_version=11.0
    => Test suite correctly executed
So, I see 3 different possibilities:
* There's an issue with the Firefox and IE Web drivers
* There's an issue with Selenium interacting with them (but then why doesn't Chrome fail as well ?)
* There's an issue with Siesta interacting with Selenium and the Web drivers

Any ideas on what are my investigation options here? Any command line that I can run to check how Selenium is behaving, without the Siesta wrapper?
Bruno

Post by nickolay »

My guess is, Chrome's headless mode is more headless than Firefox's one :) So it can run in console mode fine. Valuable finding as well.

To verify "raw" selenium behavior you can try this nodejs script:
var webdriver   = require('selenium-webdriver'),
    By          = require('selenium-webdriver').By,
    until       = require('selenium-webdriver').until;

if (process.platform == 'win32')
    process.env.PATH    = process.env.PATH + ';bin\\binary\\chromedriver\\windows'
else
    process.env.PATH    = process.env.PATH + ':bin/binary/chromedriver/linux64'

if (process.platform == 'win32')
    process.env.PATH    = process.env.PATH + ';bin\\binary\\geckodriver\\win64'
else
    process.env.PATH    = process.env.PATH + ':bin/binary/geckodriver/linux64'

if (process.platform == 'win32')
    process.env.PATH    = process.env.PATH + ';\\binary\\iedriver_win64'

var driver = new webdriver.Builder()
    .forBrowser('firefox')
    .forBrowser('chrome')
    // .forBrowser('ie')
    .build();

driver.get('https://www.google.com/ncr');

driver.quit();
Need to save it in the Siesta folder and run with node. You can use the nodejs from the Siesta package: /bin/binary/nodejs/win64

Post by bperel »

Here are the results of running the command
"C:/siesta/bin/binary/nodejs/win32/node" temp.js > log.txt
, where temp.js is the following:
var webdriver   = require('selenium-webdriver'),
    By          = require('selenium-webdriver').By,
    until       = require('selenium-webdriver').until;
...
var driver = new webdriver.Builder()
    .forBrowser('BROWSER')
    .build();

driver.get('https://www.google.com/ncr');

driver.takeScreenshot().then(
    function(image, err) {
        require('fs').writeFile('out.jpg', image, 'base64', function(err) {
            console.log(err);
        });
    }
);
driver.quit();
"run whether user logged on or not":
  • BROWSER=chrome: Screenshot saved
  • BROWSER=firefox: Screenshot not saved
  • BROWSER=ie: Screenshot not saved
"run only when user is logged on":
  • BROWSER=chrome: Screenshot saved
  • BROWSER=firefox: Screenshot saved
  • BROWSER=ie: Screenshot not saved
I saw this recent blog post about pure headless mode in Firefox, I didn't have time to investigate but maybe it could help to provide a "real headless mode" for Firefox: https://hacks.mozilla.org/2017/12/using ... n-firefox/
Bruno

Post by nickolay »

Thanks for sharing. Yes, we use the same "-headless" config line option for firefox headless mode.

Post Reply