Get help with testing, discuss unit testing strategies etc.


Post by bjones »

Using Siesta 5.1.1 and launching from a CentOS 7 system

Command Line output:
$ ./webdriver [test_url] --browserstack [username],[key] --cap os="OS X" --cap os_version=Mojave --cap browser=Safari --cap browser_version=12.0
Launching local tunnel to BrowserStack: [username]
[ERROR] Setup failed: Error: Error while opening project page: UnsupportedOperationError: Parameter 'x' has a boolean value, which is not allowed.
Closing BrowserStack tunnel
BrowserStack logs:
BrowserStack Safari error.png
BrowserStack Safari error.png (61.58 KiB) Viewed 2751 times
The logs from BrowserStack above show that the last instruction to execute before the error occurs is changing the window position. This can be traced to the siesta-launcher-all.js code starting at line 14045 (the open function of Siesta.Launcher.Page.WebDriverNodeJS).

Relevant code from siesta-launcher-all.js:
open : function (url, callback) {
    var me          = this
    var driver      = me.driver

    driver.get(url).then(function () {
        // setPosition also updates window size (on MacOS) for some reason, so, on MacOS it must be done before setSize
        // on other platforms its the opposite
        //return me.launcher.isMacOS ? driver.manage().window().setPosition(1, 1) : Promise.resolve()
        return driver.manage().window().setPosition(1, 1);
    }).then(function () {

        return driver.manage().window().setSize(me.width - 1, me.height - 1)

    }).then(function () {
        return Promise.resolve()
        // return !me.launcher.isMacOS ? driver.manage().window().setPosition(1, 1) : Promise.resolve()

    }).then(function () {
        callback()
    }, function (e) {
        callback(e)
    })
},
Another interesting thing I noticed was that in my BrowserStack logs, the command to change the window size executed before the change position command. Looking at the code above, this should only happen if me.launcher.isMacOS evaluates to false.

What was happening here is that the system where ./webdriver was called determines isMacOS / isWindows / isLinux / etc. Since we’re calling our BrowserStack tests from a Linux system, isLinux will always be true regardless of the OS / browser user on BrowserStack.

However, even when I changed the source so that setPosition was always called before setSize (as the comments indicate is the correct order for Mac OS), I got the same error of “Parameter 'x' has a boolean value, which is not allowed.” So it seems setPosition(1,1) doesn't work with Safari.
BrowserStack Safari error.png
BrowserStack Safari error.png (61.58 KiB) Viewed 2751 times
In order to get Safari BrowserStack runs to work, I modified the source to skip the setPosition(1,1) call when the OS / browser specified via capabilities was OS X Safari but maintained the same behavior as the original source in all other cases. Now all the OS / browser combinations we are testing via BrowserStack are working without error. Hopefully this helps anyone who comes across similar issues.
Attachments
BrowserStack Safari error 2.png
BrowserStack Safari error 2.png (63.22 KiB) Viewed 2751 times

Post by nickolay »

Thanks for sharing! We'll investigate and include the fix for this issue in the next release.

Any reason you are not on Siesta 5.2.x ?

Post by nickolay »

This seems to be a bug in Safari 12:

https://github.com/web-platform-tests/wpt/issues/15625
https://github.com/web-platform-tests/w ... a27240318d

According to the links above, it was already fixed, but will take some time until the fresh version will be available in the BrowserStack. We'll add a workaround (which is moving a window to position (2, 2))

Post by nickolay »

The workaround pushed to sources. Will be available in the tomorrow nightly build and in 5.2.3 (to be released this week).

Post by bjones »

Thanks for looking into this and adding a workaround. Will update to 5.2.3 when it's released.

Post Reply