Error when launching OS X Safari tests on BrowserStack

Get help with testing, discuss unit testing strategies etc.
Post Reply
User avatar
bjones
Premium Member
Premium Member
Posts: 5
Joined: Thu Nov 01, 2018 8:22 pm

Error when launching OS X Safari tests on BrowserStack

Post by bjones » Tue Apr 23, 2019 10:29 pm

Using Siesta 5.1.1 and launching from a CentOS 7 system

Command Line output:

Code: Select all

$ ./webdriver [test_url] --browserstack [username],[key] --cap os="OS X" --cap os_version=Mojave --cap browser=Safari --cap browser_version=12.0
Launching local tunnel to BrowserStack: [username]
[ERROR] Setup failed: Error: Error while opening project page: UnsupportedOperationError: Parameter 'x' has a boolean value, which is not allowed.
Closing BrowserStack tunnel
BrowserStack logs:
BrowserStack Safari error.png
BrowserStack Safari error.png (61.74 KiB) Viewed 115 times
The logs from BrowserStack above show that the last instruction to execute before the error occurs is changing the window position. This can be traced to the siesta-launcher-all.js code starting at line 14045 (the open function of Siesta.Launcher.Page.WebDriverNodeJS).

Relevant code from siesta-launcher-all.js:

Code: Select all

open : function (url, callback) {
    var me          = this
    var driver      = me.driver

    driver.get(url).then(function () {
        // setPosition also updates window size (on MacOS) for some reason, so, on MacOS it must be done before setSize
        // on other platforms its the opposite
        //return me.launcher.isMacOS ? driver.manage().window().setPosition(1, 1) : Promise.resolve()
        return driver.manage().window().setPosition(1, 1);
    }).then(function () {

        return driver.manage().window().setSize(me.width - 1, me.height - 1)

    }).then(function () {
        return Promise.resolve()
        // return !me.launcher.isMacOS ? driver.manage().window().setPosition(1, 1) : Promise.resolve()

    }).then(function () {
        callback()
    }, function (e) {
        callback(e)
    })
},
Another interesting thing I noticed was that in my BrowserStack logs, the command to change the window size executed before the change position command. Looking at the code above, this should only happen if me.launcher.isMacOS evaluates to false.

What was happening here is that the system where ./webdriver was called determines isMacOS / isWindows / isLinux / etc. Since we’re calling our BrowserStack tests from a Linux system, isLinux will always be true regardless of the OS / browser user on BrowserStack.

However, even when I changed the source so that setPosition was always called before setSize (as the comments indicate is the correct order for Mac OS), I got the same error of “Parameter 'x' has a boolean value, which is not allowed.” So it seems setPosition(1,1) doesn't work with Safari.
BrowserStack Safari error.png
BrowserStack Safari error.png (61.74 KiB) Viewed 115 times
In order to get Safari BrowserStack runs to work, I modified the source to skip the setPosition(1,1) call when the OS / browser specified via capabilities was OS X Safari but maintained the same behavior as the original source in all other cases. Now all the OS / browser combinations we are testing via BrowserStack are working without error. Hopefully this helps anyone who comes across similar issues.
Attachments
BrowserStack Safari error 2.png
BrowserStack Safari error 2.png (63.43 KiB) Viewed 115 times

User avatar
nickolay
Core Developer
Core Developer
Posts: 3329
Joined: Mon May 16, 2011 10:48 am

Re: Error when launching OS X Safari tests on BrowserStack

Post by nickolay » Wed Apr 24, 2019 8:55 am

Thanks for sharing! We'll investigate and include the fix for this issue in the next release.

Any reason you are not on Siesta 5.2.x ?
We offer training in both Ext JS and our products, read more here.
Read the API documentation

User avatar
nickolay
Core Developer
Core Developer
Posts: 3329
Joined: Mon May 16, 2011 10:48 am

Re: Error when launching OS X Safari tests on BrowserStack

Post by nickolay » Wed Apr 24, 2019 9:18 am

This seems to be a bug in Safari 12:

https://github.com/web-platform-tests/wpt/issues/15625
https://github.com/web-platform-tests/w ... a27240318d

According to the links above, it was already fixed, but will take some time until the fresh version will be available in the BrowserStack. We'll add a workaround (which is moving a window to position (2, 2))
We offer training in both Ext JS and our products, read more here.
Read the API documentation

User avatar
nickolay
Core Developer
Core Developer
Posts: 3329
Joined: Mon May 16, 2011 10:48 am

Re: Error when launching OS X Safari tests on BrowserStack

Post by nickolay » Wed Apr 24, 2019 9:50 am

The workaround pushed to sources. Will be available in the tomorrow nightly build and in 5.2.3 (to be released this week).
We offer training in both Ext JS and our products, read more here.
Read the API documentation

User avatar
bjones
Premium Member
Premium Member
Posts: 5
Joined: Thu Nov 01, 2018 8:22 pm

Re: Error when launching OS X Safari tests on BrowserStack

Post by bjones » Wed Apr 24, 2019 6:32 pm

Thanks for looking into this and adding a workaround. Will update to 5.2.3 when it's released.

Post Reply