We’ve just released Siesta 4.2.0 – an important Siesta milestone and in this blog post we’ll highlight its most important features.
Introducing Ariadne – The New Query Finder
The internals of the Siesta event recorder have been completely rewritten (more specifically, the target finding algorithm). In previous versions, the recorder sometimes found either a very long CSS query or it could not figure out a stable CSS query at all. We decided that this feature was important enough to warrant a new implementation.
In this new release, we have moved the query finding functionality to a new project called Ariadne. Ariadne finds the “optimal” CSS query for any given DOM element and it handles Ext JS component queries just as well. Let’s check the key requirements we had for this task and design choices made during the development.
First of all, how does one define the “optimal” query, seen from the event recorder’s perspective?
The “optimal” query has to be stable – it should survive reasonable markup changes. This means the recorder should try to avoid all types of positional selectors such as: “>” (direct child), “nth-child()”, “nth-of-type()”. Those are very fragile and will break as soon as new element is added to the DOM, or when the position of an element changes. In the same time there are situations when it is not possible to build a unique query, without using those selectors. In such cases Ariadne falls back to a positional selector, but it uses them as little as possible. Ariadne prefers “nth-of-type(1)” selector over the “nth-child()”, since it survives addition of element with another tag name.
The “optimal” query also has to be short and readable. This means only specific CSS classes (or DOM attributes) should be included in the query, and as few as possible of them. This requirement also improves the stability of the query, since the bigger query is – the higher risk that a markup change will break the query.
And lastly of course, the finding query should be fast.
We have implemented a quite sophisticated algorithm to achieve all these requirements and here are some examples from ExtJS kitchensink (action targets are marked with red cross):
Previous implementation finds some huge, unreadable CSS query:
#content-panel .kitchensink-example .x-panel.x-panel-default-framed.x-resizable.x-panel-resizable.x-panel-default-framed-resizable.x-border-box .x-panel-body.x-panel-body-default-framed.x-column-layout-ct.x-resizable.x-panel-body-resizable.x-panel-body-default-framed-resizable .x-container.x-column .x-form-layout-wrap.x-form-layout-auto-label .x-field.x-form-item.x-form-type-text.x-form-form-item .x-form-item-body.x-form-text-field-body .x-form-trigger-wrap .x-form-text-wrap .x-form-text
>>#content-panel form-multicolumn[title=Multi Column Form] textfield[inputType=text]
#content-panel form-multicolumn[title=Multi Column Form] textfield[inputType=text] => .x-form-text
Now the same recording repeated with Ariadne – resulting in a much shorter, more stable and more readable query:
#content-panel .x-container:nth-of-type(1) .x-field:nth-of-type(1) .x-form-text
>>#content-panel textfield[fieldLabel=First Name]
#content-panel textfield[fieldLabel=First Name] => .x-form-text
Previous implementation was not able to find any query and only generated coordinate: [ 546, 411 ]
Ariadne demonstrates stable results:
#content-panel .x-container:nth-of-type(1) .x-field:nth-of-type(2) .x-form-text
>>#content-panel textfield[fieldLabel=Last Name]
#content-panel textfield[fieldLabel=Last Name] => .x-form-text
Previous implementation only found a component query: >>#content-panel layout-cardtabs tabbar tab[text=Tab 2]
Ariadne finds all 3 types of queries, and the component query is shorter:
#content-panel .x-tab-inner:textEquals(Tab 2)
>>#content-panel tab[text=Tab 2]
#content-panel tab[text=Tab 2] => .x-tab-inner
In general you should notice a significant improvement in the quality of generated queries.
While improving the quality of the queries, we’ve also put some effort into improving the quality of the playback, so that recorded actions could be replayed 100% reliably. Notably Siesta now respects the caret position in text fields, supports HOME/END key caret navigation, properly simulates “click” events in certain edge cases. All these minor improvements should allow you to successfully replay what you’ve recorded.
As you know Siesta includes SlimerJS as a headless test launcher. However as of version 0.10.0, SlimerJS is no longer headless and it requires Firefox to be installed in the OS along with a graphical server (xvfb or similar). We did not upgrade SlimerJS for a long time because of this reason, but now the Firefox version it implements is too old and outdated (39 and current is 49). So we had to upgrade SlimerJS for our users to get consistent results with the real Firefox browser. If you are using SlimerJS for running your tests in a server environment you will need to install
Firefox. It should be just a matter of running
apt-get install firefox xvfb on Debian-like systems.
The good news is that once you have xvfb installed on your machine, you can significantly increase the test suite execution speed (see next section)
Xvfb – running several browser sessions simultaneously on local machine
As your test suite grows, execution time can become a bottleneck. The solution for that is parallelization – running your test suite in several parallel “threads” or “workers” (as it’s called in Siesta). Since version 4, Siesta supports the
--max-workers command line config. However, when several browser windows are opened simultaneously on the same desktop, they start “focus fighting” – because focus can only be in one place by definition. Because of that, previously the primary use case for
--max-workers config was in cloud testing providers, since on local machine it didn’t work reliably.
Starting from this release, Siesta supports a new command line switch
--xvfb. It requires the presence of “xvfb” in the host system, which is basically a virtual desktop server. With this switch, Siesta will wrap every launched browser process with a virtual desktop, meaning the “focus fighting” problem goes away and one can safely specify reasonably big values for
--max-workers config (limited by CPU mostly). Values between 5-10 works great for us:
Execution time for
--xvfb --max-workers 1 ~ 13 min:
Execution time for
--xvfb --max-workers 7 ~ 3 min:
This option is supported on Linux only for now. Support for MacOS may be added in future releases. Unfortunately we couldn’t find any xvfb-similar tool for Windows (those that exists don’t have command line interface), so Windows support is currently not feasible.
We encourage you to give this release a try and use the new features and improvements to speed up your test suite. Please also share your feedback in our forums.
Enjoy the Siesta!