How to get application tests stable for slimerjs / teamcity?

Get help with testing, discuss unit testing strategies etc.
Post Reply
User avatar
nilsdehl
Posts: 5
Joined: Tue May 26, 2015 11:49 am

How to get application tests stable for slimerjs / teamcity?

Post by nilsdehl » Wed May 09, 2018 1:01 pm

In a current customer project we use Siesta to test an Ext JS modern 6.5.3 application.
All test run green locally in chrome using the siesta UI.
Unit and UI test run smooth in FF and remote on teamcity via slimerjs as well.
But the application tests fail remote in slimerjs while run green if I run them locally in chrome or FF.

I already tried:

Code: Select all

runCore: 'sequential',
pauseBetweenTests: 200,
in harness config for the application tests. As result the run more stable in local FF.

Failure example:
030_device_registration.t.js
If wrong server password credentials are used an error message should appear and the user stays on the same view.
fail 5 - Waited too long for: componentQuery: errormessagebox to return a visible component
Failed assertion `waitForComponentQueryVisible`
Condition was not fullfilled during 30000ms
After a succesfull device registration the user should be redirect to the personalisation view.
fail 3 - Waited too long for: componentQuery: personalization to return a visible component
Failed assertion `waitForComponentQueryVisible`
Condition was not fullfilled during 30000ms
Harness:

Code: Select all

var harness = new Siesta.Harness.Browser.ExtJS();

harness.configure({
	title: 'Test Suite',
	testClass: CustomTestClass,
..
});

harness.start(
	{
		group: 'Test Suite',
		items: [
			{
				group: 'Unit Tests', // runs stable
				...
			},
                        {
				group: 'UI Tests', // runs stable
				...
			},
			{
				group: 'Application Test',
				pageUrl: '../index.html',
				autoCheckGlobals: false,
				runCore: 'sequential',
				pauseBetweenTests: 200,
				items: [
					'application/010_app_sanity.t.js',
					{
						group: 'Device registration',
						pageUrl: '../#setup/device/registration',
						items: [
							'application/030_device_registration.t.js'
						]
					},
					....
				]
			}
		]
	}
);
030_device_registration.t.js

Code: Select all

describe('Device registration', function (t)
{
       // Init sim manager to fake backend 
	t.initFakeBackend();

	t.beforeEach(function (t, _next)
	{
		// clear localstorage to have a clean environment
		localStorage.clear();

		// reset route to registration
		t.setHash('setup/device/registration', _next, t);
	});

	t.it("Device shouldn't be registered", function (t)
	{
		t.expect(dilocsync.core.Master.isRegistered()).toBeFalsy();
	});

	t.it("Device registration view should be rendered", function (t)
	{
		t.chain(
			{
				waitForCQVisible: 'registration'
			}
		);
	});

	t.it("If wrong server password credentials are used an error message should appear and the user stays on the same view.", function (t)
	{
		t.chain(
			{
				waitForCQVisible: 'registration',
				desc: "Registration form should be visible"
			},
			{
				target: 'textfield[name=server]',
				setValue: 'server-does-not-exist'
			},
			{
				target: 'textfield[name=password]',
				setValue: 'wrong-password'
			},
			{
				waitFor: 100,
				desc: "Wait that bind can update the button"
			},
			{
				click: 'button[reference=register] => button'
			},
			{
				waitForCQVisible: 'errormessagebox',
				desc: "Error message should be shown"
			},
			{
				click: 'errormessagebox button => button'
			}
		);
	});
...

});

Most application tests fail at some point on slimerjs on teamcity cause if an "waitForComponentQueryVisible" time out. How can I make the application tests more stable?

Aiming for all tests green everywhere :)

Thanks
Nils

User avatar
nickolay
Core Developer
Core Developer
Posts: 3405
Joined: Mon May 16, 2011 10:48 am

Re: How to get application tests stable for slimerjs / teamc

Post by nickolay » Wed May 09, 2018 1:04 pm

Aiming for all tests green everywhere :)
Great attitude! :)

Try using webdriver + headless firefox/chrome instead of slimerjs? Or puppeteer launcher? There's plenty of headless options now, slimerjs is somewhat outdated project.
We offer training in both Ext JS and our products, read more here.
Read the API documentation

User avatar
nilsdehl
Posts: 5
Joined: Tue May 26, 2015 11:49 am

Re: How to get application tests stable for slimerjs / teamc

Post by nilsdehl » Wed May 09, 2018 1:13 pm

Thanks Nikolay we will try it with one of the alternatives.

Post Reply