Get help with testing, discuss unit testing strategies etc.


Post by Nigel »

Hi All
I've just started using Siesta and love it already! I've got some basic tests working that load up my (ExtJS v3.4) application and simulate logging in and logging out again - all works great.

Before I go much further, I want to separate out my tests and build a suite of re-usable functions - logging in and out are prime candidates as they would be called at the beginning and end of most of my tests!

I am struggling slightly and I think my problem is related to the scope passed to all functions called in e.g. t.waitFor etc - ideally I want to get hold of 't' again in a function defined elsewhere (rather than within the scope of the function passed as a parameter to StartTest itself as in all the examples I've found so far).

Conceptual example of what I'm attempting (partial):
StartTest(function (t) {
    t.diag('Sanity');

    t.ok(Ext, 'ExtJS found');

    t.waitFor(xampe.Test.loginTest);
});
where xampe.Test sits one of the files included as part of the preload, as follows:
Ext.ns('xampe.Test');

xampe.Test = {
    loginTest: function () {
         var w = MyApp.loginWin, f,
		nameField,
		passwordField;

	t.diag('Login');

	t.ok(w, 'Login Form Found');
    
        //etc (code that works fine when structured within the scope of the original function as per examples)
    }
}
I've tried setting a global variable at the beginning of the function call, but it doesn't seem to be in scope if I call another function.
var test;

StartTest(function(t) {
    test = t;

    // etc
});
then refer to 'test' rather than 't' in my test suite file. That doesn't work, presumably because the 'test' global variable is not in the same scope as that of the test function.

Another attempt was based on a look at some of the Siesta code - I thought that perhaps the 'global' variable of the scope in the function would contain the global variables of the original scope, so I tried replacing 'test' with 'this.global.test', but no joy.

Anyone have some suggestions on how to create and structure a set of re-usable test functions that can access the same Test object? I'd like to start as I mean to go on so would welcome any ideas.

Thanks
Nigel

Post by nickolay »

What you need is custom assertion methods. Please check a section about this in the "Getting started guide" ("Creating your own assertions"). It contains a simple example and you can read the Siesta.Test.ExtJS sources for additional inspiration :)
There's also "examples/023-extjs-grid" which contains a "YourUtilities" test class with additional "getGrid" helper method.

Things to note:
- "this" inside of assertion methods corresponds to "t"
- test class will be executed in its own scope (to not interfere with any test's globals). So, "window" inside of assertion methods is different from the "window" inside of StartTest.
To get the "window" of test page, use "this.global"
- To get the Ext object from the scope of test page use `this.getExt()`, which is a shortcut for `this.global.Ext`
- new test class should be included in the harness file - index.html

I'll expand that section in the docs with this additional info.

Post by Nigel »

Nickolay

Thanks for your prompt response and the good advice. I had seen (briefly) the example you mention and the info about extending the test class, but I hadn't expanded my thinking to realise that this was the appropriate place to add the sort of methods I'm describing i.e. effectively test sub-routines that will be re-used, and utilities, rather than just additional 'assertions'. Now that I've turned by brain round 90 degrees, I can see how I'll approach this :)

I'm pleased you will add more to the documentation on this topic and, in particular, on the complexities of how scope works across the various layers as it is, naturally enough for this application, more complex than most users normally have to deal with - I can't be the only one who has struggled a bit so some good advice and best practice will be most welcome!

I'll update this thread when I've successfully created some of my test subroutines and let you know how I get on

Regards
Nigel

Post by nickolay »

Ok, great. I moved the content of the "Creating own assertions" to the new guide (will be available in 1.0.5). If you feel it can be further expanded - please let us know.

Post by Nigel »

Hi Nickolay

I can't actually see the new version of the guide yet, but my main comment at this stage would be that the title, 'create your own assertions', is too limiting. My big realisation was that it isn't JUST new assertions that can be created, but entire test sub-routines, utility functions to help find the right components in your app, etc etc. I think there's an opportunity to widen the topic to clarify that the extension of the Test classes is the appropriate, and recommended, way of providing ALL of these options for making a more robust and re-useable test suite specific to your own environment.

Happy to comment on exactly what's written so far if you want to send me a private message with the link to it.

Regards
Nigel

Post by nickolay »

Hi Nigel,

Ok, thanks for feedback. I changed the guide title to "Extending test class with your own assertions and utility methods" - will be available 1.0.5 (scheduled for release today). The content is mostly the same - just added couple of examples and link to /examples/023-extjs-grid

Post by Nigel »

Hi Nickolay

I've made some progress, but hit some new issues.

Firstly, I've spotted a problem with handling ExtJS v3 components - I had mentioned this to Mats but thought it was solved again - alas no!

When I pass an ExtJS v3 component to the Test.click method, it fails with the following error:

Test suite threw an exception: Ext.Element.fromPoint is not a function

I determined the problem - the 'fromPoint' method appears to be new in ExtJS v4, I can't find it in v3. I made a temporary fix by adding the following in my (v3.4) application code-base:
Ext.Element.fromPoint = function (x, y) {
    return Ext.get(document.elementFromPoint(x, y));
};
but, unless I've done anything else wrong, I'd suggest that the Siesta code should detect whether the Ext.Element.fromPoint method exists before using it, and if not, provide the above code override (lifted straight from the v4 code-base) to do the job cleanly.

-------------------------

Secondly, I've begun the implementation of my Test class override, as you recommended. This is set-up and working fine, so based on the code below I am calling t.loginTest() at the beginning of the function passed to StartTest.

With the fix for 'fromPoint' in place discussed above, everything works as far as the call to t.click(nameField... )

The call to t.type fails with the first parameter to type being undefined, even though I have confirmed that nameField still held the correct value (using the console.log(nameField) check).
After much head-scratching, I still don't fully understand why, but inserted console.logs into the code for the type method to confirm that the first parameter was undefined, and inserted a console.trace that shows the action of the Joose override (line 810 of siesta-all.js, v1.0.4). Is this 'messing up' the scope before passing the arguments to type? I can't see any other reason why the nameField variable has the correct value in the line before, but the value is lost on passing it as a parameter to the type method.

I didn't have this problem when I used the nesting of t.click and t.type in code directly in a function within the StartTest function as per the examples (such as 010_basic_form).
Class('xampeTest', {
    isa: Siesta.Test.ExtJS,
    has: {
        userName: 'nigel.dahl',
        password: 'XXXXXXXX'
    },

    methods: {
		loginTest: function () {
			var t = this,
				g = t.global,
				app = g.MyApp;

			t.waitFor(function () {
				return app.loginWin;
			}, function () {
				var f,
					nameField,
					passwordField;

				t.diag('Login');

				t.ok(app.loginWin, 'Login Window Found');

				f = app.loginWin.findByType('xloginform')[0];

				t.ok(f, 'Login Form Found');
				if (!f) {
					return;
				}

				f = f.getForm();

				nameField = f.findField('loginUserName');
				t.ok(nameField, 'Name Field found');

				nameField.setValue('');

				t.waitForComponentVisible(nameField, function () {
					t.click(nameField, function () {
						console.log(nameField);

						t.type(nameField, t.userName, function () {
							/*t.click(passwordField, function () {
								t.type(passwordField, t.password + '[ENTER]');
							});*/
						});
					});
				});
			});
		}
	}
});
Hope you can shed some light on this!

Thanks
Nigel

Post by nickolay »

Nigel wrote:
Ext.Element.fromPoint = function (x, y) {
    return Ext.get(document.elementFromPoint(x, y));
};
but, unless I've done anything else wrong, I'd suggest that the Siesta code should detect whether the Ext.Element.fromPoint method exists before using it, and if not, provide the above code override (lifted straight from the v4 code-base) to do the job cleanly.
Hi Nigel,

Thanks for pointing, this dependency is generally not required, I replaced it with new method "elementFromPoint" in the Siesta.Test.Browser.
Nigel wrote: The call to t.type fails with the first parameter to type being undefined, even though I have confirmed that nameField still held the correct value (using the console.log(nameField) check).
After much head-scratching, I still don't fully understand why, but inserted console.logs into the code for the type method to confirm that the first parameter was undefined, and inserted a console.trace that shows the action of the Joose override (line 810 of siesta-all.js, v1.0.4). Is this 'messing up' the scope before passing the arguments to type? I can't see any other reason why the nameField variable has the correct value in the line before, but the value is lost on passing it as a parameter to the type method.

I didn't have this problem when I used the nesting of t.click and t.type in code directly in a function within the StartTest function as per the examples (such as 010_basic_form).
No, its not related to that, scope if left intact.

Perhaps previously you were using `t.type(nameField.el, ..)` ? Because this is the cause of the troubles - you are using Siesta.Test.ExtJS test class, which we are planning to rename to Siesta.Test.ExtJS4 soon. Siesta.Test.ExtJS is fine-tuned for Ext4 and it only works with Ext3 till certain degree.
For example in `t.type` method, it checks if the passed argument is the instance of Ext.form.Field (same class exists in Ext3) and if so, it performs typing on the `arg.inputEl`, which in Ext4 contains reference to <input> dom element. But in Ext3, "inputEl" is undefined.

Perhaps at this point, for you will be better to use Siesta.Test.Browser class, which is a generic test class, w/o dependecies on Ext or other libraries - it will be much less magic happening. You can inherit from Siesta.Test.Browser, and copy various methods from Siesta.Test.ExtJS with little modifications along the way. This will give you a Ext3 testing layer.

Post by Nigel »

Nickolay

Thanks for the comprehensive response, which has at least helped remove some of my immediate concerns about my own sanity!!

I had tried various combinations before such as nameField.getEl().dom, which worked fine, when I was struggling with this first time round. I probably got round this problem without even realising it.

I will follow your advice and revert to Test.Browser for now, extending as necessary with Ext3-specific enhancements.

I look forward to your release of a fully integrated Ext3 test class - the race will be whether you get that out before I get our app fully converted to Ext4 - I suspect you might win the race :)

Regards
Nigel

Post Reply