Customer Spotlight: IntelliSurvey
About IntelliSurvey
IntelliSurvey is the premier online survey platform for professional market researchers and consultants. We run some of the most complex surveys that go far beyond “how would you rate your experience on a scale of 1 to 10”. Our web application is used to construct advanced surveys and analyze data, in real time and long after a survey is taken.
Our flagship product is a huge Single Page Application, with over 5 MB of minified JS in combined size of desktop and mobile manifestations, that allows researchers to slice, dice, and analyze the massive amounts of data that we collect. It is visually stunning with multiple themes and has a wide variety of views and behaviors:
Why Choose Bryntum Siesta?
Like so many other companies that have been developing software in-house for almost two decades, we’ve devised a home-grown test automation system based on technologies that were a lot more relevant during the dot com boom. The system has served us well, but is showing its age. It’s unstable, expensive to maintain, and hard to find developers with a matching skill set.
After evaluating a number of modern test automation frameworks, we have decided to go with Siesta for the following reasons:
100% JavaScript
There is no question that JavaScript is the most popular programming language today. It is the easiest software technology to recruit and train for, with the greatest available talent pool.
Bryntum Siesta tests are written and executed 100% in JavaScript. We use the ES6 variation, which allows us to write “modern” JS, which is what most code schools and bootcamps use in their training materials.
The actual tests are so straightforward and declarative that even new JS developers are able to hit the ground running and be productive within only hours:
Runs Directly in Browser
This is a huge point. Our home-grown test automation is only able to execute in Linux Chrome, which is a platform-OS combo that (almost) none of our users actually use. On top of that, it requires a “full stack VM”, which is rather heavy and requires a day of onboarding to set up for a new developer.
One of the biggest benefits of Siesta for us was the ability to run our test automation in all supported desktop browsers. After all, what is the point of going through the trouble of writing all these tests, if you can’t actually run them in the desktop environment that your users are using?
Full CLI Support
It goes without saying that a proper Test Automation system should integrate with your Continuous Integration system. Ours happens to be a home-grown Perl-based system that’s been evolving for almost two decades. It is run multiple times a day on various QA and staging servers, as well as dozens of recyclable VMs that our devs create for their own Git branches, as well as on the devs’ own machines. Thanks to Siesta’s out-of-the-box Command Line Interface, we were able to easily integrate it with our CI:
And if developers get Siesta test failures when testing their changes, they can follow the link in the results report to see the failed test execute directly in the browser. With this new workflow, developers save time and can spend more time developing.
BrowserStack Integration
Our application browser vs. OS matrix supports 10 combinations in total. It would be cost prohibitive to attempt to run a device lab this large in-house, and maintain QA resources to operate and aggregate all the test result reports.
It was imperative that our new test automation allows us to cover all of our supported desktop devices, and allows us to execute tests and aggregate results in a fully automated fashion.
We refused to settle for a lower grade of QA and began leveraging BrowserStack with the out-of-the-box Siesta integration. With BrowserStack, we can cover all our supported desktop browsers and OS’s. Since new browsers and devices are always being added, it’s easy to update our supported devices.
As we grow our mobile app, we look forward to leveraging the Siesta and BrowserStack integration to test our mobile experience on real mobile devices in the upcoming Siesta 6.
Deep Link Sharing
Perhaps the greatest and the most valuable feature of Siesta, that tends to go unnoticed to an untrained eye, is the ability to share deep-link URLs down to specific tests. Since the Siesta test suite is able to execute in all of our supported desktop browsers, we are now able to usher in an unparalleled level of collaboration between the QA, dev, and PM elements of our organization. Consider the following exchange that took place between our QA & PM:
What happens if there is a regression in your app in a specific OS / browser? How would remote QA be able to demonstrate the behavior in question to a PM, if not with a wall of text of repro steps? In our case, the PM simply clicks the link in their desktop messenger and watches the test play out with step-by-step instructions being displayed right as the test is executing. Consider the overall efficiency improvement in the organization when team members are able to pass a link around and see exactly what executes and fails (or passes), and are able to communicate without having to write up walls of repro steps or spend hours setting up some test infrastructure.
How Siesta could be improved further
Siesta works well for testing our supported desktop browsers / OS’s, but currently, Siesta’s web interface is not optimized for mobile devices. So at this time, we are unable to test our app in mobile devices in a way that simulates the actual user’s mobile experience. As we continue to build out our mobile application, it is increasingly important that we test our mobile experience on the actual mobile devices that our users use.
Siesta already has an integration with BrowserStack, which supports testing on real mobile devices. If the Siesta browser interface was optimized for mobile, testing our supported mobile devices through BrowserStack would be an easy next step.
Conclusion
The web browser has come a long way since the early 2000s. Back then, few people could conceive the idea of an entire test automation executing right in your device browser, without having to spend time setting up various infrastructure pieces. Now, thanks to Bryntum Siesta, all you have to do is click the URL in your desktop messenger and click Run Test!
Resources
For full details on what changed please see the change log.
Ivan Jouikov is a Software Architect at IntelliSurvey. Ivan has been involved with Ext JS for almost a decade and is passionate about developing scalable, maintainable, and extensible front-end codebases.