SF Selenium Meetup Testing Tools Demo Night

Several months ago I downloaded a copy of Selenium to evaluate how we could use it to improve automated acceptance or functional testing at Spoke. It was quite an useful tool but we didn't have cycles for setting up some sort of infrastructure for running the tests. When I saw there was going to be an upcoming meetup, I thought it would be a good opportunity to try to get inspired on using Selenium again and also learn what is new in the world of Selenium. On April 20, 2010 I attended the San Francisco Selenium Meetup at Sauce Labs (the folks behind Selenium) where the topic of the evening was "Testing Tools Demo Night." The night included 4 speakers and demos and an opportunity to talk to the folks at Sauce Labs about their products. I even won 1000 minutes of Sauce OnDemand.The topics that were covered were two demos of frameworks that put a wrapper around Selenium tests to define test in an alternative closer-to-natural grammar. One demo of how Selenium is integrated into a continuous integration environment. Finally, a demo of an up coming Selenium tool to test Flash applications. The tool for testing Flash was impressive but I don't really have a use for it. Therefore, from my point of view, the most applicable demos were FitNesse and Robot Framework.

Both FitNesse and Robot Framework are automated and integrated (with Selenium) frameworks for acceptance level testing and "acceptance test-driven development" (ATDD). Each have their own simplified grammar for defining tests that map to actual Selenium tests. FitNesse is a Wiki based solution and Robot Framework is keyword-driven using text files to define the acceptance tests. The objective of both frameworks is to make it easier for the non-technical "customer" or product management team to write acceptance tests. According the speakers and some of the folks in the crowd, the reality is that this does not happen. These tools do, however, make it easier for the non-technical and technical teams collaborate on formulating the acceptance tests.

Almost just as important, More important than the speakers that night was the opportunity to talk to the folks at Sauce Labs. They are very friendly and are really seem to be interested in making you a successful user of Selenium or Sauce. They even offered to come to Spoke and talk abou the best practices for using Selenium or Sauce and I'm definitely taking them up on that offer.

Below are the notes that I took during the event. I used my iPad to take notes with Evernote. There seems to be a bug in Evernote on the iPad where some combination of saving, syncing, and opening a note causes your most recent revision to go away. So I lost some notes about the Flash testing tool.

Notes

* Announcements * Redfin is having a tech talk on April 30, 2010 blog.redfin.com

Next meetup is at Mozilla in Mountain View. Selenium and HUdson Continuous Build Integration. Linkedin Ruby-based page model something

* Selenium + FitNesse - A QA Multiplier Effect * Ray Vizzone and Matt Krapivner test labs.com

FitNesse is web based collaborative tool, write and execute tests in a wiki Extensible through fixtures Easily integrates into continuous build processes

Fixtures - maps a test API calls to a cusomter friends grammar Web testing withing selenium or others

Test cases written in a friendly grammar - less programming resources required, widens pool of test case writers Single code repository - reduces maintenance cost Uses Selenium - broardens browser compatibility activity versus task focused test cases - readable by customers and stakeholders

The FitNesse test cases look like plain English that runs the selenium code. "sign in with username and password" "add my picture (location) to existing album Test Album 2" ^neat

The demo starts With a page that look likes a wiki that describes the test suite. Uses a wiki like syntax and a grammar to describe the test

Started up a selenium grid or remote control. Works with Sauce OnDemand and will provide links to the video of the test.

How is Bromine different from FitNesse? They are both open source and might be similar.

Comment was that the promise of getting non-technical people testing is empty. The presenter said this is a step in the right direction. Wiki is still too difficult to non-technical people and not technical enough for other. Another comment was that FitNesse could be good to be used for the technical people and the non technical people to collaborate together to create the tests and the requirements. But handing it over to non-technical people doesn't work. (Tough crowd!) FitNess can be started from the command line and can therefore be used with any continuous integration including Cruise Control.

The FitNesse web page will update as the test is performed.

* Integrating Selenium into a continuous test harness * David Vydra, Tester from GuideWire

GuiideWire builds core systems for the insurance industry (except medical). Each company does something slightly different so had to build the system very flexible and extensible. They have their own IDE and language.

An intranet application.

Have an IDE screen for creating the user interface using a static language that executes dynamically (Scala?) Presenter says it is somewhat similar to Scala. The IDE and language seems more like a RAD.

The test seems to be written in the proprietary language with their own object model. Objects define various UI elements. Then it executes the Selenium tests. You can run it locally in the browser. Remotely on a container.

The language literally uses a lambda symbol.

The recorder can record a test or. You can use a more functional programming style to write more maintainable tests.

* Sneak preview of new Selenium capabilities *