Posts Tagged ‘Testing’

Make Tellurium / Selenium work with Firefox and Snow Leopard

Tuesday, September 28th, 2010

TDL’s been playing around with Tellerium / Selenium for functional web-based tests for a little while now. Unfortunately everyone who’s been messing around with it has been running them on Ubuntu and the others of us on the team have OS X. When you run the Tellerium tests on a Mac with Snow Leopard they fail to start Firefox with the following output from the Selenium server:

Preparing Firefox profile...
dyld: Library not loaded: /usr/lib/libsqlite3.dylib
Referenced from: /System/Library/Frameworks/Security.framework/Versions/A/
   Security
Reason: Incompatible library version: Security requires version 9.0.0 or 
   later, but libsqlite3.dylib provides version 1.0.0

What is Wrong?

The problem Firefox is complaining about is the version of libsqllite3.dylib found. Snow Leopard ships with a version of libsqllite in /usr/lib and Firefox also provides it’s own version of libsqllite3. Unfortunately there is a bug in Selenium 1.0.1 with how it calls Firefox that was patched in next version 1.0.2, however because of other bugs Tellerium is staying with 1.0.1 for now. You can read more about that decision in the mailing list thread:

http://www.mail-archive.com/tellurium-users@googlegroups.com/msg02149.html

The actual problem, at least from an outside view point, is dirt simple. When Selenium calls Firefox it sets up a set of environmental variables, one of which is:

DYLD_LIBRARY_PATH="null:/Applications/Firefox.app/Contents/MacOS"

As you can probably see there’s a problem with the path with the “null” that snuck in there. So somewhere in Selenium there’s some place where they forgot to check for a null value. When Firefox starts up it’s not able to find it’s local copy of libsqllight because of the invalid path arguments. But Firefox will happily guess a good set of paths if you forget to set the DYLD_LIBRARY_PATH, so a simple solution is to place a simple bash script in-between Selenium and Firefox that simple removes the corrupted path.

(more…)

DSpace Functional Tests?

Sunday, April 4th, 2010

The Texas Digital Library has been focusing on testability for our projects. Since DSpace is related too or part of most of our projects we’ve been looking for a way to increase DSpace’s testability. Traditionally this would mean adding unit tests and integration tests. However as DSpace currently stands is hard to break it up into individual components that can be tested in isolation. You’ll quickly find that writing tests for DSpace pull in the entire system, plus databases, and a file system. To address this problem we’ve created a simple framework for adding both integration tests and functional tests which improve the reliability of our projects. I’m interested to see if this is something the greater DSpace community would be interested in?

The goals of our project were to create a mechanism where we could run complete functional tests. Functional tests evaluate the entire system as the end user would use it, so think of it as opening a web browser and evaluating the output – but completely automated. They test everything all together. Ideal it would be better to test each component individual, but this is in practical for DSpace for two reasons 1) DSpace is highly integrated and nearly impossible to separate from the database and file systems, 2) Creating unit test for all of DSpace is very time consuming it is simpler to write a few functional tests that cover a wide set of features over the whole application. It gets you to a point where you can reliably verify the software quicker. If you’re working on unit tests for DSpace please do not let this stand in your way.

(more…)

Grails HTTP-based test plugins

Wednesday, November 18th, 2009

grails-logoThere are two leading frameworks for doing http-based integration tests within the Grails framework: functional-test and webtest. Both are packaged as grails plugins and accomplish the same task in a similar manner. The main library behind the scene is HtmlUnit. This is a well thought out library for abstracting web conversations, i.e get this url, check that it has a form, click the submit button. It’s basically a headless browser which even supports Javascript. HtmlUnit appears to have taken the mantle from HttpUnit as the premeir library in this small arena. There has only been one release since 2006 for HttpUnit while HtmlUnit has been very active with 11 releases in the same period.

Webtest is the more established project with its first release in 2007. Webtest is provided by Canoo as a free open source plugin to Grails. The tests are run via an Ant script that packages together the test cases and runs through each one.

Functional-test is a relatively new project with its first release in early 2009. Functional test is basically the same as webtests however instead of using the Ant-based scripts everything is based on Junit and implemented in plain old java or groovy.

We’ve chosen to proceed with using the Junit-based plugin for our HTTP integration tests. Our primary reason in deciding to use Junit is that it integrates well with our other unit-based tests which also use Junit instead of using disparate frameworks for the various types of tests. While this plugin may be newer it’s traffic on the mailing lists is growing and substantial.