Test Automation Planet

Testing is Overrated

2009-01-06T22:20:28+00:00

I stumbled across a presentation by Luke Francl called: Testing is Overrated. He could have called it: "programmer testing is overrated" but that doesn't have as much zing to it. His slides display a refreshing reminder to look for balance...

Getting closer...

2009-01-06T20:08:09+00:00

We've been working legal ropes for awhile to get this result: Sun Approved: Merge from BSD Java to OpenJDK. There was a funky little legal snarl in that the BSD Java team has been working on Java ports to xyzzyBSD for years but it was under the JRL or SCSL license and they couldn't directly offer their changes to the OpenJDK project. One little bit of interesting is the SoyLatte work by Landon Fuller, which was based on that very same BSD Java port. The legal unwrangling now allows Landon to post this: Sun has approved merging the JRL-licensed BSD Java to the GPLv2+ClassPath OpenJDK-6. That means that all of the BSD Java changes, including the SoyLatte Mac OS X port, can now be sent to the OpenJDK project, and an official BSD porters group can be proposed:

Cheers and 13949712720901ForOSX

But wait, that's not all... Dalibor noted a couple days ago that Openjdk 6 is in Debian Main... and our esteemed Linux strategist, Barton George notes OpenJDK 6 is heading into OpenSUSE 11. And there appears to be interest to include OpenJDK in Mandriva.

Great milestone reached by OpenJDK on Fedora

2009-01-06T20:08:09+00:00

Java is finally Free and Open .. hey, The Fedora/IcedTea team have reached a great milestone. Their build of the OpenJDK has passed the (extensive) JCK test suite. I bet they now have a huge appreciation the extent to which we took the slogan "Compatibility Matters!".

Since this inspired Rich to take a walk down memory lane... in May 2006 I was in the audience for the JavaOne keynote. I was real excited because a project I'd been working on intensely was about to be announced by Jonathan Schwartz and Mark Shuttleworth from the stage. This project, the Operating System Distributor License for Java (DLJ) was our latest hope to improve the state of Java on Linux by giving the world the JDK's rebundled with a more permissive license. It was going to usher in an age of nirvana and happiness, but it seems that Jonathan Schwartz and Rich Green had a different idea in mind. Jonathan and Mark did announce the DLJ project that day, and DLJ derived JDK's have been shipping in several Linux's and Solari since then. However Jonathan and Rich a few minutes after the DLJ announcement completely upstaged it with "It's not a matter of if but when" (or something like that).

Thus ensued a huge amount of work. The initial open sourcing happened on the first Java Freedom day, November 13, 2006. The second Java Freedom Day happened at JavaOne 2007 when the rest of the OpenJDK was released ... except... there was two issues, first that some code was encumbered and we weren't able to open source it right away. This made for a more complex build situation where replacement code was available as binary to plug in the holes left by the encumbered source. The second is we never got to open sourcing the plugin or java web start.

I see the slashdot thread, IcedTea's OpenJDK Passes Java Test Compatibility Kit, has it's usual set of mistakes and misconceptions so I'll be answering some of them as I go through the rest of this.

The IcedTea project formed almost immediately and initially its purpose was to reuse Classpath code to fill the gaps of the encumbered code. At some point in (?late?) 2007 they had a complete unencumbered IcedTea build derived from the OpenJDK7 source tree. At the same time in the OpenJDK team work was done to replace the encumbered code by various means. For some code we were able to get permission to release it under open source, in other cases we chose open source alternative implementations. For example a remaining encumbrance is the font rendering code, and in the OpenJDK we are using open source font rendering code (FreeType) but the closed source code is continuing to be used in the commercial JDK builds.

Performance .. the OpenJDK uses the exact same HotSpot as is used in regular JDK builds. Performance should be 99.9% the same. Since some of the GUI rendering code differs the GUI performance is likely different.

"Only 4% of the Sun Java code wasn't released. So IcedTea only had to implement the 4% of Java that wasn't GPLed." This is no longer true... the OpenJDK6 source tree is 100% unencumbered.

OpenJDK7? OpenJDK6? In the beginning there was OpenJDK, and it was pleasing to Richard Stallman and he smiled. However there was a promise made to have an open source implementation compatible with Java 6, and thus was OpenJDK6 born. OpenJDK6 is an example of making reverse progress by moving forward into the past ... OpenJDK6 is a fork of OpenJDK7 where features and code was removed until it could pass the JCK tests. If you look in the IcedTea project there are two branches, the one derived from OpenJDK7 and the one derived from OpenJDK6. It's the OpenJDK6 which the IcedTea/Fedora project have made pass the certification tests.

What's the big deal? This is Sun's own code base so of course it's going to pass the compatibility tests ???

Not so fast, as there are some differences from any other release which passed JCK6a. The last version to pass JCK6a was the Java6 release (which is up to Java 6u6). Remember that OpenJDK 6 started life as a fork of OpenJDK 7. While OpenJDK7 started from the JDK6 source tree ... some changes had been made. There was 20 builds worth of Java7 new feature development and bug fixes, and there was source tree rejigglings related to open sourcing. Further there were changed introduced by removing the encumbrances, and there were changes introduced by IcedTea/Fedora. This means there's a lot of ways a conformance failure could have been inserted.

Hey, great news and I hear there is some celebration at FUDCON. Wish I'd been able to attend.

Re-released Microsoft Compatibility Testing Virtual PC Images

2009-01-06T03:20:32+00:00

The Microsoft compatibility Virtual PC images have had a re-release. These time out on April 2009 (even when they time out you still get a grace 1 hour usage as I found out this morning). Windows XP SP3 with IE6 Windows XP SP2 with IE7 Windows XP SP3 with IE8 Beta 2 Vista Image with IE7 At no [...]

It will be (Open)JDK7 where OpenJDK==JDK

2009-01-05T22:48:15+00:00

I'm listening to the 'holidays 2008ish' episode of Javaposse and in reviewing their last years predictions they have enough fumbling around the status of OpenJDK that I want to do a little bit of explaining.

OpenJDK 6 != JDK 6

There are circuitous historical reasons for this but the fact is that OpenJDK 6, as good as it is, does not correspond to JDK 6uN for any value of N. OpenJDK 6 began as a fork of OpenJDK 7, and the team stripped out code from the OpenJDK 7 fork until it became compatible with the Java6 specification as verified by the JCK 6 test suite.

Joe Darcy has posted several blogs about this OpenJDK 6 Genealogy, OpenJDK 6 and 6u10 features, OpenJDK 6: Logistics of Partial Merge with 6u10, and the most important of these is: Forward to the Past: Toward OpenJDK 6

If you're scratching your head going "huh??", well, I don't blame you, just read Joe Darcy's blog postings and it will be clear as, well, uh, I just hope it's clear why.

The bottom line is that OpenJDK 6 works rather well, several Linux distros are using it as their JDK, it can be built to pass the JCK6b test suite, meaning OpenJDK 6 can be used to build a compliant JDK, and unfortunately it is this evolutionary sidetrack which we hope will be relatively short-lived. It serves a purpose, namely of having an fully open OpenJDK compliant with Java6 ....

OpenJDK ~=~ IcedTea

It seems some people really like typing "./configure" rather than setting up environment variables and running "make". The IcedTea project originally began due to incompleteness in the OpenJDK (gaps due to encumbrances) and the community requirement of a fully open source toolchain and code base. IcedTea has long been a set of patches applied to OpenJDK, along with a different build system that is "./configure" based as I just said.

In OpenJDK we've replaced the encumbered code such that there are no gaps any longer. As we've done so I believe the IcedTea project has shrunken the number of patches they use. One nice thing seems to be that the IcedTea configure script makes it easy to build OpenJDK in many different modes such as using the Zero Assembler port to support compiling on non-x86/sparc chipsets, etc.

A big piece IcedTea provides is a plugin/java-web-start infrastructure. We haven't been able to open source our plugin and of course for 6u10 we completely rewrote the plugin. It is hoped we will open source the new plugin into the OpenJDK project but to my knowledge that decision has not been made concrete set in stone.

I don't spend much of my time looking at IcedTea so don't be alarmed if I've missed something ...

OpenJDK 7 == JDK 7

Okay, given that there is not a Java7 JSR and all the JCP stuff related to that issue ... Hopefully all that will be straightened out and we do have a proper Java7 JSR, so for the following please assume that's the case, that Java7's JSR status is straightened out ... the plan is that beginning with OpenJDK7/JDK7 that the code bases will be nearly identical. It is obviously expensive to maintain a fork, and if JDK7 were to diverge strongly from OpenJDK7 it would do two things: a) be very expensive, b) undermine our efforts at an open source ecosystem.

But "nearly identical" does mean it's likely there will be some differences.

Remember those gaps due to encumbrances? Some of those gaps were simply code we couldn't open source as of May 2007 but have since done so, other gaps are for code we still have to gain approval on (e.g. SNMP), but there is some code where there are open source replacements where we still use the old closed source code. This is primarily in font and graphics rasterization. The old closed source rasterisation code, while being encumbered, has had 10+ years of bug fixing and fine tuning etc and, for any open source replacement to displace that code in productized JDK builds, it would have to be provably as fast and stable and good quality as the existing closed code.

What took so long?

If we had followed normal release patterns JDK7 would be shipping about now. That is, Java6 shipped in December 2006 and our normal pattern is 18-24 months between major releases meaning the JDK7 launch would have been 2-5 months ago. What happened? Well, clearly there was a resource crunch on several angles.

For example getting the nearly complete OpenJDK source release in May 2007 took a lot of effort from lots of people. But there was also certain announcements at JavaONE in May 2007 (a little thing named JavaFX) which also turned into a major rethink of Java (as some have said, this ain't yer Dad's Java) and also a major piece of work to accomplish.

In other words instead of producing JDK7 we did JDK6u10 and JavaFX.

How not to behave as a test contractor

2009-01-05T21:38:08+00:00

Sometime back, Linda Wilkinson had a good post on Test Contractor behaviour I shall add a few more 'things not to do' to the list based on some recent (and slightly less recent) experiences with contract testing staff. A few things to bear in mind before we start. When you contract as a tester you [...]

Testability and Reproducibility

2009-01-05T13:23:29+00:00

Doug Szabo has asked me two interesting questions, and we thought we would share my answers.

Doug: Can you point me to some guiding literature that explains how to make code testable?

Explanation: I noticed at least one mention in Perfect Software: and Other Illusions about Testing of making code testable. I have seen that language in a couple of other software testing books, including Software Testing Techniques. I have even been told by more than one developer that they needed to "make the code more testable" before they wanted me to start testing. Since the developers who told me this didn't have the faintest clue what testing was about, I really don't know what they intended to do with their code. Apparently, neither did they - I eventually asked, "what do you need to do to make it more testable?", to which they replied that they didn't know. I was and still am confounded by the lack of explanation for what it is that makes code testable. I understand when code is very difficult to test, like when you have a "horrible loop" (Software Testing Techniques), or when multithreaded code is not first tested in single thread contexts, but I don't understand what needs to be done to code to make it testable. Are we talking about instrumenting the code with Debug statements, assert statements, or some symbol that a test tool can detect?
Jerry: I share your dismay at the lack of publications about how to make code testable. There are many, many little techniques (like initialization on entry, not exit; eliminating as many special cases as possible; and general simplification for a reader), but they're not as important as three things:

1. All code should be open code that anyone in the project can read and critique.

2. All code must be reviewed in a technical review (see my The Handbook of Walthroughs, Inspections, and Technical Reviews)--in which at one professional tester is present and fully participating.

3. Same as 2, except for design reviews and requirements reviews).

If you do these thing, the organization will quickly learn how to make code testable.

But, yes, someone should write the book and start with these three things.

Q2: Do you have some strategies for triaging bugs that do not reproduce consistently?

Explanation: I was a developer before my current role of tester. Hey. Don't roll your eyes at me. I was an engineer (a real one, with a professional license and all that hoopla) before I became a developer. So I already knew the value of testing, even if I didn't know what software testing really was. Well, as an engineer it was crucial that tests be designed such that the results would be reproducible, and measurable against a control. When I got into software development, I tried to stick to the engineering principle with respect to testing. Unfortunately, as I worked on larger software projects, particularly those where my code talked to other processes, and also where I had multiple threads running, I found that bugs were starting to occur where steps to reproduce did not consistently reproduce the bug. Oh oh. I knew that meant there was something wrong, I just didn't have something that I could run under a debugger and know where to set a breakpoint. As a developer, it seemed like there were always enough reproducible bugs that I had lots of excuses to avoid trying to solve those that might not reproduce. Now, as a tester, I am empathetic to developers and have a self-imposed guideline to make an entry of any non-reproducible issue, but at the same time I don't assign it to the pool of issues to fix until a set of steps to consistently reproduce is found. What I got out of Perfect Software is that perhaps I should be passing the tough to reproduce issues over to the pool to fix, but then what would you recommend for a triage approach, to convince stakeholders to take those issues as seriously as the ones that do consistently reproduce?

Jerry: Great question, Doug.

First answer. Try changing "not reproducible" to "I am unable to reproduce."

Then parse the second category into sub-categories like:

a. I saw this one, but was never able to make it happen again.

b. I see this when I run with this setup, but sometimes I don't.

c. This is the same anomaly I see under several setups, but not all the time.

d. Under X setup, this happens sometimes and not other times. There may be something different from one X to the other, but I' not seeing it.

In each case, you are unable to pinpoint the fault underlying the failure. There may be several faults producing the same failure, or different failures appearing from the same underlying fault. Since you don't really know the frequency of this failure, the way I use to triage the failure is to apply the question: "What would it cost us every time this failure appears in the field?"

If that number is high, then get a team working on the bug. If it's low, let the bug rest while (perhaps) new data accumulates or the bug disappears mysteriously as other changes are made to the code.

Code Coverage and the Zune Bug

2009-01-05T10:20:30+00:00

Many of you have probably heard of the Zune bug and some of you may have seen the source code of the bug. Specifically, the bug resides in this:

BOOL ConvertDays(UINT32 days, SYSTEMTIME* lpTime) { int dayofweek, month, year; UINT8 *month_tab; //Calculate current day of the week dayofweek = GetDayOfWeek(days); year = ORIGINYEAR; while (days > 365) { if (IsLeapYear(year)) { if (days > 366) { days -= 366; year += 1; } } else { days -= 365; year += 1; } } // Determine whether it is a leap year month_tab = (UINT8 *)((IsLeapYear(year))? monthtable_leap : monthtable); for (month=0; month<12; month++) { if (days <= month_tab[month]) break; days -= month_tab[month]; } month += 1; lpTime->wDay = days; lpTime->wDayOfWeek = dayofweek; lpTime->wMonth = month; lpTime->wYear = year; return TRUE; }

Regardless of whether or not you know this programming language (C), you should be able to see what caused the Zune player to freeze. In fact, it's a common code smell: the arrow anti-pattern.

Still don't see it? Take a close look at that while/if/if loop. Still don't see it? Well, there was some discussion of this code over at Reddit. There was some talk about whether or not it would be possible to find this bug automatically and that's really an interesting topic. Some type inference systems can catch infinite loops (the paper that Mark Jason Dominus drew on for his Strong Typing and Perl talk). However, I doubt such inference could catch this bug.

Using type systems to find infinite loop bugs requires that you have a sound type system for your language ("sound type systems" has a very specific meaning), something Perl 5 does not have (Perl 6 won't, either). Microsoft has Terminator, software which looks for infinite loops and, while admitting it can't handle everything, can find many infinite loop bugs. But Terminator probably isn't going to work for us -- it appears to be geared towards C programs and in any event, dynamic languages cause all sorts of havoc with many analysis tools.

So what are we to do? Given that the code above is probably a small sample in many hundreds of thousands of lines of code, it's quite understandable how that could be missed. But can we avoid that?

Well, yeah, actually. It wouldn't be fully automatic, but the simple answer is more than just "write tests for you software". It's "use code coverage tools". Rewriting the problem in Perl (and reformatting to avoid the excessive horizontal whitespace):

1: while ( $days > 365 ){ 2: if ( is_leap_year($year) ) { 3: if ( $days > 366 ) { 4: $days -= 366; 5: $year += 1; 6: } 7: } 8: else { 9: $days -= 365; 10: $year += 1; 11: } 12: }

Now does the problem stand out? What happens if $day == 366? Now we have a problem, but code coverage would have caught that.

Specifically, "conditional coverage" would have caught that the condition in line 3 was never false. Then, a sharp programmer could examine the coverage report and for those conditions which are easy to test, test them! It doesn't guarantee that your software is bug free, but it would have helped Microsoft avoid an embarrassing PR fiasco like this one.

Buccaneer-Scholar Site is Up

2009-01-05T02:20:38+00:00

My book “Secrets of a Buccaneer-Scholar” will be published in September by Simon and Schuster. It’s a book about how I approach self-education, personal branding, and original thinking. It’s the story of how I got started and how I got along all these years without any formal education or certification. It’s not a book about testing, but it will be highly relevant to testers. It will be relevant to any knowledge worker who wants to compete better in a down economy.

The book is finished and delivered, it just takes a long time to get through production and sell into book stores. While I’m waiting and writing the next buccaneer book in the series, I’ve started a new site at buccaneerscholar.com and a new blog. There I will talk about the more general topics of thinking and learning that are not specific to testing.

I’m also creating a class to teach self-education (not as ironic as it sounds… but it does sound ironic, eh?).

This blog will continue to deal exclusively with testing and issues directly related to the software development industry.

The Great Implication of Context-Driven Methodology

2009-01-05T00:31:16+00:00

Sometimes I hear people react to context-driven methodology with a shrug. “Yeah, everything depends on context. So what?”

Here’s the so what: If all practice depends upon context, then the competent practitioner must know how to invent, evaluate, criticize, and modify practices. In other words, focus must shift from merely copying and memorizing practices to developing skill in the craft.This is a profoundly different focus.

Wouldn’t it be absurd for someone to claim that the best way to commute to work, for everyone in the software industry, is to take a horse and buggy? Wouldn’t it still be absurd even if they added the words “but you should take context into account and do what’s right for you” at the end of such a strange claim? When consultants talk to people from different practical contexts, and they treat context variables as a trivial concern, they do their audience little service. That’s the trouble with “best practices.”

To say that context matters but that “almost always X is the right thing to do” is to speak like a parent dictating to a child. I fear that many testing bloggers and consultants are happy to do just that. But if what we want is a meeting of minds and sharing of wisdom, we need to practice talking about context-dynamics and the dynamics of practices that serve those contexts.

The most recent time I practiced this, myself, was in writing the previous paragraph. I almost wrote that you should not, as a methodologist, speak to your clients as a parent dictating to children. But then I noticed that very statement is just such a dictum. I don’t want to be like that. So I had to add “But if what we want is…” to at least acknowledge that even here there are different ways to be, depending on context.

A Suggestion

A great way to automatically avoid the perils of best practice talk is simply to talk about your own experiences and preferences, rather than make general prescriptions. This is why, in peer conferences such as LAWST, we focus on experience reports (actually we called them “war stories” until our terminology was attacked by bloodthirsty pacifists) rather than advertisments for good practice.

WPF Application Quality Guide

2009-01-04T21:38:38+00:00

Microsoft have released v0.4 of the WPF Application Quality Guide

Introducing “WPF Application Quality Guide v.0.4″

In light of the rapid adoption of the WPF and the continuous requests from partners and customers, the WPF team at Microsoft is happy to present the fourth preliminary release of the “WPF Application Quality Guide”. We plan to release the Guide in stages, updating and fine-tuning the content, based on feedback provided by you. This fourth release contains a updated reading roadmap that helps guide you through the sections based on your needs, data-driven testing, stress and stability testing, accessibility testing, updated globalization and localization testing, as well as TestApi in tools section.

Happy New Year and....Teach Me Web Testing!

2009-01-04T14:10:27+00:00

Happy New Year everybody! I hope 2009 won't rush by as quickly as 2008 did...

Now for the 'Teach Me Web Testing' part: Steve Holden graciously offered to be the host of an Open Space at PyCon 2009 on this topic. Steve started the 'Teach Me...' series at the last PyCon, with his now famous 'Teach Me Twisted' session.

For this format to work, we need to put together an audience which is formed of at least 3 types of people:

1) people interested in learning about Web testing in Python
2) people who write Python Web testing tools for fun and profit
3) people who use Python Web testing tools extensively for fun and profit

My role here is to rally people in categories 2 and 3. So if you're either a Web testing tool author or somebody who uses Web testing tools extensively in your job, please either comment on this post, or send me email at grig at gheorghiu dot net and let me know if you'd be interested in attending this Open Space session. Knowing Steve, I can guarantee it will be LOTS of fun.

Credo

2009-01-04T12:07:40+00:00

If you want to understand what context-driven testing is all about, James Bach and Cem Kaner have recently collaborated on a statement of what it is, and a lengthy explication. It's posted on Cem Kaner's blog at http://www.satisfice.com/kaner. The posting covers the territory admirably, and I wholeheartedly agree with it.

In particular, the statement emphasizes that adaptation to the needs of the project is the first step in the context-driven approach. This makes context-driven distinct from other approaches that may acknowledge the importance of context, but make something else—a standards focus, iterative development and unit tests, "best practices", a particular process model—paramount.

Context-Driven Testing Clarified

2009-01-04T07:24:17+00:00

Cem Kaner has just posted a clarification of context-driven testing. I contributed to it, and it also represents my point of view.

One reason we’ve come up with this new description is that several people (we don’t really know who) have posted or copied descriptions of context-driven testing that bear little resemblance to the vision of the founders of the school. These descriptions typically claim that context-driven is a flavor of Agile. No, no, no, no, no, it isn’t. If context-driven testing is a flavor of anything, it’s a flavor of problem-solving.

What is context-driven testing?

2009-01-04T04:20:08+00:00

James, Bret and I published our definition of context-driven testing at http://www.context-driven-testing.com/. Some people have found the definition too complex and have tried to simplify it, attempting to equate the approach with Agile development or Agile testing, or with the exploratory style of software testing. Here's another crack at a definition: Context-driven testers choose their testing objectives, techniques, and deliverables (including test documentation) by looking first to the details of the specific situation, including the desires of the stakeholders who commissioned the testing. The essence of context-driven testing is project-appropriate application of skill and judgment. The Context-Driven School of testing places this approach to testing within a humanistic social and ethical framework. Ultimately, context-driven testing is about doing the best we can with what we get. Rather than trying to apply "best practices," we accept that very different practices (even different definitions of common testing terms) will work best under different circumstances.

"Don't Shout at your Disks" and Performance Testing

2009-01-02T19:07:32+00:00

I get asked about performance and load testing a great deal. It's fascinating work, and sometimes the strangest causes of performance problems can occur in a system. When people ask me about what goes into performance and load testing, it's...

A Little Parrot Task for the Interested

2009-01-02T07:20:15+00:00

If you have some spare time in the next couple of weeks, the Parrot TODO/skip test review is an embarrassingly parallel milestone which is suitable for anyone who can read code written in Perl 5 and skim code written in PIR.

All you need to do is pick a test file not listed on that page which contains either TODO or SKIP (capitalization does not matter) and list it on the page, with a count of the affected tests and a marker whether there's an RT or TT referenced in the reason.

That's it.

You can mail the Parrot developer list directly or tell someone in #parrot on irc.perl.org.

Reviewing one test file is a help. Reviewing more is wonderful. If you have five minutes, you can do it!

New Year's Resolutions

2009-01-01T12:02:03+00:00

I already had one a couple of month ago Shana Tova - New Year's resolution so now a few other items.

Keep writing Padre, the perl IDE.
Offer the Test Automation using Perl 5 training class in Europe.
Send more articles to the Test Automation Tips.
Prepare a one-day training class for Perl 6 and start offering it.
But most urgently, help preparing the annual accounting papers on time

Refactoring Finds Dead Code

2008-12-31T21:08:55+00:00

One of the many things that I just love about my job as a consultant/mentor is when I actually get to sit down with programmers and pair program with them. This doesn’t seem to happen nearly as often as I would like, so when two developers at a recent client site asked me if I could look at some legacy code to see if I could figure out how to get some tests around it, I jumped at the opportunity. We acquired a room equipped with a projector and a whiteboard. A laptop was connected to the projector and we were all able to comfortably view the code.

I visit a lot of different companies and see a lot of absolutely ghastly code. What I was looking at here wasn’t all that bad. Variable names were not chosen to limit keystrokes and method names appeared to be descriptive. This was good news, as I needed to understand how this was put together before I could offer help with a testing strategy.

As we walked through the code, I noticed that there were classes in the project directory ending in ‘Test’. This took me by surprise. Usually when I’m helping people with legacy code issues, there aren’t any tests. Here, there were tests in place and they actually passed. Very cool, but now my mission wasn’t clear to me as I thought my help was needed getting tests in place around legacy code.

The developers clarified that they wanted help in testing private methods. Ah ha, the plot thickens.

The question of testing private methods comes up frequently whether I’m teaching a class or consulting on a project. My first response is a question. “Is the private method being tested through the public interface to the class?” If that’s the case, then there’s nothing to worry about and I can steer the conversation away from testing private methods to testing behaviors of a class instead of trying to test individual methods. Note that a private method being tested through its public interface would be guaranteed if the class was developed TDD style where the test is written first, followed by one or more public methods to make the test pass, followed by one or more extract method refactorings, which would be the birth of the private methods. This is almost never the case. My client didn’t know how the code was developed, but by inspection they concluded that the parameters of the test were adequately exercising the private method.

It looked like my work was done here. But not so fast.

I have a policy that says that whenever I have code up in an editor, I have to try to leave it just a little bit better than when I found it. Since we had put the testing matter to rest and we still had some time left in the conference room before another meeting started, I suggested that we see if we could make some small improvements to the code we were examining.

As I said earlier, the code wasn’t horrible. The code smell going past my nostrils was Long Method and the cure was Extract Method.

The overall structure of the method we were examining was

    
    if( conditional_1 )
    {
        // do lots of complicated stuff
    }
    else if( conditional_2 )
    {
        // do even more complicated stuff
    }
    else
    {
        // do stuff so complicated nobody understood it
    }

where conditional_1 was some horribly convoluted expression involving lots of &&’s, ||’s, and parentheses. Same for condition_2, which also had a few ^’s thrown in for good luck. To understand what the method did, one would have to first understand the details of how the method did it.

I asked the developers if they could come up with a nice, descriptive method name that described what I’m calling condition_1 so that we could do an extract method refactoring and the code would look like:

    
    if( descriptive_name() )
    {
        // do lots of complicated stuff
    }
    // etc

Now there were less details to understand when trying to determine what this method did. If we were to stop here and check in the code, we could leave the session feeling good as the code is better than when we started. But we still had time before we had to abandon our conference room so I pressed on.

“Can you summarize what this code does as a descriptive method name,” I asked. The developers pondered a few moments and came up with what they felt was a good name. Excellent. We did the same procedure for the “else if” clause. When we finished that, one of the developers said something along the lines of, “That was the easy part, I have no idea what this [else] remaining code does.” I was going to pat everybody on the back and call it a day because the code had improved tremendously from when we started, but the developers seemed to have a “we can’t stop now” attitude. They studied the code, pondered, cursed, discussed some more, and then one of them said, “This code can never execute!”

I’d like to use the expression, “You could have heard a pin drop,” to describe the silence in the room, but since there were only three of us, the phrase looses its power. As it turns out, now that the if() and else if() conditionals were given descriptive names and people could grok them at a glance, it became obvious that the business rules didn’t permit the final else – the first two conditions were all that could exist and the most complicated code of all was never being called. This was downright comical!

I asked if the tests would still pass if we deleted the code and after a short examination of the tests, the developers weren’t as confident that the test parameters actually hit that area of code. There was a log() statement in that code and one of the developers was going to examine the production logs to see if the code ever executed.

So there you have it, refactor your code and the bad parts just go away!

Fudge anyone?

2008-12-31T16:33:49+00:00

Back in September, when I was just staring the Slim project, I made a crucial architectural decision. I made it dead wrong. And then life turned to fudge…

The issue was simple. Slim tables are, well, tables just like Fit tables are. How should I parse these tables? Fit parses HTML. FitNesse parses wiki text into HTML and delivers it to Fit. Where should Slim fit in all of this?

Keep in mind that Fit must parse HTML since it lives on the far side of the FitNesse/SUT boundary. Fit doesn’t have access to wiki text. Slim tables, on the other hand, live on the near side of the FitNesse/SUT boundary, and so have full access to wiki text, and the built in parsers that parse that text.

So it seemed to me that I had two options.

Follow the example of Fit and parse wiki text into HTML, and then have the Slim Tables parse the HTML in order to process the tables.
Take advantage of the built in wiki text parser inside of FitNesse and bypass HTML altogether.

I chose the latter of the two because the Parsing system of FitNesse is trivial to use. You just hand it a string of wiki text, and it hands you a nice little parse tree of wiki widgets. All I had to do was walk that parse three and process my tables. Voila!

This worked great! In a matter of hours I was making significant progress on processing Slim decision tables. Instead of worrying about parsing HTML and building my own parse tree, I could focus on the problems of translating tables into Slim directives and then using the return values from Slim to colorize the table.

Generating html was no problem since that’s what FitNesse does anyway. All I had to do was modify the elements of the parse tree and then simply tell the tree to convert itself to HTML. What a dream.

Or so it seemed. Although things started well, progress started to slow before the week was out. The problem was that the FitNesse parser is tuned to the esoteric needs of FitNesse. The parser makes choices that are perfectly fine if your goal is to generate HTML and pass it to Fit, but that aren’t quite so nice when you’re goal is to use the parse tree to process Slim tables. As a simple example, consider the problem of literals.

In FitNesse, any camel case phrase fits the pattern of a wiki word and will be turned into a wiki link. Sometimes, though, you want to use a camel case phrase and you don’t want it converted to a link. In that case you surround the phrase with literal marks as follows: !- FitNesse-!. Anything between literal marks is simply ignored by the parser and passed through to the end.

Indeed, things inside of literals are not even escaped for html! If you put hi into a wiki page, it will escape the text you’ll see hi on the screen instead of a bold “hi”. On the other hand, if you put !- hi-! on a page, then the HTML is left unescaped and a boldfaced “hi” will appear on the screen.

I’m telling you all of this because the devil is in the details—so bear with me a bit longer.

You know how the C and C++ languages have a preprocessor? This preprocessor handles all the #include and #define statements and then hands the resulting text off to the true compiler. Well, the wiki parser works the same way! Literals and !define variables are processed first, by a different pass of the parser. Then all the rest of the wiki widgets are parsed by the true parser. The reason that we had to do this is even more devilishly detailed; so I’ll spare you. Suffice it to say that the reasons we need that preprocessor are similar to the reasons that C and C++ need it.

What does the preprocessor do with a literal? It converts it into a special widget. That widget looks like this: !lit?23? What does that mean? It means replace me with the contents of literal #23. You see, when FitNesse sees !- FitNesse-! it replaces it with !lit?nn? and squirrels FitNesse away in literal slot nn. During the second pass, that !lit?nn? is replaced with the contents of literal slot nn. Simple, right?

OK, now back to SLIM table processing. There I was, in Norway, teaching a class during the day and coding Slim at night, and everything was going just great. And then, during one of my tests, I happened to put a literal into one of the test tables. This is perfectly normal, I didn’t want some camel case phrase turned into a link. But this perfectly normal gesture made a unit test fail for a very strange reason. I got this wonderful message from junit: expected MyPage but was !lit?3?

I knew exactly what this meant. It meant that the value MyPage had been squirreled away by the first pass of the parser. I also knew that I had utterly no reasonable way of getting at it. So I did the only thing any sane programmer would do. I wrote my own preprocessor and used it instead of the standard one. This was “safe” since in the end I simply reconvert all the colorized tables back into wiki text and let the normal parser render it into HTML.

It was a bit of work, but I got it done at one am on a cold Norwegian night. Tests passing, everything great!

Ah, but no. By writing my own preprocessor, I broke the !define variable processing – subtly. And when I found and fixed that I had re-broken literal processing – subtly.

If you were following my tweets at the time you saw me twitter ever more frantically about literals. It was the proverbial ball of Mercury. Every time I put my thumb on it¹ it would squirt away and take some new form.

I fell into a trap. I call it the fudge trap. It goes like this:

forever do {“I can make this work! Just one more little fudge right here!”}

I was in this trap for two months! I was making progress, and getting lots of new features to work, but I was also running into strange little quirks and unexpected bizarre behaviors caused by the fudging I was doing. So I’d fudge a little more and then keep working. But each little fudge added to the next until, eventually, I had a really nasty house of cards (or rather: pile of fudge) ready to topple every time I touched anything else. I started to fear my own code². It was time to stop!

I knew what I had to do. I had to go back to my original architectural decision and make it the other way. There was no escape from this. The FitNesse parser was too coupled to the wiki-ness, and there was no sane way to repurpose it for test table processing.

I dreaded this. It was such a big change. I had built so much code in my house of fudge. All of it would have to be changed or deleted. And, worse, I needed to write an HTML parser.

I was lamenting to Micah about this one day in late November, and he said: “Dad, there are HTML parsers out there you know.”.

Uh…

So I went to google and typed Html Parser. Duh. There they were. Lots and lots of shiny HTML parsers free for the using.

I picked one and started to fiddle with it. It was easy to use.

Now comes the good part. I had not been a complete dolt. Even when I was using the FitNesse parse tree, I ate my own dogfood and wrapped it in an abstract interface. No part of the Slim Table processing code actually knew that it was dealing with a FitNesse parse tree. It simply used the abstract interface to get its work done.

That meant that pulling out the wiki parser and putting in the HTML parser was a matter of re-implementing the abstract interface with the output of the new parser (which happened to be another parse tree!). This took me about a day.

There came a magic moment when I had both the wiki text version of the parser, and the HTML version of the parser working. I could switch back and forth between the two by changing one word in one module. When I got all my tests passing with both parsers, I knew I was done. And then the fun really began!

I deleted ever stitch of that wiki parse tree pile of fudge. I tore it loose and rent it asunder. It was gone, never to darken my door with it’s fudge again.

It took me a day. A day. And the result is 400 fewer lines of code, and a set of Slim tables that actually work the way they are supposed to.

Moral #1: “Fudge tastes good while you are eating it, but it makes you fat, slow, and dumb.”

Moral #2: “Eat the damned dog food. It’ll save your posterior from your own maladroit decisions.

¹ I do not recommend that you actually put your thumb on any Mercury. Never mind that I used to play with liquid Mercury as a child, sloshing it around from hand to hand and endlessly stirring it with my finger. Wiser heads than I have determined that merely being in the same room with liquid Mercury can cause severe brain damage, genetic corruption, and birth defects in your children, grandchildren, and pets.

² Fearing your own code is an indicator that you are headed for ruin. This fear is followed by self-loathing, project-loathing, career-loathing, divorce, infanticide, and finally chicken farming.

Tests, Technical Debt, and Christmas Holidays

2008-12-31T11:31:11+00:00

I'm just about done with my major cleanup of how our files are organized in our system. So far the tests pass (and the Heisenbug has mysteriously gone away). When I'm done, I'll have moved 903 files (a combinations of 403 XML test fixtures and 490 Perl libraries) and have edited almost 1000 files. It's taken almost a week to get this done and the only thing which has made it taken this long is that the full test suite takes over an hour to run right now :/

Still, having those tests means that I'm able to make a huge, sweeping reorganization of our system with little fear. All xml fixtures are now organized into three simple directories (rather than the multiple ones they were randomly scattered across) and all of our core class have had their class names changed to be cleaner. So why on earth would I ever again want to work for a company which doesn't test? I can't think of any reason, other than being brought in specifically to build their test system.

In other news, the latest Parrot doesn't pass its test on my system. Bug reports might be forthcoming.

Update: I just needed a fresh checkout of Parrot. I must have broken something internally.

PHPUnit 3.4.0 - Alpha 1

2008-12-31T09:10:00+00:00

In an effort to end the year on a high note, I have released a first preview release of PHPUnit 3.4 today.

The highlights of this new release series include:

Support for optionally executing each test using a separate PHP process
Support for exploiting dependencies between tests to improve defect localization
Support for mocking SOAP web services based on their WSDL specification
A rewrite of the code generator that is used by the mock object system

The complete ChangeLog is also available.

PHPUnit 3.4.0alpha1 is not feature-complete and should not be used in production.

Please have a look at the new features and test whether this new version of PHPUnit can run your test suite without problems.

A Wish List for the Next Mainstream Programming Language

2008-12-29T21:21:33+00:00

It’s been fun watching the reactions to new features in C# 4.0. Some people love them. Others wonder, legitimately, where it is all going to end. The argument for feature addition is simple: over time we find better ways of doing things and our languages, as tools, should allow us that flexibility. The only downside to this strategy is that you end up with sprawling, complex languages over time – you never get to revisit the foundations.

Fortunately, however, people design new languages all the time and some of them do eventually enter the mainstream. We get a chance to start over and address foundational problems. And, that’s nice because we can do better than Java and C# for mainstream development and I don’t think there is any way to mutate either language into a better foundation.

Before I launch into the wish list, however, I want to set the context.

When I say “mainstream language” I am talking about languages which are in the Java/C#/VB market space – languages which are light on rocket science, seen as suitable for large-scale development, and don’t scare people. So, I’m not going to suggest dynamic typing or (on the other side of the coin) tight Hindley-Milner type systems and lazy evaluation. I love those approaches and I’m happy (in particular) that Ruby is gaining widespread acceptance, but I’m not going to fight that fight here. In the immediate future, for whatever reason, there will be development shops which feel much more comfortable with traditional static typing – the kind found in Java, C#, and VB. Given that, the question becomes: what can we do to make that sort of language better?

Here’s my list:

Immutability by Default – Over the past few years, a rather broad consensus has emerged around the idea that code is easier to understand and maintain when it has less mutable state. This isn’t a new idea; it’s been around for as long as functional programming, but our recent concerns with concurrency and our move toward multi-core computing just underscore the state problem. A mainstream language should, at the very least, make mutable data something special. References should be immutable by default and mutable state should be marked by a special keyword so that its use leaps out at you. It’s too late for the current crop of languages to make such a pervasive change, but the next mainstream language could.
Support for Components – In large-scale development, teams have to manage usage and dependency across an organization. The notions of public, protected, and private are too coarse as protection mechanisms, and we really need scopes larger than class, assembly, and package. In the end, this sort of protection is a social issue, so we should have mechanisms which make use very easy within a team (3-10 people working together) and somewhat more manageable between teams. It’s odd that language-based support for this work stopped with Ada and the Modula family of languages. Java’s recent move toward support for modules seems to be an exception.
Support for Scoped and Explicit Metaprogramming – In the past, language designers avoided adding meta-programming support to their languages because they were scared it would be abused. However, we’ve learned that without meta-programming support, people create yet another distinctive type of mess. If there is a middle ground for mainstream languages it probably involves scoping the use of metaprogramming and making it far more detectable. If, for instance, all of the code which modifies a given component had to be registered with some easily locatable component-specific construct, maintenance would be much easier.
Support for Testing – This one is only a matter of time, I think. In the last 10 years we’ve seen an explosion of mocking tools and testing frameworks. It’s not clear to me that we’ve reached any sort of consensus yet, but I suspect that at the very least we could add constructs to a language which make mocking and stubbing much easier. It’s also about time that languages attempt to solve the problems that dependency injection addresses.
Imposed I/O Separation – This is the controversial one. The more I work with Haskell, the more I notice that there is a beneficial discipline that comes from factoring your program so that its points of contact with the outside world can not be mixed with the pieces doing the work. When you first start to work that way, it feels like a straitjacket, but I think the benefit is apparent to anyone who has had to go on a spelunking expedition in an application to find and mock parts of the system which touch the outside world. Is that discipline too much for a mainstream language? I hope not.

So, that’s my list. There is no “grand language planning board” which decides these things. We will move forward chaotically like ever other industry, but I do hope that some of these features make it into the next mainstream programming language.

Syntax highlighting for Perl 6

2008-12-29T17:17:39+00:00

Following the Grant accepted for Integrating Padre with Parrot and Rakudo and Plans for Integrating Padre with Parrot and Rakudo I can already let you know that the first steps has been made. Padre is already capable to provide correct albeit slow syntax highlighting for Perl 6.

Actually it was implemented by Ahmad M. Zawawi, a neighbor of mine, who is also the maintainer of Syntax::Highlight::Perl6.

That's good news. I have not been playing with Perl 6 in many months now as some time ago I decided that I'll start to write Perl 6 again when I can get syntax highlighting in Padre and when I can run my Perl 6 scripts from within Padre.

That time has come so I've started to write some Perl 6 code but I need the documentation to be handy. So we are now discussing it on the #padre IRC channel on how to intgerate Padre with the existing Perl 6 documentation and how to provide a real-world use case.

I would like to be able to press on F1 and get an explanation of the element where my cursor is currently located. I'd like to get the calltips directly from the Perl 6 documentation. As the Perl 6 documentation can be linked to the tests implementing the specific elements using the smartlinks in the test files we could open the specific test files for editing right from the documentation.

Bring Your Own Ruler Internet Measuring Contest

2008-12-27T10:41:11+00:00

Freshmeat indicates that, relative to Python and other languages, Perl isn't seeing as much code released.

-- David N. Welton, Python "Surpasses" Perl?

Given that CPAN and perl.org provides announcement services, project history, bug tracking, documentation hosting, smoke testing, and download mirroring and all you have to do is upload a tarball to PAUSE, I wonder instead why a Perl developer would make an announcement on Freshmeat. I could likewise prove that there are no good writers on the Internet by surveying popular fan fiction sites, seeing as how that's exactly where they don't hang out.

I enjoy a good rant as much as anyone, but if you're going to ruin it with research, at least make a puzzle out of how you propped up your conclusions.

DB2 on Mac officially released

2008-12-24T18:43:47+00:00

As pre-announced in my two previous posts, DB2 for Mac OS X Leopard is finally available for download. It’s now official, DB2 on Mac is here.

Reflections on DB2 on Mac

Several people, including myself, would happily ditch their virtual machines and start introducing DB2 into their native Mac development stacks. But this milestone represents much more than the immediate implications would have us believe. A few years ago, the idea of giving away DB2 for free would have been met with rejection. Yet, DB2 Express-C came along, and unlike the other “express” databases, it’s a true production-ready DB2 version that can be used free of charge.

Likewise, the idea of having a DB2 version for Mac was unthinkable up to a few years ago. Yet today we finally have a copy of DB2 Express-C for Mac OS X that’s available for download. Aside from this being an acknowledgment of the growing importance of Mac as a development and business platform, I feel it underlines IBM’s ability to change. The desire that a few of us mac addicts had, coupled with reasonable pressure from the community, was sufficient enough to make DB2 on Mac a reality. This matters and appeals to both the developer and the technical evangelist in me.

In the list of downloads, you’ll notice that the Mac download is only 138 MB, versus the 412 MB of Linux’s 64-bit. The reason for this difference is that DB2 Express-C for Mac currently ships in English only, and at this stage it doesn’t include either DB2 Text Search or the Java based tools like the DB2 Control Center. This lighter package is, in my opinion, a welcome side effect of this brand new beta release.

Getting started with DB2 and Rails on Mac OS X

Since the first download went live on Friday, a newer release that includes a guide for installing DB2 on Mac OS X was published and it incorporates a few changes that will make the lives of developers easier, as they approach building and using drivers (e.g. the ibm_db Ruby gem). If you downloaded this beta version over the weekend, do not worry: just grab - and execute - this shell script (e.g. sudo fixlib.sh). If you are downloading DB2 on Mac now, you won’t need this script of course.

Once you’ve downloaded DB2 for Mac OS X Leopard, please proceed to read this PDF guide, which will tell you everything you need to know (and more) about installing DB2 on your Mac, as well as providing extra details. It’s best not to skip over reading this document, as the installation on Mac OS X requires a few more steps than simply running the setup wizard.

With DB2 installed and started (db2start), and the SAMPLE database created (db2sampl), you’re ready to start playing with this power horse. For details about SAMPLE’s structure you can read this article in the InfoCenter.

To run the DB2 console (known as the Command Line Processor or CLP for short), run:

$ db2

To connect to the SAMPLE database, from within the CLP run:

db2 => connect to sample

Unless you get an error, you should now be ready to query the database. For example, run the following query:

db2 => select count(*) from staff

Then to exit from the CLP, simply run:

db2 => quit

If this sanity test worked well you can proceed with installing the ibm_db gem (which includes the Ruby driver and the Rails adapter for DB2). To do so, run the following, adjusting the path to your own username of course:

$ sudo -s
$ export IBM_DB_INCLUDE=/Users/acangiano/sqllib/include
$ export IBM_DB_LIB=/Users/acangiano/sqllib/lib32
$ export ARCHFLAGS="-arch i386"
$ gem update --system
$ gem install ibm_db
$ exit

The ibm_db gem will be installed on your system and is ready to be used. To verify that this is the case, run a small Ruby program with the following code:

require 'rubygems'
require 'ibm_db.bundle'

conn = IBM_DB.connect("sample","my_username", "my_password")
if conn
 stmt = IBM_DB.exec(conn, "select count(*) from staff")
 count = IBM_DB.fetch_array(stmt)[0]
 puts "The staff table contains #{count} records."
else
  puts "Connection error: #{IBM_DB.conn_errormsg}"
end

If everything is fine and dandy, you should see the message “The staff table contains 35 records.”.

Now that Ruby can talk with DB2, we can move on to Rails. Assuming you have Rails 2.2.x installed, run the following to create a sample bookshelf application:

$ rails books -d ibm_db

This generates a Rails application (as usual) with a config/database.yml file customized for DB2. You’ll notice that unlike with MySQL, the database names are not books_development, books_production and books_test. The names are truncated by default due to the fact that DB2 currently only allows for database names that are up to 8 characters long. Feel free to change the development database in database.yml simply to ‘books’.

As a Rails developer you may also be accustomed to running rake db:create to automatically create the development database, yet this feature is not available for DB2 at this point, so instead you can create the database using the db2 command, as follows:

db2 create database books

DB2 allows you to specify all kinds of options for the creation of databases, but in its simplest form, the line above will work just fine.

Once the development database has been created, you should be able to use Rails with DB2 as you normally would with other database management systems. For example, you could scaffold a resource as follows:

$ ruby script/generate scaffold Book title:string
 author:string isbn:string description:text loaned:boolean

Start the webserver with:

$ ruby script/server

And then visit http://localhost:3000/books to perform CRUD operations on book records.

At this stage, the only caveats are that you’ll have to use the db2 command, rather than ruby script/dbconsole, and that you won’t be able to use the rename_column method in your migrations. On the plus side, you’ll have the XML datatype (t.xml in your sexy migrations) at your disposal, to natively store XML documents and retrieve them through XQuery and SQL/XML.

I really hope that you’ll enjoy DB2 on Mac! Don’t be afraid to ask for help, if you need it, in the DB2 Express-C forum. Oh and we are trying to get the word out there. Your help is highly appreciated. You can promote this story on Twitter, Hacker News, Reddit, DZone, StumbleUpon and Digg.

Disclaimer: The opinions expressed in this post are mine and mine alone, and do not necessarily represents the opinions of my employer, IBM.

Interview on Video

2008-12-24T08:24:32+00:00

I gave an interview at the STAR West 2008 conference about agile testing. They’ve just put it online. Here it is! Once again I give answers that are too long. I’ll never make it as soundbite artist.

How We Used to Do Unit Testing

2008-12-23T20:48:24+00:00

Reader Charles sends a question I've frequently had when consulting with testing organizations who are weary of having code thrown at them "over the wall."

Charles

From your experience in software, what are some good practices you have found for what is now known as "Unit Test?"

I have mine, I want to see if we have similar experiences.

I also have a motive, I get the opportunity to write our "Unit Test Procedure", so I want to capture "Good Practices" that developers can use.

Ack!--Unit Test Procedure!!! I wish we could have names like, "Menu of Good Unit Test Practices, Choose One or More that Make Sense for your Unit, or Use and Add a New Practice to the Menu of Good Practices (Include Provenance or Example)."

Jerry

Honor Unit Testing

1. Well, that's probably number one, the name and prestige you give to the process. If you belittle it, you will have poor results. If you reward it ... finish the sentence. I don't recall if we even had a name for it in the early days. It was just assumed that any pro would do a damn good job of this activity, not to be embarrassed by unit bugs found in integration test or system test--or god help us, in production.

Build Small and Simple

2. The next most important thing is to design to build in testable pieces. Small as possible, but no smaller. Cleanly structured, with no testing traps like hidden memory that might make the code not really reentrant.

Be Completely Open

3. Next is complete openness, getting as many eyes on it as you could get, but certainly not just one pair.

Test First

4. Today what we did is called Test Before Code, or perhaps Test Before Design. Ideally, we sketched out a set of test cases before putting pencil to coding pad. (We didn't have terminals or PCs in those early days.) These were punched into cards and put in the permanent test case library.

Take Advantage of Individual Skills and Help Each Other

5. After that, I recall things breaking down into tactics favored by each individual. Generally, we unit tested each other's code, and anyone having trouble could and did call on anyone else to help out.

Don't Green Dot

I think if people did these things today, we'd be in a lot less trouble. I know because there are people who do these things today, and they tend to stay out of trouble. On the other hand, people who just throw code at a compile and say "it's unit tested" as soon as they see a green dot, tend to be in trouble all the time.

Comments welcome!

The Seductions of Scala, Part I

2008-12-23T17:51:27+00:00

(Update 12/23/2008: Thanks to Apostolos Syropoulos for pointing out an earlier reference for the concept of “traits”).

Because of all the recent hoo-ha about functional programming (e.g., as a “cure” for the multicore problem), I decided to cast aside my dysfunctional ways and learn one of the FP languages. The question was, which one?

My distinguished colleague, Michael Feathers, has been on a Haskell binge of late. Haskell is a pure functional language and is probably most interesting as the “flagship language” for academic exploration, rather than production use. (That was not meant as flame bait…) It’s hard to underestimate the influence Haskell has had on language design, including Java generics, .NET LINQ and F#, etc.

However, I decided to learn Scala first, because it is a JVM language that combines object-oriented and functional programming in one language. At ~13 years of age, Java is a bit dated. Scala has the potential of replacing Java as the principle language of the JVM, an extraordinary piece of engineering that is arguably now more valuable than the language itself. (Note: there is also a .NET version of Scala under development.)

Here are some of my observations, divided over three blog posts.

First, a few disclaimers. I am a Scala novice, so any flaws in my analysis reflect on me, not Scala! Also, this is by no means an exhaustive analysis of the pros and cons of Scala vs. other options. Start with the Scala website for more complete information.

A Better OOP Language

Scala works seamlessly with Java. You can invoke Java APIs, extend Java classes and implement Java interfaces. You can even invoke Scala code from Java, once you understand how certain “Scala-isms” are translated to Java constructs (javap is your friend). Scala syntax is more succinct and removes a lot of tedious boilerplate from Java code.

For example, the following Person class in Java:

    
class Person {
    private String firstName;
    private String lastName;
    private int    age;

    public Person(String firstName, String lastName, int age) {
        this.firstName = firstName;
        this.lastName  = lastName;
        this.age       = age;
    }

    public void setFirstName(String firstName) { this.firstName = firstName; }
    public void String getFirstName() { return this.firstName; }
    public void setLastName(String lastName) { this.lastName = lastName; }
    public void String getLastName() { return this.lastName; }
    public void setAge(int age) { this.age = age; }
    public void int getAge() { return this.age; }
}

can be written in Scala thusly:

    
class Person(var firstName: String, var lastName: String, var age: Int)

Yes, that’s it. The constructor is the argument list to the class, where each parameter is declared as a variable (var keyword). It automatically generates the equivalent of getter and setter methods, meaning they look like Ruby-style attribute accessors; the getter is foo instead of getFoo and the setter is foo = instead of setFoo. Actually, the setter function is really foo_=, but Scala lets you use the foo = sugar.

Lots of other well designed conventions allow the language to define almost everything as a method, yet support forms of syntactic sugar like the illusion of operator overloading, Ruby-like DSL’s, etc.

You also get fewer semicolons, no requirements tying package and class definitions to the file system structure, type inference, multi-valued returns (tuples), and a better type and generics model.

One of the biggest deficiencies of Java is the lack of a complete mixin model. Mixins are small, focused (think Single Responsibility Principle ...) bits of state and behavior that can be added to classes (or objects) to extend them as needed. In a language like C++, you can use multiple inheritance for mixins. Because Java only supports single inheritance and interfaces, which can’t have any state and behavior, implementing a mixin-based design has always required various hacks. Aspect-Oriented Programming is also one partial solution to this problem.

The most exciting OOP enhancement Scala brings is its support for Traits, a concept first described here and more recently discussed here. Traits support Mixins (and other design techniques) through composition rather than inheritance. You could think of traits as interfaces with implementations. They work a lot like Ruby modules.

Here is an example of the Observer Pattern written as traits, where they are used to monitor changes to a bank account balance. First, here are reusable Subject and Observer traits.

    
trait Observer[S] {
    def receiveUpdate(subject: S);
}

trait Subject[S] { 
    this: S =>
    private var observers: List[Observer[S]] = Nil
    def addObserver(observer: Observer[S]) = observers = observer :: observers

    def notifyObservers() = observers.foreach(_.receiveUpdate(this))
}

In Scala, generics are declared with square brackets, [...], rather than angled brackets, <...>. Method definitions begin with the def keyword. The Observer trait defines one abstract method, which is called by the Subject to notify the observer of changes. The Subject is passed to the Observer.

This trait looks exactly like a Java interface. In fact, that’s how traits are represented in Java byte code. If the trait has state and behavior, like Subject, the byte code representation involves additional elements.

The Subject trait is more complex. The strange line, this: S => , is called a self type declaration. It tells the compiler that whenever this is referenced in the trait, treat its type as S, rather than Subject[S]. Without this declaration, the call to receiveUpdate in the notifyObservers method would not compile, because it would attempt to pass a Subject[S] object, rather than a S object. The self type declaration solves this problem.

The next line creates a private list of observers, initialized to Nil, which is an empty list. Variable declarations are name: type. Why didn’t they follow Java conventions, i.e., type name? Because this syntax makes the code easier to parse when type inference is used, meaning where the explicit :type is omitted and inferred.

In fact, I’m using type inference for all the method declarations, because the compiler can figure out what each method returns, in my examples. In this case, they all return type Unit, the equivalent of Java’s void. (The name Unit is a common term in functional languages.)

The third line defines a method for adding a new observer to the list. Notice that concrete method definitions are of the form

    
def methodName(parameter: type, ...) = {
    method body
}

In this case, because there is only one line, I dispensed with the {...}. The equals sign before the body emphasizes the functional nature of scala, that all methods are objects, too. We’ll revisit this in a moment and in the next post.

The method body prepends the new observer object to the existing list. Actually, a new list is created. The :: operator, called “cons”, binds to the right. This “operator” is really a method call, which could actually be written like this, observers.::(observer).

Our final method in Subject is notifyObservers. It iterates through observers and invokes the block observer.receiveUpdate(this) on each observer. The _ evaluates to the current observer reference. For comparison, in Ruby, you would define this method like so:

    
def notifyObservers() 
    @observers.each { |o| o.receiveUpdate(self) }
end

Okay, let’s look at how you would actually use these traits. First, our “plain-old Scala object” (POSO) Account.

    
class Account(initialBalance: Double) {
    private var currentBalance = initialBalance
    def balance = currentBalance
    def deposit(amount: Double)  = currentBalance += amount
    def withdraw(amount: Double) = currentBalance -= amount
}

Hopefully, this is self explanatory, except for two things. First, recall that the whole class declaration is actually the constructor, which is why we have an initialBalance: Double parameter on Account. This looks strange to the Java-trained eye, but it actually works well and is another example of Scala’s economy. (You can define multiple constructors, but I won’t go into that here…).

Second, note that I omitted the parentheses when I defined the balance “getter” method. This supports the uniform access principle. Clients will simply call myAccount.balance, without parentheses and I could redefine balance to be a var or val and the client code would not have to change!

Next, a subclass that supports observation.

    
class ObservedAccount(initialBalance: Double) extends Account(initialBalance) with Subject[Account] {
    override def deposit(amount: Double) = {
        super.deposit(amount)
        notifyObservers()
    }
    override def withdraw(amount: Double) = {
        super.withdraw(amount)
        notifyObservers()
    }
}

The with keyword is how a trait is used, much the way that you implement an interface in Java, but now you don’t have to implement the interface’s methods. We’ve already done that.

Note that the expression, ObservedAccount(initialBalance: Double) extends Account(initialBalance), not only defines the (single) inheritance relationship, it also functions as the constructor’s call to super(initialBalance), so that Account is properly initialized.

Next, we have to override the deposit and withdraw methods, calling the parent methods and then invoking notifyObservers. Anytime you override a concrete method, scala requires the override keyword. This tells you unambiguously that you are overriding a method and the Scala compiler throws an error if you aren’t actually overriding a method, e.g., because of a typo. Hence, the keyword is much more reliable (and hence useful…) than Java’s @Override annotation.

Finally, here is an Observer that prints to stdout when the balance changes.

    
class AccountReporter extends Observer[Account] {
    def receiveUpdate(account: Account) =
        println("Observed balance change: "+account.balance)
}

Rather than use with, I just extend the Observer trait, because I don’t have another parent class.

Here’s some code to test what we’ve done.

    
def changingBalance(account: Account) = {
    println("==== Starting balance: " + account.balance)
    println("Depositing $10.0")
    account.deposit(10.0)
    println("new balance: " + account.balance)
    println("Withdrawing $5.60")
    account.withdraw(5.6)
    println("new balance: " + account.balance)
}

var a = new Account(0.0)
changingBalance(a)

var oa = new ObservedAccount(0.0)
changingBalance(oa)
oa.addObserver(new AccountReporter)
changingBalance(oa)

Which prints out:

    
==== Starting balance: 0.0
Depositing $10.0
new balance: 10.0
Withdrawing $5.60
new balance: 4.4
==== Starting balance: 0.0
Depositing $10.0
new balance: 10.0
Withdrawing $5.60
new balance: 4.4
==== Starting balance: 4.4
Depositing $10.0
Observed balance change: 14.4
new balance: 14.4
Withdrawing $5.60
Observed balance change: 8.8
new balance: 8.8

Note that we only observe the last transaction.

Download Scala and try it out. Put all this code in one observer.scala file, for example, and run the command:

    
scala observer.scala

But Wait, There’s More!

In the next post, I’ll look at Scala’s support for Functional Programming and why OO programmers should find it interesting. In the third post, I’ll look at the specific case of concurrent programming in Scala and make some concluding observations of the pros and cons of Scala.

For now, here are some references for more information.

The Scala website, for downloads, documentation, mailing lists, etc.
Ted Neward’s excellent multipart introduction to Scala at developerWorks.
The forthcoming Programming in Scala book.

“Tearing Down the Ivory Tower of Testing” at SQuAD 2009

2008-12-23T15:51:37+00:00

I will be giving a keynote speech titled “Tearing Down the Ivory Tower of Testing” at the SQuAD 2009 Conference, hosted by The Software Quality Association of Denver. The conference in March 11-12, 2009, in Denver Colorado. I know several people who are SQuAD members but I haven’t been to one of their meetings before, so I’m excited to be going.

I will also be giving tutorials at the conference that will be announced later.

Perl Hell

2008-12-23T11:30:07+00:00

Here's what has me stumped today:

#!/usr/bin/env perl use strict; use warnings; use Test::Most 'no_plan'; use Test::XML::XPath; my $xml = do { local $/; <DATA> }; my @xpath = qw( /p:pips/p:version/p:segment_events/p:segment_event[@pid="p0000004"]/p:ids/p:id /p:pips/p:version/p:segment_events/p:segment_event[@pid="p0000004"]/p:ids/p:id[1 ] /p:pips/p:version/p:segment_events/p:segment_event[@pid="p0000004"]/p:ids/p:id[2 ] not(/p:pips/p:version/p:segment_events/p:segment_event[@pid="p0000004"]/p:ids/p: id[3]) count(/p:pips/p:version/p:segment_events/p:segment_event[@pid="p0000004"]/p:ids/ p:id)=2 ); for my $xpath (@xpath) { like_xpath $xml, $xpath, 'whee!'; } __DATA__ # our XML

Know what it takes to get that to pass? This:

for my $xpath (@xpath) { my $anything = undef; like_xpath $xml, $xpath, 'whee!'; }

Or this:

for my $xpath (@xpath) { like_xpath $xml, $xpath, 'whee!'; } ok 1;

Trying to reduce that to a minimal test case for a bug report has failed miserably. It appears that any sort of assignment anywhere in the like_xpath code will cause these tests to pass and the mere act of moving the affected files to other directories also appears to make tests pass. Running these tests under the debugger also makes them pass. Printing out any affected variables makes these tests pass. Hell, I suspect that breathing heavy would make these tests pass.

This is Perl 5.8.8 on Solaris. Earlier this month, Andy Wardley started a London.pm thread about this version of Perl segfaulting due to a comment. Some people could reproduce his error and others couldn't. Versions of Perl other than 5.8.8 seemed fine (and not all 5.8.8 seemed affected). I'm beginning to suspect that 5.8.8 has a parser issue. Since Perl's parsing is heuristic-based, this is a bugger (ha!) to track down.

Update: Pardon me, but F****************K! As mentioned, any assignment of a variable in the like_xpath code will fix this bug -- but only for my minimal test case. One of our actual tests still fails roughly 1 out of 5 runs. The XML returned is always the same, aside from timestamps which aren't in the xpath. I'm going insane.