Test Automation Planet

In these days when money is short, lots of people cannot afford to participate all the conferences they might have attended last year. Money may be the first criterion for choosing conferences, but it's not the only one. I'm going to three conferences this year, each chosen by different characteristics. For me, money doesn't enter into it, so my reasons might be helpful for those who can afford to be in at least one conference, but are trying to choose. my first step in choosing a conference is to eliminate the majority of conferences by applying the following guides.

What Makes Conferences Less Attractive

I generally eliminate conferences that

- overschedule event with no time or place for spontaneous meetings

- overcrowd, usually to maximize profits, with just too many people, which encourages people to hang out only with their old pals

- lack adaptability so opportunities pass by without notice or care

- offer too much lecturing, not enough interaction, and insufficient experiential work--or none at all

- invite presenters of widely varied and untested skill and preparation

- do not name their presenters in advance, or give biographical information

- provide insufficient time and space for socializing, meeting new people

- allow little or no interaction with the presenters (In some conferences, presenters eat in a special area, intentionally separated from the participants. In others, presenters speak and run.

- schedule sales pitches instead of teaching presentations

- schedule canned pitches instead of original material

- offer too many plenary sessions, when participants have no choice of what to attend

Few conferences meet all my criteria, but I look for those conferences that do, like the three (below) that I am attending this year. I have long-ago reached a stage in my life where I cannot tolerate several days sitting in an uncomfortable chair listening to someone read bullet points from PowerPoint slides.

CAST

[http://www.associationforsoftwaretesting.org/drupal/CAST2009]

I participated in the Conference of the Association for Software Testing (CAST) last year, and I'm returning this year because the subject of the conference is precisely focused on my current interest: promoting and improving the practice of software testing.

The sessions I attended were all of high quality and interest to me. Also, it's a reasonably small conference with numerous opportunities to participate in spontaneous hall sessions.

BizConf

[http://bizconf.heroku.com/]

BizConf is a new conference this year, and I'm participating primarily because of the other participants, who, like me, are small entrepreneurs running technology businesses. It's a small conference, limited to 75 participants, and scheduled once again to encourage spontaneous hall and meal sessions. All of the presenters I know are of the highest quality.

AYE (Amplifying Your Effectiveness)

[http://www.ayeconference.com]

You might say I participate in AYE every year because I'm a host--one of the people who created the conference. But I wouldn't have been a host in the first place if I had been satisfied with most of the conferences available. When we designed the conference, we had several issues in mind in addition to the ones listed above. We wanted the conference to be reasonably priced, easy to reach, and easy to learn more about than could be found on a simple website. We created a wiki that registrants could write on and anybody could read. We retained a small staff of intelligent, personable people (Lois and Suzy) to give information and solve problems over the phone. I hope we've made it easy to get to AYE, and I hope to see you there or at one of the other two conferences I'll be attending.

Ara Pulido will be presenting Mago in Gran Canaria Desktop summit. Eitan Isaacson will also be attending the conference.

Eitan has done all the base ground work for LDTPv2. Eitan also did the ground work with Javier and Ara on Mago too :) alrounder !!!

Any one interested in GNOME / KDE automated testing, I recommend you to attend the session by Ara.

Happy hacking Ara, Eitan.

To run terminal application automated test scripts under RFT, we need a third party that will help RFT recognize objects of green screen UI. For this purpose we use Terminal Extension component.

Up till now, Terminal Extension works only with RFT 6.1 under windows xp. There is an updated version that works with higher version of RFT and under windows Vista or Windows 7. However, I don’t have them. So I am obliged to use the extension under windows XP and with RFT 6.1.

After your administrator installs the extension, you will see its icon in the Rational Tester Perspective; as the figure shows:

1. Click on the terminal icon as shown in the figure.


2. Fill in the required fields as follows:
Host: Server name, this is the current server name.
Terminal Type: TN5250

3. Save the connection as .conn file for later use.
4. Now click Connect! You’re now connected to the system from within RFT. Now you’re ready to record, edit and playback your automated scripts.

1. Ask your system administrator to install CVS server component. This location will be used as your scripts repository.

2. Now open Rational Functional Tester

3. Go to RFT->Window->Open Perspective ->CVS Repository Exploring

4. Click on Add CVS Repository icon

5. The following windows displays

1. Fill in the required fields. Here’s an example:
HOST: server name
Repository Path: /fill in the path
User: domain\username
Password: *********

2. Click finish

3. Now the repository is created

4. Right click on the repository and select: Refresh Branches. This will connect you with the branches you check from the retrieved list.

5. To checkout a project to your workspace, right click on a folder, and then click Check Out. An instance of the project will be checked out to the workspace. Usually the workspace is located in the following path: C:\Documents and Settings\username\IBM

6. To connect to the check out project, change the perspective to Functional Test (Default)

7. Now go to File->Connect to a Functional Test Project

8. Select the checked out folder

9. Click ok

10. Now you’re connected to the project and you can edit it.

11. Once you’re done editing, you need to sync your changes to the repository. Change the perspective to Team Synchronization

12. Right click on the project Folder and click Synchronize

13. Your changes now are saved on the server.

14. If you want to work offline, you can check in a project. Then you need to change the folder name of the project in the workspace. And go to File->Connect to a Functional Test Project. And choose the copied folder, and work offline.

I came across this note from the Engineering team at Facebook which talks about how they managed to smoothly launch their recent 'pick a username' feature. The title of the note is, appropriately enough, 'Hammering usernames' -- this is of course because they were expecting their infrastructure to be hammered.

In the note I saw for the first time a name for a strategy that teams I've been involved with have applied before: 'dark launching'. Essentially, dark launching is releasing a new feature to a subset of your users, mostly with no UI changes, but otherwise exercising all the parts of your infrastructure involved in serving that feature. A good strategy to apply when you're dealing with massive, large-scale deployments, and when you want to see how your infrastructure behaves in conditions that are as close to production as possible. Because remember, there's nothing like production! Your careful load/stress testing exercises in a lab environment ain't gonna cut it.

The note from Facebook has all sorts of other nuggets of wisdom related to massive infrastructure deployments. I recommend subscribing to the RSS feed for 'Engineering @ Facebook's Notes'.

While googling for 'dark launching', I also came across this very good post by Dare Obasanjo. Recommended reading.

I'd like to have a system that can hold all the artifacts of a test execution and that also knows a lot about TAP. Smolder is a good start as it knows about TAP but there are tons of other things I'd like to have in it.

Some of them are hard to add, others are harder.

Anyway, earlier I wrote about Reporting Test Results. Now let me write about my wish-list:

1) I'd like Smolder to collect all the artifacts of a test run. Look at the data Buildbot collects or what is collected by the PostgreSQL test farm! Each run has several stages and the actual test run is only one of them. In a Test Reporting system I'd like to have access to all the output and error files of all the stages.

2) The tests might create some additional log files, I'd like to be able to include those too.

3) One major issue we have is test counting. TAP encourages you to have a plan - the number of test units you are going to execute in a test run - so that it can check the number of actually executed test to that number. Keeping the plan up to date is annoying, people don't like it so many use no plan. Some TAP producers don't even let you plan ahead (e.g. the ones I saw in PHP) but I think generally those that are more xUnit like won't have a test plan.

In larger settings this plan does not even help as it does not provide any tools to avoid reducing the test plan accidentally along with a shortcut in the test script: A developer can comment out several tests temporarily and adjust the plan for the time while he is trying to fix a bug forgetting to revert these changes after he is done.

Smolder could help a bit here: On the first result report the administrator of the project should look at the reports and mark them as having the correct number of tests. Then if Smolder notices a test report with a different number of test for the same test file it should raise an alert. The project administrator can examine the change and decided if that is the new correct number of tests setting the new expected number to this or if it was changed by mistake.

4) History of unit tests. If we would like to be able to provide a history based on the actual test units (calls to an ok() function) then we need to have some unique identifier for each test as we cannot assume that the test number or the test name will remain the same. After all people sometimes insert new tests or remove old test at the beginning of the test scripts and the names can change.

Smolder could also help here too. It could track the test numbers and test names. If it sees a change it can alert the project administrator who can then use some GUI to connect the old test units to the new ones.

Both involves some manual work but I think large organizations will be ready to invest that work in order to get the benefit of greater control and better reports.

What would you like to have in your test reporting system?

The latest developer release of Test::More allows subtests. Subtests are great in that they solve a lot of problems in advanced Perl testing, but they have required a change in Test::Builder. Previously you could do stuff like this:

package Test::StringReverse;

use base 'Test::Builder::Module';
our @EXPORT = qw(is_reversed);

my $BUILDER = Test::Builder->new;

sub is_reversed ($$;$) {
    my ( $have, $want, $name ) = @_;

    my $passed = $want eq scalar reverse $name;

    $BUILDER->ok($passed, $name);
    $BUILDER->diag(<<"    END_DIAG") if not $passed;
    have: $have
    want: $want
    END_DIAG

    return $passed;
}

1;

And you've have a simple (untested ;) test for whether or not strings are reversed.

The reason that worked is that Test::Builder->new used to return a singleton. This is no longer true. If someone uses your test library in a subtest, the above code would break. Instead, you want to do this:

sub is_reversed ($$;$) {
    my ( $have, $want, $name ) = @_;

    my $passed  = $want eq scalar reverse $name;
    my $builder = __PACKAGE__->builder;

    $builder->ok($passed, $name);
    $builder->diag(<<"    END_DIAG") if not $passed;
    have: $have
    want: $want
    END_DIAG

    return $passed;
}

It's a minor change, it's completely backwards-compatible and it supports subtests. There's a work-around being planned, but it's not out there yet.

mnot == Mark Nottingham (RESTful Web Services guru and blogger)

redbot == Resource Expert Droid == a testing tool written in Python that "checks HTTP resources to see how they use HTTP, makes suggestions, and finds common protocol mistakes"

Use this tool if you want to see how well your Web application does in terms of HTTP connections, content negotiation, caching and validation. Very useful both for traditional Web sites and for Web services.

A few weeks I went down to the U.S. Consulate in Toronto to register Ariel, my daughter, as an American citizen born abroad. (She's a dualie, because she was born in Canada to an American parent: me. I'm a dualie too, born in the U.S. to Canadian parents. Being born a dual citizen is a wonderful example of a best practice. You should follow it. But I digress.)

The application process is, naturally, fraught with complication and bureaucracy. There's also a chilling and intimidating level of security; one isn't allowed to bring anything electronic into the Consulate at all. No cell phones, no PDAs, and certainly no laptop computers. That means no electronic records, and no hope of looking anything up. So one has to prepare.

There's a Web site for the Consular services. One of the first items that one sees on the site is a link for telephone inquiries. Note a couple of things here: the telephone services are for visa information, not for general information; and that visa information costs USD$0.90 per minute for a recorded system with no operator. (Oddly, that's the price for calls from the U.S.; calls from Canada are cheaper, at CAD$0.69 per minute.) I didn't test that.

With only a little digging, I was able to find information related to registering a birth abroad. I gathered the information and documents that I figured I needed, and took it all down to the Consulate. I was getting ready to travel the next day, and so in typical fashion, I pushed things out to the noon deadline for receiving applications. I watched the clock on the car anxiously, parking at 11:53 and getting to the Consulate at 11:55. "Wow, that's pushing it," said the security guard. "Last one today."

When I spoke to the friendly, helpful lady behind the counter (I mean that; she was genuinely friendly and helpful) she they told me some things that the Web site didn't.

• The application form itself is online, and these days it's one of those PDFs that has input fields, so everything can be nice and tidy. Again, though, there are some fields in the form that have several possible answers. There is some helpful information available, but I still had questions.
• The consular officers want to see original documents, but accept and keep only photocopies of them. You need to come with your own photocopies. If you don't, it costs you $1.00 per document—and there are lots of documents. This isn't noted anywhere on the Web site that I could see.
  
• On one of the Web pages listing documentation requirements, it says "In certain cases, it may be necessary to submit additional documents, including affidavits of paternity and support, divorce decrees from prior marriages, or medical reports of blood compatibility." Well, what cases? The page doesn't tell me, and getting it wrong means an extra trip. The lady behind the counter reviewed what I had brought, answered a number of questions, and told me exactly what to bring next time.

As I travel around, I sometimes see an implicit assumption that documents tell us all we need to know. Yet documents are always a stand-in for some person, an incomplete representation of what they know or what they want. They're time-bound, in that they represent someone's ideas frozen at some point in the past. They can't, and don't answer followup questions. As Northrop Frye once said, "A book always says the same thing." Yet if we look more closely, not even ideas that are carefully and thoroughly debated can be expressed unambiguously. That's why we have judges. And lawyers.

The next thing that happened emphasized this. After I left the Consulate, I returned to my car. At the collection booth, the posted time was 12:20. I'd been less than half an hour, which is good because parking at that garage costs $3.00 per half-hour. I handed the attendant my ticket. The charge was $6.00.

"What?! I've only been gone for 25 minutes."

She looked at the ticket. "Sorry, sir. You checked in at 11:40."

"No way," I said. "I know what time I checked in; I was running late. It was at least 12 minutes later than 11:40. I got to the entrance to the Consulate, just over there, at 11:55. No way I could have taken 15 minutes to walk 75 metres!" She showed me the ticket. It said 11:40. "That's impossible. I want to check the clock."

The difference was only $3.00, but I was furious. I exited the garage, drove around to the entrance and check the display. It read 12:24, the correct time. I pushed the button and pulled out a ticket; it too read 12:24. To her credit, the attendant appeared and checked the clock, and asked to see the ticket I had just printed. "12:24. I'm sorry, sir, there's nothing I can do." Quite true, no doubt.

In this case, the (clearly fallible) machinery and the (clearly fallible) documentation were more credible than I. I didn't check the ticket on the way in. And yet I know when I arrived, and I know that there must have been some kind of failure with the machinery. A one-off? A consistent pattern? Happens only at a certain time of the day? A mechanical problem? A software problem?

All the way home, I pondered over how the failure had occurred, and how one might test for it. But what impressed me most about my experience with the Consulate's Web site, and the consular officer, and the the parking ticket machine, and the parking attendant, was the way in which we invest trust, to varying degrees and at various times, in machines and in documents and in people. When is that trust warranted, and when is it not?

Postscript: Just now, as I attempted to publish this post, the net connection at this hotel was suddenly unavailable. Again.

Thanks to the energetic James Lyndsay, a bunch of us are meeting at the conclusion of his Exploratory Testing class and my Rapid Testing class on Friday, June 3 2009, in London, UK. I expect we'll be there somewhere between 5:30 and 9:00pm. The venue is the Prince Arthur pub, 80-82 Eversholt Street, Euston, London, NW1 1BX, right across the street from Euston Station, north of Euston Road. Three large tables have been reserved. All are welcome; please spread the word!

After that, I'm off to The Magnet, a pub near the Archway station, where a gang of folks gets together every Friday from 9:30pm until late to play Irish traditional music. I'll be armed with my mandolin. All are welcome there, too!

“Added a validation in the service codes to make sure either "DSP" or "Already in GE" is true for Accession.”


Do your devs write bug resolution comments? If not, ask for them. All bug tracking systems have a spot for devs to write notes about what they did to fix the bug. It’s tough to get devs to write their resolution but it’s well worth the pain.

When you verify fixed bugs you should be asking yourself what regression testing is necessary; what did this bug fix break? If you don’t know how the dev fixed it, you don’t really know what could have broken as a result. Sometimes, devs are cool enough to provide tips of where other bugs may be lurking…


“This problem is bigger than the shoot value hiding. In fact, the entire preCat is not reloading after a save which is hiding other bugs (UI refresh and service layer issues.)”


Some of my devs are terrible at writing resolution comments. Sometimes I have no idea what they did to fix it. My emotions tell me to Reopen the bug. But after listening to my brain, and asking the dev what they did, I usually discover an unexpected fix that still solves the problem. Dev comments would have been nice.

You may also discover your devs share some of your angst when dealing with scatterbrain stakeholders, as evident in comments I’ve recently seen such as the following:


“UI now allows multiple preCats and DSP check box, even though earlier requirements conflict with both of these.”

“Yet another requirements change. The need dsp check box has always been there. It was expressed that it should only be visible when there IS and accession, but I guess we are changing that.”


Poor dev guy. I feel your pain. I miss the good old pre-agile days too.

I've found that one of the best ways to try new testing ideas is to run them against our test suite at work. We've over 30,000 tests at this point, with several test harnesses integrating together, along with two fundamentally different test systems. This means that when I throw something at this test suite, I often shake out bugs. My new Test::Aggregate::Nested combined with our test suite managed to find a rather serious issue with the new subtest function in Test::More. The following, for example, fails:

    use Test::More tests => 2;
    use Test::XML;

    ok 1;
    subtest 'FAIL!' => sub {
        plan tests => 1;
        is_xml '<foo/>', '<foo/>', 'Singleton fail';
    };
    __END__
    xml.t ..
    1..2
    ok 1
        1..1
    Cannot run test (Singleton fail) with active children at /home/ovid/pips_dev/work/Pips3/branches/rights_modeling/deps/lib/perl5/Test/XML. pm line 57.
        # Child (FAIL!) exited without calling finalize()

There's nothing wrong with the code as it's written, but Test::XML knows that the Test::Builder object is a singleton, so there's a false optimization in the code. Near the top of the package, you see this line, defined outside of all functions:

    my $Test = Test::Builder->new;

If every Test::XML function simply had that line in the function rather than trying to share this across all test functions, the subtest code would work fine. Instead, the author knew he had a singleton and there's no point in re-instantiating, is there?

To be fair, I've done the same thing before (see Test::JSON), even though I knew it might be a bad idea. Heck, lots and lots of testing libraries have this issue. Now we need to figure out how to deal with this problem or else subtests aren't going to play well with a lot of code. Damn.

Boy oh boy, does nested TAP make Test::Aggregate much cleaner. It's not uploaded yet -- still documenting and working out corner cases for the new Test::Aggregate::Nexted -- but here's what aggregated test output used to look like. Remember, this is five separate test programs. Can you tell where each starts and ends?

Test-Aggregate  $ prove -lv t/pre_post.t
t/pre_post.t .. # ******** running tests for aggtests/check_plan.t ********

ok 1 - aggtests/check_plan.t ***** 1
ok 2 - aggtests/check_plan.t ***** 2
ok 3 # skip checking plan (aggtests/check_plan.t ***** 3)
ok 4 - env variables should not hang around
ok 5 - aggtests/check_plan.t ***** 4
ok 6 - findbin is reinitialized for every test
ok 7 # skip Testing skip all
#     ok - aggtests/check_plan.t (1 out of 5)
# ******** running tests for aggtests/findbin.t ********
#     ok - aggtests/findbin.t (2 out of 5)
# ******** running tests for aggtests/skip_all.t ********
#     ok - aggtests/skip_all.t (3 out of 5)
# ******** running tests for aggtests/slow_load.t ********
ok 8 - slow loading module loaded
ok 9 - env variables should not hang around
ok 10 - subs work!
ok 11 - Startup should be called once
ok 12 - ... as should shutdown
ok 13 - Setup should be called once for each test program
ok 14 - ... as should teardown
1..14
#     ok - aggtests/slow_load.t (4 out of 5)
# ******** running tests for aggtests/subs.t ********
#     ok - aggtests/subs.t (5 out of 5)
ok
All tests successful.
Files=1, Tests=14,  1 wallclock secs ( 0.02 usr  0.01 sys +  0.11 cusr  0.01 csys =  0.15 CPU)
Result: PASS

Now let's run that with Test::Aggregate::Nested (requires the development version of Test::More):

Test-Aggregate  $ prove -lv t/pre_post_nested.t
t/pre_post_nested.t ..
1..5
    1..5
        1..0 # SKIP Testing skip all
    ok 1 # skip Testing skip all
        1..1
        ok 1 - findbin is reinitialized for every test
    ok 2 - aggtests/findbin.t
        1..1
        ok 1 - subs work!
    ok 3 - aggtests/subs.t
        1..2
        ok 1 - slow loading module loaded
        ok 2 - env variables should not hang around
    ok 4 - aggtests/slow_load.t
        1..5
        ok 1 - aggtests/check_plan.t ***** 1
        ok 2 - aggtests/check_plan.t ***** 2
        ok 3 # skip checking plan (aggtests/check_plan.t ***** 3)
        ok 4 - env variables should not hang around
        ok 5 - aggtests/check_plan.t ***** 4
    ok 5 - aggtests/check_plan.t
ok 1 - nested tests
ok 2 - Startup should be called once
ok 3 - ... as should shutdown
ok 4 - Setup should be called once for each test program
ok 5 - ... as should teardown
ok
All tests successful.
Files=1, Tests=5,  2 wallclock secs ( 0.03 usr  0.01 sys +  0.11 cusr  0.02 csys =  0.17 CPU)
Result: PASS

Much, much nicer. As an added bonus, plans can now be cleanly asserted. I hope to have it on github soon, and later on the CPAN.

Weave Sync is a prototype that encrypts and securely synchronizes the Firefox experience across multiple browsers, so that your desktop, laptop and mobile phone can all work together. It is part of the Weave project, which aims to integrate services more closely with the browser.

Major Features

What is Weave Sync all about? In short, Weave Sync lets you securely take your Firefox experience with you to all your Firefox browsers — including our mobile browser, codenamed Fennec. It currently supports continuous synchronization of your bookmarks, browsing history, saved passwords and tabs. For example:

• Get the same results on the Smart Location Bar on each of your Firefox browsers, so you can get to your favorite sites with just a few keystrokes
• Continue what you were doing: have the ability to open any tab you have open on any of your Firefox browsers
• Keep the same list of bookmarks on all of your Firefox browsers
• Easily sign in to all your favorite sites using your saved passwords (this is especially handy on mobile phones, where it’s hard to type in complex passwords)
• Do it all securely: Weave Sync encrypts user data before uploading it to Mozilla’s servers, so that only you can access your data

What’s new in 0.4.0?

If you have not looked at Weave recently, now is a great time to jump in and try it out! This release includes a major rewrite of many of Weave’s key components since the last major release in June. A few of the major changes are:

• Preference syncing (including Personas)
• Identity support (automatic login, including use of OpenID logins)
• Better support for addons using Weave
• Support for Fennec 1.0 beta 2 on Maemo Linux
• Significant improvements in performance during startup and opening new windows.

Getting Involved with Testing and Development

• Install Weave 0.4.0 (requires the Firefox 3.5 release candidate)
• Learn more about Weave
• Discuss, debate, and add to the design in the Weave forum
• Join us in #labs on irc.mozilla.org

– Mike Connor, on behalf of the Weave development team

The CAST testing conference is happening in Colorado Springs, July 13-16. I mention this for two reasons:

1. I will be teaching a testing tutorial there. I also will be wandering around with my various testing games and challenges hoping to do them with anyone who wants to see what I mean by “testing skills.”

2. CAST is the only conference that is organized in the true spirit of the Context-Driven School of Testing. It’s kind of our home conference. If you find my thoughts interesting, or those of Cem Kaner, Michael Bolton, Ben Simo, etc. Then you’ll want to be there, if you can.

Recently I sold my old, damaged MacBook Pro on eBay, and in doing so I claimed that there was a chance that it could be repaired (by Apple) for free. But how, you may be wondering, could I make such a bold claim? Was it all a strategy to over-sell my broken laptop? Not in the least. Back when I first found out about this Apple’s KB article, the contents of which appeared as though they would entitle me to a free repair, I headed straight to the Apple Store at the Fairview Mall in Toronto. I’d had a fairly positive customer care experience there just a week before, and I was rather optimistic that they’d repair it for me.

After I arrived at the Genius bar, I had to wait for quite a while before having the displeasure of dealing with an uncooperative “genius”, a young guy whose unfriendly attitude far outweighed any technical know-how he may have had. He immediately denied any knowledge of video issues on MacBook Pros from 2007 and only agreed to check my laptop after I’d showed him a printout of the knowledge base issue mentioned above. He essentially humored me in a rather reluctant way, and after a very short while told me that my laptop didn’t qualify for the free repair. Annoyed by this guy’s lack of care regarding my problem, I left the store.

Around the time when this situation arose, I had already visited the Genius Bar several times regarding various matters and was feeling a bit tired of dealing with a broken laptop. Ultimately I gave up on my old MacBook Pro (my first Mac ever) and when it was economically possible, I purchased a replacement laptop. Though my old MacBook Pro had cost me time and money, I felt that its tale was done and over with once I brought my new laptop home. That is until the day I decided that someone else could get more use out it than I was (as it was just sitting unused in my office), and that I could get a few bucks by selling it on Ebay. The old Mac’s auction wrapped up with a selling price of $578. It was bought by a fellow from Ontario, who was a very pleasant person to deal with.

If fact he was so kind that he sent me a follow-up regarding his own attempt to get the laptop fixed. Here’s what he wrote:

Just a quick update. The testing on the Macbook Pro was conducted this week, and found to conform to the warranty exception Apple identifies with the nVidia chip problem. A replacement is being installed now, but likely won’t be ready until next week. While you’re probably more content to wash your hands of the whole matter, I’d seriously consider a complaint against the apple store you took it to, and more specifically against the “genius” who served you, given he was clearly no genius at all, and likely cost you significantly in time and money as a result. Hope you’re having a good day.

I am genuinely happy for the guy. He took a risk by buying a broken laptop from me, trusted my story to be true (as it was!), and ended up scoring a working laptop that’s (now) worth far more than what he paid for it. His bet paid off and I wish him all the best.

The truth of the matter is that, as he says, I’m glad to “wash my hands of the whole matter”. At this point there is little Apple would do, even if I made a fuss about it (which I’m not going to, of course). Next time I walk by that particular Apple store, I may have a word with the manager (regarding the Genius I’d dealt with), but that’s about it.

The moral of the story is not that Apple’s customer care sucks or rocks. Apple Genii are just people, some are very good, others exceptionally bad, and most of them are somewhere in between. The take-home lesson for me here was that when dealing with Apple, if you get turned down by one store, you shouldn’t stop there! Take the time to visit a couple more and to persue the matter as far as you can.

Whether you're talking about git, Mercurial, SVK or some other distributed version control system, it's very important to realize that one of its strongest benefits is "distributed". Many people don't care about this, but it can tremendously boost productivity when combined as a part of overall development strategy.

Recently I got so fed up with our dev box that I install Ubuntu on my work computer so I could work locally. This was because our dev box was routinely running with loads around 20, and on one occasion hit a load of 57. Seems the admins thought it would be a good idea to let a bunch of other teams develop on that server but not tell us about it. I just couldn't work like that. Instead of using my computer as a dumb workstation with Windows, I now develop locally and am (usually) not dependent on our dev box being available.

That's also the reason why I have MySQL server installed locally, rather than using our test database. Sometimes that test database goes down, so I don't want to depend on that, either. And the admins thought it would be a good idea to let a bunch of other teams develop on that server but not tell us about it.

In fact, sometimes I break down. With a recent back injury, I was working from home for a week. It was slow, frustrating and painful because I wasn't really set up to work from home and this caused me to lose even more productivity. Had I been able to work from my laptop, there would have been no performance hit. Of course, even without a back injury, we had thousands of employees unable to work full hours because of a Tube strike.

Which brings me back to version control. More than once I've been unable to check in code because the subversion server has been down (never mind when the repository simply freaks out and tells me I can't commit). With something like git, I could do all of my work locally and if I can't push to the central repository, oh well.

So yeah, you know all this, but seriously, do the math. Let's be really generous and assume that our dev box, subversion server, mysql server and ability to be physically present at work all run at 99% availability. This means that your chance being "available" to work is .99^4, or about 96%. If the average availability of all of those things seems around 95%, then your "availability" drops to about 81%. Before I started working so hard to "disconnect" from the network at work, my availability was often lower than 81%. It was awful.

Now imagine sitting in a job interview and telling them that you're only going to work 81% of the time. You're not going to get the job. But the reality is, many corporate environments enforce 81% availability. This is insane, but common. And of course, we also know that those four items aren't the only things which can disrupt your productivity, but those are the four which most impact mine, so that's why I chose them.

When you're setting up a dev environment for programmers and you want it to be as productive as possible, don't force them to depend on things they can't control, such as network availability. Of course, as an employee, it's always fun when the network goes down and you see everyone wandering around with a coffee cup in hand, having been electronically muzzled. What on earth could convince people that this is a sane development environment?

Evaluating this framework brings back memories… the year was 2005, I had been working for a small IT firm as one of the first developers hired, which as of 2005 included three developers. I had zero process in place and had never had any experience elsewhere as this was my first job. Somehow, from reading Royce’s Software Project Management to a book on RUP to a book on Scrum, I finally found my way to Extreme Programming and was trying my best to implement it. I had just picked up unit testing and was using SimpleTest (after trying PHPUnit out and deciding I liked SimpleTest better) and felt that I should be Test Driving all code, javascript included. So I joined #javascript on efnet and asked “Hey, what should I use? I’ve been trying jsUnit out…”

The reply I got was that I should stop using jsUnit from jsunit.net and start using the one on berlios.de instead as it was a more faithful port of JUnit, was more Object Oriented, and it was possible to run the tests from Rhino (which led me to asking what rhino is, and then a 2 hour longer conversation ensued).

Anyway, jsunit is just that… it’s a faithful port of JUnit 3.8.1 and is designed in a more object oriented way than it’s counterpart. In fact, the syntax looks similar to the current built in test runner in js-test-driver. With that out of the way, let’s take a quick look at the dirty details.

The Dirty Details

License
Apache Software License 2.0

Last Updated
The last stable release was 1.3 in October of 2008.

Support
The jsunit mailing list link on the page links to the other jsunit, but you can email the author directly.

Let’s Dive In!

First let’s download the tarball for jsunit 1.3, and extract it to a directory. It seems a maven2 and ant plugin are included for plugging into continuous integration cycles, but thats a topic for another post so we’ll delete those.

First, I admit it’s been a long time since I used this so I’m a bit disoriented, but in the samples directory there are many sample tests, and an example runner named AllTests.html. This is more of an example on how to implement a runner versus “the de-facto runner” … it’s very basic. Here’s the javascript code driving it:

var writer = new HTMLWriterFilter( JsUtil.prototype.getSystemWriter());
var printer = new ClassicResultPrinter( writer );
var runner = new EmbeddedTextTestRunner( printer );
var collector = new TestCaseCollector( this );
runner.run( collector.collectTests());

and here’s the output of opening it up in firefox:

Yep… very basic.There are also various examples in other languages, such as JScript in AllTests.asp, AllTests.jsp, and more. Alright, enough of that… let’s go ahead and write a test case for the old HTML5 placeholder example.

Getting started is pretty simple… we’ll just modify the existing sample code and get started with the test case. The examples use a format of defining functions prefixed with the test name (i.e. Html5PlaceholderTest_setUp()) and then calls glue to add those methods in. I don’t like this though, and you don’t have to do it as long as you set the protototyp to TestCase before adding test methods. Anyhow, here’s the first test:

function Html5EmulatorTest(name){
    TestCase.call( this, name );
}
Html5EmulatorTest.prototype = new TestCase();
Html5EmulatorTest.prototype.setUp = function(){
  var sandbox = $('


')
  this.emulator = new Html5Emulator(sandbox);
  this.PLACEHOLDER_TEXT = "placeholder text is here";
  this.input = sandbox.append('

With the exception of having to call TestCase in the “constructor” and assigning the prototype to TestCase, you might notice that that as I mentioned previously the syntax is similar to that found in js-test-driver. Here is my current directory structure.

So with the basic runner I open it and firefox and see some plain text describing that I have a failure… wirte some code to make it pass and it passes. Simple enough, but very ugly… when you get to multiple test cases (as you surely will) it’s hard to distinguish the results.

Anyhow, as usual I have completed test driving some example code and provided it on github for your viewing pleasure.

What I Liked

• A lot more object oriented than the other jsUnit
• Pretty faithful adaption of JUnit
• It’s possible to run from any javascript interpreter, and includes an ASP example

What I Didn’t Like

• Hasn’t been updated forever… not sure if it’s a live project anymore
• The included runner is very plain… it’d be nice to at least have red/green bar
• You have to remember to set the prototype to TestCase and invoke TestCase.call(this) in all your test cases

Summary

Overall I did like the fact it feels like a straight adaption of JUnit, and like I said a similar syntax survives in js-test-driver. It’s overall a good framework, but sadly in today’s arena it really doesn’t have anything special going for it. But it was definitely a start in the right direction.

What’s Next?

Wow… who woulda thunk there were so many javascript TDD frameworks out there!? And there’s still more… Crosscheck, jsunittest, FireUnit, SugarTest, and many many more. But for now I think we’ve covered enough and although I’ll cover these as the chance arises I will now be moving into the next phase of this series.

The most important aspect of test driven development is automated Continuous Integration. All these frameworks are fine and dandy, but they’re useless if you can’t have them running in daily (or even hourly.. or even every 10 minutes) so that you know when something is broken. So for the next several blog posts I’ll be investigating how each of these frameworks (or at least the ones I like working with the most) stack up in ease of integration in CI. Stay tuned!

By James A. Whittaker

Schwern has released Test::Simple 0.89_01. Nested TAP is now available. In case you don't recall (or haven't heard about this), you might write a subtest like this:

  use Test::More tests => 3;

  pass("First test");

  subtest 'An example subtest' => sub {
      plan tests => 2;

      pass("This is a subtest");
      pass("So is this");
  };

  pass("Third test");

And get the following nested TAP:

  1..3
  ok 1 - First test
      1..2
      ok 1 - This is a subtest
      ok 2 - So is this
  ok 2 - An example subtest
  ok 3 - Third test

(Adrian, I believe you have a branch of Test::Class which uses this? :)

Today Mozilla Labs is releasing a preview of the next major version of Ubiquity. Ubiquity 0.5 brings a slew of new features like localization and more natural command names (along with the inevitable new bugs). Before we release Ubiquity 0.5 proper, we first need your feedback on the new features and your help with testing and finding bugs.

Download the extension here. Works in Firefox 3.0 and 3.5.

The First International Ubiquity

Ubiquity is an experiment in natural-language interfaces, but “natural language” doesn’t mean just English! Since before the first release of Ubiquity, we’ve been pondering how it could be generalized to work for non-English speakers.

The original Ubiquity input parser — the part of Ubiquity that takes what you typed and matches it to what you want to do — was based on a lot of assumptions about English that are not necessarily true in other languages. Some languages don’t use spaces between words; some have the verb come at the end of the sentence. Some languages put different endings on nouns to identify what part of the sentence they are. (If all this grammar talk makes your head spin with fuzzy memories of high-school Latin, you’re not alone.)

Ubiquity 0.5 features a new input parser re-written from scratch to support many languages. Adding new languages to it does not require writing new code, so people don’t have to be programmers to contribute to localization. A new parser localization tutorial will walk you through teaching Ubiquity the grammar of your language.

Ubiquity 0.5 also supports the localization of commands bundled with Ubiquity. At launch, Ubiquity 0.5 will have commands localized in Danish, English, and Japanese. More languages are planned for the near future. We also have a new tutorial that shows how you can get involved in translating Ubiquity commands to your language.

New Interactive Tutorial

We’ve improved the first-run experience of Ubiquity fairly dramatically, based on some user testing. In particular, inspired by the tutorial levels in video games, we’ve created an interactive tutorial that takes you by the hand and walks you through each step of using Ubiquity. It’s aware of where you are, so it can always tell you exactly what to do next.

No More Hyphens

Due to a limitation in the old parser, command names couldn’t have spaces in them. That’s why we had a lot of commands with hyphens: “add-to-calendar”, “close-all-tabs”, etc. These hyphens were ugly, hard to type, and not very natural-language. Thanks to the fact that the new parser supports spaces, we can now have much more natural command names. We’ve also standardized the command names in a way that we hope will make Ubiquity overall easier to learn.

Suggestions while you wait

Ubiquity suggestions are now fully asynchronous. What does that mean? Previously, Ubiquity could only show you suggestions based on what it already knew. But with the new improved architecture, it is capable of making network calls to help figure out what your input means, and it can then update the suggestion list in real time to show you new possibilities that it finds. For example, you can now type “pasta” into Ubiquity; before, it wouldn’t have known what to do with this input. But now, by making network calls to web services, it recognizes “pasta” as a type of restaurant, and suggests the Yelp command to find pasta places near you:

Updating Your Commands

The new version of the parser has a new API for command developers. Unfortunately, this means that custom Ubiquity commands will need to be updated in order to work with the new version. Fortunately, the required updates are very minor; they mostly have to do with how commands declare their arguments. We have written a tutorial on how to convert your existing commands to work with the new parser. There is also an updated Command Development tutorial if you are writing new commands from scratch for the new version.

How To Get Involved

• Check out Planet Ubiquity for the latest and greatest Ubiquity happenings.
• Read and contribute to Ubiquity’s documentation on the Mozilla wiki.
• Join the ubiquity-firefox mailing list to discuss Ubiquity with others.
• Join the #ubiquity channel on irc.mozilla.org to talk to Ubiquity community members in real-time over IRC.
• Browse the command library to find Ubiquity commands all over the Internet.
• Report or help fix bugs at our bug tracker.
• Discuss translation and localization issues on the Ubiquity-i18n (internationalization) mailing list.

Thanks

Special thanks to our amazing community members, who really stepped up to make this release possible. They put their blood, sweat, and tears (and weekends) into making Ubiquity 0.5 what it is:

Thanks in particular to:

• Mitcho (Michael Yoshitaka) the linguist of the Ubiquity team, for doing the majority of the design and implementation work on this new parser and localization infrastructure.
• Seth Bindernagel of the Mozilla localization department for his valuable advice.
• Christian Sonne, for working very hard to single-handedly create the Danish translation.
• Masahiko Imanaka (marsf), for contributing the majority of the Japanese translations.
• Heather Kofke-Egger for writing documentation, tracking bugs, and giving technical support to users.
• Brandon Pung for his work on the asynchronous noun detection features.
• Satoshi Murakami (satyr) for numerous bug fixes.
• Blair McBride (Unfocused) for many contributions.
• Fernando Takai (fern) for working on the command search engine.
• Zac Lym (indolering) for his usability research.

Release Notes

The full release notes for 0.5 list many fixed bugs and minor features not mentioned above.

–Jono DiCarlo, on behalf of the Ubiquity team

In an open source project - especially one that involves Perl - most of the people just want to write code. Nearly no one wants to do PR. It even has a bad reputation among many open source developers.

In Padre I constantly struggle if I should spend my time adding a new feature or trying to recruite more developers. The former works better in the short term, the latter works better in the long term. Eventually there is a ballance I try to keep.

One of the things I really miss from the perl 5 porters - the people who maintain and improve perl itself - is the communication.

In the last couple of weeks we saw lots of blog-posts and mailing lists posts - sometimes not in the nicest way - why there is no regular release of perl 5 and how a speed related bug that was fixed 18 months ago was not released in an intermediate version. In these discussions several of the p5p people got involved but unfortunatelly this is mostly deffensive talk. Not good PR.

I also got fed-up with the lack of release and asked on the p5p list what is stopping them from releasing the next version of perl and what is so hard in the release process that they cannot do it more regulary?

I got several related replies. One of the them on the mailing list others on IRC.

According to those the next perl 5 release is within a small number of weeks. That is certainly good news. I would prefer to see a Release Candidate already that people can take for a ride but in the meantime I can point you at the three major issues around the release that need to be done:

1. Fix a bunch of outstanding issues.
2. The release must be stable.
3. The actual release process should be automatic.

For item no. 1 you need to talk to the perl porters or look at the topic on the #p5p IRC channel where they point you at the list of outstanding issues

Item no. 2 was addressed in an e-mail of David Golden and the thread following it. That should be summarized in a document and the test automation system should be implemented.

For item no. 3 we first need to have a written document of what needs to be done and then we can attempt to automate it. The document can be based on the e-mail of Nicholas Clark. Actually I already created a pod version of that file that I hope soon will be added to the source tree of perl and then it can be fine tuned and implemented.

The interesting issue is that 2. and 3. need to be implemented only once and they can be reused for every future release of perl 5. So if you'd like to see more frequent releases that's where you need to help.

Luckily those, especially no. 2, are issues that any perl hacker can get involved easily. They only need perl coding.

I know I am not a good PR person and I hardly know anything about what's going on in the perl 5 porters but I hope this post helped a bit. At least it might trigger someone to write a better explanation of the situation.

Oh and a last note, if you'd like to get involved, the description on how to deal with the Git repository can be found here

By James A. Whittaker