Testing of software systems is hard. Testing of convergent systems is much harder. Every additional device, every additional style of interaction, every additional class or role of users adds a whole extra dimension of tests. Pretty soon you get to a point where a new release of the system is impossible to test with the test team and time available.

Historically, there have been a few kinds of responses to this.

• Deny that the problem is as bad as it seems, stretch the test team by a bit, stretch the test time a bit, work a bunch of overtime, then make testing compromises and ship an essentially untested and probably buggy product.
• Admit the problem, and scale back the product vision so that the product is largely testable, but miss out on potential sales and marketing opportunities.
• Dither and refuse both to to reduce the vision or to ship an untested product, until the market window has long passed and nobody wants the product anyway.

Now it appears that there may be a way around this problem. Dionysios G. Synodinos has written an ineresting article at InfoQ about “crowdsourcing” testing of such applications.

InfoQ: Crowdsourcing JavaScript Integration Testing with Test Swarm.

A cynical view might be that this is little more than a formalisation of the approach of releasing an untested product, calling it a “beta”, and waiting for customers to complain. There is a chance, though, that engaging a wide range of people explicitly as testers and providing them with the information they need to test the system thoroughly might result in a better, cheaper, and (most importantly) scalable answer to this kind of problem.