Frank Carver ~ Convergent Visionary

The measurement and comparison of web site and web application popularity is vital for any business model which depends on advertising for some or all of its income. Every advertiser wants to get the best value for money spent. In general, higher popularity implies higher value for advertising, although this can be offset by knowing more about visitors, and selecting appropriate advertising.

The world wide web has been around for decades. It would be nice to assume that measuring and comparing popularity of web sites is a solved problem. It most assuredly is not. With the growth of convergent applications and the spread of application usage across an increasingly diverse range of devices the situation can only become more complex.

There are several major ways to try and gain insight into web popularity. The oldest way is to examine the server logs of the systems hosting the application. Servers typically log every page request from every user. In theory, all that is needed is to determine which requests originated from which users, and the count of distinct users will give the popularity.

Unfortunately, there are several problems with this approach. The first problem is how to determine which logged requests belong to which users. Some applications use “cookies” (small nuggets of information stored by a web browser and passed back to the server on every request). However, at best a cookie only identifies a particular web browser on a particular machine – a user who visits a site from home and from work will likely have several distinct cookies. Other problems with cookies include some visitors deliberately switching off cookies, or clearing stored cookies and appearing like a new visitor. In a multiple device world, may devices simply do not support cookies.

Other approaches to correlating log entries and unique visitors, for example using IP addresses or URL parameters are even less reliable. It is also important to realise that many log entries will not be caused by real human visitors at all, but rather by “robots” such as the system Google uses to index web pages for its search engine.

The single biggest problem with relying on server logs for measuring popularity is that it is almost impossible to compare different systems. Not only will the information logged often be different, but server log files are private data. Each organization will probably be able to compare the popularity of its own web sites and applications with each other, but probably not be able to compare them with competitors or other established web sites.

As we have seen, there are significant problems with server logs for popularity measurement. This has led to a range of alternate techniques and services, each of which has its own advantages and disadvantages.

A familiar approach to advertisers is the one taken by Nielsen. Just as for TV ratings, a select group of people are issued with devices to track their internet usage, and the results are treated as statistically similar to the broader population. This has the advantage of being similar to the traditional way of rating TV, but it also has the same disadvantages of group bias and generalising from a small group to a much larger one. This approach is unique among web traffic analysis in that the company knows more than just the web activity of its sample group. Nielsen collects demographic and personal data about its test panel, and also interviews members to obtain more subjective information.

Nielsen’s reputation as a source of web rankings took a significant blow in 2007, when they arbitrarily decided to change the way of calculating overall popularity. There is also a strong argument that the statistical problems with using a small test group are magnified in the case of the web. Unlike TV where viewers choose from at most a few hundred channels, on the web there is an almost limitless number of sites to visit. For this reason, Nielsen web figures probably only make sense for the most popular few web destinations.

In an attempt to take a similar approach but to gather much broader statistics, Alexa have been busy for years encouraging web users to install monitoring software in the form of a “toolbar”. Alexa rankings suffer from some of the same problems as the Neilsen approach. Alexa generalises from a relatively small set of users, suffers potential bias toward the kind of people who will install their toolbar and, significantly, lacks data from anyone prevented from installing the Alexa software by company policy.

A third approach to tracking site usage and popularity is the one exemplified by Google Analytics. With this technique the site operator is required to place some special code in every page of the site. Whenever a visitor views a page, Google is notified, and compiles both specific and overall statistics. In general, this is considered a more accurate way of tracking and measuring overall site usage than the statistical approach taken by Nielsen and Alexa. However, it faces its own problems. Google’s embedded code relies on a specific browser feature (JavaScript) to be present and enabled and for the web browser to be able to effectively communicate with Google’s logging servers. If either of these are not working, then no activity is recorded from that user. Most significantly, though. Google Analytics shares a problem with the traditional analysis of server logs; the data gathered for any particular site is private, and unsuitable for comparison with competitors.

So, for now, there is no single best way to measure and compare web popularity. Attempts are continuing to try and solve this overall problem, but I don’t hold out hope for an answer any time soon.

As for other devices and communication methods, there is next to nothing. Some web-based techniques happen to work with a small number of mobile devices, and some TV-based techniques might work with some set-top box interfaces, but for now user and popularity tracking for convergent applications is still back in the “stone age” of reading log files.

I predict that there is a major business opportunity for anyone who can crack this problem.

One way or another it’s important to pay attention to Google. In this case Vic Gundotra, vice president of mobile and developer platforms at the company, has indicated that they value cloud computing over mobile applications.

“If the Web emerges as the ‘metaplatform’ across all these devices, that’s music to Google’s ears,” he said.

InternetNews Realtime IT News – Google: Cloud Trumps Apps on Mobile Phones.

This is interesting from a convergence perspective. Enabling a market where converget applications can simply and effectively make use of the capabilities of a wide range of differing devices is a key convergence point.

On the other hand, over-reliance on the emergence of a “metaplatform” is an easy way to avoid a lot of profitable niches right now, as is the tendency for such an approach to result in a “lowest common denominator” application.

Some researchers at the UC Berkeley Reliable Adaptive Distributed Systems Laboratory have produced a white paper attempting to pin down a definition of “cloud computing”.

Cloud Computing is a new term for a long-held dream of computing as a utility, which has recently emerged as
a commercial reality. Cloud Computing is likely to have the same impact on software that foundries have had on the hardware industry.

Read more at abovetheclouds.pdf.

Some visitors may have noticed that the “subscribe by email” option was not working properly, and would give an unhelpful “feedburner” error in many cases. This was very annoying, as allowing simple subscription by email as well as RSS and twitter is just the kind of convergence feature I recommend to my customers.

I have now fixed this, so please feel free to subscribe by email if that is what you prefer.

Thanks for your patience.

Convergent technology can seem complex. Particularly if you embrace the idea of different people using the most appropriate devices they have, at whatever time and place that suits best. An application, service or product is not just a web application or a mobile product, or a facebook plugin. It’s all of them according to the needs and preferences of the users and customers.

But there’s a point where convergent solutions become even more complex. When all these different modes of use are happening at once it can seem like chaos.

Last year, on another blog, I wrote about the untapped potential of the apparently simple “clicker” devices becoming popular in education. From a convergence point of view the response from potential users has been mixed. Institutional inertia is a major barrier to the uptake of even such simple possibilities.

On a more positive note, Olivia Mitchell wrote How to Present While People are Twittering, which encourages presenters to embrace the “back-channel” offered by services such as twitter.

As yourself – how open are you to the possibilities offered by the convergence of disparate communications technology into a “back-channel”?

I’m sure you have heard it, or maybe even based a business on it. The idea that “legacy”, or “low spec” or “dumb” mobile phone handsets will soon fade away, as everyone upgrades to iphones, blackberries, androids and the like. The fairytale continues with the implication that this will leave the market open and desperate for whatever web-in-your-hand application is being pitched at the time.

The saddest part of this story is that the same assumption has been trotted out for years, and the promised smartphone dawn has never come. Headlines shouting about billions of devices and the huge potential of mobile business can easily be read the wrong way. Globally, cheap and simple communication devices are still outselling all kinds of smart phones. Even in developed areas the proportion of users with smart phones is tiny.

The much touted 11% or 12% smartphone sales does not translate to 11% or 12% smartphone users. Current figures estimate between 4 and 6 billion active mobile phones, but smartphones are only in the tens of millions. This means that phone users are buying ten times as many “dumb” phones as “smart” ones, and there are hundreds of basic mobile phones in use for every one smartphone.

Image by Esthr

A bit of thought should help explain why this is the case. It’s a truism that people are different, and the mobile phone market has expanded to its current size largely by exploiting those differences. Advertising targets individual devices at every identifiable niche: the budget-conscious get cheap and simple, the style conscious get broad colour and accessory choices, business buyers range from sedate to pretentious, and all the other options which pack out a typical phone retailer. If what drives the sale is the colour and the association with a celebrity, why cut into your profits by jamming in expensive smartphone features? If what drives the sale is the low cost of rental, why cut into your profits by including copious data transfer?

That seems compelling enough, but even so it does not include the enormous number of older, limited devices already in use. Many reasons, including cost, familiarity, and general lack of desire to change, act to ensure that existing handsets will not go away any time sooner than a generation or so.

Basing a business model on the dream that everyone has, or will have, a smartphone merely leaves you fighting with all the other similar dreamers for a fraction of an already miniscule market. Worse than that, a market already flooded with free (or mostly free) products and services. Sure, there are a small number of people making quick cash in the iphone application market. But remember that there are also a lot of people making money selling ringtones, images and the like for old, dumb phones. And a whole lot more people losing money in these same areas.

It can be a tough choice, but for me the key value-add for any business which wishes to make a profit in mobile is to realise that mobile devices merely form part of a much broader picture. Break down the imaginary walls between device types: web users typically find it easy to click, drag, scroll and type, users of simple phones generally find it easy to speak and listen, or perhaps send SMS text messages, or even USSD. Set-top box users might find it more comfortable to view rich video information while pressing next, previous, play, and fast-forward buttons on a TV remote. And yes, smartphone users may prefer to poke a screen with a stylus, or swipe it with finger gestures.

Once you step back from the assumption that smartphones are a single market for an isolated product, you have the freedom to evaluate the markets, costs, timescales and return on investment of providing support in your application for all these different types of devices, and grow your customer base organically into untapped areas.

As a sound bite: Make smart choices, don’t just assume smart phones.

Mobiles are unique – if you want to miss out on the opportunity they represent, you could choose to see them as just slow computers with tiny interfaces and dodgy Internet connections. Then try to squeeze in your traditional applications; try squeezing the office desktop metaphor with its sedentary documents into a device the size of a mouse!

Alternatively, see them as the most personal, social and dynamic of devices that are becoming connected to the Internet. Now a multi-billion-scale global opportunity opens up to you. That’s customers and dollars! In trying to grasp this, some are calling it ‘Mobile 2.0′, by analogy with its sibling, Web 2.0.

Duncan Cragg gets the point of modern convergent technology. Well worth a read for anyone interested in this space.

Read more at The Mobile 2.0 Killer App is the App Killer.

I have heard the term “technical visionary” as a job title recently. This is a not a traditional and well-understood title, so I thought I’d look around to see if I can work out what it might entail.

Rags Srinivas, CTO of Sun’s Technology Outreach team seems to qualify as a technical visionary. His background includes teaching and software development. As for key responsibilities, he cites:

• Think ahead: What’s down the road?
• Help steer along technology releases.
• Support the rest of the staff

In the field of industrial physics, it appears there is a more rigorous definition:

Industrial technical visionaries are defined as ‘technical individuals who effectively synthesize multiple technologies and business strategy to identify new and innovative breakthrough products and processes’.

In August 2007, a paper was published by the IEEE titled Motivating Technical Visionaries in Large American Companies (login required). Although skirting around an exact definition of the role, the paper does discuss some ways to recognise or characterise a technical visionary:

Technical visionaries, while rather rare, and thus, difficult to systematically study [19], [63], are instrumental to the new product development (NPD) portfolio through their ability to spot and see the potential for breakthrough opportunities, and as such, are part of the key to success for many technology-driven organizations.

The new product development literature focusing on the fuzzy front-end of the NPD process has hinted at the idea of a key individual who is responsible for creatively melding the technology and market insights that occur during this stage [36], [41]. The opportunity recognition literature supports this view as well [8], proposing that it takes a very unique person who possesses both technical and market vision to be successful at identifying opportunities.

Because technical visionaries are, by definition, different from the typical population of engineers, we also expect them to be motivated somewhat differently than more typical or average engineers. The creativity literature suggests that they will be motivated by the job or problem at hand [55], [58] and by the amount of freedom that they receive [58], and that these factors will influence them more than they influence other technical professionals.

Outside the corporate and academic world of definitions and job titles, there exist people popularly accepted as technical visionaries, such as Wayne Green, characterised by radical thought:

Wayne … still has the dynamism and ability to ‘think outside the box’ that has driven him throughout his life.

and strong opinions

Wayne seems to have had views on everything, so it’s a certainly that almost everyone will disagree with him on some point or other.

There don’t seem to be many job openings on the job sites for technical visionaries, but I did find one which describes the desired skills as:

• Excel in multiple programming languages
• Both the left and right sides of your brain work! (well-rounded)
• Not afraid of taking risks and challenging conventional wisdom
• Have an entrepreneurial, whatever-it-takes spirit
• Like the excitement of working within a small start-up team
• Have high energy and enthusiasm towards what you do

Although these sources all differ on some details, the general sweep of the descriptions seems to be the broadly the same. I’ll try and summarise them into a single definition.

A Technical Visionary has a varied background (technical, teaching, a bit of business etc.), possesses both marketing vision and very good technical skills, has enthusiasm and strong opinions, is not afraid of challenging conventional wisdom, has a whatever-it-takes spirit, and is motivated by challenges and the freedom to address them.

Above all, such people are rare and very valuable. So if you find any, do whatever it takes to keep them and use them to benefit your business!

User interface design is more than just screen design, it is about the journey the user takes to accomplish their goals.  Remembering that could have saved this particular organisation a lot of money.

Marc McNeill  keeps an eye on the business impact of usability, in particular the difference between interface design and interaction design. The user journey | dancingmango.

Jason at productivity software startup “37 Signals” posts a reminder that chopping between tasks has a cost in both time and quality.

If you want to do great work, focus on one thing at a time. Finish it and move on to the next thing.

Read more at Multitasking is the fastest way to mediocrity.