Browsing, transcoding and intelligent proxies

I’m currently on vacation (just for a week, though), and therefore very much limited to browsing on my Nokia 9300 smartphone. It’s a good phone with a qwerty keyboard, a reasonably sized display, support for faster EDGE data transmission and relatively good email client. One area which I’m not too happy with is, however, regular internet browsing. I should be able to log into bloglines with it (this is a no-go as bloglines uses frames), check the news (without waiting a minute for the page to load), and in general have a much better experience. No wonder mobile browsing hasn’t taken off as projected…

We recently implemented full support for content distribution with handset recognition and some basic content transcoding functionalities, and this got me thinking..

Content transcoding

It’s very much possible, even on a bigger scale, to do behind the secenes content optimization and transcoding to make the mobile browsing experience a lot faster. Consider the usual browsing scenario: loading a page consisting of HTML, some JavaScript, a stylesheet and a lot of images. Typical page sizes asily exceed 100KB, which usually chokes the bandwidth and processing powers of a mobile handset. I guess that with some intelligent back-end infrastructure, carriers could make pages load 2-10 times faster.

What to optimize?

Now, I haven’t give this a lot of though, but here’s my list off content types that can be optimized, and operations that can be performed on them (in order of difficulty):

• Caching of content (this is so basic that it probably shouldn’t be listed..)
• HTTP optimization (keep-alive connection re-usage, gzip compression)
• Images (format conversions, bit depth, scaling, removal).
• Character set conversions
• Markup conversions (HTML, cHTML, WML, XHTML, SMIL -> client compatible markup)
• (X)HTML DOM transcoding
• Content removal (ads, embedded objects, etc.)
• Audio/Video clip conversions (format, bitrate, etc.)
• On-the-fly conversion of streamed video & audio
• Other optimizations?

The above mentioned techniques are pretty much browsing related, but they can also be applied to e.g. multimedia messaging (MMS), an area which has been suffering from compatibility and usability problems as long as it has existed. From what I know and have heard of, content transcoding is increasingly used to solve these problems.

Operating modes

As in compression, I think transcoding can have two different modes of operation. Either the original content is preserved fully or it is modified. Caching and HTTP optimizations do not alter the content, but can boost the performance of the actual transmission of the data. All the other types of transcoding listed above do change the actual data. Usually this isn’t a problem, the initial goal of transcoding is to make the end-user experience better, but consider e.g. ad removal. While it certainly makes loading of a page faster, its author might not be too happy about that.

Existing solutions

Google has for a long time provided a mobile version of it’s search engine that has provided on-the-fly HTML to WML conversion. Opera provides a service called Opera Mobile Accelerator which seems promising, and a company called Greenlight Wireless has a similar service name Skweezer. The Google service is very basic, but free, the two others have a monthly fee (well, a basic version of skweezer is available free). What I’m surprised at is that I found very few content optimization and transcoding solutions targeted towards mobile operators (e.g. as an intelligent proxy solution). I believe this is something that should be provided free-of-charge and seamlessly by the network operators to their end-users (something worth considering is whether they want to provide something that, in the end, reduces the amount of transferred data through optimizations.. many carriers charge by the kilo/megabyte…).

Other thoughts

What I haven’t seen done is automatic JavaScript transcoding. This would be increasingly beneficial, as dynamic HTML is used more and more. I recently did some cross browser DHTML work, and most of the time was spent on sorting out differences between browser implementations (I only had three to worry about: FireFox, Internet Explorer and Safari). In the wireless world there are a lot more browser variations that have some sort of support for JavaScript and DHTML. It would be interesting to dig into the possibilities of creating a intelligent proxy solution that could (besides other transcoding operations), automatically re-write JavaScript to match the DOM (document object model) of the browser that is used. Easy? Of course not. Possible? Certainly. This functionality could be offered as a service to web developers; consider uploading a web page consisting of HTML and JavaScript and getting back a cross-browser compatible version. Would I pay for that? Yes.