the art and observations of Ranjit S. Mathoda

Google’s mission is to organize the world’s information and make it universally accessible and useful. Edufire.com founder Jon Bischke asks a great question: How close is Google to attaining their mission?

There appear to be three components to Google’s mission: (1) getting access to all of the world’s information, (2) organizing the world’s information in a useful way, and (3) making sure everyone can access the information.

How much of the world’s information does Google access presently and how can this be expected to change in the future? Google faces a number of problems in getting access to the world’s information. These challenges vary based on where the information is stored (in the Internet cloud, on a computer, or generated in real life but not stored), the privacy of the information (public, private, secret), the legal status of the information (public domain, copyright but shareable, copyright but illegal to share), and the type of media (text, audio, video).

According to a paper on Google’s storage system (pdf), as of November of 2006 Google stored approximately 850 terabytes of information for its search crawler, 70.5 terabytes of information for its maps software, 4 terabytes of information in terms of what users search history is, 2 terabytes in Google Base, and 9 terabytes in Google’s Orkut. This sounds like alot, but as of today, the Internet is 3,000 to 5,000 petabytes of Internet traffic (with about 3 gigabytes of that data being monthly American usage) (see MINTS survey). Even accounting for the discrepancy between 2006 and 2008, and for traffic generated versus actual content of websites, that’s a more than 3,000 fold gap.

One significant problem Google faces with the Internet are that there are significant parts of the Internet that it is prevented from accessing. For example, Google is not allowed by Facebook to crawl most of the information on Facebook’s pages. Google doesn’t know your click history on Yahoo’s websites. Also, alot of our usage of the Internet isn’t actually captured in the Internet very well. Patterns of mouse movements and web page navigation, for example. Furthermore, as Jon Bischke points out, Google is also not very good at indexing audio and video files (although they certainly are working on such problems; see NYTimes on Google’s new image search).

To Google’s benefit, the utility of the Internet keeps pulling more and more information into it. Yet there is a tremendous amount of information that is generated every moment that is on only personal computers or mobile devices, and that amount of information is dwarfed by the information that is not on any computer. While these challenges can be tackled with new tools (like Google’s book scanner, or Google desktop search, or a Google backup service for your computer), Google is far from having access to all of the world’s information. There’s plenty of information companies generate each day that they do not put where Google’s tools can reach the information, from oil company geological data to a consumer products company’s internal documents. For Google the solution likely lies in helping create new tools to capture (life recorders) and manipulate (personal computer software, Google enterprise search) that information, even if the information is never put into Google’s cloud infrastructure.

Will Google be the place people go to find the world’s information organized in a useful way?

The second problem that Google faces is that no matter how useful it is in creating easy to use methods to organize information, there are new ways of organizing information that Google won’t control (Facebook, Friendfeed, etc.). To some extent Google is trying to route around this problem by encouraging software developers to create programs that interface easily with Google’s systems (Google Apps Engine) but such a solution seems limited at best. Like Apple, Google can try to create powerful interfaces that have the best utility for users, but they will be competing against the whole world, including their own former employees. They will never have the best system for accessing all information.

Can everyone access Google’s services?

The third problem Google has is that as ubiquitous as it is, and the Internet allows it to be, Google is not the most commonly used search system everywhere. Google has taken a number of steps to make sure it is easily accessible, from placing their search box wherever they can (including on Dell computers, the Firefox web browser, and the iPhone), to creating their own cell phone software (Android). Although much of the world is not on the Internet yet, technological development proceeds, wireless technology is proliferating across the developing world, and Google may have a shot at being the mobile device of choice in the developing world. The challenge for Google will be to create the best tools, both on the Internet and off it, both connected to Google’s overall architecture and stand alone so people can maintain their privacy, in a world of consistently expanding choice.

For all of the utility that Google provides to people every day, their mission statement is beyond a stretch goal. They are far from achieving it, and likely never will. Perhaps the economics of storage and access mean that it’s not worth storing certain kinds of information at all. Yet it is great that Google has such an audacious goal. Without trying to solve significant problems, no great accomplishments would ever be made. Clearly organizing and making useful even a small portion of the information generated by the world can be fantastically worthwhile and rewarding.

The Financial Times reports:

Apple is in discussions with the big music companies about a radical new business model that would give customers free access to its entire iTunes music library in exchange for paying a premium for its iPod and iPhone devices

While the music industry appears to be asking for a $100 premium, Apple is bargaining for a $20 premium (see the Financial Times article).

How much would you pay up front to have unlimited access to music on the portable device you carry around every day?

Most people have their personal devices for about 2 years.  A subscription service of $8 a month costs $192 over the life of the device, and is a lot more complex to use.  Thus by one measure the music industry’s price seems very attractive to customers.

Yet the average amount of songs sold through iTunes for every iPod is about $20. Any amount over $20 is bringing the music industry revenues it doesn’t have now. And as bands make more and more of their money from live events, it makes sense from an advertising perspective for their music to be distributed broadly, listened to frequently, and consumed with abandon. So while the music industry may hold out for more, at anything over $20 they are gaining revenues, usage and exposure.

Contrast one up front fee to the current system. Buying each song individually on Amazon.com or in iTunes requires many more purchasing decisions, which reduces the total amount of music listened to legally.  Downloading free music from the Internet is fraught with legal risks. Since many listeners to music are children or young adults, with a choice of asking their parents for more iTunes money for song purchases or downloading songs illegally, a device with an unlimited access to music also solves a significant problem for parents.

The future of media. The future of the music industry could be bolstered by such a deal. A steady form of legitimate revenue that its users will actually use, supplemented by ancillary revenues from greater live venue attendance, could make the industry stronger. The future of handheld media devices, particularly devices connected to fast wireless Internet networks, is looking fantastic. They are rapidly becoming the most powerful form of distribution for any form of digital media, from music, to books (see the Amazon.com Kindle), to software (see the iPhone software keynote). What’s next?

Here’s a quote that struck me for how really wrong it was:

“[According to Steve Jobs] Apple was supposed to become a wonderful consumer products company. This was a lunatic plan. High tech could not be designed and sold as a consumer product.”
John Sculley, CEO of Apple from 1983 to 1993, incorrectly understanding the potential of Apple, as stated in his 1987 memoir “Odyssey”

You can find my other favorite quotes at http://mathoda.com/quotes

As iPhone excitement reaches a fever pitch, let’s take a moment to ponder what Steve Jobs called Apple’s “hobby”, Apple TV (product page). Jobs explained to Walt Mossberg that Apple TV is not another set top box or a set top box replacement, but is “sort of a new DVD player for the Internet age.” Jobs also said Apple TV is a product that will evolve over time.

Many people in the developed world have a DVD player. It’s not as big a market as cell phones, but it is a huge market. It’s historically been a pretty bad business, unless you’re the low cost manufacturer, because the format in which DVD discs are encoded is a standard that any manufacturer can create a device for.

Toshiba and Sony each saw a solution to the commoditization of DVD players: create and control a new standard for a new type of DVD disc that carries a lot more data, enough to carry movies in high definition. Toshiba, with its HD DVD standard, and Sony, with its Blueray standard, are fighting a pitched battle over who gets to control that high definition DVD disc future. A good amount of their fight has been over convincing the movie studios to back their respective standards. Yet the confusion over which type of player to buy has kept many people from buying either type of player. And when hackers cracked the digital rights protection on high definition DVDs, Fox stopped distributing high definition movie discs.

Meanwhile, Apple and startups like Vudu (see my prior post on Vudu) are seeking to use Internet connected boxes that would replace the DVD player entirely. Is an Internet connected box a better approach than a high definition DVD player? For any type of content, to me it seems the Internet is a better medium than a physical disc in 4 ways and a worse environment in 3 ways.

The ways the Internet is superior are:

1. Marketing. It’s cheaper to do marketing. You can tease people with parts of content and it’s easier for people to share what they like. With community features like YouTube ratings and the Facebook social graph, you can let people know what content their communities prefer.
2. Upload, storage, distribution. It’s easier to allow people to upload. In the physical world, only a select few get DVDs made. In the Internet world, anyone can share their video. The technological costs of storing and distributing that video are rapidly decreasing as the Internet evolves.
3. Encryption. It’s technologically easier to create an encrypted environment on the Internet because you control the software on both the server, client, and data packet sides. In the physical world it’s harder to update broken encryption systems once part of the encryption scheme has been shattered.
4. Update the interface. The interface for Internet based solutions can be revised and updated, as we see in the rapid changes happening to websites. This advantage of the Internet will spread to Internet connected consumer devices.

The ways the Internet is inferior are:

1. Technologically it’s cheaper to move large amount of data by moving discs than by copying bits. High definition video takes up a lot of data space. For a long time it has been cheaper to move that data around by truck than by a network because each physical disc can store tremendous amounts of data. The advantage trucks enjoy is diminishing, however, because the Internet is evolving swiftly and peer to peer file sharing systems dramatically lower the technological costs of distribution.
2. Legally it’s cheaper to move discs then to copy bits. For professionally produced content it’s a bit more expensive to buy or rent movies than it is to share physical discs (thus the current disc through the mail business models of Netflix, Blockbuster, Gamestop, LaLa). This is because although technology makes it cheaper to copy bits electronically then to move them physically, the law of copyright requires you to obtain rights to copy bits electronically but often lets you move them around physically without seeking permission. Therefore, despite the greater technological costs of moving discs, they have less legal costs, and therefore potentially less total cost.
3. Painful to hook up. It’s a bit difficult to connect the Internet to your high definition television type displays. The interfaces on the devices currently used to make this hookup are generally pretty poor and keeping the Internet connection live adds a potential point of failure.

Yet that’s just where things are currently. In the spirit of Wayne Gretzky, let’s ask where the puck is going.

Are physical discs going to become better at marketing, upload (openness), storage, encryption, or changing their interface, than the Internet? It seems to me self evident that the answer is a resounding no.

Is it going to get technologically and legally cheaper to move discs onto the Internet relative to the legal cost of moving physical discs? The discrepancy in technological and legal cost will likely diminish, but it’s unlikely for the legal cost advantage of physical distribution to disappear unless more and more content is unencrypted.

There are multiple approaches to dealing with the legal cost of moving content around. Apple is trying its best to get around this disadvantage by creating one environment (iTunes) to serve content (music, video, games, software) to a multitude of devices (the desktop, laptop, ipod, iphone, Apple TV). Internet stores selling professional content in a digital form may be able to leverage their growing market share (see story) to diminish their legal cost disadvantage. Like Apple, Joost has created an encrypted hard to upload environment, but there are also unencrypted easy to upload environment (think YouTube), or encrypted easy to upload environments (think Brightcove), which all circumvent some of the legal cost disadvantages that the Internet has over physical distribution. The growing size of the Internet advertising market also potentially will draw more professional content onto Internet based distribution systems.

Is it going to get less painful to hookup the Internet to your high definition television type display? Undoubtedly. User interface design in consumer devices is something that Apple is very good at, but it’s not beyond the capabilities of a TiVo, Microsoft, Google, or other party to innovate in such a space. And one significant advantage of building consumer electronics devices with a built in Internet service to distribute media is that you can continuously upgrade the interface of that device. We’ve seen this in the desktop, the laptop, and hints of this with Apple TV and the iPhone. Device development goes from a relatively slow iteration hardware model to a super fast iteration web site like model.

Although leadership in the Internet age can change rapidly (see my post on Murdoch’s statement “They’re all moving to Facebook now”), it is possible to create competitive advantage in the Internet. Apple has shown they can do it by coupling beautiful hardware, elegant software interfaces, a minimalist aesthetic approach to what a customer actually finds most useful, and a growing library of professional content that it has the rights to distribute (iTunes) and amateur content (YouTube through Apple).  Internet connected consumer electronics devices is certainly a better business than being a commodity DVD player manufacturer, if you can get traction with the consumer, and can maintain a proper pace of innovation. This is a point that Toshiba and Sony, with their massive initiatives on a waning medium, would do well to heed.

Update, 1-9-08: An alternative to Apple TV or the various Windows extender devices is to attach a full computer to the television.  See http://scobleizer.com/2007/12/27/the-macmini-hdtv-revolution/

Update, 3-14-08: An even better alternative may be to get Netflix’s streaming movie service ($13.99 a month for 2 DVDs in the mail AND unlimited streaming of about a third of Netflix’s movies) onto your television. Currently this requires a Windows PC (thanks to Apple’s refusal to license digital rights management to Netflix), but in the near future this Netflix service is expected to appear on other consumer products devices.