Steve Jobs is not like you and me. He has millions of customers, 32,000 employees, and a board of directors who think he can do no wrong. Running a company that is immensely profitable, gaining in market share, has no debt and $20 billion in cash, he can afford to make bold moves, the most recent of which is his decision to replace Tony Fadell, until moments ago head of the division that produces Apple’s iPod. Like everything Jobsian, Fadell’s departure is part of an Apple GRAND PLAN. The variables at work here are (in no particular order) ego, competitive advantage, ego, management technique, ego, strategic thinking, and ego. To say that Steve Jobs’ ego can expand to fill any known space might be an understatement but I’ll stand by it anyway. Fadell’s failing in this regard is his being hailed as the “father of the iPod.” What does that make Jobs? Who made THE BIG DECISION? Who committed the company? Who – most importantly of all – seduced all the record companies? That last guy would be James Higa, but since I don’t want to get HIM fired, too, let’s just attribute it all to Steve Jobs – for all intents and purposes the REAL father of the iPod. All hail Steve. Apple exists solely as an extension of Steve Jobs. Remember that. Anything attributable to Apple is really attributable to Jobs. Other people work at Apple, of course, and excel at their positions, but that is primarily because they were chosen, anointed, or inspired by Jobs. Not that Jobs doesn’t make the occasional mistake. Look at the Mac Cube, for example. But that was our mistake as consumers, not realizing that it really ought to have been worth an extra $500 to us to have a computer with no cooling fan. Steve Jobs makes very few such mistakes, in fact. That, and his total domination of Apple at every level allow the company to be literally the only PC vendor to have anything like a strategic plan. Dell and HP have the odd strategic initiative, like getting into or out of media players or TVs, but the idea of a comprehensive corporate strategy, well that’s too much to expect from companies that are managed, not led. Steve Jobs is a leader 100 percent in the mold of General George S. Patton. Rent the movie and it will start to make sense. Heck, rent it on iTunes. So here’s what’s going on with Tony Fadell. First, he was vulnerable as a charismatic leader in his own right who has been talked about in the press as a possible heir to Jobs. That alone meant he had to die, but it wasn’t enough to mean that he had to die just now. That decision required an external variable in the form of former IBM executive Mark Papermaster. Steve Jobs wants to give Tony Fadell’s job to Papermaster. It’s not that Papermaster would be any better at the job than Fadell, but there are two over-riding factors here: 1) Jobs can only have so many direct reports, and; 2) he thinks putting Papermaster in Fadell’s job is the best way to get past any legal objections from Papermaster’s former employer, IBM. Papermaster most recently ran IBM’s blade server division and in the mind of Steve Jobs blade servers and iPods couldn’t be farther apart. One is an enterprise sale while the other is consumer. One is a clear IT sale and the other has nothing to do with IT, really, since iPods and iPhones aren’t aren’t computers or computer peripherals. Jobs thinks Apple can make this point stick with a judge and he might well be correct. Papermaster has to be gone from IBM for a year before he can take a job that clearly competes with his last position at IBM. But Jobs doesn’t want Papermaster for blade servers, nor does he even want him for iPods. Jobs wants Papermaster for the expertise he showed two jobs ago at IBM running Big Blue’s PowerPC operation. Jobs wants Papermaster to lead Apple’s PA Semi acquisition and create a new family of scalable processors optimized for Snow Leopard and beyond. Having Papermaster run iPod hardware is a placeholder to let him get used to Apple and get ready to take over the Apple processor job, some of which will be used in iPods and iPhones, so the job isn’t a total waste. But for the few months he’ll be running iPod hardware, Papermaster will mainly be overseeing the implementation of Fadell’s strategy. If that seems like a game of musical chairs in the Cupertino executive suite, well it is. It’s also a game we’ve seen played over and over again. Back to point 1 from five paragraphs ago: Jobs can have only so many direct reports. Steve Jobs believes the key to his success is in finding, hiring, retaining, then firing the best talent in the world. He would maintain in the very moment he’s firing Fadell that Tony is better at his job than anyone else on Earth. Yet still Fadell must go and that’s because – ego issues aside – Jobs had to make room in his inner circle for Papermaster. Everyone close to Jobs is under continual analysis: is this person really (or still) the best in the world? If they aren’t, or if someone else is just as good but more important for some additional reason, then the incumbent has to go. Steve Jobs ultimately betrays all of his direct reports in this manner. It’s just the way he is. And if it costs Apple a few million to remove one extra head from the room, well that’s okay with a board that KNOWS (as we all do, to put it fairly) that Jobs really is the secret of Apple’s success. His system may be brutal, but it works. So Fadell was already in danger because he had become known as an individual. Remember that when PortalPlayer (now part of nVidia) was making the guts of every iPod the company was forbidden by Apple to acknowledge that. Even in its financial reports PortalPlayer was forbidden to use the “A” word and simply had to attribute to some unnamed company 85 percent of PortalPlayer’s revenue. Just as Jobs was scourging Fadell, though, he was seducing Papermaster. Jobs can be VERY seductive. And he was hardly going to seduce the IBM executive with a promise to put him two levels down. So as the most vulnerable person in Jobs’ inner circle, Fadell had to go. That Fadell’s wife was head of Human Resources for Apple and was forced, essentially, to terminate her own husband, well that was just gravy and yet another reason for Apple employees to take the stairs rather than risk sharing an elevator with Jobs. Fear can be a remarkable motivator. Don’t feel bad for Fadell, though. His $8+ million golden parachute stock grant is coming at a time when Apple shares are depressed and could easily double by the time he can sell them in 2010. He get’s $300,000 per year to “advise” Jobs (I’d like one of those jobs, too, Steve) and then there’s his wife’s departure package, which hasn’t been mentioned. Clearly out of the picture as an heir to Jobs, Fadell will next appear in 2010 as a CEO somewhere in the South Bay. Of course IBM with its largest corporate legal department on earth has filed suit against Apple, trying to block Papermaster from taking the Apple position. Apple’s legal department is fairly accomplished, too, and Cupertino is a much stronger company than Armonk, which will lead to the ultimate solution to this legal problem. Apple still hopes to convince a judge that it is correct about Papermaster. But if Apple fails in that, Steve Jobs will just pick up the phone and choose IBM Microelectronics as the fab to build the next generation of Apple’s PowerPC processors – a contract worth billions, but ONLY if IBM drops all legal action. Apple will win in the end -- I guarantee it. And the way Jobs negotiates, Big Blue will probably end up losing money on the chip deal, too.

It isn't very often I get to apply Moore's Law to a non-Information Technology business and rarer still that I can then relate the whole thing back to Microsoft, so I'm going for it. Here's what the solar power industry can teach us about Microsoft: The wonderful thing about Moore's Law is what the lady at the bank called the "miracle of compound interest." That halving of manufacturing cost every 18 months (the OTHER way of looking at Moore's Law that we generally don't use) has little apparent impact in the first few years, but eventually the halving and re-halving takes a real bite out of the cost side until substantial performance is very, very cheap. That explains why there is more computing power -- a LOT more -- in your iPod than was required for the Apollo Moon missions. Well this applies to ALL silicon-substrate photolithography applications, not just computer chips. It applies equally well, for example, to silicon solar cells. There are many types of solar cells. Some solar cells involve crystalline silicon just like computer chips and others use amorphous silicon, but all types benefit from Moore's Law. In fact one especially good aspect of solar cells is that they can make use of older process technologies that are obsolete for computer work. So every time Intel or AMD builds a new fab there is a market in the solar industry for their old machines. Look at those round solar cells used in many arrays today and you'll notice the smaller wafer sizes favored in Silicon Valley 15-20 years ago. That's no coincidence. The result of this relentless application of Moore's Law to the solar industry is that we can see a time in that near future when the cost of producing a watt of electricity from a solar cell on your roof will be approximately the same as the cost of delivering that same watt over a power line from an electric utility. And of course that means that 18 months after that point the solar watt will cost HALF of what the same power would cost from the electric company, which will completely change the game. The time when that electricity cost parity will be reached, I'm told, is seven years from now. Just think of the impact that will have on electric utilities! Why would any of us continue to buy our power from them? We might use them as a giant storage battery and possibly for backup on cloudy days, but why would we use them at all for power if we can generate it cheaper at home? You can bet that's a question the electric power generating industry is asking itself. The whammy for the power companies is two-fold, because not only will power be cheaper but, by definition, the cost of building and installing solar panels will be substantially cheaper, too, than it is today. If it costs $40,000 on average to refit your house today, a lot of homeowners can't afford that, but what if it becomes $10,000? That's what worries electric companies that are used to having easier access to capital than do their customers. But once installing solar power costs relative chump change (the cost of a nice Ski-Doo or remodeling a bathroom), we'll see massive conversion and the power companies know that. So what can they do? They can find ways to get us to use more power than can possibly be generated from the roof of a typical American home. And that's why this week the Electric Power Research Institute proposed that we all get plug-in hybrid cars. It would save billions of barrels of oil, they say, lower greenhouse gas emissions, clean the air, oh and by the way require more electricity than your solar cells can produce, thanks. And it will work -- for a while. But Moore's Law is relentless, you know, and the role of electric utilities will change dramatically over the next decade as a result. As far as I can see, this is all for the better. But what does it have to do with Microsoft? Well that brings us to Windows Azure, which was still called Windows Cloud when I first mentioned it a couple weeks ago. Like all Microsoft strategies, Windows Azure is a reaction to external competitive pressures. And it is important, VERY important. Here's how a source of mine at Microsoft put it a few days ago, before the Azure announcement: "The cloud stuff isn't just another enterprise product. It is going to impact everything we do -- all of the product groups -- consumer and enterprise -- are going to have to figure out where they fit in to the cloud paradigm. The shift to cloud-based computing is analogous to our shift to the Internet in the late '90s. It changed the direction of the company and impacted everything we did." Wow, that's a big deal! Yet based on the Microsoft announcement this week, all Windows Azure looks like to me is Microsoft's effort to sell web services or maybe cut the sticker shock for smaller businesses adopting SQL Server. But more properly, it likely means Microsoft's acceptance that computing clients may eventually be free or nearly so. In short, Windows Azure is an insurance policy against the possible Vista-like failure of Windows 7. Last week, for example, I wrote about Microsoft's Windows Mobile technology, predicting that it would die simply because Redmond would realize that it could never be first or second in market share. That was no big scoop from me, though some news people took it as one -- it was just common sense. And so what happened this week? Well here's a report from a reader attending Microsoft's Professional Developer Conference, where Windows Azure and Windows 7 were introduced this week. "Windows Mobile has (a) near zero presence at MS PDC," wrote the reader. "Their Live Mesh platform has Windows Mobile as an integral component but otherwise no mention, no sessions. There was one session scheduled but it was cancelled at the last minute. Hmmm." When the body is under stress, it eventually sacrifices entire limbs to keep the internal organs working. Windows Mobile is just an appendage to Microsoft and always has been. Yet mobile is clearly the client of the future, so what's to be done? Windows Azure. Control the back end through industry standard -- even open source -- protocols. Make money from subscriptions and ads -- make money any and every way in the hope of leveraging a global infrastructure investment into a continuing business strategy. Can you see the connection here? There is almost no difference between Microsoft trying to become our computing utility and the electric company trying to power our next-generation cars. Both are coping strategies, both are risky, but neither Microsoft nor the electric utilities see that they have any real choice. And maybe they don't. For Microsoft, at least, it could be a strategy with legs. While the utilities will be undercut more and more by Moore's Law, Microsoft as a computing utility won't be. But that doesn't mean they'll be any good at the job. It means fighting a war on two fronts -- with Google as a provider of applications and with Apple as a provider of content. MAYBE Microsoft has a shot against Google, which is becoming more Microsoft-like itself by the day, but to compete with Apple as a content provider? Forget it. Microsoft simply isn't the class act it needs to be to dominate that space, so look for acquisitions to (maybe) fill that void. And all this means that Windows 7 had darned well better hit a home run or Microsoft is in BIG trouble.

I am not a very sophisticated mobile phone user. I don't use most of the bells and whistles on my phone, probably because I don't know what they even are. But just because I'm an idiot about USING mobile phones doesn't mean I don't understand the emerging mobile market, to which I have been paying a lot of attention of late. And why not? As personal computers fade from what Al Mandel called "ubiquity to invisibility," something has to take over. And everyone I respect thinks the new dominant platform will be mobile. So it's my job to tell you, then, that Windows Mobile is probably doomed. Interestingly, this conclusion isn't based on any personal preference or subjective analysis. I'm not saying that Windows Mobile is bad, just that it is probably doomed. It's a simple matter of market economics. There is generally room in any technology marketplace for three competing standards. Notice I say "standards," not "brands." There can be many brands of road vehicles, but they generally come down to cars, trucks, and motorcycles -- each a standard. In personal computers we have Windows, Macintosh, and Linux (or similar Unix workstation variant). In HVAC systems, just to stretch the point, there are radiant, forced air, or evaporative systems -- again three standards. And among those three standards there tends to be a market-share distribution that is more or less 85-10-5. These numbers can jump around a bit and one can argue that the Mac is now more than 10 percent of recent sales, though not of the installed PC base, so I hope you get my point. This magic 85-10-5 distribution also happens to mirror what happens at the racetrack or in the casino, where 85 percent of gamblers lose, 10 percent break-even, and 5 percent are winners, which explains all those big buildings in Las Vegas. The mobile phone marketplace shows a similar distribution, though that now appears to be in some transition. One could argue that the old 85-10-5 came down to basic or dumb phones (85), smartphones (10), and specialized or vertical phones like the old Nextel (5). Moore's Law now seems to be inexorably turning all phones into smartphones, so we're probably moving toward an 85-10-5 based on programming platform. Let's consider this smartphone migration for a moment, first with Samsung in mind. Last week Samsung announced that it would no longer be making high-end phones and would stick to basic phones in the future -- going for higher volumes at lower cost. This makes a lot of sense given that sophisticated phones must cost more to develop yet tend to be more expensive as a result and therefore have lower sales numbers. So why bother? This was the message Samsung sent out and everyone bought, but I really think that if you look at it in the context of a dynamic market the announcement means something else altogether. Deconstructing the Samsung announcement we'd have to wonder how the company sees itself and its competitors. That answer is pretty simple: Samsung sees itself as a global electronics company competing with outfits like Sony. Samsung has been for 40 years all about copying and eventually crushing Sony. Now given that's how Samsung sees itself (nobody I know contests this vision, by the way), how can the company possibly afford to let Nokia, Motorola, Sony, and Apple make high-end phones, which is to say smartphones, without Samsung competing in that space? That would be giving up a lifelong dream and Samsung just won't do it. So were they lying? No, Samsung wasn't lying, they were just doing what my old friend, PR man Martin Quigley, called "dissembling." Samsung probably has no intention of abandoning the smartphone market because ALL phones are becoming smartphones. What they truly intend to do, however, is make smartphones that are generally inexpensive, hoping to gain market share as a result. We'll see this trend from Apple, too, which will push iPhone prices down and introduce cheaper models next year and beyond. While there are many ways for Samsung to make smartphones less expensive, the easiest way to do so and yet remain competitive on features is by no longer using software that costs money. In the smartphone space there are, at present, only three operating systems that are being broadly licensed on an OEM basis -- Android, Symbian, and Windows Mobile. Of those three, two are free -- Android and Symbian. Symbian didn't used to be free but times change and now it is. So Samsung was announcing that it would be ending development of Windows Mobile devices at some point, though they never said that directly. Sticking with Samsung for a moment, then, which of the two free software platforms is the company likely to endorse? That's a good question. Symbian has a very strong presence in Japan, which is an important market for Samsung, so I don't see them abandoning Symbian immediately. But in the longer term I think Samsung WILL abandon Symbian, as will most of the rest of the world. Here's why (donning flameproof clothing): Symbian is simply too old. The OS is getting slower and slower with each release. The GUIs are getting uglier and are not user-friendly. The development environment is particularly bad, which wouldn't hurt if there weren't others that are so much better. Symbian C++, for example, is not a standard C++. There is little momentum in the Symbian developer community, maybe because coding for Symbian is a pain. Yes, there are way more Symbian phones in circulation, but those phones will be gone 18 months from now, probably replaced by phones with a different OS. Lately, Symbian's success has been primarily based on the high quality of Nokia hardware, on the loyalty of NTT DoCoMo, and now on the lure of being recently made open source and therefore free. But if open source developers don't flock now to Symbian (they aren't as far as I can see -- at least not yet) then the OS is doomed. My guess is that in time Samsung, like Motorola, will devote its smartphone development 100 percent to Android. Maybe, but what about Apple and RIM, what will happen there? This is not a time to bet against the iPhone, which is changing the entire landscape of not just smartphones but mobile phones in general. For all its teething problems, there is a new sheriff in town and his name is iPhone. We'll see nothing but progress and market-share gains there for at least another two product cycles or three years. RIM is another story altogether. What RIM has going for it are loyal users, good keyboards, and push mail. Most mobile phone users still think RIM is the only platform that has push mail. But given that push mail will soon be everywhere and the market will eventually figure that out, RIM is facing a huge challenge. I'm not saying they won't meet that challenge, I simply don't know. If I had to bet right this moment on the mobile 85-10-5 of 2011 I'd say iPhone, Android, then RIM, Symbian, or something completely new from behind Door Number Three. Why iPhone over Android? For exactly the same reason why the iPod holds that approximate 85 position among music players, including ones using open source software. iPhone has a really great SDK (light-years ahead of any other right now). The App Store distribution platform is great, but locked on too many points. This is a careful timing issue for Apple. If they open the APIs too quickly they risk being blocked. They need to open an API once they are perfectly sure it is the right one and the right way to export that function. Apple is going to relax the restrictions progressively when they better understand the use cases and what are the best APIs. In the meantime it is giving an advantage to Android, but one that I think a year from now Apple will have reclaimed. And where will Windows Mobile be in 2011? There way things are headed now, given that Microsoft can't really afford to be anything but first or second on the platform that supplants Windows, I'd say Windows Mobile will be dead.

Apple last week introduced a pair of very nice notebook computers that, not at all surprisingly, looked like riffs on the MacBook Air. The company in a separate announcement released 600 high-definition television episodes through the iTunes Store. This week Apple will reportedly release new 20-inch and 24-inch iMacs, also for the Christmas season. Two weeks, three announcements, but what strikes me (and apparently only me at this point) is what won't be announced -- the big surprises that are missing. What happened? A MacBook and a MacBook Pro are nice, but not overwhelming. I like the dual GPU in the Pro and I hate the lack of a FireWire port in the MacBook, but beyond that there is little to say about these products except that the glass screens (on the iMacs, too) are better for houses like mine filled with LCD screen-destroying pre-school boys. These new products don't appear to break any price or performance barriers and sure as heck don't allow time travel or make me more handsome. We were led to expect more -- a lot more. And I am not talking about rumors. Back on July 21st in his regular conference call with industry analysts, Apple Chief Financial Officer Peter Oppenheimer said that Apple's profit margin would likely shrink from 34.8 percent in the just-concluded quarter to 31.5 percent in the quarter ending in September. "We've got a future product transition that I can't discuss with you today," Oppenheimer said as he spelled out the reasons for the anticipated profit reduction. "One of the reasons that we see gross margin being down sequentially is because of a product transition." What kind of Apple product could be expected to come along, taking a $244 million profit hit for the company? It certainly isn't any of the products we've discussed so far, nor is it the iPhone 3G or the new iPod Touch, which have both been publicly dissected and found to have gross margins in the 56 percent range. It's something else that was probably intended to be announced this week but wasn't. The change of plan could have come for many reasons. Maybe the revolutionary product wasn't ready in time. Maybe introducing an aggressive, low-margin product in the middle of a global financial crisis was considered a bad risk. Maybe some strategic alliance had to be in place and wasn't ready. Whatever it was, the same analysts will ask about it this Tuesday when Apple has another such conference call scheduled. But of course none of this keeps me from speculating about what's missing from Apple's announcements and the reasons it might be missing. I think the delayed product has everything to do with Apple's desire for Blu-ray DVDs to die as a standard. Apple CEO Steve Jobs took a swipe at Blu-ray in last week's announcement -- a swipe that felt out of sync with the rest of the program. Steve has no difficulty at all NOT talking about subjects he wants to avoid, so leaning into Blu-ray was not at all offhand or without strategic importance. Don't expect Blu-ray drives on Apple computers, Steve said, yet he didn't offer a clear alternative. The alternative Jobs would like to offer, of course, is full 1080p HD video distribution on iTunes, but that's not currently possible. It will happen in time, of course, but certain prerequisites have to be in place. Apple hardware has to support it in a practical sense, for one. Interestingly, users of the new Apple notebooks began reporting that CPU utilization for H.264 decoding on their new machines dropped from 100 percent on an earlier model with the same processor to sub-20 percent on the new aluminum MacBooks. Though it wasn't announced, Apple seems to have (finally) enabled H.264 decoding on the Nvidia GPUs in these new machines. Equally significant is the fact that ONLY H.264 appears to be accelerated. HD content using the MPEG-2 or VC-1 codecs seem to be not accelerated, which means this improvement is aimed specifically at Apple (iTunes) content, NOT physical media. This is something we might have expected Apple to trumpet, but they don't offer any 1080p content yet other than movie trailers. Maybe it is better, Apple might imagine, to pre-seed this capability so more machines can take advantage of it when Apple 1080p content finally does appear. What's yet to come, I'm guessing, is Apple's next OS X release, Snow Leopard, with QuickTime X -- the first version of QuickTime supposedly optimized for H.264 hardware decoding. Snow Leopard is late, but then operating system updates are always late, no matter the vendor. This delay could be for any number of reasons and there are probably several, but one of them I can guarantee you has to do with H.264. More than a year ago I made a big point of predicting that Apple would go to H.264 hardware acceleration, though I pinned it on a specific chip from NHK and NTT in Japan. This was after the usual evening of drinking with Japanese executives that typically reveals such information. Oh the sacrifices my liver makes for journalistic integrity! So what happened to that NTT chip? I don't know. Maybe it was too expensive and fell out of the plan. Maybe it's in there still and Nvidia licensed technology from NHK and NTT to enable the new hardware acceleration (this is just a speculation -- I'm not at all saying they did). Maybe -- and this is the one I believe -- the discrete NTT chip was overtaken by a snowballing Apple strategy involving PA Semi, Apple's recently acquired division that designs microprocessors. Here's the reasoning, which isn't all mine by any means. I have the smartest readers in the world and they are constantly giving me new things to think about. First we see Apple moving away from Intel chipsets with these new MacBooks and probably with the iMacs to be introduced this week. Apple has always been involved in its own chipset design and giving up that capability to Intel until now didn't make much sense, especially considering the crappy Intel integrated graphics. It is logical to assume that Apple would reclaim its right to design or at least specify the chipset as soon as its internal engineering capability could support that. Turning to Nvidia isn't the same as doing the chipset itself, but you can bet Apple's fingers are all over the chipsets in these new machines and that they are significantly different from those in notebooks from companies like Dell and HP. There is a reason to be different here that goes beyond performance. That other reason is Psystar, the would-be seller of Mac clones with which Apple is now locked in a legal battle that could go either way. What if Psystar comes out on top and has the right to sell Mac clones based on the Hackintosh model? Then Apple will have to break that model by becoming more proprietary and therefore harder to emulate. Enter the third-party chipset, which is just the first step in Apple's effort to become immune to Psystar-type clones no matter what the court decides. Second is Snow Leopard, itself. Apple has suggested that Snow Leopard won't have many new features but will be Apple's effort to more fully take advantage of multi-core processors. This harkens back to my parallel computing column of a couple weeks ago. Snow Leopard, I'm told, will make seamless use of as many cores as are available. It isn't clear whether applications will have to be rewritten to take advantage of this capability, but I'm guessing they will have to be. This is just a guess, mind you, but is consistent with the sort of demands Apple likes to place on developers. Apple's own applications will be Snow Leopard-compatible you can bet, and will set a daunting performance standard in iMovie and Final Cut Pro. Where PA Semi fits in is by providing to Apple a modular, Intel-compatible multi-core architecture that can scale to cover entire future Apple product lines. By dishing out more responsibility to the GPU, Apple can enable a much simpler CPU with as many cores as needed. Imagine a single core in an iPhone, two cores in an Apple TV, 2-4 cores in a notebook, 4-6 in an iMac, and 8+ in a Mac Pro. Wait a year then refresh all those platforms by doubling the number of cores with no change in software. Moving to its own microprocessors would maintain Windows compatibility (though possibly at some lower performance level), cut hardware costs by $200 or so, and make it that much harder for others to build Mac clones in the future. And what about the jump to 1080p video distribution on iTunes? That will require faster hardware, especially on the Apple TV, which really needs a refresh. It would have been nice to introduce all of this for Christmas, but I'm not surprised it slid. And maybe January MacWorld is better, anyway, if Apple can also introduce new Mac Pros for content creation and those rumored giant Apple displays (HDTVs) with their built-in Apple TVs.

A couple of columns ago we touched on the practical rebirth of parallel computing. In case you missed that column (it's in this week's links), the short version is that Moore's Law is letting us down a bit when it comes to the traditional way of increasing the power of microprocessors, which is by raising clock speeds. We've hiked them to the point where processors are so small and running so hot that they are in danger of literally melting. Forget about higher clock speeds then; instead we'll just pack two or four or 1000 processor cores into the same can, running them in parallel at slower speeds. Instantly we can jump back onto the Moore's Law performance curve, except our software generally doesn't take advantage of this because most programs were written for single cores. So we looked back at the lessons of parallel supercomputers, circa 1985, and how some of today's software applies those lessons, such as avoiding dependencies and race conditions. But we didn't really talk much in that column about the use of threads, which are individual processes spun off by the main CPU. Each time the microprocessor adds a new task, it creates a thread for that task. If the threads are running on the same processor they are multiplexed using time-slicing and only appear to run in parallel. But if the threads are assigned to different processors or different cores they can run truly in parallel, which can potentially get a lot of work done in a short amount of time. Most traditional PC applications are single-threaded, meaning the only way to make them go faster without a completely new architecture is to run the CPU at a faster clock rate. Single-threaded apps are simpler in that they are immune to the dependencies and race conditions that can plague true parallel code. But they are also totally dependent on tasks being completed in a quite specific order, so in that sense they can be dramatically slower or dramatically more resource-intensive than multi-threaded apps. For an example of where single-threaded applications are just plain slower, consider Eudora, which is still my favorite e-mail client (I'm old, you don't have to tell me). Until not very long ago Eudora still couldn't send mail in background, so everything (and everyone, including the user -- me) had to wait until the mail was sent before completing anything else, like writing a new message or replying to an old one. I KNOW THIS IS NO LONGER THE CASE, SMARTY-PANTS -- THIS IS JUST AN EXAMPLE. The program was single-threaded and, since sending mail is a very slow activity, users were generally aware that they were waiting. Today Eudora sends mail in background, which is the same as saying "in another thread." Multithreading has been great for user interactivity because nothing should ever stop the input of data from typing, mouse movements, etc. There are many ways to use threads and before we consider some let's think about scale -- literally how many threads are we talking about? To run at true clock speed we'd have only one thread per CPU core, but a fast processor can multiplex hundreds or even thousands of threads and multi-core processors can do even more. So the EFFICIENT shift to multi-threaded programming requires a significant change in thinking on the part of developers. Here's an example: A hard problem with programming games is when you want something to happen every so often. That's not very efficient to code because it traditionally requires a program loop that spins as fast as the CPU will let it (making the CPU go to 100 percent) and keeps checking the time to see if it is time to do that thing. But threads are different: With threads you can very easily put them to sleep for any period of time, or even put them to sleep indefinitely until some event occurs. It's not only easier for the programmer, it takes nearly no CPU power compared to the looping system. How do you use threads to write an e-mail server that handles thousands of simultaneous incoming e-mails? Well, you write it as if you were writing a server that can only handle ONE e-mail at a time. Just write very simple code that knows how to accept e-mail, then test it by sending in an e-mail. It works? Cool. Now send 1000 threads into that same piece of code. Each thread has its own state as to what e-mail (FROM, TO, SUBJECT, etc.) it is receiving, but despite the fact that that the content is different, the process is exactly the same. Now you have an e-mail server capable of serving a thousand simultaneous connections. See, writing multi-threaded apps may require a different approach but the benefits from doing so can be fantastic. A new area of multi-threaded programming that is REALLY hard (hard even for developers who do normal multi-threaded programming really well) is the use of optimistic concurrency. Two columns ago I alluded to this in my example of Appistry's decision to forego using a database for a credit card processing application. I said I would show a hack that was yet another approach to the same problem. Well here comes the hack. Let's consider this problem: My wife (the young and lovely Mary Alyce) and I happen to be in different parts of town, each of us standing in front of a bank ATM machine. I am a thread, Mary Alyce is a thread, and the ATM is main memory. Contrary to our usual behavior in which we only take money OUT of the bank, we are paradoxically planning near-simultaneous bank deposits. Our balance starts at $1000. I am depositing $200 while Mary Alyce is depositing $300. The ATM does the process like this:

1. Retrieve existing balance
2. Add new deposit to that balance
3. Update new total balance

The trick is that we are both doing this same thing at the same time. What if this happens: Bob starts deposit transaction Mary Alyce starts deposit transaction Bob's ATM grabs the balance ($1000) Mary Alyce's ATM grabs the balance ($1000) Bob deposits $200 (his ATM updates the balance to $1200) Mary Alyce deposits $300 (her ATM updates the balance to $1300) Bob's ATM updates the main database with the new balance ($1200) Mary Alyce's ATM updates the main database with the new balance ($1300) This is a race condition. Mary Alyce updated the balance last so my deposit is lost completely. There are many ways for this to work out still, but also many ways for it not to work out OK. And any traditional solution requires a LOT of back-end reconciliation and computation. We need something simpler. Classic concurrency control would basically LOCK the database. This is called, not surprisingly, "pessimistic concurrency." So I go up to the ATM and my ATM requests the database to be locked for me. If Mary Alyce then went to another ATM it would tell her she has to wait because the database was being updated elsewhere. This ensures that the race condition can't happen, but it also holds up Mary Alyce, who does not like to be kept waiting. Optimistic concurrency control says, "We KNOW there could be a race condition, but we'll add a very cheap way to detect if it occurred. And if so, we'll pay the very expensive cost of restarting one of the transactions from the beginning." The only changes from the above sequence are in the last two lines: Bob's ATM updates the main database with the new balance ($1200) Mary Alyce's ATM updates the main database with the new balance ($1300) These become: Bob's ATM updates the main database with the new balance ($1200) so long as the current balance is still $1000. Mary Alyce's ATM updates the main database with the new balance ($1300) so long as the current balance is still $1000. That's easy to write but trickier to implement. The check of the balance and the update of the new balance must occur within the microprocessor as an atomic action. It all must happen basically as one single operation. Some processors have had these instructions for a long time, but they're pretty common now and called "test-and-set" or "compare-and-set" instructions. If the "so long as" part fails, the whole transaction must be restarted from scratch. In our example, Mary Alyce's $300 would pop back out of the machine and she'd have to start over. That's very expensive for Mary Alyce, but the actual occurrence of the race condition is very rare. So although the redo is expensive it hardly ever happens, so no one has to wait for another person doing an update operation. Apply this to 25,000 ATMs and suddenly the database is decoupled from transaction processing and the system is additionally controlled for internal race conditions such that it can run with less code and at full speed, which is saying something. Suddenly the system can be 100 times faster (cascading 10X improvements) or run 10 times faster on one tenth the hardware (take your pick), all thanks to the timely embrace of clever multi-threaded programming.

My promised column on threads will appear in this space on Friday. It would have appeared here today but the crumbling global financial system suddenly seemed a more appropriate topic. We're in trouble and by "we" I mean the whole darned planet. What started as a mortgage problem in the U.S. has blown into global financial paralysis that threatens us all with recession and maybe even with depression. I know I'm feeling depressed, how about you? The crisis seems immune to any and all efforts to fix or end it. NOT passing a $700 billion mortgage bailout can send Wall Street into a tailspin, for example, but then finally passing the bailout didn't seem to improve things, either. The Federal Reserve and Treasury Department are running out of tools and time yet still the system flirts with suicide. So I say it is time to take a completely different view of the problem and to look to a new leader to solve it, in this case Jack Welch. Jack Welch is the retired chairman and CEO of General Electric who took the company during his 23-year tenure from being worth about $14 billion to about $410 billion by really MANAGING the business and concentrating on creative use of capital. I've written columns and columns deriding managers as a profession but none of that applies to Jack Welch and GE, where managers really manage -- they manage the heck out of the place and to generally good effect. Jack Welch built that system, he has time on his hands, I say let's give him a new job. There is very little difference, in fact, between the global financial system and General Electric. Welch saw GE entirely in terms of cash flows and the application of capital to those parts of the business where it would do the most good. Welch also thought in terms of continuous quality improvement, which is virtually unknown on Wall Street OR in Washington, where such things aren't even talked about, much less measured. I've been thinking about this crisis and a lot of it comes down, I believe, to a fear of failure especially on the part of the banks. There is no credit available to anyone, anywhere, no matter what the credit rating or score. This is because the banks are frozen by fear to the point where they won't even lend to each other much less to customers. This fear of failure seems to be pretty much guaranteeing failure. And the regulators are now throwing what will soon be trillions of dollars at trying to break these bankers out of their paralysis. But I think there is a better way: use this very fear of failure as a motivator. Before we get to Jack let's deconstruct this current psychological crisis on the part of the banks. They aren't lending money because they are afraid it won't be repaid. They won't lend even to each other because banks seem to be failing all over yet there hasn't been an instance yet when an overnight loan has resulted in default. So what's the problem? More properly, what is the outcome the banks fear? They fear going out of business either through honest failure or through being forced to merge or having their deposits taken away by the Federal Deposit Insurance Corporation (FDIC). In short it comes down to fear of losing their licenses, because as a highly regulated industry the banks can only do business at all with the permission of government. Right now they are totally fixated on the idea that if they lend money and it isn't repaid the government will pull their licenses. Yet the government has made it clear that the most important thing is to LEND MONEY, breaking this credit paralysis. All the banks know this but none of them want to be the first to take the big risk of lending money. You do it! No, you!! Enough of this crap. What if Jack Welch was the U.S. banking czar? We know the result of all such crises these days in the U.S. is the appointment of a czar -- a new government official drawn from industry and charged with cutting across agency lines and streamlining an ultimate solution. We did it in energy and terrorism and I'm sure we'll do the same thing now so let's just think ahead a bit. Let's assume that's the case here and that whatever President is in office names Welch. What would Jack Welch do? If Welch ran the U.S. banking system like he ran GE, he'd kill the bottom 10 percent of banks every year and fire the lowest 10 percent of bankers. What impact would that have on the system? Would it make it better or worse? Well it couldn't be worse, could it? Doctors have a way of measuring pain. It seems our brains can only focus on one pain source at a time, so if you have a pain in your gut you really don't notice the pain in your ankle. So there is a device called a palpometer that goes on your arm or leg and is calibrated to produce standardized and replicable levels of pain. Put this gizmo on, turn up the pain, and when you get to the point where the patient suddenly goes from saying, "My head hurts," to "My leg hurts," then you know how much pain they are in. This was all pre-waterboarding mind you. Right now all the bankers are afraid to lend money because they are afraid of failure. As the new banking czar Jack would much rather have them be afraid of HIM. If the bottom 10 percent of bankers were fired every year and the bottom 10 percent of banks had their branches and deposits redistributed, wouldn't they be more afraid of THAT than of making bad loans? Their motivation would still be to make GOOD loans, but the penalty for making NO loans would be there, too. Our problem then is that we're throwing money at something we should be handling using a different regulatory tool -- licensing. U.S. bank regulators should go to all the banks this afternoon and say, "You aren't making loans, which is part of the definition of what it is to be a bank. If you aren't acting like a bank by tomorrow we'll take away your banking license and transfer your deposits to another bank that WILL make loans." Problem solved overnight. It's only one part of the problem, of course, but this solution will cost a lot less than $700 billion. It will cost nothing. And if you think it won't work, then you don't know Jack.

Last week I was in Boston to moderate a panel at the MIT Technology Review’s Emerging Technologies Conference — one of those tech shindigs so expensive I can only attend as hired help. My panel was on parallel computing and it produced this column and another I’ll file early next week. This week is about databases and next week is about threads. Isn’t this a grand time to be a nerd? Thanks in part to Larry Ellison’s hard work and rapacious libido, databases are to be found everywhere. They lie at the bottom of most web applications and in nearly every bit of business software. If your web site uses dynamic content, you need a database. If you run SAP or any ERP or CRM application, you need a database. We’re all using databases all the time, whether we actually have one installed on our personal computers or not. But that’s about to change. We’re entering the age of cloud computing, remember? And clouds, it turns out, don’t like databases, at least not as they have traditionally been used. This fact came out in my EmTech panel and all the experts onstage with me nodded sagely as my mind reeled. No database? No database. Parallel computing used to mean scientific computing, where hundreds or thousands of processors were thrown at technical problems in order to solve them faster than Moore’s Law might otherwise have allowed. The rest of us were relying on rising clock rates for our performance fix, but scientists — scientists with money — couldn’t wait so they came up with the idea of using multiple CPUs to solve problems that were divided into tasks done in parallel then glued back together into a final result. Parallel computing wasn’t easy, but sometimes that was the whole point — to do it simply because it was so difficult. Which is probably why parallel computing remained a small industry until quite recently. What changed was Moore’s Law put an end to the clock rate war because chips were simply getting too hot. While faster and faster chips had for the most part linear performance increases along with cost and power consumption decreases, the core temperature inside each microprocessor chip was going up at a cubic rate. Back in 2004 Intel released a chart showing that any clock speed over 5 GHz was likely to melt silicon and Moore’s Law would, by 2010, make internal processor temperatures similar to those on the surface of the Sun! For those, including me, who think that’s pretty darned hot I’ll point out one of my astronomer readers immediately had to mention that the Sun’s chromosphere is actually much hotter than the surface. Forgive him, he means well. Faced with this absolute thermal performance barrier, Intel and AMD and all the other processor companies had to give up incessant clock speed increases and get us to buy new stuff by putting more than one CPU core in each processor chip can. Now chips with two and four processor cores are common and Intel hints darkly that we’ll eventually see hundreds of cores per chip, which brings us right back into the 1970s and ’80s and the world of parallel computing, where all those principles that seemed to have no real application are becoming very applicable, indeed. And that’s exactly where databases start to screw up. Bob Lozano, chief visionary, evangelist, father-of-eight (same woman) at Appistry came up with the first database example I’d heard and it was eye opening. Appistry (I’ve written about them before — it’s in the links) specializes in distributing what would normally be mainframe applications across tens, hundreds, or even thousands of commodity computers that act as one. If a government agency wanted to do meter-scale analysis of very high-resolution images from spy satellites, it might use Appistry, I’d imagine (just a guess, of course). The database example Bob used at MIT was of an unnamed customer that is a credit card transaction processor. Their core application — the processing of credit card transactions — was happening on IBM Z-series mainframes (the biggest of big iron) and the client wanted to port the whole mess to commodity PCs. Google did it, why not them? But the first time they tried it at Appistry, it didn’t work. “We hired a technical team that had done similar applications before so they started with replicating the mainframe architecture on the commodity computers, database and all,” Lozano recalled. “But when they finished it wasn’t appreciably faster than the mainframe it replaced. Even worse, it wouldn’t scale. So we fired the team and started over.” The problem, it turned out, was in the database. The way the original mainframe application functioned was by first receiving transactions, writing them to the database, reading them from the database, then doing the actual processing before writing them again back to the database. That’s read-write-read-process-write. The second time through the Appistry team tossed the database, at least for its duties as a processing platform, instead keeping the transaction — in fact ALL transactions — in memory at the same time. This made the work flow into read-process-write (eventually). The database became more of an archive and suddenly a dozen commodity PCs could do the work of one Z-Series mainframe, saving a lot of power and money along the way. If this sounds like a risky way to do business — not writing the data to disk until things slow up enough — remember that’s the way Google runs its search engine and why it is so darned fast. Google has THE ENTIRE INTERNET IN MEMORY AT ONCE. If the application slows down they just add more hardware. This is good news for cloud computing and bad news for mainframes, because systems like Appistry and its competitors (there are several) are going to eventually bury the mainframe by putting the “cloud” into cloud computing. Suddenly a Storage Area Network with a relatively weak database controller is good enough for archiving while the parallel or even massively parallel cloud does the real work. Later this month Microsoft will announce a cloud version of Windows Server and it will be very interesting to see how it handles database integration and dis-integration. The database problem is much more than just slow reads and writes. Relational databases also create false dependencies between pieces of data. Dependencies of any kind break parallelism, and therefore make an application hostile to commodity platforms. That is, if one chunk of data (A) is dependent on another chunk of data (B), then no work can be done on A until all work on B is complete. If the dependency is real, like when A and B are both withdrawals from the same bank account, then there are hacks we can try like one I will describe in my next column, but most programmers just choose to have a cup of coffee and wait for B to finish. But if these are withdrawals from different bank accounts, or maybe even different banks, then no true dependencies exist. Unfortunately, if all of this has been stored in a single relational database then we unintentionally create a false dependency, since that database can only handle a fairly limited amount of items concurrently — we’ve created a bottleneck that will choke the application. While the database guys are busy figuring out how to add more and more concurrency internally, in reality when you take a few steps back and think of a large set of commodity boxes all executing a single data munching app, then no matter how sophisticated we get, the relational database will still effectively be a single thread to that app. A traditional response is to pour dollars into the data tier, buy faster, more concurrent SANs, better interconnects, and bigger database servers. That works up to a point and pleases Sun and IBM no end. Somewhat more helpful though are data grid products like Coherence or eXtreme Scale (IBM), or Appistry’s Fabric Accessible Memory (FAM). For many applications those can take more of the lifetime of each chunk of data and keep it in memory in the middle tier — hopefully on the same boxes where this large data-munching app resides. But this still doesn’t completely solve the problem because there remain limits on how far the relational database can scale. Here’s how Google attacks this problem, which goes beyond simply keeping the entire Internet in memory. The problem, of course, is that you can’t keep the entire Internet in memory in every server because then you’d need more memory chips than even Google can afford to buy. To scale the Google search service, then, they figured that many large problems did not intrinsically require doing actions one at a time. But Google first had to free itself of the false dependencies. So they coined the term MapReduce and created both a set of operations and a way to store the data for those operations natively, all while preserving the natural independence that is inherent in each problem, building the whole mess atop the remarkable Google File System, which I’ll cover some other day. Google led the way but many other companies have followed suit, opening doors to a wide range of new ways of thinking about large-scale data manipulation. Suddenly there are different ways to store the data, new ways to write applications, and new places (thousands of cheap boxes) to run such applications. What this does for Larry Ellison and his libido is a great question, because it looks like he’s bought up most of the traditional database-centric software industry just in time for it to be declared obsolete. Sorry Larry.

Last week I was in Boston to moderate a panel at the MIT Technology Review's Emerging Technologies Conference -- one of those tech shindigs so expensive I can only attend as hired help. My panel was on parallel computing and it produced this column and another I'll file early next week. This week is about databases and next week is about threads. Isn't this a grand time to be a nerd? Thanks in part to Larry Ellison's hard work and rapacious libido, databases are to be found everywhere. They lie at the bottom of most web applications and in nearly every bit of business software. If your web site uses dynamic content, you need a database. If you run SAP or any ERP or CRM application, you need a database. We're all using databases all the time, whether we actually have one installed on our personal computers or not. But that's about to change. We're entering the age of cloud computing, remember? And clouds, it turns out, don't like databases, at least not as they have traditionally been used. This fact came out in my EmTech panel and all the experts onstage with me nodded sagely as my mind reeled. No database? No database. Parallel computing used to mean scientific computing, where hundreds or thousands of processors were thrown at technical problems in order to solve them faster than Moore's Law might otherwise have allowed. The rest of us were relying on rising clock rates for our performance fix, but scientists -- scientists with money -- couldn't wait so they came up with the idea of using multiple CPUs to solve problems that were divided into tasks done in parallel then glued back together into a final result. Parallel computing wasn't easy, but sometimes that was the whole point -- to do it simply because it was so difficult. Which is probably why parallel computing remained a small industry until quite recently. What changed was Moore's Law put an end to the clock rate war because chips were simply getting too hot. While faster and faster chips had for the most part linear performance increases along with cost and power consumption decreases, the core temperature inside each microprocessor chip was going up at a cubic rate. Back in 2004 Intel released a chart showing that any clock speed over 5 GHz was likely to melt silicon and Moore's Law would, by 2010, make internal processor temperatures similar to those on the surface of the Sun! For those, including me, who think that's pretty darned hot I'll point out one of my astronomer readers immediately had to mention that the Sun's chromosphere is actually much hotter than the surface. Forgive him, he means well. Faced with this absolute thermal performance barrier, Intel and AMD and all the other processor companies had to give up incessant clock speed increases and get us to buy new stuff by putting more than one CPU core in each processor chip can. Now chips with two and four processor cores are common and Intel hints darkly that we'll eventually see hundreds of cores per chip, which brings us right back into the 1970s and '80s and the world of parallel computing, where all those principles that seemed to have no real application are becoming very applicable, indeed. And that's exactly where databases start to screw up. Bob Lozano, chief visionary, evangelist, father-of-eight (same woman) at Appistry came up with the first database example I'd heard and it was eye opening. Appistry (I've written about them before -- it's in the links) specializes in distributing what would normally be mainframe applications across tens, hundreds, or even thousands of commodity computers that act as one. If a government agency wanted to do meter-scale analysis of very high-resolution images from spy satellites, it might use Appistry, I'd imagine (just a guess, of course). The database example Bob used at MIT was of an unnamed customer that is a credit card transaction processor. Their core application -- the processing of credit card transactions -- was happening on IBM Z-series mainframes (the biggest of big iron) and the client wanted to port the whole mess to commodity PCs. Google did it, why not them? But the first time they tried it at Appistry, it didn't work. "We hired a technical team that had done similar applications before so they started with replicating the mainframe architecture on the commodity computers, database and all," Lozano recalled. "But when they finished it wasn't appreciably faster than the mainframe it replaced. Even worse, it wouldn't scale. So we fired the team and started over." The problem, it turned out, was in the database. The way the original mainframe application functioned was by first receiving transactions, writing them to the database, reading them from the database, then doing the actual processing before writing them again back to the database. That's read-write-read-process-write. The second time through the Appistry team tossed the database, at least for its duties as a processing platform, instead keeping the transaction -- in fact ALL transactions -- in memory at the same time. This made the work flow into read-process-write (eventually). The database became more of an archive and suddenly a dozen commodity PCs could do the work of one Z-Series mainframe, saving a lot of power and money along the way. If this sounds like a risky way to do business -- not writing the data to disk until things slow up enough -- remember that's the way Google runs its search engine and why it is so darned fast. Google has THE ENTIRE INTERNET IN MEMORY AT ONCE. If the application slows down they just add more hardware. This is good news for cloud computing and bad news for mainframes, because systems like Appistry and its competitors (there are several) are going to eventually bury the mainframe by putting the "cloud" into cloud computing. Suddenly a Storage Area Network with a relatively weak database controller is good enough for archiving while the parallel or even massively parallel cloud does the real work. Later this month Microsoft will announce a cloud version of Windows Server and it will be very interesting to see how it handles database integration and dis-integration. The database problem is much more than just slow reads and writes. Relational databases also create false dependencies between pieces of data. Dependencies of any kind break parallelism, and therefore make an application hostile to commodity platforms. That is, if one chunk of data (A) is dependent on another chunk of data (B), then no work can be done on A until all work on B is complete. If the dependency is real, like when A and B are both withdrawals from the same bank account, then there are hacks we can try like one I will describe in my next column, but most programmers just choose to have a cup of coffee and wait for B to finish. But if these are withdrawals from different bank accounts, or maybe even different banks, then no true dependencies exist. Unfortunately, if all of this has been stored in a single relational database then we unintentionally create a false dependency, since that database can only handle a fairly limited amount of items concurrently -- we've created a bottleneck that will choke the application. While the database guys are busy figuring out how to add more and more concurrency internally, in reality when you take a few steps back and think of a large set of commodity boxes all executing a single data munching app, then no matter how sophisticated we get, the relational database will still effectively be a single thread to that app. A traditional response is to pour dollars into the data tier, buy faster, more concurrent SANs, better interconnects, and bigger database servers. That works up to a point and pleases Sun and IBM no end. Somewhat more helpful though are data grid products like Coherence or eXtreme Scale (IBM), or Appistry's Fabric Accessible Memory (FAM). For many applications those can take more of the lifetime of each chunk of data and keep it in memory in the middle tier -- hopefully on the same boxes where this large data-munching app resides. But this still doesn't completely solve the problem because there remain limits on how far the relational database can scale. Here's how Google attacks this problem, which goes beyond simply keeping the entire Internet in memory. The problem, of course, is that you can't keep the entire Internet in memory in every server because then you'd need more memory chips than even Google can afford to buy. To scale the Google search service, then, they figured that many large problems did not intrinsically require doing actions one at a time. But Google first had to free itself of the false dependencies. So they coined the term MapReduce and created both a set of operations and a way to store the data for those operations natively, all while preserving the natural independence that is inherent in each problem, building the whole mess atop the remarkable Google File System, which I'll cover some other day. Google led the way but many other companies have followed suit, opening doors to a wide range of new ways of thinking about large-scale data manipulation. Suddenly there are different ways to store the data, new ways to write applications, and new places (thousands of cheap boxes) to run such applications. What this does for Larry Ellison and his libido is a great question, because it looks like he's bought up most of the traditional database-centric software industry just in time for it to be declared obsolete. Sorry Larry.

In the early 1980s I was a volunteer firefighter for a tiny community in the Santa Cruz Mountains of Northern California. We all lived in a beautiful redwood forest and our task was to keep that forest from burning down in a huge conflagration, taking us all with it. The job was made all the harder because our little part of paradise hadn't burned since the 1920s, so there was 60+ years of flammable undergrowth just waiting to light off. The current financial crisis facing the United States and the world really isn't much different from that. An unmanaged forest, one without the sort of fire control we attempted to provide, would naturally burn every few years. The undergrowth would build up, reach a critical mass, some source of ignition would come along -- usually lightning -- and all that undergrowth would burn. The redwoods themselves would be scarred but not really threatened, as we could see from the charring that marked them from countless such fires over centuries. Of course burning undergrowth threatened homes and property, too, so there was a natural desire on the part of that community to want the next burn to not come this year, please not this year. So there came a policy of aggressively fighting fires with the result that we eventually faced 60 (now 90!) years of flammable material growth rather than six or eight years. And the probable fire fueled by 60 years of undergrowth would have been so bad that our job changed to one of trying to prevent fires from happening, well, ever. This was an impossible task, of course. Eventually the stars would align the wrong way and the whole place would burn down, we all knew it. Just let it not happen on our watch. Does this sound familiar? Now America and much of the world face the possibility of recession and we handle that by first arguing about the definition of the term. Are we or are we not in recession? This distinction appears to be very important to some who view it like a forest fire: is it burning or not? Implicit in this distinction, I suppose, is the idea that if we're not burning -- if we are not in recession -- that maybe through some miracle we'll never face that problem. Whether this is a practical attitude or not depends entirely on your event horizon -- how soon you expect things to change. The people in power in this country have a relatively short event horizon. Politicians tend to think of two, four, or eight years as the longest periods of time that matter. Corporate honchos might look out further, you could guess, but they don't since the average duration for a U.S. Fortune 500 CEO is under four years. So while the intent of the fire chief and the mayor and the governor and the Congressman and the President and the CEO is that there be no unpleasant surprises during their watch, all of them know such surprises are coming. This short-term focus in the face of long-term difficulties leads to odd behavior at times. In the forest it is usually better to let smaller fires burn or even to deliberately set them, yet few fire chiefs are willing to take that risk, even though NOT taking the risk is so much worse. In financial markets, as companies crumble and governments prepare bailouts, short sellers pile on in scrums of doom that make the shorts rich yet hurt society. Traders with a trading mentality, they can't help themselves any more than Ralph Nader can keep from running for President. "But George Soros did it to the Bank of England," they say, as if that makes everything okay. The American economy is at the end of its longest-ever period without a recession. Through sleight of hand and a fair amount of financial fraud we've managed to keep the "R" word out of our communal vocabulary for 14+ years. We did this through a succession of bubbles -- first the Internet bubble and then the real estate bubble with a little war bubble thrown in between. Financial bubbles are unnatural enough in themselves, but piling bubble atop bubble defies logic, yet still we managed to make it happen, helped somewhat by half a trillion dollars in war spending, all with borrowed money. By rights the end of the Internet bubble should have sent us into recession, burning the financial undergrowth and setting up the next period of prosperity. But we avoided that and the result is what we see now -- something far worse. Recession is inevitable at this point, yet still we talk about avoiding it. This is crazy talk. If we as an economy somehow push the next recession back a few more years, it will be all the worse when it comes. AND IT WILL COME. Everybody in power knows this; none of them argue against it; they all know recession is inevitable; they know that forestalling a recession at this point will only make it worse. Yet still they pray, "Not on our watch, please." So we plan $700 billion bailouts on top of hundreds of billions in other knee-jerk measures all intended simply to forestall the inevitable and therefore ultimately bound to fail. "Just not on our watch, please." The cost of delaying the inevitable is high, but we won't pay it, our children and grandchildren will. We can argue policy all day and whether this matters or not, but it is hard to argue that any such cost doesn't matter IF IT DOESN'T WORK. So I propose The Cringely Plan -- a relatively small attempt to point public policy in a direction that makes sense for a change. The Cringely Plan won't avoid a recession because that's impossible. The Cringely Plan is intended, rather, to look beyond the inevitable recession and assist with the ultimate recovery and beyond. What we need are energy and economic policies that play to the strengths of government, not its weaknesses. Governments are good at big moves that force changes of direction, not little moves intended to, for example, create the economic "soft landing" we'll start to hear about in a couple months. Governments are best at turning super tankers, not at docking them. And while governments can print money and thereby pay for a lot of stuff, they are actually most effective just telling us what we can or can't do, more than anything else. We can see that, for example, in the Clean Air and Clean Water Acts of the 1960s that significantly improved the lives of all Americans without the government having to pay for it. The Cringely Plan does nothing for banks or mortgages. That's a problem that will have to work itself out, I think, and will if we give it a chance to do so. But we won't give it such a chance, I'm guessing, because it is an election year and because government is viewed as stupid and able to be threatened into paying for the most amazing things that it shouldn't. So The Cringely Plan has to look past banks and mortgages to energy policy and economic stimulus. It would be interesting to know what the mortgage market would be like had the price of a barrel of oil not hit $140+. There still would have been a mortgage bubble bursting, but I wonder when? Whatever the banking situation, though, we still need an energy policy and simply have not had one for decades. So get ready, here it comes; The Cringely Plan calls for: Prohibiting the manufacture and sale of incandescent lights. Kind of a letdown, eh? That's it. No ethanol subsidies, no drilling in wilderness preserves, no tax credits, no artificial price supports, no enriching good ol' boys from the oil patch. And no cost to government at all. As our incandescent lights wear out we'll have to replace them with something else, primarily compact fluorescents and LEDs. Yes, this will lead to the closure of light bulb and filament plants in Ohio and elsewhere, costing a few thousand jobs. At the same time it will create jobs in the non-incandescent light industries as those ramp up. But most importantly, it will within a year (based on a 700-hour life for incandescent bulbs) lead to an 18 percent drop in U.S. electrical demand. Dropping electricity demand by 18 percent will have an interesting effect on the electric utility industry. A simplistic view would suggest that our electric bills should drop by 18 percent. Cynics will point out that fixed overhead built into the regulated industry will keep bills from dropping that much simply because doing so would unduly hurt utility profits. I don't think so. Our utility bills pay for power that comes from many sources, some more efficient or profitable than others. Our bills also pay for building new power plants, some of which are again more efficient or profitable than others. An 18 percent cut in demand will have a huge impact on the production strategies and capital spending plans of every U.S. utility. More expensive plants built to serve marginal demand -- plants like smaller gas turbines for example -- could be taken off-line, dropping the average cost per kilowatt. Large new plants that have been in the works (and that we've been paying for) for years can be eliminated or delayed. Cancelled plants would lead to higher profits that lead to lower electrical rates. So our electric bill won't go down 18 percent, they'll go down 25 percent, which is a savings of $22 per month for the average American home. Twenty-two dollars per month!? Big deal. Was there an economic component of this energy policy? Well, it's a savings of $29 billion per year EVERY YEAR FROM NOW ON, which pales the recent and easily forgotten economic stimulus package that sent $600 checks generally not to the people who needed them most. How much of a positive effect those checks had on the economy is open to debate, but how much effect they had on reducing energy consumption isn't, because that effect was nada, zilch, zero. $29 billion also amounts to about the annual debt service cost of either the Iraq war or the coming $700 billion bailout. Take your pick. Want another energy policy with positive economic implications? Make single-phase lighting illegal for businesses with ceiling heights above 15 feet. Three-phase power at 380 or 440 volts is more efficient, requires less copper, and saves money overall. Once you have it in place for lighting it's a no-brainer to also use it for electric motors, where the savings are HUGE. The financial payback period for such conversion is under two years, which is an imputed annual interest rate of 50 percent. What investment can you reliably make that pays back 50 percent year after year after year? Still no government money has been spent. Take this a step further and require all new residential construction to use 380V three-phase power for EVERYTHING, which is to say turn the U.S. into Europe, where homes already use less power for the same level of service because of three-phase efficiency. Now this is a big change, of course, because it means getting all new electric appliances, TVs, everything. What's wrong with that? It's not like the products don't exist, since the same manufacturers are already making them for sale in Europe. Market expansion leads to lower prices. Swapping out 10 percent of the U.S. appliance market per year ON TOP OF NORMAL ATTRITION would create a huge boom in electronics and electrical goods. Okay, maybe this one will cost some money, but no more than those $40 digital decoder subsidies the government is already handing out. These are just a few examples of what can thoughtfully be accomplished. There's a natural role for government here and that's setting rules. They are good at that. Government hasn't always been as good at enforcing rules as setting them, but that doesn't mean rules aren't worth having. Let's just set a few new ones that make sense as we recover and rebuild after the economic fire to come.

I’ll begin this third and (I promise) last column on IT management with a confession: I have been fired from every job I have ever held. This is certainly not something I set out to do, nor did I even realize it until one day my young and lovely wife mentioned that I had never told her about voluntarily leaving any position. It’s not that I’ve had so many jobs, either. This one and the one before it have kept me going for more than 20 years. But they always seem to end the same way. This one might, too. You can never tell. Most of the times I have been fired it’s because I’ve been judged to be unmanageable, which is to say I won’t shut up. The ultimate reason given is usually something minor. The last time around, for example, I was fired because I didn’t transfer the cringely.com domain to my employer. They asked me to do it and I said “no.” Had they said, “Transfer the domain or you will be fired,” I might have decided differently. But they never said that — never gave a hint of the consequences — so I assume the real goal was less to get the domain and more to get rid of me. The guy who had me fired, Stewart Alsop (maybe you’ve heard of him), ultimately lost his own job for firing me, at least according to International Data Group Chairman Pat McGovern, who told the story to 300 people once at a DEMO conference. Back when I was a kid and working at WWST Radio in Wooster, Ohio, I was fired for writing those seven unspeakable words in the middle of a livestock auction report. Another kid had been playing similar tricks on me for weeks, but when I finally retaliated he turned me in. I guess I was a threat to him and didn’t know it. One company hired and fired me three times and another company hired and fired me twice. And somehow in all this I’ve never received severance or unemployment compensation. I just found another job or it found me. There’s a point to all these firing stories and they actually do relate to IT. I’m typical of a lot of IT types. You know us. We are useful but sometimes a pain in the ass. We have opinions and speak our minds and don’t suffer fools at all. We stand up to authority from time to time. Sometimes we’re wrong. We get fired a lot and hired a lot, too, because we are generally useful, though dangerous. What do you do with folks like me if you are a manager? At a traditional newspaper you’d either fire me or make me a columnist. And, sure enough, look where I am. In an IT shop you give me a task to do and let me do it, generally on my own, and never EVER put anyone under me, because I am hopeless as a manager. But it turns out I’m not so bad as a leader. Weird, eh? The last two columns have shown that IT is the Cousin It of American industry. We serve the company but often don’t feel part of it. Certainly the value structures and lines of authority that function perfectly well for most of the rest of the company don’t work at all well for IT. We’re vital but at the same time, well, so different that it’s hard to imagine a CEO emerging from the IT ranks. It happens from time to time. Everyone points to John Reed, who rose from IT to CEO of Citicorp, but Reed was an exceptional case. He succeeded because his predecessor, Walter Wriston, had an unusual interest in IT and mentored Reed. Reed succeeded, too, because he didn’t really come from IT but from Data Processing, which was more hierarchical. And ultimately he didn’t succeed at all, by some measures, because John Reed was fired. So right now let’s just accept that it is very unlikely that, coming from IT, you’ll ever become the CEO of your company. That means you are instantly off the traditional management track and have the option of either eventually moving on to some other organization or taking what’s behind Door Number Three. Remember Door Number Three from Let’s Make a Deal? It could reveal a sports car or a donkey, but whatever was behind Door Number Three was unlike anything you could imagine. We need a Door Number Three for IT professionals. I have a friend of 20 years who is in a key technical role at a very large company. He’s too vital to the company to risk losing but too geeky to fit in. He’s on the craft (non-management) salary scale, but way higher than he ought to be for having no direct responsibility. All he does, in fact, is from time to time save his company from ruin. And even more rarely, he saves all the rest of us from ruin, too, in ways I am not at liberty to explain. How do you manage such a guy? Where he works they have him report to the CEO. The Big Guy has 5-6 direct reports and one of them — my friend — doesn’t manage anyone or anything. THAT’S Door Number Three. We’re in an important transition period not just for IT, but also for business in general. Everything seems to be in flux. And that means the old ways of doing things are changing and ought to. And in this way IT is leading — or ought to lead — the way. Later this week I’ll be making a dramatic shift and proposing the Cringely Energy/Economic Policy, but first I need to drive home the point that, however different it is from the rest of the company, IT is generally the vanguard for a new corporate culture and whole new ways of doing business for the world. We’re in a mess. The world is screwed up and some of that can be traced to the improper use of IT as a financial weapon. But the people of IT actually present many of the answers we need, because they are living much deeper in technology than other parts of the company or of our society. Think about it. There has nearly always been a class of eggheads showing us a path toward new business models, whether it was Edison and Firestone, Hewlett and Packard, Noyce and Moore, Gates and Allen, or Brin and Page. It takes in each case a generation to happen, but ultimately we all (and I mean ALL — everyone in the total organization) come to look like the geeks of the generation before. So let’s lean into that, get on with the transition, and get past this place we’re in right now where nobody wants to be. Let’s consciously embrace the next model that’s generally running fitfully right now inside every company, down in the more functional parts of the IT department. What I mean by this is that times have changed and the world can no longer afford even John Reed’s world view with its needs analysis, design, debug, test, rollout strategy — whether we’re talking about a new app or a new marketing campaign. By the time the app (or the campaign) is rolled out, the world changed from HTML to Javascript/SOAP/Ajax (or from financial regulation is bad to financial regulation will save us). At the heart of this is a concept completely foreign to traditional business — Open Source. What the open source community has demonstrated is the superiority of a strategy that emphasizes early proof of concept, early release, and frequent releases with features added as needed — probably totaling 20 percent of the features identified in a needs assessment. This is the new IT strategy we live with every day — 80 percent solutions because they are fast, increasingly reliable, and keep the end users in the loop from almost the beginning. All made possible because of an open Internet (at least until Comcast succeeds and enslaves us), easily grasped standards and impressive demonstrations by companies like Amazon, Google, Facebook, and a ton of start-ups. Wall Street back offices figured this out long ago, they just never got their boss’s bosses to understand. Last week’s column was a utopian vision that simply requires all the old managers to be reprogrammed or accept a bullet in the head. But it is not at all utopian if applied solely (or initially) to IT, where this stuff actually works pretty well. IT people are most of the time building fortresses or feeling unappreciated — often both at the same time. Yet to our discredit, we’ve done a very poor job of explaining or demonstrating or outright selling our utility to the broader organization. Where are our Geek Appreciation Days? Take a Geek to Lunch? Bring Your Geek to School? Taciturn, we disparage our co-workers for not appreciating us while giving them little obvious reason why they should appreciate us. That has to change. Door Number Three isn’t just an escape hatch for nerds, it is the way business and culture and civic life will be for most of us a generation further into this information age. We’re just leading the way. And if we’re leading the way let’s embrace that role and become leaders. If, like me, you are likely to be fired, anyway, there’s no real downside to this strategy. Let’s give it a try.

Last week’s column on bad IT management and the strong response from readers that followed show this to be a huge issue. There are WAY too many IT managers who either can’t or shouldn’t manage technical teams. Last week I maintained that having a firm technology base, or at least the ability and willingness to acquire one, was essential for good managers. While readers got carried away with which technical test is the best, I don’t think there is much dispute that there are certain aspects of technical management that are helped by the manager being a code god. But that’s far from all there is to the job. So this week I want to go deeper and look at what’s really missing in nearly every instance of such bad management, which is leadership. The distinction between management and leadership is a critical one. Management is — at its very best — an exercise in coping while leadership is so much more. Last week’s simple idea that the manager ought to at least be able to tell good work from bad is exemplified by Bill Gates, who liked to claim that he could tell good code from across the room and that whatever task a team was facing was something he could code in Visual Basic over a weekend. Both statements are nonsense, of course, but Bill knew he had to talk the talk, making him at least an adequate manager. Does it make him a leader? I don’t think so. But let’s not blame Bill for that. Let’s blame Charles Simonyi. Charles is the guy who came up with Microsoft’s development process — an outgrowth of his research at Xerox PARC. I covered this extensively in my book, Accidental Empires, but the short version is that Charles came to advocate a strong program manager as the central controller of any development group. One person made all the decisions and as long as that one person was correct 85 percent of the time, it was better to have a dictatorship than a democracy or even a meritocracy. This was an effective way to extend Bill’s will to Microsoft programmers Bill would never even meet. And to Charles’ credit the system worked well enough if the dictator was really, really smart and the task at hand wasn’t too complex. It was perfect for the 1980s. But it is far from perfect today and represents one of the fundamental reasons why Windows Vista was so late to market and such a mess when it finally shipped. Vista had plenty of management, but not very much leadership. When I think of leaders what comes to mind first are political and military leaders. We use the term “leadership” to describe those roles far more than we do for what ought to be similar roles in business or technology. This week former Hewlett-Packard CEO Carly Fiorina said that John McCain, Barack Obama, Sarah Palin, and Joe Biden were all ill-suited to be CEOs of major corporations. However badly the statement went over (Carly supports McCain, by the way), her real point was that there are different skill sets for leaders than managers. And she’s right to an extent, but it also says a lot about her own tenure at H-P, which was long on management and short on leadership. Management is telling people what to do, which is a vital part of any industrial economy. Leadership is figuring out what ought to be done then getting people to do it, which is very different. It is a vital part of any successful post-industrial economy, too, but most managers don’t know that. Let’s use the U.S. military involvement in Iraq as an example of the difference between leadership and management. As more books are written and stories come out we can see that there is a lot of arguing that goes on inside the military. Officers are onboard or not. They are proposing various strategies and taking positions generally advocating what’s perceived as safest for both the mission they have accepted and the preservation of life among their troops. Eventually someone makes or imposes an order, but even then there is a lot of second-guessing in the military, which has to be ready with Plans B through G just in case they are needed. This is a lot different from the image many of us have of General Patton pointing toward Berlin, imposing a singular view and forcing it through. In contrast to the military, most businesses do a lot less explaining and pondering and a lot more laying down edicts. That’s management, which works fine on an assembly line, but not at all well building a big software application or winning a war. Janna Raye is someone I worked with at InfoWorld half a lifetime ago who has built a consulting career on understanding this stuff and helping companies to transcend 20th century corporate hierarchies and become what she calls “fractal organizations.” Janna’s consulting business is called Strategems and here is her take on this issue: “Modern corporations suffer from systemic-level issues that emerge in top-down hierarchies. Managers are there to control staff and budgets, not to lead. Although you can make valiant and often successful attempts to control things and processes, you will never again be able to control people. We’ve evolved, basically, and the information age has had a lot to do with it. So we still “manage” companies the same way as when we actually operated assembly lines in America—the good old days! Now, people need leaders, not managers, and that’s what a fractal organization enables. “In fractal organizations, it’s the staff deciding how to continuously improve processes in their functional areas for efficiency of time and resources. These organizations thrive with a new pay model also, based upon results or value of work delivered and not how much time it takes to do the task. Those who are really good will get to go home early! These are not the organizations that are shrinking. Like galaxies, they continue to expand, actually aided by a strong gravitational pull of the leaders at the center. Those who do it well create a compelling vision and keep it alive. They allocate resources to projects that align with the vision, and reward arm- and team-cluster leaders for the creative ideas their staff bring to the organization. It’s a shared vision and collective goals that are missing from the vast majority of organizations, which is why failing projects continue to drain resources. Really caring about what you do and feeling proud to be a part of something special and wonderful is what every human desires, even if they say they don’t.” So what’s Janna’s model for the ideal 21st century organization? Pixar. “They let creativity run wild at Pixar!” says Janna. “Ed Catmull, Pixar’s president, wrote an article for the Harvard Business Review on their collective creativity. Ed and John Lasseter (and sometimes Steve Jobs) are in the center of the galaxy, keeping the gravitational pull strong and the company rotating, so to speak. Around them are the directors, who in the fractal org model lead the arms (no more divisions!). Each film team or cluster, from the storyboard artists to the renderers, goes through the iterations necessary to achieve the best results. Everyone is on board with creating exemplary films and they are relentless in demanding only the best. But in this process, as they say, they have to get all the “sucky” ideas out first. If they don’t reveal all the ideas, they don’t get all the perspectives and therefore might miss something important. In quantum physics and information theory, this relates to the observer effect and the importance of acknowledging the perspective influence of everyone in a scene. “In top-down hierarchies, the opposite occurs, yet the people on the front lines are the ones dealing with the evolutionary changes going on around them and are the best source of ideas for solving tactical issues. You wouldn’t need “change management” if you made continuous improvements at the functional level the responsibility of every individual and team cluster. Yet Pixar makes incredible films in this manner, so you could certainly accomplish it anywhere, even in a supermarket! In fact, some of Whole Foods’ leadership practices are fractal — in-store teams make decisions about products and their placement, based on their observations of customer patterns. “Most start-ups are fractal in their nature, especially those that have exciting visions and get everyone on the same page with collective purpose, goals, and objectives. Most investors, however, are bought into the conventional org chart; when the company devolves into top-down, the turnover begins. That’s because of the internal competition that emerges in top-down organizations. The perception is that there’s only so much room at the top. At each level of management, the competition increases as cooperation decreases. Thus are created the ubiquitous “silos” of information that thwart collaboration and encourage redundant, wasteful business practices. “Managers are supposedly promoted because of their ability to outperform others and not because of an intention to provide inspiration, guidance, and mentoring to their staff, nor are they openly rewarded for this behavior, even though it usually produces a healthier bottom line. The usual way of rewarding based upon meeting financial goals and managing budgets keeps the focus on short-term financial results only, whereas continuous improvement leadership by frontline staff creates more long-term successes. “When managers don’t mentor staff, focusing only upon numbers and bossing people around, it leads to an illusion of control, of which there’s no such thing. In these situations, they begin to feel they must continually prove their worthiness and so defend their territories against possibly brilliant staff working “beneath” them. This is a systemic issue, not a personality quirk, though some personalities are more susceptible than others. In most companies, the idea of climbing over others on your way to the top and throwing people who get in your way under buses is de rigueur. The top-down hierarchy was designed to manage industrial-age processes, not information-age challenges. You didn’t want the door guy getting creative when attaching the door. Nor did he need to collaborate with the bumper dude. The information age is vastly different. Each scene we’re in presents new circumstances and opportunities. “Pixar claims they have a meritocracy. This is a good description of the atmosphere that emerges in fractal organizations. Google was likely more fractal in the beginning, before they brought in managers trained in top-down hierarchies and engrained with the accompanying behavioral patterns, such as knee-jerk resistance to ideas they haven’t thought of themselves. Not everyone is like this, of course, but usually those who aren’t have had confident mentors themselves who encouraged creative participation. In Catmull’s HBR article, he says they insist that everyone in the company contribute ideas, across all functions and levels, or they will perish. Interestingly, he mentions the difficulty in getting new hires to feel comfortable with this process. This results from cultural-level systems that keep people in competition rather than cooperation. Though Catmull tells readers what they do at Pixar and why, he doesn’t instruct on how to make the organizational changes that enable this approach to creativity.” I guess that last part is Janna’s job. All this fractal stuff is interesting, but I’m guessing it is also difficult to implement, because it may describe Pixar but it DOESN’T accurately describe Apple — the other place where Steve Jobs is king. Still, Apple’s product success shows that something is being transferred from one company to the other. It’s just that at Apple, Steve Jobs can’t make himself stay out of the way.

This week marks the seventh anniversary of the 9/11 terrorist attacks in the U.S. This week is also a time when the world economy is under stress comparable or greater to that imposed seven years ago. Whatever you are feeling in your wallet, I can’t overemphasize the impact the current global credit crunch is having on our economy and that of other nations, including Germany and Japan. We’re in a mess — one that is at least TWO YEARS from being resolved no matter who is the President. This matters to a technology columnist because it is something the technology community will eventually have to address, just as we did 9/11. I think some coping skills are in order. First let’s take a look at a small part of my column from September 13, 2001 — a column that wasn’t especially popular with readers at the time, but I think stands up pretty well with time: ” ‘To a man with a hammer, everything looks like a nail,’ wrote Mark Twain. In the current context this means that the organizations charged with reacting to this catastrophe will do so by doing what they have always done, only more of it. Congress, which controls the budget and passes laws, will want to pass laws and to allocate more money, lots of money, forgetting completely about any campaign promises. The military, which is the nation’s enforcer, will want to use force, if only they can find a foe. The intelligence community, which gathers information, will want to be even more energetic in that gathering, no matter what the cost to the privacy of the millions of us who aren’t thinking of terrorist acts. And agencies like the Federal Aviation Administration, which regulate, will want to create more stringent regulations. Now here is an important point to be remembered: All these parties will want to do these things WHETHER THEY ARE WARRANTED OR USEFUL OR NOT.” In 2008 we’re facing cuts in IT that are prompted by economic decline. Many of the IT shops I talk to are in denial about this. Many more, while not in denial, are making bad decisions. I think this is a good opportunity to do some housecleaning that probably should have been done years ago. If you have to cut your budget by 10 percent, where do you cut? What if you have to cut by 30 percent? As I have written before, one of the great problems in IT management is that the big bosses typically haven’t a clue what is happening, what is needed to happen, and what it all should cost. There is a role for trust here, but if the Big Guy is signing off on a budget he can’t even read, much less understand, well something is wrong. Some IT departments like this, of course, just like my students liked it when class had to be cancelled (they liked getting LESS for their money), but in tough times, facing reality and speaking the truth is usually the best course. Because power in IT organizations tends to be based on head count, preserving jobs takes a priority. And when jobs have to be eliminated, they tend to come off the bottom of the organization when they should more logically come off the top — or at least from near the top. A tech who directly helps users is more important than a manager who can’t manage. This is especially true if that manager is making 2-3 times as much as the tech. If your boss doesn’t understand your job enough to describe it in technical detail, that boss is in the wrong job. If you are managing an IT shop and can’t write the code to render “hello world” in C, html, php, and pull “hello world” from a MySQL database using a perl script, then YOU are in the wrong job. I should point out that these latter tasks can be copied and pasted straight from properly composed Google queries. They aren’t a test of programming knowledge at all, just of the ability to use the Internet. Yet many technical managers will fail and should get the boot as a result. You can’t manage what you can’t understand. Think about whole projects that can be chopped, too. What’s really needed, after all? That knowledge is in your organization, though often not where it is available to the decision makers. The essence of efficiency is doing only the parts that are absolutely needed and almost every shop has at least one project that everyone except the big boss knows is either pointless or hopeless. Cut it. One can argue, of course, that MORE IT, not less, is in order, and maybe that argument will work. But make it only if it is true. I think a good argument can be made for embracing cloud computing, even to the extent of eliminating data centers and facilities. We’re very close to the point where relatively few organizations really ought to have their own data centers. This could also be a good time to embrace open source tools. Yes, there is a learning curve, but the price is right and I can argue that open source quality is substantially better. Oh, and cancel those contracts with Gartner, Forrester, IDC, etc. You’ll feel better in the morning. Some folks who WON’T feel better in the morning are the next class of IBM employees to see their jobs moved overseas. I understand that there is a new round of cuts coming in October and it will be different from the “death by a thousand cuts” that has been happening for the past year. The technique is the same, of course — cutting unneeded workers in the U.S. while suddenly needing virtually identical (if younger and cheaper) workers in India and Argentina. It’s pure coincidence. Yeah, right. If you were disappointed with Apple’s product announcements this week, take cheer from knowing that more announcements are coming, including new MacBooks and iMacs. You don’t have to take my word for this — just look at the closeouts Apple is offering on current models. Christmas is still the most important quarter for Apple so they won’t let these announcements wait too long. I just wonder how they slipped out of this week’s event. And finally, I am surprised to admit that the latest version of 64-bit Windows Vista seems to be running pretty darned well on my desktop. No driver problems, 32- and 64-bit apps seem to be running well — why hasn’t Microsoft been shouting about this? 32-bit Vista still sucks, of course.

This was the week Google surprised the world with Chrome, its own open source web browser. Just imagine the deadly effect that had on a dozen or more browser-specific start-ups in Silicon Valley. Lots of readers are wondering what I think of Chrome, like my opinion really matters. Chrome is okay — faster, but not faster enough to make me change for that reason alone. It’s better than IE and almost better than Firefox except there are no plug-ins to speak of. What I really wonder, though, is why Google bothered to do a browser at all? Now I know. It’s not like there aren’t enough web browsers in the world. There are plenty. And though Internet Explorer still dominates the Windows market, Firefox (not to mention Opera, Safari, etc.) is there to keep Microsoft honest,. So why did Google even bother? There are two general opinions on this and they are not mutually exclusive. Naturally one opinion is widely held and the other is held mainly by me. The first reason why Google had to do its own browser comes courtesy of my friend Dave: “People are looking at Google Chrome and actually think Google is competing in the so-called Browser Wars,” said Dave. “This is not the case at all. Google doesn’t care what happens to Chrome. And, in fact would be absolutely thrilled if Firefox and Opera enhanced their browsers to the point where they trounce Chrome into extinction. Google doesn’t make a dime off of Chrome. Its money comes from people using the web browser — any browser. “What Google does not want is Microsoft creating a browser that sucks. Actually, Google doesn’t mind if Microsoft’s browser sucks. What they really don’t want is Microsoft to make a browser that sucks and everyone ends up using it. And, if the IE8 beta shows us anything, making a really sucky web browser is Microsoft’s true ambition. “Google’s main concern is quite simple: Browsers should render pages accurately, and the JavaScript engine in the browser should be fast, efficient, and bug free. On both counts, IE8 is an abomination. JScript just doesn’t behave very well and is buggy. And, IE’s page-rendering engine simply does not follow the standard. Because of this, Google has to keep development on their Google Applications quite generic and simply cannot implement the features they want. You’ll also notice that Microsoft recently has been putting on some very compelling web content that is only available if you use Windows and IE.” Now back to Bob. Everything Dave says makes sense and I agree with it, but it doesn’t answer my real question, which is not “Why did Google have to do a browser?” but rather, “What made it impossible for Google NOT to do a browser?” The answer to this latter question begins with Dave noticing Microsoft’s recent IE- and Windows-specific web content, which cracks open the door on Google’s greatest fear — that Microsoft will turn off ads in IE. Microsoft can’t do that, can they? Microsoft can do pretty much whatever it wants in this area. There is plenty of browser competition. They can hobble their own product if they like, though it would drive users away from IE — from a product that brings Microsoft no direct revenue anyway — so what’s the risk? Microsoft turns off the ads in IE and what happens? Google takes a huge revenue hit, is knocked down three pegs in the eyes of Wall Street, while pretty much nothing happens to Microsoft, which would have just shown the world who is still the sheriff. I am not saying this is going to happen, but I AM saying that it COULD happen — and that very remote possibility is, by itself, enough to make Google have to produce its own browser. Let me be clear that there doesn’t have to be any subterfuge here on Microsoft’s part. They can simply turn off the ads in IE, declaring it a non-commercial product. If you don’t like it, get another browser — there are plenty to choose from. Microsoft’s revenue would go almost unchanged while Google’s would plummet, if only for a few weeks or months — just long enough for Microsoft to come through with a second punch, that is if they have thought that far ahead. If you are wondering whether people really sit around Google asking if Microsoft would actually do something like this, well they do. So to avoid that eventuality (and to do all the other things that Dave said, above) here we have Chrome, Google’s attempt to direct the future of browser development and take some momentum away from IE. Chrome promotes WebKit rendering, which is also done in Safari. It would not surprise me if WebKit didn’t make some inroads shortly with Firefox and Opera, helping somewhat to turn the tide away from IE. Yet WebKit will change, too, by adopting Google’s V8 JavaScript engine, replacing JavaScriptCore in both WebKit and Safari. Thus all the open source browsers (and Safari) become better and more alike, which helps them against IE. A rising tide floats all (open source) ships. Google needs open source browsers to become even more competitive with IE, hence Chrome is a reference design that Google knows will work brilliantly with all Google Apps. So much for Chrome: Now for something REALLY scary. I’ve been hearing that peer-to-peer file sharing has declined a bit. Actually, it’s the rate of growth that has declined, but in a market where volume is always rising and prices always falling, even a decline in growth can be significant. This is happening for lots of reasons (market saturation, summer vacation, etc.) but the effect appears to be real, much to the relief of the RIAA and MPAA, which hate people sharing music, TV shows, and movies that they see as violating the intellectual property rights of their members. But I think something else is actually happening. People are just finding new ways to share files — ways that are harder to detect and even more chilling for society to prohibit. Look at where P2P came from in the first place. The idea behind BitTorrent and similar programs was that many people wanted the same content and few users could afford the bandwidth to run their own dedicated servers, so sharing files by caching and re-serving small pieces of files was very efficient, especially with flat-rate bandwidth. Depending on your point of view, P2P has been a huge success or a huge pain in the ass. But all the while, the cost of Internet bandwidth has come down A LOT. Remember P2P was born in the 1990s when most users still had dial-up connections. With the cost of Internet backbone bandwidth dropping 50 percent per year for the last decade or more, the economics have changed dramatically and it has become reasonable to effectively have your own server. No, I’m not talking about YouTube, I’m talking about dedicated servers used in large part to distribute movies and music. I’m talking about any of a number of Internet backup services. The poster child for this new kind of service is RapidShare, a German file-sharing service that will let you distribute files up to 200 megs each for free and up to two gigs for not much money — 55 Euros per year — with no limit on the total number of files, total storage, total downloads or even total simultaneous downloads. Rip your copy of The Dark Knight, store it on RapidShare, then send the download URL to anyone you like or simply post it somewhere on the web. It’s not as efficient as P2P, but it sure is easier AND harder to detect since nothing but http is used. Can you see where I am going with this? How are the MPAA and the RIAA likely to respond if this technique becomes really popular? They are going to want to spy on us more, even to the point of auditing (or attempting to audit) our network backups. More lawsuits, more grandmothers and little kids being sued, less privacy. I’m sure the RIAA and MPAA will fail in the long run. Once custom protocols and ports are dropped and you can’t tell the difference between a spreadsheet and I Am Curious (Yellow) the game is up. But we’re still years — and a lot of pain — away from that.

While to regular readers this may seem an odd time of the week to see a new column from me, get used to it, because I’m deliberately increasing the frequency of I, Cringely columns to something greater than one per week yet still possibly less than two. In part this is my response to having more than ever to say. It’s also an attempt to create more opportunities for you to view the ads we don’t run. But the one thing this IS NOT is a knee-jerk response to the fact that Hurricane Hanna is right now bearing down on my home in Charleston, South Carolina, determined to drown us sometime on Friday. Even without Hanna to inspire me, you’d still be reading this column today. What has me riled up earlier in the week than usual is Comcast’s decision to limit its customers, which might include me (more on that later), to no more than 250 gigabytes of total bandwidth per month. Other pundits have called it “the end of the open Internet” and a betrayal. But I’m not so sure. When I started writing this column in 1997 my Internet connection was a business DSL line from Covad rated at 384 kilobits per second for both uploads and downloads. And in the fine print of that agreement 11 years ago the account had a maximum download limit of 3 gigabytes per month. I know this for sure because I violated the limit several times and was penalized for it by Covad, which charged something like $4 for every gigabyte in excess of the limit. So was there ever a truly wide-open Internet? Not for me. Of course in 1997 dial-up was still the norm and V.90 modems were the bomb at 50-60 kilobits per second if line conditions allowed. If you take that nominal 56 kbps and multiply it out, what we were buying was about 18 gigabytes of DOWNLOAD POTENTIAL per month. Nobody could use all of that, of course. Or rather nobody could use exactly that amount because you’d have to take a gulp of digital air from time to time, pumping white space into the pipe. In fact the maximal duty cycle expected of even the most strenuously exercised dial-up connection back then was about 15 percent, which worked out to around 2.7 gigabytes per month — remarkably close to the 3-gigabyte limit I had at Covad. So what I was buying from Covad back then was a faster pipe, sure, but in a sense not a larger one. We were, after all, mining diamonds of data, not coal. My 384 kbps could have technically allowed me to pump up to 124 gigabytes per month if I could have figured how to do that at the time using my 386-90 PC. So how does this relate to what Comcast — America’s largest broadband ISP — is trying to do now by limiting its customers to a total upload and download of 250 gigabytes of data per month? The new number is more than 80 times my old limit but only twice my old pumping potential. The new limit would be comparable, then, if my 2008 Internet connection was 80 times as fast as my 1997 Internet connection. Is it? That’s hard to say. I’m a Comcast customer and know how fast my Internet connection is, but if I were just a prospective customer, all I’d get from the Comcast web site is that their cable modem is “4X faster than DSL.” Where Comcast used to give you a number like 3, 4, or even 5 megabytes per second, now all they claim to be is four times faster than DSL. Yeah, but WHAT DSL? Which version? I think they must mean the old 1.5M/128K DSL of days of yore, and which pretty much nobody has today. To be 30 times faster than my old Covad DSL my current Comcast connection would have to run at 11.5 megabits-per-second, but it doesn’t. My Comcast Small Business Internet link runs at 8 megabits down and 2 megabits up for which I pay what seems to me to be too much money, though I do get five static IP addresses in the mix. Most Comcast residential customers get about 5 megabits down and 1 megabit up, but they pay less than I do. Now we probably have enough data to come to some conclusions. That 250 gigabytes per month is a lot of data (Comcast claims it is enough for 99 percent of its customers), but it isn’t proportionally as much as I got from Covad back in 1997 (for just about the same price, I might add). That means this 250-gigabyte limit actually IS a limit and Comcast is not being as generous as it could be. Still, there are many unanswered questions here, and my discussions with Comcast suggest they don’t yet have all the answers, either. For one, as a Comcast Small Business customer, am I even held to the 250-gigabyte residential Internet limit? Comcast doesn’t know. The bandwidth limit is for consolidated uploads and downloads, but does it include Comcast extra-cost services like VoIP phone service? Comcast doesn’t know. This feels to me like a trial balloon. Eventually I’m sure we’ll learn that Comcast phone service and Comcast Video-On-Demand movie downloads are excluded from the limit. Go over your limit renting from NetFlix and you’ll be punished, but not if you are renting from Comcast. It’s all about TV and movies, you know, and what if cable TV as we know it dies and is replaced entirely by digital downloads? If that’s the case we can make a similar calculation coming from a different direction. How much bandwidth capability would we need if cable did die? Say the average house has two TVs, one is running four hours per day and the other two hours. Say HD downloads require 2 megabits per second, which is actually quite an aggressive number assuming H.264 compression. How much bandwidth would those two TVs suck up in a month? It looks to me like 162 gigabytes, which is within the 250-gigabyte maximum, but just barely. What Comcast has done here is draw a line in the sand that it thinks it can justify. They are trying to change the game with DSL from one of comparing download speeds. And while Comcast seems to allow its customers just barely enough bandwidth to survive in the radically changed environment of a no-cable world, that’s only the case if peer-to-peer file transfer technologies such as BitTorrent aren’t used. Comcast doesn’t say so, but these new rules effectively kill P2P in the long run, that is unless Comcast somehow sanctions that P2P, calling it Comcast P2P. And THAT’s what this is really all about.

Like a few million other people I recently bought an iPhone 3G. But unlike a few million other people I bought TWO of them — one for my young and lovely wife. That puts me in the rare position to actually speak from experience about a current networking issue: what’s the deal with these iPhones? Is it the phone or the network that is causing problems? And the answer is: both. Our experience with the iPhone 3G is not good, though not terrible. It is a fantastic device, if flawed. The main flaw is the phone, rather than the iPod Touch bits that comprise the rest of the unit. Voice service is not good, calls are dropped, and the iPhone won’t work places where my old Nokia N80 easily did. But my iPhone, oddly enough, works better than does my wife’s. This variation is how I know at least part of the problem is with the phone, not just the network. But the network sucks, too. We switched from Verizon (I know you can’t use an N80 on Verizon, smarty-pants — I have more than one cell provider) which claims the best network but had the nagging problem of delivering the odd voicemail 7-10 days late. Verizon claims never to have heard of this problem but ask a few Verizon users and they’ll confirm. Now that we are full-time on AT&T we might blame the lousy service on the phones except I also got a Samsung AT&T phone for Mimi, my mother-in-law, who is on our family plan. Her service sucks, too, so it is not just the iPhone. I’m sure AT&T has oversold their network. You can tell because the worst service of all is from one iPhone to the other. If the call doesn’t spontaneously disconnect half the time you often still can’t understand what the other person is saying. Service is somewhat better going to landlines or other mobile providers. I’m sure AT&T will fix this eventually but I don’t like being treated this way. No wonder they are so hot to keep that iPhone exclusive, since half the iPhone users I know would jump to T-Mobile if they easily could. Last week’s column about the population of CCIEs and global development raised quite a ruckus — a word I include to confuse the non-native English speakers who saw last week’s column as discriminating against them. We can argue a bit about the numbers and their meaning, but I think it is fairly obvious that: 1) Cisco dominates the Internet core router business, so this is a real issue no matter what your language, and; 2) CCIEs are NOT just network techs. It is an extremely difficult certification to get and typically costs in the neighborhood of $30-40,000 by the time you are finished, sometimes a lot more. While it may be patently obvious that China and Korea will be more important 30 years from now than India and Japan, that wasn’t my point. Anyone can express that opinion. What I was trying to do was to show a reason WHY that might be the case as evidenced by this CCIE data. Why shouldn’t India be just as successful as China? Their populations are comparable and they both have good educational systems with large numbers of graduates. They both value science and technology. India even uses English as one of its official languages. Both have booming economies with plenty of room for growth. Well this CCIE analysis gives one empirical reason why it should be so. While the Indians are developing their IP expertise, the Chinese are developing their IP networks, simple as that. Another reason to talk about this subject is because there is far too little actual thinking on the Internet these days. The blogosphere is full of opinions but not very much solid discussion of why things are the way they are. Agree with me or not — I don’t care — but I’ll always make you think. The whole point of this column is getting people to think and discuss. Now to the problem from last week of how YouTube and similar video services can better appeal to advertisers. I foolishly thought last week’s tease might coax a few bucks out of the bushes at a time when I could use them, but no. So I’ll reveal my solution anyway, even without a reward. YouTube would love to make lots of money from ads that would play before its 100 million videos per day, but they have had difficulty appealing to traditional advertisers, not because of the quality of the viewers but because of the quality of the videos, themselves. There is a huge variety of content on YouTube and while advertisers are willing to stretch a bit in what they’ll sponsor, they are afraid of making a mistake and sponsoring the wrong videos, like those that contain nudity or other objectionable content. Short of watching all the videos, how do we best avoid this problem? That’s what has Google scratching its GoogleHead. It all comes down to the quality of the metadata — the data describing each video. I think the answer is obvious and is composed of three parts: 1) notification, 2) structure, and 3) standardization. I’m probably listing these backward, but I want to get notification out of the way first. Whatever system YouTube chooses to manage its metadata won’t be perfect. There will be errors — the Internet equivalent of Janet Jackson’s “wardrobe malfunction.” So there should be a facility for viewers to notify YouTube if they feel that there is a significant mismatch between the likely target of the content and the target of the accompanying ad. No condom ads before Dora the Explorer clips, for example. No bacon commercials with vegan cooking shows. Having a way to report such errors will diminish them in both number and importance. Next comes structured metadata. It floors me that YouTube doesn’t enforce this already. If you ever took a journalism class you learned that the first paragraph of any news story (called the “lead” or “lede” — same pronunciation) is supposed to answer the questions who? what? why? where? when? and how? YouTube could use a form for each video submission that used these categories, possibly minus the highly suggestive “why.” To submit your video you have to fill in all five blanks. Leave any blank unfilled and your video bounces. These five metadata categories really ought to cover the gamut of describing most any video. If they don’t, or if they are improperly used, then we are back to notification and getting users to help fix our mistakes. Finally there is standardization. Even within the five structured metadata categories there can be great variation in the meaning of the chosen terms. That’s why we need to standardize those terms. Rather than reinvent the wheel, I would rely on the best current system of standardized metadata, which is Wikipedia. Behind each structured entry would be those Wikipedia terms that would seem to be appropriate, along with links to their definitions, just to be sure. With this system it would take a minute or two longer to submit each video, but what the videos were about would be much clearer to viewers and advertisers alike. Paris Hilton is a “who.” The Paris Hilton is a “what” or perhaps a “where,” though the system would probably force them to be separated. Either way you end up with a hotel if you want a hotel and a sex video if you want a sex video. Now it’s time to recharge my iPhone (again).

Leading indicators are measurements that change over time and suggest future trends for important second-order results like population growth and economic development. Economists in particular are often looking for indicators that have been known historically to lead the overall economy. If unemployment goes down, for example, it is a good bet that shortly thereafter income will rise and the economy will improve. It’s for this very reason, then, that economists and Wall Street fund managers are always looking for newer and better leading indicators. But such indicators needn’t be limited to the economy: they can apply to technology and technical culture, too, which has its own feedback loop to economic development. My friend George Morton, who figured this all out, says that by knowing the right numbers to look at we can have a good idea what countries will be leading in technology — and presumably in economic development and power — in the years ahead. The measure George likes is the number of Cisco Certified Internetwork Experts or CCIEs. The CCIE is Cisco’s top certification category and VERY hard to earn. Being a CCIE doesn’t mean you have Len Bozack on speed dial, but it might as well. Cisco products dominate the Internet and CCIEs are Cisco gurus, so if you are serious about the Internet as a nation you’ll have CCIEs hanging about, or that’s the theory. Conversely, if you just talk a good game as a country with technological aspirations, maybe you won’t have many CCIEs at all — maybe none. It’s one way to determine who the posers are. Cisco publishes the total number of CCIEs and their geographical distribution four times per year and George used the Internet Archive to track down the last nine years of data to look for trends. All of this is behind one of this week’s links. Where I took a step further was to divide the number of CCIEs into each country’s population, then do the same for each country’s Gross Domestic Product and correct for widely varying populations and states of economic development. For a baseline, then, the U.S. has at present 5,863 CCIEs, which is 1.947 CCIEs per 100,000 population and $2.2 billion of GDP per CCIE. It is logical to assume that nations with adjusted numbers that exceed those of the U.S. for CCIEs per 100K are Internet up-and-comers and ought to fare well in the decades ahead. Beyond the population statistic, countries that have significantly less GDP per CCIE are those that would seem to have made networking a national priority. Countries that are significantly behind the U.S. on one measure or another are just that — behind the U.S. — which is not good. Here, then, are some of the numbers I calculated: Canada, not surprisingly, is similar to the U.S. with 2.2 CCIEs per 100K and $2.45 billion per CCIE, as is the UK with 1.5 CCIEs per 100K and $2.12 billion in GDP per CCIE. Ireland is very similar to the UK with 1.48 CCIEs per 100K and $2.95 billion in GDP per CCIE. The really interesting European numbers to me come from Germany, with 0.74 CCIEs per 100K and $4.3 billion per CCIE, and France, with 0.36 CCIEs per 100K and $8.11 billion per CCIE. Both of these countries appear to be underinvesting in network technology, with France especially lagging. Latin America is dramatically behind Europe and North America, though I found it interesting that Argentina, with 0.17 CCIEs per 100K and $8.94 billion in GDP, is 50-100 percent ahead of both Mexico and Brazil. It is especially interesting to compare India with China and Japan with South Korea. India has 0.036 CCIEs per 100K to China’s 0.22 per 100K — a 7X differential — while India has $10 billion in GDP per CCIE to China’s $3.3 billion. There is no doubt that there is plenty of network expertise in India, but these numbers show that expertise isn’t making it out of the technology centers to the rest of the country. China, on the other hand, is developing its IT infrastructure much more broadly. This also doesn’t take into account the simply huge numbers coming out of Hong Kong, where there are 3.3 CCIEs per 100K and $1.13 billion in GDP per CCIE — numbers that might properly be added to the rest of China in some accounts. Japan has 1.23 CCIEs per 100K to South Korea’s 1.9, but the significant difference between these two countries is the $4 billion per CCIE in GDP for Japan compared to $1.28 billion in South Korea, which is clearly investing massively in network infrastructure. Looking 30 years into the future I think it is clear that the regional leaders will be China and Korea, NOT India and Japan. Israel has numbers very similar to Korea with 1.43 CCIEs per 100K and $1.4 billion in GDP per CCIE, which is more than double on both scales that of the other Middle Eastern leader, Saudi Arabia, with 0.42 CCIEs per 100K and $3.2 billion in GDP per CCIE. And don’t count out corporate haven Bermuda with its five total CCIEs and its population of 66,000. That’s 7.5 CCIEs per100K and $900 million in GDP per CCIE. “When looking at the countries a key element for IT is the English language,” said George. “Yes I know Harvard still publishes diplomas in Latin. Where English is a first language, or an important second language is key to the number of CCIEs. Sorry C, PERL, JAVA, Cisco IOS, Basic, &t. are all English context. The Moore’s CCIE Law also brings into question the ability of countries to attract IT capital with open or closed network infrastructure. The last point — over 50 percent of all CCIEs in the US work for Cisco. In Mexico, and India I bet it is over 50%. Both are very large call centers for Cisco Support. Morton’s CCIE Law = Moore’s Law accelerated by the technological support available to exploit Moore’s Law. The degree of acceleration is measured by the available number of CCIEs. It’s Newton’s Second Law: F=ma, (or Force = Moore’s Law * CCIEs).” What we’re talking about here is the future of world power in this century, but I can understand if your eyes have glazed over a bit. Maybe something more interesting would be the big question reverberating through the hallways at Google: why the heck can’t they make any money from YouTube? This is, on the face of it, the same question that eBay must be asking about Skype. Both were fairly large investments yet neither is contributing significant revenue to the parent company. Poor Skype can’t feed us ads before or after every phone call, especially since the only way to have contextual data to make those ads more valuable would be to listen in on the calls — a no-no. But for YouTube the problem is different because it ought to be fairly easy to figure out what a video is about then sell ads against that metadata. Easier said than done, my friends. The problem with YouTube and advertisers is that the nature of the videos, themselves, is too varied and the metadata too easily wrong. A hotel chain that might well want to advertise before a video of the Paris Hilton, for example, might be extremely reluctant to advertise before a video OF Paris Hilton, yet from the perspective of metadata both are extremely similar. This is a HUGE problem for Google and all the other streamers of user-generated videos, which leads some people to believe that amateur night will eventually end and the Internet will return to being like most television — a place for predominantly professional video. What’s funny about this is I think I have the problem figured out. The answer seems obvious to me, but I’m not sure Google would even listen if I told them my answer.

One of the great strengths of the Internet as a communication and entertainment medium has always been its lack of security, a fact that seems to pass over the heads of many "experts" today. Bob Kahn and Vint Cerf could easily have added robust security to TCP/IP, but they deliberately chose not to with the idea that innovation would be encouraged by making the Internet a wide-open space. It wasn't that they prohibited security, but pushed it up the stack, effectively making it other people's business. If your application required security, there was nothing keeping you from adding it, but on the other hand there were few, if any, hints about how best to do that. We were on our own and to a great extent we still are, which means there is a lot of bad stuff happening and probably always will be. I could write column after column about Internet abuse, but my inspiration this week is actually quite prosaic. There is a fairly popular commercial website I heard about this week that has a novel way of making sure its ad revenue numbers are met. This site has its own ad sales team selling display space bringing in tens of millions of dollars per year in revenue. It is a good site and grandly profitable, but if for some reason revenue dips below target a little code kicks in and starts refreshing particular pages every 90 seconds, generating each time a new "hit" on that page's banner. The refreshes are tied directly to revenue and nothing else and it seems to me that what's happening is, well, theft. Here's how these bozos can get away with a stunt like this. First, advertisers often don't really want to know the success of their ad campaigns to a fine level of granularity. They'd rather keep the ad game a sort of dark art because, frankly, they pretty much don't know what the heck they are doing anyway and ad buying is done by the lowest-level agency employees when it properly should be done by the highest. So if something is wrong they'd really rather not know about it, thanks. The second reason why this kind of stunt goes unpunished is because it doesn't happen all the time. The site generally provides good content and good service and they only kick in this little script when absolutely needed. So if a lot of people seem to be clicking on ads but few are actually buying, well it comes back to that black art, doesn't it? And finally, the Internet isn't as sophisticated an ad space yet as are magazines, for example, where the Audit Bureau of Circulations does a pretty good job of keeping track of how many people actually read ads. The Internet, though some would claim otherwise, doesn't really have a comparable operation to the ABC, but it probably should. This is not the first time I have heard of a scam like this, but it is the first time I've heard it applied to a big and respected operation. There are lessons to be learned here by publishers, advertisers, and readers alike. More bad Internet behavior apparently took place over the last several days during the Russian invasion of the Republic of Georgia, where the Russian government was accused of cyber-terrorism. This is a good time to define "cyber-terrorism," which to me means the deliberate attack on network infrastructure with the goal of causing harm, destruction, and possibly theft. There can be little doubt about the baldly geopolitical nature of the Russian invasion, but what happened on the bits-and-bytes level I wouldn't call terrorism, more like graffiti. The affected Georgian websites were informational and were quickly moved to servers in the U.S. Whether the bad guys were employed by the Russian government, Russian Mafia, or were just high school kids as some have suggested, what they did wasn't terrorism. No systems or networks were destroyed, no bank accounts plundered, no command and control systems crushed. If the Russians had intended there to be a cyber component of this invasion you can bet they would have done a better job of pulling it off. And the attacks, such as they were, weren't strictly from one side, according to this report I got from Moscow: "Starting in the first hours of Georgia's armed attack on Tskhinvali, Georgian hackers began the war on the Internet. The servers of leading Russian mass media, including the website http://www.mk.ru, were subjected to DDoS attacks. But specialists in information security and our hackers won an unqualified victory in the Internet war. "These attacks were reported to us," Vitaliy Kamlyuk, an expert of the Kaspersky Laboratory for Computer Security, reported to Moskovskiy Komsomolets. "According to our information, the DDoS attack was organized on Friday. At that time requests from computer zombies were sent to the servers of Russian information resources, including http://www.mk.ru. These (zombies) are machines infected with viruses that one or several hackers have joined into a whole network. They sent innumerable requests, which caused a short-term glitch in the work of several sites and editorial offices. But to our knowledge these attacks did not have great success. The systems administrators of the information sites were able to deal with the virtual attack quite quickly. We are continuing to monitor the situation. Our partners also continue to intently watch the development of this cyber-war." Cyber war? I don't think so. I had yet another indication this week that the Internet is still a frontier and that came when there was information I desperately needed and nobody had yet thought to provide in a way that was useful to me. This may seem silly, but there are people who claim that the best ideas have already been taken when it comes to Internet businesses, yet every day I look for something and can't find it. In each case that's a business opportunity lost, because if I'm looking for something there are probably thousands of others looking too. This week, like millions of Americans, I was looking for cheap fuel. I packed my wife and three kids in our 1996 34-foot Winnebago motor home for a 1,600-mile cruise up the U.S. East Coast and back. The Winnie clocked in at SIX miles per gallon, so obviously I was on the hunt continuously for the cheapest gas I could get. Armed with a notebook computer and a fairly reliable cellular data connection, my wife riding shotgun surfed all the gas sites from Charleston, South Carolina to Scranton, Pennsylvania, and back while I did the driving. There was plenty of data available and it proved pretty reliable, but the question never answered by any site we could find was, "where's the best place on my route for me to stop for gas?" You can get gas prices by ZIP code (gasbuddy.com is great) or along your route (aaa.com is the best), but no site looked at my route, checked all the data and did what I really needed, which was to just tell me where to stop. Instead they expected me to click on every station or input every ZIP code, write it all down on a piece of paper and make my own damned decision. Not my style. Here is a terrific mash-up opportunity that you'd think would have been written back when gas was $2 per gallon. Ironically, I can get this very information for my airplane on my delightful Flight Cheetah display, but in this instance cars are apparently not so advanced. Someone please hurry up and get such a service going before I take my kids on another trip.

Editor's Note – As many readers have reported, we have had difficulties in recent weeks both posting the current column and especially posting reader comments. These problems are not unique to I, Cringely, but affect all PBS blogs based on the Moveable Type platform. Architecture changes are in the works as well as an entirely new version of Moveable Type that we hope will shortly fix these problems. Until then please bear with us. Companies have their own personalities, generally adopted from founders or charismatic leaders. Some companies are gentle, others competitive, some are even cruel. Once a corporate personality becomes embedded it can continue in that fashion for years -- even decades -- past the departure of the original inspiring executive, as generations of leaders and wannabe leaders act the way they think The Big Guy would have done. Sometimes corporate personalities are effective but often they just get in the way, especially when companies act in ways they no longer need or even understand. And sometimes the lingering effects are legal or regulatory, too, as companies spend years cleaning up messes inspired by executives now long gone. That's what's happening right now with Microsoft, I believe, which seems to be coming out on the losing end of a number of lawsuits. The first bit of evidence can be found in Microsoft's recent settlement with Texas-based Vertical Computer Systems Inc., which accused Microsoft of violating Vertical's patent on so-called SiteFlash technology that the company said was copied by Microsoft in .NET. The companies settled with terms as yet unannounced, but Vertical definitely won since Microsoft is buying a license to the patent in question. It's nice for Vertical that Microsoft settled, but sharp legal minds (far sharper than mine) were surprised. Microsoft isn't in the habit of settling these suits early if at all and the U.S. Supreme Court' KSR decision from April, 2007 has shifted the advantage in such lawsuits to companies with large patent portfolios (that would be Microsoft) and away from their smaller adversaries (that would be Vertical). The importance of this decision (and its dire effect on small inventors) has gone right over the heads of most people, but it is rapidly changing the way IP is handled in the tech world. So if Microsoft normally doesn't settle suits like this early and the legal terrain has actually improved for Microsoft since the suit was originally filed, what the heck are they doing settling the suit and presumably paying millions that could probably have been avoided? Some clever folks think that this is a ploy on Microsoft's part aimed at companies like Adobe, which is presumably also on Vertical's to-be-sued list. Getting a SiteFlash license for .NET puts pressure on Adobe's .NET competitor called Air. Or that's the claim. Yet viewed in a current legal light, Adobe is in as strong a position as Microsoft was, patent-wise, and so should have as good or better chance to beat the Vertical patent in court. If Vertical sues Adobe AND LOSES, then Adobe is strengthened and Microsoft is weakened. So what's really going on? Of course I have a theory. I think Microsoft settled quickly and quietly with Vertical to avoid going to court because a trial might have exposed Redmond to far more risk than one might expect from a little patent infringement suit. That's because of something that happened in an earlier case I covered extensively several years ago – Burst v. Microsoft. For those who don't remember or never knew the sad story of Burst v. Microsoft, it was a complicated suit involving not just patent infringement but also anti-trust, restraint of trade, and breach of contract. The smoking gun in the Burst case was that Burst lawyers caught during discovery a pattern of apparent destruction of e-mail evidence on the part of Microsoft. Microsoft claimed it was “too hard” to search for the lost e-mails (Burst had copies from its side so many of the messages were known to have existed) but Judge Frederick Motz finally ordered Microsoft to do whatever it took to dig up the tapes and find the e-mails. Fortunately for Redmond, the case settled (for $60 million) before Microsoft actually had to produce anything. Now, as they say, for the rest of the story….. Months after the Microsoft/Burst settlement I received e-mail from a former Microsoft contractor: “Now that Burst v. MS has moved out of the courts, I thought that I might add a little to what you know about this case. Back about two years ago when the judge told MS to cough up the rest of the emails that was supposed to have floated around between MS execs that discussed the Burst relationship, the team that I was on (Corporate tape backups) was asked to gather all together all of the tapes that were used during that time. Even though there was a corporate policy in place that any *.pst was to be excluded from backup capture, the effort failed. Not only did the Backup Exec software fail to filter out those pst files, but some of the involved blue badges (Microsoft employees) intentionally disguised their mail files so that they would not be recognized and included in the nightly backups. This last effort was even prohibited by policy from the VP level. As I was the one tasked to gather the tapes together from Arcus/Iron Mountain, I know exactly how many tapes were recalled for the involved servers. They filled several 240 tape trunks and were stored in the Building 11 tape vault.” “Several months after all of the tapes were gathered, MS legal started asking for restores of any pst files captured, the tapes 'mysteriously' went missing. Now because our team was a managed service vendor, we were held directly accountable and responsible for the loss. I can think of a lot of reasons that the tapes were removed by someone blue. It is also possible that someone on our team performing a standard purge of old media mistakenly pulled them and sent them to the shredder and even though the tapes were stored in a special section specifically marked “Do Not Touch” taped across them I find it highly unlikely.” This is Bob, back again. In case you aren’t familiar with this case let me put this guy’s statement in some context. Microsoft was saying in court that it couldn’t find the tapes and that it would take millions of man-hours to search for them when in fact the several trunks full of tapes were apparently easily gathered and already stored in the Building 11 tape vault. At the time Microsoft lawyers were claiming the tapes would be impossible to find, THEY HAD ALREADY BEEN FOUND. It would be interesting to know if Microsoft’s lawyers were aware of this. They should have been, but this may have been one of those instances when it was to their advantage NOT to know. And then THE TAPES DISAPPEARED. Funny we never heard about that, either. To my knowledge, Microsoft never presented this "the dog ate our tapes" story, most likely because they may not have wanted to acknowledge that there even WERE such tapes. The fact that there were tapes and then there weren't probably SHOULD have been mentioned, since I'm guessing not doing so violated a court order. Certainly it should probably have at least been mentioned in the discovery materials given to Burst. Why, if the tapes were found, did they sit for months in a special tape vault in Building 11 rather than be examined for the missing messages? And then there is the big question about what happened to the messages at all -- messages that were in a locked vault and labeled "Do not touch" -- how did they get thrown away and by whom? The former Microsoft contract employee who contacted me on this issue did not do so anonymously, by the way. I know his name and how to reach him. We have talked on the phone more than once. He did not hesitate to name names: "...Calvin Keaton, the blue badge who managed the team, made the appropriate loud noises about the loss to HP services, although I never saw any public disclosure about it.... Just glad I never had to depose for the issue. Can’t imagine that it would have done my career any good." So the outside vendor was Hewlett-Packard, one of Microsoft's hardware OEMs, which is to say Microsoft's bitch. The tape disappearance was blamed on HP, which reportedly accepted the blame, and the employees directly involved kept expecting there to be repercussions, especially legal ones. They expected to be deposed by Burst lawyers. But it never happened. This was, for Microsoft, a perfect ending. The damning tapes were lost in a way that could be blamed on a contractor -- a contractor over which Microsoft had great power -- power greater than just a services contract. The contractor "accepted" responsibility though there was no real evidence they had done anything wrong. It could just as easily have been a Microsoft employee who destroyed the tapes. It is possible that Microsoft never revealed to the court either that the tapes had been found or that they had later gone missing. This admission would have had to have taken place at the spoliation hearing that was scheduled for the week Microsoft instead settled with Burst for $60 million. I contacted both Burst and Burst's lawyers last year and they could not recall any aspect of this incident having been revealed to them by Microsoft or by the court. It was news to them. But of course by that time the case had been long settled. Microsoft's legal behavior in this is consistent -- consistently bad -- and the only intersting aspect of that part is that it is logical to assume Redmond would have paid ANYTHING to avoid that spoliation hearing and its need to either come clean about evidence destruction or to commit perjury by not coming clean. $60 million was nothing to Microsoft. Burst could probably have got a lot more money had they known what was actually going on. And where was HP in this? Why didn't HP file a friend-of-the-court brief explaining what had happened? Even the lowly contract workers who were blamed expected that to happen yet it didn't. Did the HP legal Department even know about the incident? My guess -- and it is only a guess -- is that HP corporate was never told about the incident, though that is not in any way an acceptable excuse. It should have been reported. Shortly thereafter, of course, HP lost the storage contract. My source was laid-off and moved away from Seattle. Microsoft (and HP by implication) got away with it... or so they thought. But since the Burst settlement there have been at least a couple more Microsoft lawsuits that may also have been affected by the lost evidence in Burst v. Microsoft. The people of Iowa cited it, I'm told, in their anti-trust case against Microsoft that settled pre-trial last year. It may have been known, too, to Vertical in this most recent case. Any claimed pattern of deception on Microsoft's part would be dragged-up in similar cases. If these claims are true, what Microsoft allegedly allowed to happen and then did not disclose (along with the suspicious timing of all of it) was illegal in a stop-the-presses, all-capital-letters way. You just don't do that to Federal judges. So Microsoft is paying and paying and paying again not just for intellectual property licenses in these subsequent cases, but also for this earlier information to be kept out of court. And for lawyers who understand such issues, it strengthens their hand substantially. The next case in which I expect this information to be a part is Gotuit Media's suit against Microsoft over the latter's Silverlight video technology, which Gotuit claims infringes on three of its video tagging patents. Gotuit's lawyer is Spencer Hosie, who also represented Burst and ought to be fairly familiar with the whole lost document issue. We started this column, remember, with the idea that companies adopt the personalities of their founders or charismatic leaders. Microsoft has always been an aggressive company willing to push the line and unafraid of litigation, which pretty much describes Bill Gates, too. It's not hard to imagine some Microsoft employee with a key to that tape vault in Building 11 asking him or herself “What would BillG do?” and making the wrong move as a result. It's very possible that today's Microsoft is very different from the one that spawned this behavior. But that doesn't matter because the company is still stuck with it. Bill Gates may be gone from Microsoft but the company will be living with his legacy – good and bad – for decades to come.

So little real information leaks out of Apple these days that we tech pundits tend to jump on any crumb we can get and munch it to death. That's certainly the case with this week's story about Apple possibly dumping Intel chipsets for the new MacBooks expected to be announced in September. What's funny to me is that the answer to what's REALLY happening has been in front of us all for more than a year. Here's how this mess of a story got started. On Monday, July 21st Apple Chief Financial Officer Peter Oppenheimer dropped a bomb on those listening to Apple's quarterly conference call on earnings for Wall Street analysts. He said that gross margins for the coming quarter, and possibly beyond, would be lower for three reasons: 1) a back-to-school special; 2) a one-time charge related to a contract manufacturer, and; 3) "a future product transition that I can't discuss with you today." That's it. That's all he said. And from that sprang a zillion stories about what that future product transition could possibly be. It settled eventually on the idea that Apple might be abandoning Intel processors, later downgraded to Intel chipsets, for the new MacBooks and beyond. Of course Apple's recent purchase of PA Semi got folded in as pundits wondered if Apple was going back to PowerPCs after all. I know how these stories develop, having written more than a few of them myself over the last 20 years. You start with one fact, get the usual suspects to speculate on what that fact could mean, throw those speculations into print, then look for an official denial of the parts that are wrong. Once that denial comes through we rinse and repeat with the goal of eventually converging on something close to the truth. It's not a very elegant way to do journalism, but that's the way it happens in the tech trades, which now include everything from blogs to the New York Times. But what's REALLY happening here? Stepping back from the carnage we can see that Apple has a "product transition" coming up that will hurt margins in the near term, but Oppenheimer also said it was dramatic and would definitely HELP margins in the long term. That's all we really have to work with from Apple, but it is really quite a bit if you parse the data carefully. First is the product transition, which quite specifically DOESN'T mean a new product. If Apple was announcing something completely new as they did last year with the iPhone and Apple TV, then Oppenheimer would have referred to it as a new product. As CFO he has fiduciary and legal responsibilities that could land the guy in hot water with the SEC, so language on these calls is important and never by chance. Second is the margin hit that will go away, which smart readers right away saw as a change of chips, because they start expensive and become very cheap over time. By making an aggressive semiconductor move Apple would be trading profit margins for technical market advantage knowing that in a few months the new chip process would come down and margins could return to normal. THAT's why all the smart money went immediately to speculating about Intel, then backed off somewhat as official denials began filtering through back channels from Cupertino and Santa Clara. As of today people are just left scratching their heads. Apple isn't changing CPU families and evidently they also aren't dumping Intel chipsets for those of Nvidia. But SOMETHING is happening because Peter Oppenheimer gets no pleasure predicting lower margins that he knew would drive down Apple's share price, if only temporarily. So now the pundits are wasting even more packets wondering what Apple is planning, at the same time generally admitting that they (the pundits) don't really have a clue. Regular readers of this column may well have an idea what's up, because I wrote about it more than a year ago. Before I drop my own bomb, though, I should say that I have no new information and what I am about to predict is based solely on my earlier reporting. Here's what I THINK Apple is about to do. I reported more than a year ago and repeated in this year's predictions that Apple would be adding H.264 hardware support to its entire line of computers. The chip they are adding comes from NTT in Japan and was developed in cooperation with Japanese broadcaster NHK. The chips began sampling a year ago and should now be available in volume, though Apple may be paying as much as $50 each for early production. This would be a major blow to gross margins because, unlike all the speculation covered above, this wouldn't be a matter of replacing one chip with another but of adding a new chip to the mix. That'll be an extra $50, thank you, with no savings from eliminating other parts. The fun part is figuring how this all fits into Apple's strategy as not just a maker of computers but also as a seller and distributor of entertainment content. The NTT chip is not just an H.264 decoder, it encodes, too, which is what makes it so special. The last I heard NHK was claiming the chip could compress a 1080p video and audio stream into four megabits per second, down from the 20 megabits normally required. If we assume Apple will apply the same kind of wink-wink, nudge-nudge transcoding to 1080p that they've already applied to 720p in the Apple TV, then it is within reason to expect they'll claim to distribute 1080p over iTunes in two megabits per second. As the dominant technology platform in television and movies today, it makes good sense for Apple to put this H.264 hardware capability into the Mac Pro line, and maybe even into the MacBook Pros for professional use, but darned if I can immediately see why such powerful and expensive compression capability is required in a MacBook, iMac, or Mac Mini, yet I was told long ago that the chips would be applied "across the entire line." We'll see. Of course this is all about taking command of the 1080p video market. Apple's strategy with iTunes will continue to evolve, but for the moment having a unique real-time 1080p capability will suck a lot of early adopters back into the Apple stores and give Apple's emerging content competitors like Netflix something new to worry about. When Apple marketers sit down to talk about the competition they discuss Netflix and MAYBE TiVo, but that's it. Hulu is something iTunes could emulate overnight so it doesn't matter and none of the other video distribution channels are seen as having the potential to achieve critical mass. What really excites me as a content creator is the amazing potential of real-time HD. Video and games are by far the greatest consumers of cycles on modern PCs. By embracing a dedicated H.264 chip THAT IT MAY WELL HAVE EXCLUSIVELY FOR A YEAR OR MORE, Apple is taking an out-of-the-box approach that will frustrate its competitors in both software and hardware. While the H.264 chips are expensive, they'll enable Apple to save money elsewhere by having slower computers that run faster video. Though it is doubtful that many will use it, you can be sure Apple will trumpet the ability to support 720p video in iChat. So why am I the only one writing this? It's because I could be wrong, of course. But I don't think so. I'll just have to take a chance and see.