Sunday, April 6, 2008

Is Google Search Appliance Appropriate for Government and Corporate Enterprises?

Question to Ponder (QTP)

Google's smarts are based on Page Rank, which is tied to popularity. Popularity on Google is heavily influenced by financial and other driving incentives. Google's strengths are based on Gi-normous machine clusters and on huge critical mass of the widest diversity of users, who are mixing it up and providing feed back into the system explicitly and implicitly by way of their behavior.

So is a Google search appliance a smart recommendation for government or corporate enterprise? I don't know exactly but last good info I had was that they cost about $150K per appliance rack that could search 2 or 3M items.  [Update 01.14.2009 - new box searches 10M docs]  One NASA center has two racks and found they chewed up 4-6M items in no time. I don't know what percentage that was, but I do know that they were want for more racks. Last I talked to them they were getting good at setting up filters to weed stuff out. (Weeding the garden is probably not a bad thing in any event.)

Now, I suppose you could - antitrust laws and fear not withstanding - scale Google's model to apply to the whole of the US federal government's information infrastructure and we'd approach some aspect of critical mass. Popularity behavior would surely be different than the commercial advertising model (I think). But that's not necessarily bad, if true.

It actually might not cost a lot of money in terms of federal budget scales. The way I figure, 1 billion records would cost $50M at full retail cost, i.e., no negotiated discount:

1,000,000,000 items/3,000,000 items per rack = 333 racks : 333*150,000 = 50,000,000

Tack on 20% to implement? +10M gets us to $60M
20% per year for maintenance? +10M per year * 10 years and we're at $160M

Double it all for 2B items. $320M. Even with no write downs for discount that's a rounding error in the federal budget.

To avoid monoply and other sticky issues with a deal like this, the government should buy Google's intellectual property and arrange a reseller license agreement so the government can contract out to the fairest qualified bidders (plural) the secure right to implement the IP on hardware on the government's behalf. Google could even be forbidden from competing in that way for some number of years. Government then leases time on the GIG to cover the bandwidth resources consumed by the huge clusters serving this purpose. (Ooh - is that in the cost?) Government already has the ability to secure this at multiple levels of security (Is that in the cost?). It is essentially, if perhaps not trivially, a matter of organizing search clusters to operate within NIPR or within SIPR or within JWICS firewalls.

[Update on 11 April based on conversation with Katharine: It might not be one big Google search farm pointed at these networks, at least not without a radical change in policy that may be intractable due to politics. There are many, many organizational firewalls. But at least get the bulk purchase order in, get a huge discount...for taxpayers, and place clusters where they need to be to get the biggest bang for those bucks. The essence is to take a strategic approach to search for the whole of government information. Search is so very fundamental that it deserves this kind of treatment.]

Ok...that demon thought is expelled now. I'm writing too many proposals. Approaching temporary burnout threshold. My brainstorming synapses may be totally misfiring at this point.


[Update 3 December 08:  2B searchable items is nary a drop in the bucket, I reckon.  Are we talking about 2 trillion?  200 trillion?  Large numbers that sound obscene when spoken?]

No comments: