Public Cloud Economics: Lessons for Enterprise IT (Part 2)

In Part 1, I illustrated how public clouds capture Moore’s Law and convert that into price decreases for customers.  In Part 2, let’s review how enterprise IT shops can capture Moore’s Law.

Event triggers

Moore’s Law can only be captured by adding new servers.  Enterprise IT, as well as Amazon Web Services (AWS) and Google Cloud Platform (GCP), are all motivated to add new servers based on two triggers: 1) fulfill new demand, or 2) refresh older servers.  AWS’s high growth rate – still exceeding 50% annually, provides amble opportunity to add fresh servers and maintain a relatively young fleet.  But, when it comes to refreshing servers, the key questions is:

What constitutes an old server? Is it the same when servers are virtualized?

The cost of time

As servers increasingly become less expensive, relative to other related costs, answering that question using a fixed rule-of-thumb (e.g., a server is 3 years, so it's eligible for refresh) starts to fail, especially if your goal is to continually reduce overall compute costs.

Let’s consider the energy cost to operate a server for a year.  Below is a different formulation of Moore’s Law, called Koomey’s Law, which states the number of computations per joule of energy doubles every 1.6 years – and is even better than Moore’s Law!


Using better servers to reduce other costs, faster

To illustrate how server costs are relative to other related costs, assume a public cloud provider can acquire a new server for just $1 – truly consumable inventory.  Further assume the annual cost of energy to operate a server is $200, the entire server fleet is one year old, and additional capacity is required.

What is the least-cost capacity fulfillment strategy?

In this hypothetical scenario, additional per server energy costs of $200 can be avoided by not adding more servers to the fleet.  Instead, replace some quantity of existing servers, regardless of their actual age, to meet the incremental capacity requirement.  This scenario provides a simplified set of variables in order to illustrate the following:

As servers become less expensive, relative to other related costs, the chronological age of a server gives way to economic-age of a server.

The higher the ratio of “other related costs” is to a server's cost, the more valuable every percent of Moore’s Law becomes toward reducing those “other related costs.”  Optimally refreshing servers reduces total compute costs!

Reduce the cost-basis of assets and increase their utilization

The practice of purchasing new, higher performant servers to reduce other related costs is referred to as asset arbitrage, and it complements the practice of placing virtual machines among hosts.  Both are solving distinct combinatorial optimization opportunities.

Virtual machine placement algorithms determine where to optimally place instances to maximize utilization rates of existing hosts, whereas asset arbitrage algorithms determine what hosts should be periodically available, when, and for what classes of instances (services), in order to reduce the cost-basis of inventory.  Since existing physical inventory will not remain static, the former’s value truly shines when it maintains high overall utilization rates – even as host inventory changes.

Asset arbitrage for enterprise shops

Many enterprise IT shops already use "labor arbitrage" – a fancy term to describe labor outsourcing from a high cost market to a low cost market like India.  The same principle applies to compute assets, albeit the mathematics are a bit more involved than labor arbitrage. 

Indeed, a server doesn’t cost $1, but it really doesn’t matter if a server is $1, $1000 or $10,000.  What is important is whether “other related costs” exceed the cost of the server.  Is the ratio 2x, 3x, or 10x the cost of server?  If so, then when to refresh servers, and capture Moore’s Law, in order to reduce total compute costs, is going to arise before the typical 3-4 year refresh cycle.  Refresh timing and quantities become a function of the cost ratio’s size (i.e., the inventory holding cost for not refreshing in a timely manner).

Are AWS’s or GCP’s server costs less than their cost for energy, facilities and power distribution and cooling?  Without them disclosing those figures, the best we can do is guess.  If you are curious,  James Hamilton’s blog walks through a data center cost model (James is VP of Engineering at AWS).  The figures are four years old and likely do not reflect AWS’s actual costs.

Are enterprise IT shops’ server cost less than their cost for energy?  Not likely.  But, that is fine because there is an even bigger cost target to go after in the data center: software.

Convert Moore’s Law into fewer software licenses

One of the largest cost in enterprise shops are core system and application software (e.g., OS, virtualization, system management, middleware, databases, app servers, etc.), where the cost of the software stack runs 3-20 times a server and are often priced per processor or core.  At these cost ratios, saving 50% for an instance at a public cloud provider doesn’t amount to a lot of savings.

However, by co-opting some of AWS’s inventory management practices, required software licenses can be reduced by 20-50%.  Instead of refreshing the host servers every 3-4 years, maintain fresher server inventory and minimize over-provisioning. It’s simply less expensive to refresh on shorter cycles than every 3-4 years.

And like AWS, that also means coordinating with procurement to secure license agreements at reduced levels to forecast.  Too often, organizations pay-up for “all you can eat” licensing contracts to compensate for poor inventory control and forecasting.  That may have worked wonderfully to overcome quantity uncertainty for a non-virtualized server fleet, but that uncertainty can be substantially reduced in virtualized environments.

But unlike AWS, reducing significant levels of compute cost does not require replicating Amazon’s retail inventory management systems.  It is already best practice to organize high cost software stack (HCSS) based services such as databases, application servers, middleware, etc. into common resource groups, or pools.  These savings can be incrementally captured by identifying and prioritizing HCSS-based services for substantial cost-takeout.

Generate additional downstream savings

The faster uptake rate of new servers, for HCSS-based services, can also reduce energy spend by improving overall performance/watt of the server fleet.  Plus, the residual servers that remain from quicker refreshes still have operating life.  They can be easily re-deployed to classes of services with no or low-cost software stacks, essentially providing “economically free” servers to those service classes – reducing overall cost per instance.

Having a strategy to capture Moore’s Law

Public cloud providers like AWS and GCP have well-defined strategies to capture the value of Moore’s Law and reduce their costs.

Enterprise IT shops can deliver lower total compute costs (including the software stack) without the need to replicate AWS’s or GCP’s supply-chain or business model.  Capturing Moore’s Law to reduce large software expenses can be implemented in stages – by types of services (e.g., SQL Server databases, or Oracle, or WebSphere, etc.), along with the cascading cost savings in energy and low-cost software stack service instances.

The question for enterprise IT shops is: Will they learn to excel at converting Moore’s Law into cost-competitive service offerings like AWS and GCP? Or will they spend a lot of precious capital trying — only to discover their own customers choosing less expensive alternatives in the public marketplace?

Some very big technology companies are betting heavily on the latter.