Design for Cost : Stop cloud over-engineering and over-provisioning

by Rahul Gupta

The public cloud providers have done a great job building so many cloud services which can be integrated with your applications with just a few clicks. Its like a fast-food joint - convenient, tempting and affordable. However, these fast food meals are full of calories, that we forget about our nutritional needs. Similarly, public cloud has also made it so tempting and easy to provision those new and shiny services for example spinning a new specialty vm, integrating AI, ML and Analytics to your applications, containerization of your applications, publishing the microservices with single click. Although in many cases these are great additions to your architecture and applications but Are you force-fitting these extra services to your applications or are you wasting your resources by over-provisioning them ?

Many studies have shown that moving to cloud will save you a lot of money but only if you follow a strict governance and control. These over-engineered applications bloated with so many superfluous services which costs a lot to build and harder to operate over time. On top of it, if these applications are over-provisioned then they become an endless money pit.

Gartner has estimated that in 2019 total public cloud market was $227.8 billion and in 2020 the revenues will grow by 17% to $266.4 billion. Studies have found that in 2019, over-provisioning has costed around $8.8 billions and over-engineered applications on cloud has wasted around $5.3 billion. This was before COVID-19 pandemic and now with so many companies moving their applications to public cloud, these number will shoot up significantly.

Recent trends shows that enterprises are also adopting Multi-Cloud Environment - combining on-premises private clouds and public cloud services, which is driven by diverse IT needs, avoiding vendor lock-in, right cloud for right applications etc. This has made it harder to properly predict the cloud spending and enterprises are up for a very rude cloud cost shock when the monthly bills start to come in. In this multi-cloud era, cloud cost optimization is a Design and Engineering problem.

As cloud architects and engineers, we all have a responsibility to not get lured into the hype of the shinny services available on Public Clouds. We should focus on the MVP - Minimal Viable Product that's sufficient to implement the core functionality. We need to draw a line between whats required and whats cool. The trick is to strike a balance between over and under engineering by understanding the requirements and avoid linking together too many tools and end up complicating a simple solution. Always try to design your solution with requirements and costs in your mind.

Designing for cost

  • Consider using cheaper alternatives such as containers to drive up utilization, smaller instances types or cheaper storage types.
  • Use Auto-Scaling with open architecture as much as you can. Start small with an open architecture where you can implement and change each layer without affecting the accessibility and implementation of other layers. Thereby, you can auto-scale the layers independently which is more in demand on a certain day rather than scaling up the whole system.
  • Whenever possible, move your small non-mission critical applications to serverless functions: Even with most of your applications running on the smallest VM (think EC2.micro on AWS or general purpose B1s on Azure or f1.micro on GCP), it can end up costing you alot with thousands of them running. Just by moving your small non-mission critical applications to serverless could save you significantly.
  • Lifecycle Management: Setup lifecycle management policies to lower the cost of storing data as the frequency of access reduces over time. Logs, backups and snapshots are perfect candidates to move to lower cost infrequently accessed storage.
  • You are paying by the hour, so shut down the services you don't need, for example schedule start and stop AWS EC2 instances automatically, a typical use case is a developer or a test instance which a developer uses only during working hours.
  • Add Resource Monitoring and Alerting: Most of the cloud providers gives basic monitoring and alerts free of cost so add these active monitoring and alerting to all your services. We must track usages and expenditures to optimize cloud cost.
  • Stick to the Requirements: Don't add unnecessary add-on's such as AI and ML to a cloud deployment if its not requested or required. For example an inventory control application doesn't need an active AI/ML while a Fraud detection application does. Collate and analyze your use-cases and evaluate if AI/ML will add any values. Are they going to provide any substantial benefits over your traditional analytics ?
  • Automation is awesome, but don't take it to an unproductive extreme. Take an inventory of all your applications, data stores and processes. This will give you an insight on your meta development processes. Then break down these meta development processes to sub-processes to identify the candidates for automation.
  • Don't burden your application by using cutting edge technologies that technology savvy companies like Netflix or Facebook or Google uses. You need to take into account of your own legacy systems and processes which might not be compatible with these cool tools. Adapt and innovate around your own products.
  • Data Transfer : try to minimize the data transfer between different zones or regions. Strategically select regions across your deployments by understanding the scale and resource consumption patterns of your applications and logistical complexity.
  • On a multi-cloud deployment, avoid deploying different components of the same application to more than one public cloud.

If you follow these simple recommendations to design by cost, you will have a much easier and happier time after your cloud application goes into operations and when you see your monthly cloud bill.