Since utilisation-based cloud services became popular, there has been a significant emphasis on optimising costs. After having recently spent months reviewing internal costs at Bulletproof, we quite quickly came to the realisation that there are many cost savings to be made across the business.
These savings are not just by reviewing our cloud expenditure – which when managing more than 700 multi-cloud accounts is a substantial portion of monthly costs – but also reviewing vendor agreements, recurring business expenses and assigning a dedicated resource to continuously audit and review expenses.
I’ll share a few of the ways we were able to save hundreds of thousands of dollars by thinking smarter, not harder, about cloud utilisation and business costs.
AWS Cost Optimisation
With AWS being the largest contributor of cloud workloads currently managed by Bulletproof, we’ve had the opportunity to work with many customers with differing environment sizes. These range from a few hundred dollars to a million dollars a month, yet both ends of the spectrum struggle with the ongoing management of cost within their environments.
Visibility is the key to managing your AWS environment from an operational perspective and even more so cost and this leads into the first ‘To Do’ of cost optimisation.
A fundamental requirement of an organisation’s ability to manage and reduce cost is accountability. I recently attended a meetup where the CIO of a large media organisation mentioned that they had introduced a policy whereby any instances running for one week, without tags, were automatically deleted. I’d recommend a notice period before implementing a similar policy, however it worked and resulted in an increased focus on the reduction of cloud spend across the business.
Tagging is quite simple, however tagging correctly, tagging everything and tagging often is important. Tags give you the capability to report on a per account basis and unlock a more holistic view of your wider cloud portfolio. The more popular tag profiles are:
- Cost Center
- Business Unit
Once you have a tagging policy in place across your organisation, it then becomes an easy task to align costs with departments and quickly dissect costs to avoid incurring unnecessary charges.
Unattached EBS Volumes
EBS volumes become a costly asset when you have many instances running with multiple volumes attached to each. There is also an added risk that EBS volumes become ‘unattached’ or remain in a provisioned state without a running instance. Unattached instances can be viewed in your AWS console or via a third party tool such as CloudHealth which provides detailed insight, automated reporting and governance across all cloud platforms.
Costs can be saved immediately by terminating all unattached EBS volumes or implementing a policy to terminate any volumes which remain unattached for a specified period of time. In the example above, all volumes which are in an ‘available’ state are unattached and can most likely be deleted.
When configured and left to flourish, snapshots can quite easily become one of the highest recurring costs in your cloud environment. In the example below, snapshots have grown to consume more than 50% of the monthly AWS spend. While many organisations have differing snapshot requirements, in the event data is required to restore, in all likelihood your team will restore data no older than a few days or one week at most.
Zombie Assets and Rightsizing
AWS were the first to introduce per hour, minute and second billing and this presents another opportunity to reduce your ongoing costs. Zombie assets are servers which have been provisioned and essentially forgotten about. They may be running a single, lightweight function once a week or even a month, however remain operational 24 hours a day, 7 days a week and incur unnecessary costs.
As an example, if you’re running an APM such as New Relic in your environment, you can ingest instance resource utilisation data into CloudHealth and at the click of a button, identify any assets which have been running at <5% CPU utilisation for 30 days. Any instances identified can almost certainly be powered off and only switched on when required. Using this same data set will also allow you to rightsize all instances in your environment based on CPU and memory utilisation, the savings in most occasions amounts to thousands of dollars per month.
In the case of test, development or staging environments, all EC2 and RDS instances should be powered off or terminated outside of business hours and weekends. These instances can then be powered on and RDS instances reprovisioned each morning when required.
Instance Generation Upgrades
In May this year, AWS announced one of their latest in a long list of price reductions relating to M4 instances. While instance generation upgrades may not result in an increase in performance, there are cost benefits. Many customers continue to run m1, c1 and t1 instances to name a few which have all since been deprecated by many generation upgrades. The costs for these older instances can incur a hefty premium of 20%-50% as AWS look to decommission aged infrastructure.
This is Cost Optimisation 101. There are many Reserved Instance (RI) options available including reservations with no commitment, a partial upfront commitment or effectively a minimum monthly commitment (convertible reservations). With the recent announcement of Reserved Instance flexibility across both EC2 and RDS, AWS have made it even easier to reduce your ongoing costs for production workloads.
It’s important to note that if you’re powering instances on and off (see Zombie Assets and Rightsizing), there is a bigger cost saving to be made than reserving those instances. It’s also important to note that reservations will only cover a single running instance at any one time. For instance, if you’re running 2 x c4.large instances from 8am to 8pm, a reservation will only be a applied to one of the running instances during those hours and the remaining 12 hours will be forfeited and applied to your account as an underutilised or unused RI cost.
Business Cost Optimisation
During a recent internal review of costs, we identified a number of key mistakes we were making in terms of ongoing cost management and committed time and resources to fix them all.
Depending on the size of your organisation, it may be difficult to assign a dedicated resource to manage vendor agreements on an ongoing basis. However, we found a number of vendor agreements which had been active for years and most certainly not on current plans. This resulted in us overpaying for services which are available today via different vendors or with the same vendor at a better rate.
The lesson here is to maintain a good relationship with your vendors, review your costs annually and don’t be afraid to move if you find a better product offering at the same or at an even slightly higher cost.
While performing a review of credit card transactions for a person who had left the company, we found a recurring cost which had been in place for several years even though we were no longer using the service. It’s not a service provider’s responsibility to notify you if you’re not using their service and so it’s important to review your recurring expenses on a bi-yearly or, preferably, a quarterly basis.
It’s best to think of cost optimisation as a holistic exercise. As you can see above, it’s easy in the noise of day-to-day business activities for optimisations to take time to be identified and then corrected. That’s normal, regardless of your organisation size and vertical. It is worth remembering two final things. Firstly, cost optimisation takes commitment. Secondly, that when it comes to your cloud expenditure, it usually requires specific cloud expertise and an external view to pinpoint the wins that you can achieve.
By Ghassan Zalaf – Head of Customer Enablement at Bulletproof.