In the past couple of years we have been witnessing an accelerated shift of the Big Data market to a managed cloud model, with the on-premise Big Data platforms segment undergoing considerable consolidation (for example Cloudera and Hortonworks, or MapR and HPE). This trend has left organisations that have invested time and money into on-premise solutions, unsure what their next move should be.
Why are organisations moving away from on-premise platforms?
This move towards cloud-managed solutions is happening for a variety of reasons, the most common ones being:
- The cost, effort and time required to manage the infrastructure for on-premise Big Data platforms far outweigh the returns. You need to have a dedicated infrastructure just for that platform and an in-house team to manage and maintain it. These are costs that you would be able to save with a cloud solution, as it comes with an “off-the-shelf” service package.
- On-premise platforms come with a number of components, most of which require specialised resources. This adds to the workload of the platform administrator, as changes to any one component affect the entire platform, which raises monitoring and governance issues. In the cloud, these issues are standardised and governed by the service provider rather than in-house – allowing for greater availability, scalability and peace of mind.
- Most on-premise platforms are heavily dependant on open-source components (the Hadoop stack being by far the most widely used). Such open-source components usually don’t have a solid roadmap for upgrades, fixes, bugs etc. For enterprise applications, this increases the overall vulnerability of the platform and jeopardises performance, expandability and speed of delivery. Moreover with the accelerating shift of corporate IT investment to the cloud, the on-premise market had to consolidate and the few players left are quickly moving into a cloud-native offering. Companies that have invested in these platforms are facing the impeding risk of receiving limited or no support from the respective vendors.
Are there any risks for companies making the move from on-premise to cloud?
A mistake we often see companies make, is not properly assessing their current and future big data workloads, resulting in under-forecasting their needs and related costs and therefore running over budget soon after they migrate to the cloud.
Another point to also consider, is that when moving to the cloud, you are also moving from a CAPEX model to an OPEX model, as with cloud providers you are on a subscription/pay-as-you-go cost base.
Questions you should know the answers to before making the move:
- When will these workloads be running? How much data will I be dealing with?
- What kind of analysis is required to have a full assessment of my Big Data across the whole of my organisation?
- What are my long-term Big Data needs? Will my workloads be increasing?
When is hybrid the preferred solution?
There can be many reasons why a company would opt for a hybrid solution. The ones that we've seen more often are:
- When you have business-critical or extremely sensitive data, then it makes sense to have that data in on-premise systems that are not exposed to the service managing your cloud-native solution.
- When you have invested in on-premise platforms that are closely tied with the continuity of the business, whilst also taking advantage of the flexibility and agility of the cloud-native solutions.
- When you want to better understand how cloud-native solutions truly operate prior to making a commitment to migrate.
Final thoughts
On-premise Big Data platforms have an increasing number of associated risks and challenges that companies can no longer ignore, especially when the cloud-native market has matured so much in recent years. So what are the options for those companies that have based their Big Data initiatives on on-premise platforms?
- You can keep investing in resources and vendor or service provider support, to ensure your on-premise platform is up-to-date with no decline in performance, security or governance.
- You can evolve into a hybrid solution – keeping your sensitive data secure in your on-premise platform while still benefiting from a cloud-native technology.
- Or you can fully migrate to a cloud managed solution.
The decision you make going forward entirely depends on your current and future Big Data needs. Our advice to you is to make an accurate and inclusive blueprint of your requirements and roadmap, and choose the option that meets them most effectively. Then, look at your existing resources, know-how and skillsets are sufficient to support the solution you chose, or you’ll need external help. The higher the criticality of your data and the complexity of you projects, the more sense it makes to seek the support of experts to help eliminate risks and ensure the continuity of your projects, no matter what solution you choose; on premise, hybrid or cloud native.
If you would like to discuss how you can effectively address legacy issues and choose the best course of action for your Big Data ecosystem, then give us a call on +44 (0)203 475 7980 or email us at Salesforce@coforge.com
Other useful links:
Big Data Consulting Services
Hortonworks Data Platform Support
The 3 most impactful big data technologies of 2019
