As the world is projected to store 200 zettabytes of data annually by 2025, many enterprises are looking to the Cloud and specifically Google BigQuery for an efficient and scalable means to deal with such extreme data growth. For companies where their data is contained in large, on-prem Oracle databases, it can be especially hard to leverage the benefits offered by the Cloud and Cloud Native Analytic Platforms like Google BigQuery. Let’s look at some of the benefits of BigQuery, challenges of refactoring from Oracle as a source legacy platform, and how Gluent Data Platform can simplify the migration process through transparent data virtualization.
BigQuery Migration Benefits
While companies can scale their on-prem databases to account for rapidly increasing data production, it is often an inefficient and expensive path to choose. When you factor in the cost of enterprise storage, the timeline to procure, obtain and install, you often find yourself forecasting larger than required purchases to make sure you stay ahead of the procurement and implementation cycle. On the other hand, BigQuery enables companies to dynamically scale their environment to meet the needs of today and tomorrow, without having to pre-purchase massive amounts of storage. Some additional benefits of BigQuery include:
1. Increased Scalability
One of the main advantages of BigQuery is the ability to scale on-demand. Rather than purchasing terabytes of extra storage for expected growth several years away, you simply pay for the storage you need now and increase your storage in real-time as needed. If you want to increase query performance, you can easily allocate more slots with the click of a button. The flex slots minimum commitment is only 60 seconds, so you are no longer paying for compute that you are not using.
2. More Accessible Data
Close behind scalability is access to data. While massive amounts of data are being generated, most of that data is being held captive in various silos within an organization. This leads to significant opportunity cost as various teams do not have access to information to help them win new business, better support current customers, and accurately plan for the future. Through liberating your data out of silos and into Google BigQuery, you provide your teams with one centralized location to access data, simultaneously increasing data access and availability. This provides your teams with actionable intelligence, regardless of where the data is born.
3. Expanded Security Capabilities
For many companies, security is a major concern when evaluating cloud migration; however, this is a major benefit of Google Cloud. When you partner with a cloud provider, you are significantly expanding your security capabilities by leveraging the massive infrastructure and personnel they have fortifying their cloud, securing communications, and proactively combating bad actors every day. Whether you deal with PII, financial information, need encryption at rest, or in-transit, there is a reason companies such as Northrop Grumman have chosen to partner with Google Cloud.
4. Greater Cost Savings
If you have spoken to the infrastructure team at any major organization in the last decade, one of the first complaints you will hear is how their on-prem RDBMS cost is always increasing, seemingly with no end in sight. With more cost-effective pricing models, the ability to scale on-demand, and the opportunity to secure further reduced pricing through annual commitments, Google BigQuery significantly reduces infrastructure spend while providing greater performance and flexibility.
Cloud Migration Challenges
While there are tremendous benefits to migrating expensive legacy databases to cloud native platforms, this can be a significantly complex task as well. Many of these systems are critical to the daily operations of a business and have been in use for decades. This poses several problems for migration:
What to Move
Knowledge: The Old
and the New
1. Determining What to Move
Making the strategic decision to convert from Oracle to Google BigQuery is only the beginning. After decades of building an interconnected on-prem environment, large enterprises could have hundreds or even thousands of databases and connection points. This creates a challenge in itself as companies can spend months or years attempting to analyze what migration strategy works best for which databases and workloads. During this time, new data is still being born and support renewals continue to recur.
2. Knowledge and Skills: The Old and the New
There are two sides to this coin. Teams that have worked on these systems for years can develop deep domain knowledge and technical expertise. However, when you have legacy systems that have been in place for decades, it can create a black hole when team members leave and take their knowledge with them. The other side of the coin is the potential lack of skills around your new target platform. In migrating data to the cloud, you may be taking an accomplished 20+ year Oracle veteran and turning him into a Google BigQuery rookie.
3. Rewriting Applications and Queries
Arguably the most complex part of migrating to a cloud native database is trying to ensure applications and queries are accurately rewritten to continue performing once the data lives on a new backend. In fact, a recent study by Fortinet found that 74% of companies surveyed had migrated an application back to their on-prem environment after it failed to function properly in the cloud. The scalability, security, and reduced cost of the cloud are meaningless if any percentage of a company’s applications are no longer properly functioning.
Simplifying BigQuery Migration with Gluent Data Platform
As stated, there are numerous benefits to migrating a legacy database to Google BigQuery, however, the complexity of the project can open companies to significant risk. Rather than a manual re-write, Gluent Data Platform provides a safer, simpler, and accelerated path for Google BigQuery migration through transparent data virtualization.
1. Gluent Advisor
How do you know if a database is a good fit for migration and how do you know what data is going to be negatively impacted by latency? These are two common questions companies have to address early in their journey to the cloud. However, they can be extremely difficult to answer if you don’t have the right tools. Gluent Advisor is a free tool that gathers data from an Oracle database via a simple open text SQL script. The output generates a report providing insight into the structure and activity of data within the database and defines how much data can be safely dropped from Oracle once offloaded to BigQuery. This provides actionable information to determine what data needs to reside on-prem, what data can live in the cloud, and if cloud migration is right for the database.
2. Gluent Offload Engine
Not only does Gluent Advisor tell you what data you can offload, but it also generates predefined rulesets for offloading the data to BigQuery. These parameters can be adjusted if needed and then offloading is as simple as pushing a button. Gluent Offload Engine takes over and automatically migrates the initial offload as a background function, so your applications can continue working during the process. Once the initial offload is complete, Gluent Offload Engine continues to keep your data synchronized with ongoing scheduled incremental updates.
3. Gluent Query Engine
The final step is ensuring all of your existing applications can continue querying the data now residing in BigQuery, without needing to rewrite any code. Once Gluent Offload Engine has completed an offload, a metadata switch occurs, and all of the offloaded data can then be dropped from Oracle. This allows companies to significantly reduce their Oracle footprint and expense. Moving forward, users and applications can continue running unaltered queries and when offloaded data is needed, Gluent Query Engine wakes up, gathers the data from BigQuery, and transparently virtualizes it back to Oracle to be joined with any needed data that resides in Oracle. The user and application never know the difference.
Too Good to be True?
Often in life, things that sound too good to be true, are. However, if you want to leverage the numerous benefits of a cloud native data platform such as Google BigQuery, while mitigating the risk, time, and complexity involved with migration, Gluent Data Platform may be a perfect solution for you. Reach out to us here for your free Gluent Advisor report and to learn how Gluent Data Platform is just as good as it sounds.