Performance is a key focus of release 3.2 of the Gluent Data Platform (GDP) software suite. Offload and Smart Connector were enhanced to reduce the overall time, as well as, CPU consumption of offloads and hybrid queries. Additionally, Gluent has invested in additional support for our cloud integrations, including Cloudera and Microsoft Azure Cloud Storage. An improved user experience can be found across the Gluent software with this release. Other functional and operation enhancements are also included, of particular note, is the greater support for Apache Spark (originally added in 3.1).
Offload Performance and Smart Connector
Gluent’s 3.2 release includes significant performance improvements to Offload. For example, two new methods for offloading large datasets in parallel have been added; specifically, subpartition-based sharding for tables/partitions with many subpartitions and MOD-based sharding for large index-organized tables. The data sampling and analysis phase of an offload has improvements in the sampling algorithm, reducing the CPU consumption of the data analysis step that feeds into offload datatype auto-selection.
Multiple performance improvements have also been made to Smart Connector in Gluent’s 3.2 release. Examples include additional datetime functions, formats and expressions are supported for predicate pushdown, thereby offloading more workload to Hadoop during Hybrid Queries. Customers also have the option of reducing Smart Connector latency by enabling asynchronous metrics collection using Spark, meaning that Hybrid Queries complete without waiting to collect their query profile metrics from Hadoop (this task is handled separately by Gluent’s Metadata Daemon).
GDP has now been certified for use with a larger suite of vendors. This includes Cloudera Data Hub version 6.x, enabling customers to realize the benefits of the vendors’ latest release. Also, support for several types of Microsoft Azure cloud storage was added. Customers can use Azure Data Lake Storage Generation 1 (ADLS) and Generation 2 (ABFS(S)) storage options provided by their Azure Service Accounts to store data offloaded by Gluent Data Platform.
Functional and General Improvements
Gluent’s 3.2 release continues to deliver functional improvements across the product set. SQLMon report names can now be customized and Gluent Advisor Portal makes access to the Advisor Data Extractor easier with a link on the home screen. Offload and Present now automatically grant corresponding permissions on offload-join/present-join Hybrid Views to users with privileges on the underlying tables included in the joins. Present’s datatype detection step, performed in Hadoop against Hadoop-resident tables and views, has also been made more effective.
Gluent continues its commitment to make its products easier and more efficient to use with every release. In Gluent’s 3.2 release, the datatype analysis phase in Offload outputs the option syntax for recommended datatypes to use for offloading numeric data. All queries generated by Smart Connector are now instrumented with the source Hybrid Query details, such as RDBMS session, SQLID and so on, enabling Database Administrators to quickly identify, track and link source and target queries across both data platforms.
Gluent’s Result Cache feature now benefits from the use of Apache Arrow to access data solely from Hadoop and without requiring any intermediate storage on the RDBMS server during Hybrid Queries. In addition, several Connect checks have been streamlined to reduce the number of calls made to Hadoop DataNodes while testing for connectivity and configuration pre-requisites.
About Gluent Data Platform
Gluent Data Platform keeps enterprise RDBMS relevant in the new hybrid world and provides a data virtualization layer between traditional databases, cloud storage and modern analytics platforms that accelerate time to insight and eliminates data duplication and sprawl. Gluent allows customers to preserve their investments in their RDBMS platforms while leveraging the power, scalability, and elasticity of private and public clouds.
Enabling true data virtualization has been a cornerstone of Gluent software from the beginning. Our product allows companies the opportunity to reduce their overall investment while expanding the ability of the existing applications. If you are interested in learning more about how Gluent Data Platform can help virtualize your enterprise data, contact us at email@example.com.
An opportunity to work with your team on a pilot based upon your data components will solidify what the Gluent product line can do for your existing infrastructure.