Announcing the latest release of Gluent Data Platform, version 2.8.0. along with some nice product updates, including Hive offload and monitoring improvements, we have also greatly enhanced our dynamic schema evolution capability by introducing the Schema Sync component.
Gluent’s Schema Sync software is a new tool designed to automate schema evolution by detecting and applying source schema changes to offloaded tables. Schema Sync can be scheduled or used interactively to search one or more schemas for data model changes to offloaded tables. Any changes detected can be automatically applied to the corresponding Hive/Impala backend table or the relevant SQL and present commands can be recorded to a file to apply the changes later. Schema Sync also has a refresh mode which asserts and ensures that all Hybrid objects are aligned with their source and offloaded backend tables.
The simple command requires just one input parameter, include, which represents a list of tables or list of schemas. Wildcards are supported as well, allowing more flexibility in how the schema evolution command can be executed.
./schema_sync --include sales.*_dim -x
Schema Sync was designed to handle everything from a one-off column modification on a single table to thousands of DDL changes throughout a database. This additional versatility enables schema evolution to occur with a single command for all changes, rather than scheduling a command for each individual table or database schema.
Have a look at the short demo video to see Gluent’s Schema Sync tool in action.