Product
-
May 16, 2023

New and Improved Deltas: Comparing tables is now even easier

Our latest Deltas update adds the ability to compare against multiple tables, send notifications, and more.

Kendall Lovett
Get Data Insights Delivered
Join hundreds of data professionals who subscribe to the Data Leaders Digest for actionable insights and expert advice.
Stay Informed
Sign up for the Data Leaders Digest and get the latest trends, insights, and strategies in data management delivered straight to your inbox.
Get the Best of Data Leadership
Subscribe to the Data Leaders Digest for exclusive content on data reliability, observability, and leadership from top industry experts.

Get the Best of Data Leadership

Subscribe to the Data Leaders Digest for exclusive content on data reliability, observability, and leadership from top industry experts.

Stay Informed

Sign up for the Data Leaders Digest and get the latest trends, insights, and strategies in data management delivered straight to your inbox.

Get Data Insights Delivered

Join hundreds of data professionals who subscribe to the Data Leaders Digest for actionable insights and expert advice.

Data is constantly being updated, optimized, and migrated. Every day, more and more data is being moved into data warehouses and lakes with tools like Airbyte, Fivetran, and Talend. Your data teams are moving data from source systems to targets; from staging to production, and they’re probably starting to reverse it back to your applications. Data is moving and changing everywhere.

All of this data movement highlights a fundamental challenge: It’s really hard to tell whether the data you moved from point A landed unbroken at point B. That’s why we created Deltas, a unique and powerful Bigeye feature that allows you to compare and validate multiple versions of any dataset.

Deltas use automatic column mapping and ML-recommended data quality checks—beyond just row count—to catch any data drift between datasets. So whether you’re replicating data, migrating to a warehouse or lake, or merging data model changes, you can identify issues and fix them before they’re propagated across your data stack.  

What’s new in the latest Deltas release

Compare a source table with multiple target tables

With the latest release, it’s now possible to compare a source table against up to two target tables across all supported data sources. This is helpful for cases when data is being replicated from a single source to multiple destination tables.

Simply set your source table and then identify the A and B target tables you’d like to compare against it. Deltas will automatically map columns and recommend metrics (i.e. data quality checks). Users can then choose to customize these recommendations or add their own metrics.

Deltas also provide advanced features like group by—helpful for adding additional levels of granular analysis—and the ability to filter the data with SQL conditions. This allows users to limit the Delta to a specific time period, omit rows, choose to only analyze a particular dimension of the data, and so on.

Slack, email, and Webhook notifications

In addition to in-app notifications, Deltas can now alert users via email, chat (Slack or MS Teams), or to your destination of choice via webhook. This allows data teams to be even more proactive and eliminates the need to log into Bigeye and check on a Delta status.

Improved UI for faster, clearer insights

Finally, the Deltas UI received a complete overhaul in this release. It’s now faster to configure and deploy Deltas and easier to identify where there are issues in your migration or replication job and decide what needs your attention. A brand new hierarchical view and information summaries give you the insights you need to take action in a matter of seconds.

Deltas are just one powerful component of the Bigeye platform that helps data teams ensure that data is always accurate and up to date. If you’re tired dealing with painful data migrations or silently failing replication jobs, get a demo and see Deltas in action.

share this episode
Resource
Monthly cost ($)
Number of resources
Time (months)
Total cost ($)
Software/Data engineer
$15,000
3
12
$540,000
Data analyst
$12,000
2
6
$144,000
Business analyst
$10,000
1
3
$30,000
Data/product manager
$20,000
2
6
$240,000
Total cost
$954,000
Role
Goals
Common needs
Data engineers
Overall data flow. Data is fresh and operating at full volume. Jobs are always running, so data outages don't impact downstream systems.
Freshness + volume
Monitoring
Schema change detection
Lineage monitoring
Data scientists
Specific datasets in great detail. Looking for outliers, duplication, and other—sometimes subtle—issues that could affect their analysis or machine learning models.
Freshness monitoringCompleteness monitoringDuplicate detectionOutlier detectionDistribution shift detectionDimensional slicing and dicing
Analytics engineers
Rapidly testing the changes they’re making within the data model. Move fast and not break things—without spending hours writing tons of pipeline tests.
Lineage monitoringETL blue/green testing
Business intelligence analysts
The business impact of data. Understand where they should spend their time digging in, and when they have a red herring caused by a data pipeline problem.
Integration with analytics toolsAnomaly detectionCustom business metricsDimensional slicing and dicing
Other stakeholders
Data reliability. Customers and stakeholders don’t want data issues to bog them down, delay deadlines, or provide inaccurate information.
Integration with analytics toolsReporting and insights

Get the Best of Data Leadership

Subscribe to the Data Leaders Digest for exclusive content on data reliability, observability, and leadership from top industry experts.

Stay Informed

Sign up for the Data Leaders Digest and get the latest trends, insights, and strategies in data management delivered straight to your inbox.

Get Data Insights Delivered

Join hundreds of data professionals who subscribe to the Data Leaders Digest for actionable insights and expert advice.

Join the Bigeye Newsletter

1x per month. Get the latest in data observability right in your inbox.