How to Create a Business Focused Data Quality Assessment Dylan Jones, Editor/Community Manager editor@dataqualitypro.com Why Do We Need a Data Quality Assessment? We need to perform a data quality assessment [DQA] for many reasons... Data Warehouse Implementation Data Migration Project CRM / ERP Monitoring Integrating Billing Systems Potential Merger or Acquisition Revenue Assurance or Recovery Risk Analysis or Compliance Fraud Investigation Integration Integrating Billing Systems On a billing integration we want to know if our processes are working correctly so that we’re compliant with regulations and also protecting our revenue Migration Data Migration Project On a migration we want to know if the legacy data will support integration, transformation, load and future target functions Merger Potential Merger or Acquisition On a merger, we want to verify that the company is being accurate with its statement of accounts e.g. “We have 20,000 customers” Compliance Risk Analysis or Compliance For compliance we may want to check that a company has handed over all information of customers that meet certain criteria e.g. Terrorists/Criminals DQA helps us understand... “Does our data perform against expectations, goals or requirements?” Regulatory Performance Financial Legal What Happens During a DQA? We use technology to assess data against a series of metrics or criteria ● ● ● ● 17,121 Customer Records have a missing postal code 35% of Equipment.Install_Date values are invalid 99.2% of Financial Transactions have a valid timestamp 300 out of 192,213 customer accounts are duplicated What are some common metrics (or data quality dimensions)? Completeness Validity Conformance Accuracy Consistency Integrity Uniqueness Timeliness Dependency Coverage Business Rule Violations etc….. DQA’s can be complex... Result #1 Assessment #1 Result #2 Assessment #2 Result #3 Assessment #3 Findings #1 Findings #2 Findings #3 (Local Data Stewards) (Legal & Compliance) (Head of Operations) So, what’s the problem? ● DQA’s are often carried out by technical people who assess data in isolation (e.g. by column, table, system) ● The findings lack business impact because they present primarily on data quality metrics ● The business can’t engage because there is no business relevance to create a story around But we’ve bought a data quality tool! Yes, that’s a wise move because … ● You can leverage your data quality functionality to tell a bigger (+ better) story by integrating business metrics ● You can automate the assessment process and put it into business-as-usual How can you improve the conventional approach? Understand Your Profitability Drivers Build a Financial Performance Model 1 2 Create an Holistic Data Quality System 3 Create an Interrogative Reporting Layer 4 Step 1: Understand Your Profitability Drivers Understand Your Profitability Drivers 1 Case Study: Revenue Assurance Telecoms companies need to ensure that when they implement complex B2B customer telecoms solutions they are optimising their internal cost centres, recovering all client revenues and ensuring service level continuity. American Bank requests an international telecoms solution Understand Your Profitability Drivers 1 Telco initiates multiple processes Understand Your Profitability Drivers 1 An example of a Telecoms process System System System System You need to understand the profitability of your core process ● ● ● ● ● Cost of installing 3rd party equipment Cost of leasing international lines Cost of hiring contractors or local workers Cost of lead generation and sales Revenues generated from service Understand Your Profitability Drivers 1 Performance is also useful... ● ● ● ● How long does it take to service a line? Which team receives the most orders? Which engineers perform the most revisits? What is the Process Cycle Efficiency? Understand Your Profitability Drivers 1 “Huh? But we don’t have this information to hand because…” ● ● ● ● ● Some of it belongs to a different owner Some of it is in different systems Some of it is in paper records Some of it doesn’t connect Some of it... Understand Your Profitability Drivers 1 Good. Poor information gives you a perfect opportunity to add value. Understand Your Profitability Drivers 1 Flex your data quality muscles to help create a better view of costs, revenues and performance Engineering Planning Data Quality System Procurement Fulfillment Billing Understand Your Profitability Drivers 1 Once you start linking and enriching your data you can gain a clearer view of profitability and performance Understand Your Profitability Drivers 1 Example: Telecoms Order System Links to other systems, you can enrich with financial and performance data Performance Metrics and data quality defects Step 2: Build a Financial Performance Model Build a Financial Performance Model 2 We need to connect clusters of financial, performance and data quality metrics Data Quality Management Reporting System Build a Financial Performance Model 2 How do you do it? Demonstration... Build a Financial Performance Model 2 But isn’t this just business intelligence? Precisely. It’s where modern data quality technology is heading. Data Quality metrics and business performance metrics are measured the same way but for years they’ve been separated. You need to bring them into the same environment if you want to engage and focus the business. Build a Financial Performance Model 2 Step 3: Create an Holistic Data Quality System Create an Holistic Data Quality System 3 Why bother with an holistic data quality system ? Each system can have great quality data when measured against isolated data quality metrics but... … if you measured data quality and financial impact across a broader process it could still result in significant lost revenue and profits Key Tip: Assessing the quality of each system in isolation it won’t give you the full story Create an holistic data quality system 3 For example: Completeness Completeness in the [Planning.Inventory] column tells us if there are any missing values but... Value Complete? Juniper M320 YES Juniper M320 YES Juniper M312 YES Juniper m320 YES JUNIPER (M320) YES Create an holistic data quality system 2 3 For example: Completeness ... it doesn’t tell us if there are other values that should be there from [Stocks.Inventory] Master Stock Item Qty Juniper M320 10 Juniper M312 7 Juniper M315 3 Juniper M325 2 Juniper M327 15 Lack of coverage Value Complete? Juniper M320 YES Juniper M320 YES Juniper M312 YES Juniper m320 YES JUNIPER (M320) YES Create an holistic data quality system 2 3 For example: Completeness ... it doesn’t tell us if there are other values that should be there from [Stocks.Inventory] Procurement Catalogue Cost Juniper M320 40000 Juniper M312 90000 Juniper M315 890 Juniper M325 500 Juniper M327 100 Lack of price context Value Complete? Juniper M320 YES Juniper M320 YES Juniper M312 YES Juniper m320 YES JUNIPER (M320) YES Create an holistic data quality system 2 3 Step 3: Create an Interrogative Reporting Layer Create an Interrogative Reporting Layer 4 Reporting Layer By this point you will have a consolidated view of your: ● Financial Metrics ● Data Quality Metrics ● Performance Metrics Create an interrogative reporting layer 4 What does the overall architecture look like? Dashboards and Visualisation DQ Assessment Rules Data Profiling/ Discovery Data Repository and Storage Information Chain Management DQ Monitoring and Alerts Data Integration/ Movement Create an interrogative reporting layer 4 This gives you much greater clarity of the impact of poor quality data ● How long does it take to complete an order? ● What staff costs are involved in fulfillment? ● What costs are incurred when data quality errors force a re-design? ● What is the frequency of delayed orders over the last 12 months? Create an interrogative reporting layer 4 Adding performance and financial data creates focus One telecoms company found their fulfillment processes were taking longer and were increasing their stock of unplanned equipment Create an interrogative reporting layer 4 The issue lay with a rogue software release that introduced defects into the information chain Software Change Added Defects enter the information chain Holistic Reporting System monitors data quality, financial and performance metrics Alarm Raised Create an interrogative reporting layer 4 Summary Takeaway #1: Stop thinking of data quality assessments in terms of isolated analysis Summary Takeaway #2: Understand how your business model operates and create an end-to-end view of it, identifying data sources along the way Summary Takeaway #3: Focus on the goal of your data quality assessment and create an architecture to support that objective e.g. Data Quality impact on Service Levels Summary Takeaway #4: To create engagement and focus from the business make sure that you’re integrating business metrics into your reporting and underlying assessment architecture. Summary Takeaway #5: Don’t spoon-feed the business with canned reports, give them an environment that they can interrogate, let them discover the issues that matter to them most (hint: it will change over time too) Questions? Contact me: Dylan Jones (editor@dataqualitypro.com Search on Data Quality Pro: http://dataqualitypro.com Data Quality Assessment Guide: http://bit.ly/dq-assess