QC Technology Decisions Inc.
  • About QC
  • Professional Services
  • Contact

Managing Quality in an Education Data Warehouse

8/14/2015

0 Comments

 
Modern Education Data Warehouses require that data be conformed across multiple sources and in some cases even across multiple state agencies such as Early Childhood Development, Public Education (K12) Agencies, Private Education Agencies, Higher Education Agencies and Workforce Development.   Emerging trends in “Big Data” indicate that agencies will want to incorporate other data such as public data sets to perform statistical analysis to improve the Educational System.

Education Data Warehouses need to maintain accurate sets data and cohorts over time that are sourced from a variety of systems.   This poses a unique challenge with data integration and quality control to create a sustainable and efficient process capable of moving data faster into the hands of the decision makers that need it.

Enterprise data integration capabilities are the foundation for any data warehousing solution. Using reliable, enterprise class technology from Informatica, the solution accesses, integrates, and delivers data of any volume, for any application from virtually any source in any format at any latency.  Armed with these capabilities, IT organizations can break down the silos wherever data is held enabling seamless data sharing across multiple state agencies and commissions.

Although education agencies (both local and at the state level) have developed have developed processes to match student records, generate IDs, and conform data to meet their unique data warehousing requirements; they continue to struggle with data collection processes.   This bottleneck is a barrier to “Velocity”, which when removed and by definition of the three V’s (Volume, Velocity, and Variety), would bring Education Data Warehousing into the “Big Data” arena.

Addressing Data Quality Challenges

Data quality is not a one-time effort; However, in traditional data warehousing projects a significant (if not most of their time) is spent cleansing and organizing the data in a way that makes sense to business users.  The events and changes that allows data anomalies to be introduced into an environment are not unique; however, addressing anomalies becomes critically important when users are relying on accurate student records and demographics for making decisions. It is necessary for the data management teams to not just address acute data failures, but also baseline the current state of data quality so that one can identify the critical failure points and determine improvement targets.

The ability to monitor data quality and react quickly to changes demonstrates a level of organizational maturity that views information as an asset and rewards proactive involvement by delivering on the promises of business intelligence;   Data Trust,  Business Value,  and Process Alignment.

Address data quality challenges through:
  • Assessment
  • Definition
  • Validation and Cleansing
  • Monitoring and Managing Ongoing Quality of Data

Quality Scorecards

A data quality scorecard is a management tool that captures a virtual snapshot of the quality levels of your data, presents that information to the user, and provides insight as to where data flaws are impacting business operations and where the most egregious flaws exist within the system. Using data quality rules based on defined dimensions provides a framework for measuring conformance to business data quality expectations.

Your quality score card helps control:

  • Validity of Data
  • Thresholds for Conformance
  • Ongoing Process Control
  • Proactive Monitoring and Alerting
  • Data Standardization


Although this post refers to an education data warehouse, the same principles could be applied to virtually any data warehouse to ensure data quality.   For additional information please contact us anytime!
Picture
Accelerating Education Data Warehousing with Informatica Data Quality. A framework to define, control and monitor quality of data.
Download Whitepaper
0 Comments

    Author

    Shane Quigley is an expert in data warehousing, business intelligence, systems analysis, and solution architecture.

    Archives

    April 2017
    July 2016
    September 2015
    August 2015
    July 2015
    June 2015
    September 2013
    January 2013

    Categories

    All
    Big Data
    Cognos
    DevaaS
    Education Data
    Healthcare
    Integration
    PowerBI
    SharePoint
    SQL Server

    RSS Feed


QC Technology Decisions Inc.

Quigley and Company Technology Decisions Inc.  is a group of experienced professionals who offer open and practical advice and technology solutions by applying critical thinking to our designs.   ​

info@qctechnology.com 

Services

Service Catalog

Portfolio Highlights

About us

About QC 

​Contact Us


​Blog
  • About QC
  • Professional Services
  • Contact