|
A scalable Data
Acquisition Management application for the Sales and Marketing Data
warehouse initiative of a large global Pharmaceutical company.
Overview
Various divisions
within Sales & Marketing department of a global Pharmaceutical
company subscribe to syndicated data from various sources for
tactical and strategic decision making. Over a period of time
multiple localized point solutions existed for data QA/QC and
reporting. This resulted in the following problems:
Problems
- Lack of a centralized
repository of all data resulted in the same data being acquired by
multiple divisions leading to redundant expenditure
- Multiple localized
point solutions resulted in lack of integration, scalability and
difficulty in maintenance
- Inconsistent business
rules and different methods of QA/QC to review data in different
systems produced reports which were inconsistent. This led to a
lot of time and cost spent in investigating discrepancies and
there by delaying the delivery of critical information to end
users
- High cost of system
maintenance
Solution
Our solution comprised
of building a scalable application consisting of multiple
sub-systems (see illustration) using Industry Standard technology
architecture and system software. The various components of the
application include:
Data
cataloging and Meta data capture Sub-system
This module captures the
profile and processing attributes of all data files being acquired
and allows users to search and view relevant information through
an intuitive browser interface.
Data
Scheduling, monitoring and notification sub-system
This sub-system
controls the set of processes that capture, integrate, transform,
QA/QC and load source data into a staging area. Real time
processing status is provided through a browser interface and
notifications are sent to stake holders through automatic e-mails.
Data
Quality Analysis and Reporting Sub-system
All incoming data is
subjected to pre-defined QA/QC process (like control/hash total,
Data trending and data archival comparison) and any data anomalies
are automatically notified to the users. Standard QA/QC reports
are also generated and available to users for further analysis.
Data
Transformation, Standardization and Loading Sub-system
This module delivers a
consistent and standard view of all acquired data to internal
users. Uniform business rules for data transformation and
standardization is applied and industry standard ETL tools are
used to load and deliver the data.
Benefits
|
