PyQdeck LogoPyQdeckSubjectsSyllabusBlog

Data Mining Fundamentals and Association Rule Mining · Data Mining

Architecture of a Typical Data Mining System

Watch lectures

Architecture of a Typical Data Mining System

1. Block Diagram

                        +-------------------------------+
                        |       USER INTERFACE          |
                        | (query, browse, visualise,    |
                        |  ad-hoc mining requests)      |
                        +---------------+---------------+
                                        |  patterns / visualisation
                        +---------------v---------------+
                        |   PATTERN EVALUATION MODULE   |
                        | (interestingness measures,    |
                        |  threshold filters, pruning)  |
                        +---------------+---------------+
                                        |
                        +---------------v---------------+
                        |      DATA MINING ENGINE       |
                        |  functionality modules:       |
                        |  characterisation | association|
                        |  classification   | clustering |
                        |  outlier | evolution analysis |
                        +-------+---------------+-------+
                                |               |
        +-----------------------+               +-----------------------+
        |                                                              |
+-------v--------+                                          +----------v-------+
| KNOWLEDGE BASE |<-- domain knowledge, concept hierarchies,| DATABASE or      |
| (rules, levels,|    user beliefs, interestingness limits   | DATA WAREHOUSE   |
|  constraints)  |                                          | SERVER (SQL/OLAP)|
+----------------+                                          +----------+-------+
                                                                       |
                    +--------------------------------------------------+
                    |                        |                         |
            +-------v------+      +----------v-------+     +-----------v-----+
            | DATABASES /  |      | DATA WAREHOUSE   |     | WORLD WIDE WEB /|
            | OLTP SYSTEMS |      | (cleaned,        |     | OTHER SOURCES   |
            |              |      |  integrated)     |     | (files, sensors)|
            +--------------+      +------------------+     +-----------------+

(Note: data cleaning & integration is applied while preparing data for the database/data warehouse server.)

2. Components Explained

Component Role
Database / Data Warehouse Server Fetches the relevant, cleaned, integrated data on demand; answers SQL/OLAP/multidimensional queries
Knowledge Base Stores domain knowledge used to guide the search: concept hierarchies, user beliefs, metadata, constraints, thresholds
Data Mining Engine The heart — a set of functional modules (characterisation, association, classification, clustering, outlier, evolution analysis) that actually run the algorithms
Pattern Evaluation Module Filters discovered patterns using interestingness measures (support, confidence, lift, novelty, actionability); turns raw patterns into knowledge
User Interface Interaction layer — query specification, drill-down/browse, visualisation of rules, decision trees, clusters, cubes
Data preparation layer Cleaning, integration, selection, transformation (often folded into the server or the warehouse ETL)

3. Coupling of Data Mining with a DBMS / Data Warehouse

Coupling Description Pros / Cons
No coupling Mining reads flat files extracted from the DB Simple; poor performance, data staleness, duplicated code
Loose coupling Mining uses DB/warehouse as the data source, results stored back Uses existing storage and query optimiser; not tuned for mining
Semi-tight coupling In addition to loose coupling, some mining primitives/aggregates are pushed into the DBMS Better performance with modest DBMS changes
Tight coupling Mining primitives implemented inside the DBMS kernel, deeply optimised Best performance and scalability; complex, intrusive, rarely fully done

4. Layers of a Complete DM System

  Layer 4  USER / APPLICATION  (dashboards, CRM, fraud alerts, BI reports)
  Layer 3  MINING + PATTERN EVALUATION (engine + interestingness filters)
  Layer 2  DATA MANAGEMENT (DBMS / warehouse server, knowledge base, metadata)
  Layer 1  DATA STORAGE & SOURCES (databases, files, web, streams)

5. Typical Working Sequence

  1. User expresses a mining request through the UI ('find customers likely to churn').
  2. The server supplies the relevant cleaned data from the warehouse.
  3. Concept hierarchies and constraints from the knowledge base steer the engine.
  4. The engine applies the chosen technique(s) and produces candidate patterns.
  5. The evaluation module prunes uninteresting patterns against thresholds.
  6. The UI presents surviving patterns visually and the user may refine the query (multi-step, interactive mining).

6. Exam Points

  • Draw and explain the architecture of a typical data mining system (all components) — 10 marks (very frequent).
  • Explain the role of the knowledge base and pattern evaluation module — 5 marks.
  • Explain coupling schemes of DM with DBMS with merits/demerits — 5-7 marks.
  • Why is coupling needed at all? (performance, no data duplication, use of optimiser) — 3 marks.
Official AppAndroid App

PYQDeck for Android

Solve BEU previous-year exam papers anywhere, anytime. Fast search, offline paper viewing, and step-by-step solutions designed specifically for your phone.

Get on Google Play
Previous topicData Discretization and Concept Hierarchy GenerationNext topicClassification of Data Mining Systems

← All Data Mining modules