2021 question paper

Data Mining

26 questions

  1. Q1a. Discuss whether or not each of the following activities is a data mining task. Give briefly proper justification : (i) Dividing the customers of a company according to their profitability (ii) Monitoring and predicting failures in a hydropower plant20212m

    Module 2: Data Mining and Association Rule Mining

    Discuss whether or not each of the following activities is a data mining task. Give briefly proper justification :

    (i) Dividing the customers of a company according to their profitability
    (ii) Monitoring and predicting failures in a hydropower plant

    View this question on its own page →
    Worked Solution

    Is it a Data Mining Task?

    (i) Dividing customers according to profitability

    Yes, this is a data mining task.
    This is a clustering/segmentation problem — grouping customers into meaningful segments based on profitability-related attributes (purchase frequency, spend, margin) without predefined labels. Discovering these natural groupings from raw transaction/customer data is a core data mining functionality (unsupervised clustering).

    (ii) Monitoring and predicting failures in a hydropower plant

    Yes, this is a data mining task (specifically the "predicting" part).

    • Monitoring alone (just observing sensor readings in real time) is not mining — it's plain data collection/surveillance.
    • Predicting failures, however, requires learning patterns from historical sensor/operational data to forecast future failures — this is a classification/prediction task, a core data mining functionality.

    So both qualify as data mining tasks, but for different reasons: (i) because segmentation without labels = clustering, and (ii) because predicting future failures from historical patterns = predictive modeling — not because "monitoring" itself is mining.

  2. Q1b. Classify the following attributes as binary, discrete or continuous. Also classify them as qualitative (nominal or ordinal) or quantitative (interval or ratio). Some cases may have more than one interpretation, so briefly indicate your reasoning if you think there may be some ambiguity : (i) Distance from center of campus (ii) Ability to pass light in terms of opaque, translucent and transparent20212m

    Module 2: Data Mining and Association Rule Mining

    Classify the following attributes as binary, discrete or continuous. Also classify them as qualitative (nominal or ordinal) or quantitative (interval or ratio). Some cases may have more than one interpretation, so briefly indicate your reasoning if you think there may be some ambiguity :

    (i) Distance from center of campus
    (ii) Ability to pass light in terms of opaque, translucent and transparent

    View this question on its own page →
    Worked Solution

    Attribute Classification

    (i) Distance from center of campus

    • Type: Continuous (can take any real value, e.g., 0.5 km, 1.27 km).
    • Scale: Quantitative — Ratio (has a true zero — "0 distance" means literally at the campus center — and ratios are meaningful: 4 km is twice as far as 2 km).

    (ii) Ability to pass light: opaque, translucent, transparent

    • Type: Discrete (a small, finite set of category values).
    • Scale: Qualitative — Ordinal. There is a natural order — opaque (passes no light) < translucent (passes some light) < transparent (passes most/all light) — but the "distance" between categories isn't numerically meaningful (we can't say translucent is exactly halfway between opaque and transparent in any measurable unit).

    Ambiguity note

    (ii) could arguably be seen as nominal if one ignores the light-passing order and treats the three labels as unordered categories, but since the attribute is explicitly defined by increasing "ability to pass light," the natural/expected interpretation is ordinal.

    Attribute Discrete/Continuous Qualitative/Quantitative
    Distance from campus center Continuous Quantitative — Ratio
    Opaque/Translucent/Transparent Discrete Qualitative — Ordinal
  3. Q1c. Differentiate between supervised and unsupervised classification.20212m

    Module 3: Classification and Prediction

    Differentiate between supervised and unsupervised classification.

    View this question on its own page →
  4. Q1d. In real-world data, tuples with missing values for some attributes are a common occurrence. List at least two methods for handling this problem.20212m

    Module 2: Data Mining and Association Rule Mining

    In real-world data, tuples with missing values for some attributes are a common occurrence. List at least two methods for handling this problem.

    View this question on its own page →
    Worked Solution

    Handling Missing Values — At Least Two Methods

    1. Ignore the tuple — Simply discard records with missing values. Only reasonable when the missing attribute is critical (e.g., the class label) and the dataset is large enough that dropping a few rows doesn't hurt.

    2. Fill in the missing value manually — A domain expert examines and fills each gap. Accurate but extremely slow and impractical for large datasets.

    3. Use a global constant — Replace all missing values with a fixed label like "Unknown" or −∞. Simple, but the mining algorithm may mistakenly treat "Unknown" as a meaningful, interesting category.

    4. Use the attribute mean/median — Replace missing numeric values with the mean (or median for skewed data) of that attribute across all tuples.

    5. Use the mean/median of the same class — Replace with the mean of tuples belonging to the same class as the tuple with the missing value (more accurate than a global mean).

    6. Use the most probable value — Predict the missing value using inference methods like regression, decision-tree induction, or Bayesian formalism — the most sophisticated but most accurate approach.

    Missing Value
         │
         ├── Ignore tuple ─────────────▶ (drop row)
         ├── Manual fill ───────────────▶ (expert input)
         ├── Global constant ───────────▶ "Unknown"
         ├── Attribute mean/median ─────▶ statistical fill
         ├── Class-wise mean ───────────▶ context-aware fill
         └── Predicted value (regression/tree/Bayes) ─▶ model-based fill
    
  5. Q1e. Describe briefly z-score normalization.20212m

    Module 2: Data Mining and Association Rule Mining

    Describe briefly z-score normalization.

    View this question on its own page →
    Worked Solution

    Z-Score Normalization

    Z-score (zero-mean) normalization rescales an attribute's values based on its mean (μ) and standard deviation (σ):

    v=vμσv' = \frac{v - \mu}{\sigma}

    where:

    • vv = original value
    • μ\mu = mean of the attribute
    • σ\sigma = standard deviation of the attribute
    • vv' = normalized value

    Effect

    • The transformed data has mean 0 and standard deviation 1.
    • Values are typically small, ranging roughly from −3 to +3 (assuming a roughly normal distribution), though technically unbounded.

    Why use it

    • Useful when the min/max of an attribute are unknown or when there are outliers that would distort min-max normalization.
    • Puts differently-scaled attributes (e.g., income in lakhs vs age in years) on comparable footing before applying distance-based algorithms (k-NN, k-means, clustering).
    Raw values:  [200, 300, 400, 600, 1000]
                        │  subtract mean, divide by std dev
                        ▼
    Z-scores:    [negative, negative, ~0, positive, large positive]
    

    A z-score of 0 means the value equals the attribute's mean; positive/negative z-scores indicate how many standard deviations above/below the mean the value lies.

  6. Q1f. Describe briefly a priori principle.20212m

    Module 2: Data Mining and Association Rule Mining

    Describe briefly a priori principle.

    View this question on its own page →
    Worked Solution

    The Apriori (A Priori) Principle

    Statement: "If an itemset is frequent, then all of its subsets must also be frequent."

    Equivalently, by contrapositive: "If an itemset is infrequent, then all of its supersets must also be infrequent" — this is the practical pruning rule actually used by the algorithm.

    Why it holds

    Support is anti-monotone: adding more items to an itemset can only keep or reduce the number of transactions containing it (support never increases as itemset size grows).

    XY    support(X)support(Y)X \subseteq Y \implies support(X) \geq support(Y)

    Diagram

            {A,B,C,D}   infrequent
           /    |    |    \
      {A,B,C} {A,B,D} {A,C,D} {B,C,D}   ← if {A,B,C,D} infrequent, ALL its
           \    |    |    /              supersets are pruned instantly
             (pruned without counting)
    

    Why it matters

    This principle is the backbone of the Apriori algorithm's efficiency: once an itemset is found infrequent, none of its supersets need to be generated or counted at all — dramatically shrinking the candidate search space compared to brute-force enumeration of all 2d2^d possible itemsets.

  7. Q1g. List at least four general characteristics of hierarchical clustering methods.20212m

    Module 4: Cluster Analysis

    List at least four general characteristics of hierarchical clustering methods.

    View this question on its own page →
  8. Q1h. Compare and contrast k-medoids with k-means.20212m

    Module 4: Cluster Analysis

    Compare and contrast k-medoids with k-means.

    View this question on its own page →
  9. Q1i. Describe the OLAP operation slice and dice.20212m

    Module 1: Data Warehousing and Business Analysis

    Describe the OLAP operation slice and dice.

    View this question on its own page →
    Worked Solution

    Slice and Dice (OLAP Operations)

    Think of a data cube as a 3D box of numbers indexed along multiple dimensions (e.g., Time, Product, Region).

    Slice

    Selects one specific value on one dimension, "cutting" a single 2D layer out of the cube.

            Product
           /|  /|  /|
          / | / | / |
         /__|/__|/__|
        |   |   |   |     <- Slice at Time = "2024"
    Time|___|___|___|
        |   |   |   |
        |___|___|___|
            Region
    

    Example: "Sales for Time = 2024" → gives a 2D Product × Region table.

    Dice

    Selects a sub-cube by picking a range/subset of values on two or more dimensions simultaneously.

    Full cube ──dice──▶  smaller cube
    (Time × Product × Region)   (Time∈{2023,2024} × Product∈{A,B} × Region=West)
    

    Example: "Sales where Time ∈ {2023, 2024} AND Product ∈ {A, B} AND Region = West."

    Key Difference

    Operation Dimensions restricted Result
    Slice 1 dimension fixed to a single value One lower-dimension "layer" of the cube
    Dice 2+ dimensions restricted to ranges/subsets A smaller sub-cube (same dimensionality)

    Both are used for interactive, ad-hoc drill-down analysis in OLAP tools without needing to re-query the source data.

  10. Q1j. Why data mining is a misnomer?20212m

    Module 2: Data Mining and Association Rule Mining

    Why data mining is a misnomer?

    View this question on its own page →
    Worked Solution

    Why "Data Mining" is a Misnomer

    The term "data mining" literally suggests "mining data" — but strictly speaking, we don't mine data itself; we mine patterns/knowledge from data, much like gold mining extracts gold from rock, not "rock mining."

    Reasoning

    • If taken literally, "mining coal from rocks" is called coal mining, not "rock mining" — the process is named after what is extracted, not the raw material it comes from.
    • Following that convention, the process should really be called "knowledge mining from data" or simply "knowledge mining" — since what we're actually after is the hidden knowledge/patterns, not the data itself (which we already have).
    • The formally correct term for the full pipeline is KDD — Knowledge Discovery in Databases — of which "data mining" is just one (pattern-extraction) step, yet the popular term "data mining" is loosely used for the entire process.

    In short: "data mining" is a misnomer because it names the process after its raw input (data) rather than its actual output (knowledge/patterns) — the opposite convention of how "mining" terms normally work (e.g., gold mining, coal mining name the output, not the source rock).

  11. Q2a. How does instance-based classifier work? List two instance-based classifiers.20216m

    Module 3: Classification and Prediction

    How does instance-based classifier work? List two instance-based classifiers.

    View this question on its own page →
  12. Q2b. What do you mean by under-fitting and over-fitting of a classification model?20214m

    Module 3: Classification and Prediction

    What do you mean by under-fitting and over-fitting of a classification model?

    View this question on its own page →
  13. Q2c. For the following vectors, x and y, calculate the indicated similarity or distance measures : \mathbf{x} = \{0, 1, 0, 1\}, \quad \mathbf{y} = \{1, 0, 1, 0\} Calculate: 1. Cosine similarity 2. Correlation 3. Euclidean distance 4. Jaccard similarity20214m

    Module 2: Data Mining and Association Rule Mining

    For the following vectors, x and y, calculate the indicated similarity or distance measures :

    x={0,1,0,1},y={1,0,1,0}\mathbf{x} = \{0, 1, 0, 1\}, \quad \mathbf{y} = \{1, 0, 1, 0\}

    Calculate:

    1. Cosine similarity
    2. Correlation
    3. Euclidean distance
    4. Jaccard similarity
    View this question on its own page →
    Worked Solution

    Similarity/Distance Measures for x = {0,1,0,1}, y = {1,0,1,0}

    Setup

    x=(0,1,0,1),y=(1,0,1,0)\mathbf{x} = (0, 1, 0, 1), \quad \mathbf{y} = (1, 0, 1, 0)

    1. Cosine Similarity

    cos(x,y)=xyxy\cos(\mathbf{x}, \mathbf{y}) = \frac{\mathbf{x} \cdot \mathbf{y}}{\|\mathbf{x}\|\|\mathbf{y}\|}

    • Dot product: (0)(1)+(1)(0)+(0)(1)+(1)(0)=0(0)(1)+(1)(0)+(0)(1)+(1)(0) = 0
    • x=02+12+02+12=2\|\mathbf{x}\| = \sqrt{0^2+1^2+0^2+1^2} = \sqrt{2}
    • y=12+02+12+02=2\|\mathbf{y}\| = \sqrt{1^2+0^2+1^2+0^2} = \sqrt{2}

    cos(x,y)=022=0\cos(\mathbf{x},\mathbf{y}) = \frac{0}{\sqrt{2}\cdot\sqrt{2}} = \boxed{0}

    2. Correlation

    Mean: xˉ=yˉ=0.5\bar{x} = \bar{y} = 0.5

    i xixˉx_i-\bar{x} yiyˉy_i-\bar{y} product (xixˉ)2(x_i-\bar{x})^2 (yiyˉ)2(y_i-\bar{y})^2
    1 -0.5 0.5 -0.25 0.25 0.25
    2 0.5 -0.5 -0.25 0.25 0.25
    3 -0.5 0.5 -0.25 0.25 0.25
    4 0.5 -0.5 -0.25 0.25 0.25
    Σ -1.0 1.0 1.0

    r=1.01.0×1.0=1r = \frac{-1.0}{\sqrt{1.0 \times 1.0}} = \boxed{-1}
    (makes sense: y is exactly the complement of x, so they're perfectly negatively correlated)

    3. Euclidean Distance

    d(x,y)=(01)2+(10)2+(01)2+(10)2=1+1+1+1=4=2d(\mathbf{x},\mathbf{y}) = \sqrt{(0-1)^2+(1-0)^2+(0-1)^2+(1-0)^2} = \sqrt{1+1+1+1} = \sqrt{4} = \boxed{2}

    4. Jaccard Similarity

    Binary co-occurrence counts across the 4 positions:

    Position x y Match type
    1 0 1 f01
    2 1 0 f10
    3 0 1 f01
    4 1 0 f10

    f11=0, f01=2, f10=2, f00=0f_{11}=0,\ f_{01}=2,\ f_{10}=2,\ f_{00}=0

    J(x,y)=f11f01+f10+f11=02+2+0=0J(\mathbf{x},\mathbf{y}) = \frac{f_{11}}{f_{01}+f_{10}+f_{11}} = \frac{0}{2+2+0} = \boxed{0}

    Summary

    Measure Value Interpretation
    Cosine similarity 0 Vectors are orthogonal
    Correlation -1 Perfectly negatively correlated
    Euclidean distance 2 Maximum possible distance for these binary vectors
    Jaccard similarity 0 No 1-1 overlap at all — completely dissimilar on "presence"

    All four measures agree that x and y are maximally dissimilar — they're exact bitwise complements of each other.

  14. Q3a. Build a decision tree using the training data in the table given below. Divide the height attribute into ranges as follows : \{0, 1.6\], (1.6, 1.7\], (1.7, 1.8\], (1.8, 1.9\], (1.9, 2.0\], (2.0, 5.0\] | Gender | Height (m) | Class | |:---:|:---:|:---:| | F | 1.6 | Short | | M | 2 | Tall | | F | 1.9 | Medium | | F | 1.88 | Medium | | F | 1.7 | Short | | M | 1.85 | Medium | | F | 1.6 | Short | | M | 1.7 | Short | | M | 2.2 | Tall | | M | 2.1 | Tall | | F | 1.8 | Medium | | M | 1.95 | Medium | | F | 1.9 | Medium | | F | 1.8 | Medium | | F | 1.75 | Medium |20218m

    Module 3: Classification and Prediction

    Build a decision tree using the training data in the table given below. Divide the height attribute into ranges as follows :
    \{0, 1.6\], (1.6, 1.7\], (1.7, 1.8\], (1.8, 1.9\], (1.9, 2.0\], (2.0, 5.0\]

    Gender Height (m) Class
    F 1.6 Short
    M 2 Tall
    F 1.9 Medium
    F 1.88 Medium
    F 1.7 Short
    M 1.85 Medium
    F 1.6 Short
    M 1.7 Short
    M 2.2 Tall
    M 2.1 Tall
    F 1.8 Medium
    M 1.95 Medium
    F 1.9 Medium
    F 1.8 Medium
    F 1.75 Medium
    View this question on its own page →
  15. Q3b. What is over-fitting and under-fitting of a model? When do they happen?20216m

    Module 3: Classification and Prediction

    What is over-fitting and under-fitting of a model? When do they happen?

    View this question on its own page →
  16. Q4a. A university plans to build a data warehouse that would help them in analyzing the performance of the students in various courses in different academic sessions. They want to analyze if there is any relation between the average grade of a course and the number of students attending it. They would also like to know if there were some courses offered but did not have any students registered for them. Relative performance among boys and girls and average grades of students from various States and cities of the country for each course must be analyzed and also overall CGPA. Design a star schema for such a data warehouse clearly identifying the fact table(s) and dimension table(s), their primary key(s) and foreign key(s). Your schema should at least be able to satisfy the above-mentioned analysis requirements. You may consider other suitable attributes for the dimension table(s).202110m

    Module 1: Data Warehousing and Business Analysis

    A university plans to build a data warehouse that would help them in analyzing the performance of the students in various courses in different academic sessions. They want to analyze if there is any relation between the average grade of a course and the number of students attending it. They would also like to know if there were some courses offered but did not have any students registered for them. Relative performance among boys and girls and average grades of students from various States and cities of the country for each course must be analyzed and also overall CGPA.

    Design a star schema for such a data warehouse clearly identifying the fact table(s) and dimension table(s), their primary key(s) and foreign key(s). Your schema should at least be able to satisfy the above-mentioned analysis requirements. You may consider other suitable attributes for the dimension table(s).

    View this question on its own page →
    Worked Solution

    Star Schema Design — University Performance Data Warehouse

    Requirements Analysis

    We need to analyze: grade-vs-enrollment relation, courses with zero registrations, gender-wise performance, state/city-wise average grade per course, and overall CGPA. This means our grain (finest fact-table row) should be one student's performance in one course, in one academic session.

    Fact Table

    FACT_PERFORMANCE

    Column Type Key
    student_key FK → DIM_STUDENT PK (composite)
    course_key FK → DIM_COURSE PK (composite)
    session_key FK → DIM_SESSION PK (composite)
    grade_point Measure
    sgpa Measure
    is_registered Measure (flag, 1/0)

    Using a is_registered flag (or simply having a row exist) lets us detect courses offered but with zero enrolled students via a LEFT JOIN from DIM_COURSE × DIM_SESSION against the fact table.

    Dimension Tables

    DIM_STUDENT
    | student_key (PK) | student_id | name | gender | state | city | cgpa |

    DIM_COURSE
    | course_key (PK) | course_id | course_name | department | credits |

    DIM_SESSION
    | session_key (PK) | academic_year | semester_type (Odd/Even) |

    Schema Diagram

                      DIM_STUDENT
                     (student_key PK)
                      gender, state, city, cgpa
                            │
                            │FK
                            ▼
    DIM_COURSE ──FK──▶ FACT_PERFORMANCE ◀──FK── DIM_SESSION
    (course_key PK)     (student_key,        (session_key PK)
    course_name,         course_key,          academic_year,
    department           session_key,         semester_type
                          grade_point,
                          sgpa,
                          is_registered)
    

    How it satisfies the requirements

    • Grade vs enrollment relation → aggregate AVG(grade_point) and COUNT(student_key) grouped by course_key.
    • Courses with no studentsDIM_COURSE LEFT JOIN FACT_PERFORMANCE, filter WHERE fact IS NULL.
    • Gender-wise performance → join DIM_STUDENT.gender, group by course & gender.
    • State/city-wise average grade → join DIM_STUDENT.state/city, group by course.
    • Overall CGPA → stored directly in DIM_STUDENT.cgpa (or derived as AVG(sgpa) across sessions per student).

    This is a classic star schema: one fact table with numeric measures (grade_point, sgpa) surrounded by denormalized dimension tables, which keeps analytical queries fast (few joins, simple filters).

  17. Q4b. Write an SQL query that runs on the schema designed in Q4a and returns the average SGPA of boys from the State of Karnataka for each Autumn (odd) semester during the years 2005–2009.20214m

    Module 1: Data Warehousing and Business Analysis

    Write an SQL query that runs on the schema designed in Q4a and returns the average SGPA of boys from the State of Karnataka for each Autumn (odd) semester during the years 2005–2009.

    View this question on its own page →
    Worked Solution

    SQL Query — Avg SGPA of Boys from Karnataka, Odd Semesters (2005–2009)

    Using the star schema from Q4a (FACT_PERFORMANCE, DIM_STUDENT, DIM_SESSION):

    SELECT
        ds.academic_year,
        AVG(fp.sgpa) AS avg_sgpa
    FROM
        FACT_PERFORMANCE fp
        JOIN DIM_STUDENT  st ON fp.student_key  = st.student_key
        JOIN DIM_SESSION  ds ON fp.session_key  = ds.session_key
    WHERE
        st.gender = 'M'
        AND st.state = 'Karnataka'
        AND ds.semester_type = 'Odd'
        AND ds.academic_year BETWEEN 2005 AND 2009
    GROUP BY
        ds.academic_year
    ORDER BY
        ds.academic_year;
    

    How it maps to the requirement

    • st.gender = 'M' → restricts to boys.
    • st.state = 'Karnataka' → restricts to the given state.
    • ds.semester_type = 'Odd' → restricts to Autumn semesters only.
    • ds.academic_year BETWEEN 2005 AND 2009 → restricts the year range.
    • GROUP BY ds.academic_year → produces one average SGPA row per year, as required ("for each Autumn semester during 2005–2009").
    • The joins connect the fact table's measures (sgpa) to the descriptive attributes (gender, state, academic_year) stored in the dimension tables — this is exactly why a star schema keeps such analytical queries simple (only two joins, no deep nesting).
  18. Q5a. The results of an exam are recorded along with some data about the students. The results can be found in the table below : | ID | Cell No. | Language | Passed all Assignments | GPA | Passed Exam | | :---: | :---: | :---: | :---: | :---: | :---: | | 1 | 93333-11101 | Java | No | 3.1 | Yes | | 2 | 93333-11112 | Java | No | 2.0 | No | | 3 | 93333-11102 | C++ | Yes | 3.5 | Yes | | 4 | 93333-11113 | Python | Yes | 2.5 | Yes | | 5 | 93333-11103 | Java | Yes | 3.9 | No | | 6 | 93333-11114 | C++ | No | 2.9 | No | | 7 | 93333-11104 | Java | No | 1.9 | No | | 8 | 93333-11115 | Python | Yes | 3.2 | Yes | In no more than one page of text, describe the design of a K-Nearest Neighbour classifier to predict if a student will fail or pass the exam.202110m

    Module 3: Classification and Prediction

    The results of an exam are recorded along with some data about the students. The results can be found in the table below :

    ID Cell No. Language Passed all Assignments GPA Passed Exam
    1 93333-11101 Java No 3.1 Yes
    2 93333-11112 Java No 2.0 No
    3 93333-11102 C++ Yes 3.5 Yes
    4 93333-11113 Python Yes 2.5 Yes
    5 93333-11103 Java Yes 3.9 No
    6 93333-11114 C++ No 2.9 No
    7 93333-11104 Java No 1.9 No
    8 93333-11115 Python Yes 3.2 Yes

    In no more than one page of text, describe the design of a K-Nearest Neighbour classifier to predict if a student will fail or pass the exam.

    View this question on its own page →
  19. Q5b. Using the dataset from Q5a, use your K-NN classifier to predict whether the following student (who overslept and missed the original exam) will pass the re-exam : | ID | Cell No. | Language | Passed all Assignments | GPA | Passed Exam | | :---: | :---: | :---: | :---: | :---: | :---: | | 9 | 93333-11109 | C++ | Yes | 3.0 | ?? |20214m

    Module 3: Classification and Prediction

    Using the dataset from Q5a, use your K-NN classifier to predict whether the following student (who overslept and missed the original exam) will pass the re-exam :

    ID Cell No. Language Passed all Assignments GPA Passed Exam
    9 93333-11109 C++ Yes 3.0 ??
    View this question on its own page →
  20. Q6. Define any four of the following briefly : (a) Occam's Razor (b) ROC Curve (c) Vector Space Model (d) Data Marts (e) Multilevel Association Rules (f) OLAM202114m

    Module 2: Data Mining and Association Rule Mining

    Define any four of the following briefly :

    (a) Occam's Razor
    (b) ROC Curve
    (c) Vector Space Model
    (d) Data Marts
    (e) Multilevel Association Rules
    (f) OLAM

    View this question on its own page →
    Worked Solution

    Six Key Terms Defined (pick any four for the exam)

    (a) Occam's Razor

    A principle stating that among competing hypotheses that fit the data equally well, the simplest one should be preferred. In data mining/ML, this motivates choosing simpler models (fewer parameters, shallower decision trees) over needlessly complex ones, since simpler models generalize better and are less prone to overfitting.

    (b) ROC Curve

    Receiver Operating Characteristic curve — plots the True Positive Rate (Sensitivity) against the False Positive Rate (1−Specificity) at various classification thresholds.

    TPR
     1 │        ●───●
       │      ●
       │    ●
       │  ●
       │●
     0 └──────────────── FPR
       0              1
    

    The Area Under the Curve (AUC) summarizes overall classifier performance — AUC=1 is a perfect classifier, AUC=0.5 is random guessing (the diagonal line).

    (c) Vector Space Model (VSM)

    An algebraic model used mainly in text/document mining and information retrieval, where each document (or object) is represented as a vector of feature weights (e.g., TF-IDF term weights) in a high-dimensional space. Similarity between documents is then computed geometrically, typically using cosine similarity between their vectors.

    (d) Data Marts

    A subset of a data warehouse, scoped to a specific business line, department, or subject area (e.g., a Sales data mart, an HR data mart). Data marts are smaller, faster to query, and easier to maintain than the full enterprise warehouse, letting individual teams get focused analytical access without querying the entire warehouse.

    (e) Multilevel Association Rules

    Association rules mined at different levels of a concept hierarchy rather than only at the raw item level. For example, instead of just "{Bread} ⇒ {Butter}", multilevel mining can surface rules at higher abstraction levels like "{Bakery Items} ⇒ {Dairy Items}", or at lower levels like "{Whole Wheat Bread} ⇒ {Salted Butter}". This reveals patterns that might be too sparse (low support) at the finest level but become significant when items are generalized.

    (f) OLAM (On-Line Analytical Mining)

    Integrates OLAP with data mining, allowing mining algorithms to be applied directly on multidimensional data cubes rather than flat/raw data. This lets analysts interactively drill/roll/slice/dice to narrow down interesting regions of the cube, then apply mining functions (classification, clustering, association) at whatever granularity they've navigated to — combining OLAP's exploratory interactivity with mining's automatic pattern discovery.

  21. Q7a. A database has five transactions. Let min_sup = 60% and min_conf = 80%. | TID | Items_bought | |---|---| | T100 | {M, O, N, K, E, Y} | | T200 | {D, O, N, K, E, Y} | | T300 | {M, A, K, E} | | T400 | {M, U, C, K, Y} | | T500 | {C, O, O, K, I, E} | (i) Find all frequent item sets using a priori algorithm. (ii) List all the strong association rules (with support s and confidence c) matching the following metarule, where X is a variable representing customers: \forall x \in \text{transactions}, \text{buys}(X, \text{item}_1) \wedge \text{buys}(X, \text{item}_2) \Rightarrow \text{buys}(X, \text{item}_3)\ [s, c]20218m

    Module 2: Data Mining and Association Rule Mining

    A database has five transactions. Let min_sup = 60% and min_conf = 80%.

    TID Items_bought
    T100 {M, O, N, K, E, Y}
    T200 {D, O, N, K, E, Y}
    T300 {M, A, K, E}
    T400 {M, U, C, K, Y}
    T500 {C, O, O, K, I, E}

    (i) Find all frequent item sets using a priori algorithm.

    (ii) List all the strong association rules (with support ss and confidence cc) matching the following metarule, where XX is a variable representing customers:
    xtransactions,buys(X,item1)buys(X,item2)buys(X,item3) [s,c]\forall x \in \text{transactions}, \text{buys}(X, \text{item}_1) \wedge \text{buys}(X, \text{item}_2) \Rightarrow \text{buys}(X, \text{item}_3)\ [s, c]

    View this question on its own page →
    Worked Solution

    (i) Frequent Itemsets via Apriori — MONKEY Dataset

    Same dataset as before, min_sup = 60% (count ≥ 3), min_conf = 80%.

    TID Items
    T100 M, O, N, K, E, Y
    T200 D, O, N, K, E, Y
    T300 M, A, K, E
    T400 M, U, C, K, Y
    T500 C, O, K, I, E

    L1 (count ≥3)

    {M}:3, {O}:3, {K}:5, {E}:4, {Y}:3

    L2 (count ≥3, from valid C2 candidates)

    {K,M}:3, {K,O}:3, {O,E}:3, {K,E}:4, {K,Y}:3

    L3

    Only candidate surviving the apriori-prune test: {K,O,E} — check: {K,O}✅ {K,E}✅ {O,E}✅ all in L2 → count in DB (T100, T200, T500) = 3 → frequent.

    All frequent itemsets:

    {M} {O} {K} {E} {Y}
    {K,M} {K,O} {O,E} {K,E} {K,Y}
    {K,O,E}
    

    (11 itemsets total — see full step-by-step derivation in Q65's solution for the same dataset.)


    (ii) Strong Rules Matching the Metarule

    Metarule: x,buys(X,item1)buys(X,item2)buys(X,item3) [s,c]\forall x, \text{buys}(X,\text{item}_1) \wedge \text{buys}(X,\text{item}_2) \Rightarrow \text{buys}(X,\text{item}_3)\ [s,c]

    This asks specifically for rules with exactly 2 items in the antecedent and 1 item in the consequent. The only frequent itemset large enough to produce such a rule is the 3-itemset {K, O, E} (support count = 3, support = 3/5 = 60%).

    We test all three possible 2→1 splits of {K, O, E}:

    Rule Confidence = support({K,O,E}) / support(antecedent) Strong? (≥80%)
    {K,O} ⇒ E 3/3 = 100%
    {K,E} ⇒ O 3/4 = 75%
    {O,E} ⇒ K 3/3 = 100%

    Strong Association Rules (matching the metarule)

    buys(X,K)buys(X,O)buys(X,E)[s=60%, c=100%]\text{buys}(X, K) \wedge \text{buys}(X, O) \Rightarrow \text{buys}(X, E) \quad [s = 60\%,\ c = 100\%]

    buys(X,O)buys(X,E)buys(X,K)[s=60%, c=100%]\text{buys}(X, O) \wedge \text{buys}(X, E) \Rightarrow \text{buys}(X, K) \quad [s = 60\%,\ c = 100\%]

    The rule {K,E} ⇒ O fails the min_conf = 80% threshold (only 75%), so it is not a strong rule and is excluded — even though {K,O,E} itself is a frequent (frequent) itemset, not every rule derived from it clears the confidence bar.

  22. Q7b. Explain DBSCAN briefly.20216m

    Module 4: Cluster Analysis

    Explain DBSCAN briefly.

    View this question on its own page →
  23. Q8a. Why do we use ensemble methods? Describe an ensemble method.20217m

    Module 3: Classification and Prediction

    Why do we use ensemble methods? Describe an ensemble method.

    View this question on its own page →
  24. Q8b. Differentiate among OLAP, MOLAP and HOLAP.20217m

    Module 1: Data Warehousing and Business Analysis

    Differentiate among OLAP, MOLAP and HOLAP.

    View this question on its own page →
    Worked Solution

    OLAP vs MOLAP vs HOLAP

    These are three server architectures for implementing OLAP, differing in where and how the multidimensional data is physically stored.

    ROLAP (Relational OLAP)

    • Stores data in standard relational tables (star/snowflake schema).
    • Multidimensional operations are translated into SQL at query time.
    • Pros: scales to very large data volumes, reuses existing RDBMS technology.
    • Cons: slower query performance (SQL translation overhead), limited by SQL's multidimensional expressiveness.

    MOLAP (Multidimensional OLAP)

    • Stores data in a proprietary multidimensional array/cube structure (pre-computed).
    • Pros: very fast query response (data pre-aggregated), rich analytical functions.
    • Cons: cube-build time can be long, storage can explode for high-cardinality/sparse dimensions ("data explosion"), limited scalability for very large datasets.

    HOLAP (Hybrid OLAP)

    • Combines both: detailed data stays in relational tables (ROLAP), while summary/aggregated data is stored in MOLAP cubes.
    • Pros: balances MOLAP's speed for aggregates with ROLAP's scalability for detailed data.
    • Cons: more complex architecture to build and maintain.

    Comparison Table

    Aspect ROLAP MOLAP HOLAP
    Storage Relational tables Multidimensional array/cube Mixed (relational + cube)
    Query speed Slower (SQL translation) Fastest (pre-computed) Balanced
    Scalability Very high (large data) Limited (cube size) High
    Storage overhead Low High (sparse data explosion) Moderate
    Best for Very large, detailed datasets Fast, smaller/aggregated cubes Mix of both needs
    ROLAP:  [Relational DB] ──SQL──▶ query engine ──▶ result
    MOLAP:  [Precomputed Cube] ──direct lookup──▶ result   (fast!)
    HOLAP:  [Detail: Relational] + [Summary: Cube] ──▶ result (best of both)
    

    In short: ROLAP trades speed for scalability, MOLAP trades scalability for speed, and HOLAP tries to get both by splitting detail vs summary data across the two storage models.

  25. Q9a. Describe classification accuracy. How do we measure it? Differentiate classification accuracy with precision.20217m

    Module 3: Classification and Prediction

    Describe classification accuracy. How do we measure it? Differentiate classification accuracy with precision.

    View this question on its own page →
  26. Q9b. Distinguish between noise and outliers. (i) Is noise ever interesting or desirable? Outliers? (ii) Can noise objects be outliers? (iii) Can noise make a typical value into an unusual one or vice versa?20217m

    Module 4: Cluster Analysis

    Distinguish between noise and outliers.

    (i) Is noise ever interesting or desirable? Outliers?
    (ii) Can noise objects be outliers?
    (iii) Can noise make a typical value into an unusual one or vice versa?

    View this question on its own page →