Back to the 2025 paper

Unit 4: Advanced Topics & Clustering

20257m

Why is clustering important in machine learning? Show the steps of the k-means algorithm by taking a suitable example. Give the time complexity analysis of k-means.

Worked SolutionAI Assisted

Why Clustering Important:
Unsupervised, discovers hidden structure without labels. Uses: customer segmentation, gene grouping, image segmentation, anomaly detection, document clustering, feature engineering, data compression.

K-means Algorithm Steps:

Input: Data X={xi}i=1nX=\{x_i\}_{i=1}^n, kk, distance (Euclidean)
Goal: minimize J=∑j=1k∑xi∈Cj∣∣xi−μj∣∣2J=\sum_{j=1}^k\sum_{x_i\in C_j}||x_i-\mu_j||^2

  1. Initialize: Choose kk centroids μj\mu_j randomly (k-means++ better).
  2. Assignment: Assign each xix_i to nearest centroid: Cj={xi:∣∣xi−μj∣∣≤∣∣xi−μl∣∣∀l}C_j=\{x_i: ||x_i-\mu_j|| \le ||x_i-\mu_l|| \forall l\}
  3. Update: Recompute μj=1∣Cj∣∑xi∈Cjxi\mu_j = \frac1{|C_j|}\sum_{x_i\in C_j} x_i
  4. Repeat 2-3 until centroids stop moving or JJ change < threshold (convergence to local optimum).

Suitable Example:
Data points: A(1,1), B(1.5,2), C(3,4), D(5,7), D(3.5,5), E(4.5,5), k=2k=2
Init μ1=(1,1)\mu1=(1,1), μ2=(5,7)\mu2=(5,7)
Iter1 assignments: cluster1 {A,B}, cluster2 {C,D}... recompute etc. After few iterations converges to two clusters.

Time Complexity Analysis:
Per iteration: O(n⋅k⋅d)O(n \cdot k \cdot d) distance computations (nn points, kk centroids, dd dims). For ii iterations: O(nkdi)O(n k d i). Space O((n+k)d)O((n+k)d). If k,dk,d constants, O(ni)O(n i) linear in nn. Compared to hierarchical O(n2)O(n^2). Elkan/Hamerly accelerations reduce but worst still O(nkdi)O(n k d i). Typically fast, converges quickly (10-100 iterations). Mention Lloyd's algorithm.

Limitations: need pre-specify kk (elbow/silhouette), sensitive to init & outliers, assumes spherical clusters, Euclidean biased.

Similar questions

AI LearningWeb Platform

Coursify — Personal AI Course Generator

Struggling with a difficult subject or concept? Turn any engineering topic into a structured course with interactive lessons, quizzes, and research-backed explanations.