Types of Data in Cluster Analysis

Types of Data in Cluster Analysis

Clustering algorithms depend on how similarity between objects is measured, which in turn depends on the type of data. This topic defines proximity measures for every data type, plus the key issue of standardising heterogeneous attributes.

1. Interval-Scaled (Numeric) Variables

Continuous measurements ($x_{if}$ = value of object $i$ on attribute $f$).

Step 1 — Normalise to remove scale effects (so currency/km/age are comparable) using the z-score:

Step 2 — Distance measures between objects $i$ and $j$:

Euclidean distance ($L_2$):

Manhattan / city-block distance ($L_1$): $d(i,j)=\sum_{f=1}^{p}|x_{if}-x_{jf}|$

Minkowski distance (general $L_q$): $d(i,j)=\left(\sum_{f=1}^{p}|x_{if}-x_{jf}|^{q}\right)^{1/q}$

Chebyshev / supremum ($L_\infty$): $d(i,j)=\max_f |x_{if}-x_{jf}|$

Weighted Euclidean (attribute weighting): $d(i,j)=\sqrt{\sum_f w_f (x_{if}-x_{jf})^2}$

Note: $L_\infty$ is the limit of Minkowski as $q \to \infty$; larger $q$ gives more weight to the largest difference.

Numerical example — $A=(1,2)$, $B=(4,6)$:

$L_q$	1	2	3	$\infty$
distance	7	5	4.497	4

Similarity from distance: $s(i,j)=\dfrac{1}{1+d(i,j)}$, or use cosine similarity

Cosine is preferred for text/document vectors because it ignores magnitude.

2. Binary Variables

Symmetric binary (both outcomes equally important, e.g. gender):

Asymmetric binary (one outcome rarer/more meaningful — "1" usually means presence):

where $q$ = both 1, $r$ = $i$=1 & $j$=0, $s$ = $i$=0 & $j$=1, $t$ = both 0.

Jaccard similarity $=\dfrac{q}{q+r+s}$; better than overall similarity $\dfrac{q+t}{q+r+s+t}$ when 0s dominate (sparse data).

Numerical example (name, gender, fever, cough, test-1..4 from the classic table; asymmetric attributes: fever, cough, t1-t4):

Pair	q	r	s	Symmetric dist	Jaccard dist
Jack–Mary	2	1	2	(1+2)/6 = 0.50	3/5 = 0.60
Jack–Jim	1	2	1	3/6 = 0.50	3/4 = 0.75
Jim–Mary	1	1	2	2/6 = 0.33	2/4 = 0.50

3. Nominal (Categorical) Variables

Values from an unordered set with $M$ states. Simple matching:

Alternative: convert to $M$ asymmetric binary attributes (one-hot) and use Jaccard.

Example: (red, medium, A) vs (blue, medium, B) → $p=3$, $m=1$ → $d = 2/3 = 0.67$.

4. Ordinal Variables

An ordinal value can be ranked but not metrically compared (e.g. small < medium < large, ranks 1..$M$).

1. Replace each value by its rank $r_{if} \in \{1..M_f\}$.
2. Map the rank to [0,1]:

3. Use any interval-scaled distance (e.g. Euclidean) on the transformed values.

Example: {fair, good, excellent} mapped to $\{0, 0.5, 1\}$ → good vs excellent: $z=0.5$ vs $1$.

5. Ratio-Scaled Variables

Positive measurements on a non-linear scale (exponential growth/decay: bacterial population, radioactive decay). Apply a log transform first, then treat as interval-scaled:

6. Mixed Types of Variables

One distance that handles all types together. For attributes $f = 1 \dots p$, with indicator $\delta_{ij}^{(f)}=0$ if $x_{if}$ or $x_{jf}$ is missing or when $f$ is asymmetric binary with both values 0, otherwise 1:

Per-attribute contribution:

Type	$d_{ij}^{(f)}$
Interval-scaled	$\dfrac{|x_{if}-x_{jf}|}{\max_h x_{hf}-\min_h x_{hf}}$
Nominal	0 if equal, 1 otherwise
Ordinal	$\dfrac{|r_{if}-r_{jf}|}{M_f-1}$
Ratio-scaled	distance on $\log$ values, range-normalised
Binary	0/1 (Jaccard style)

Worked mixed example (test-1: nominal; test-2: ordinal with $M=3$; test-3: ratio $1.5\times10^3$ etc.)

Object	test-1	test-2	test-3
A	code-A	excellent	1.5e3
B	code-B	fair	2.5e4
C	code-C	good	1.3e5

• test-1 (nominal) contributes 1 for every pair (all different).
• test-2 (ordinal, $M=3$): excellent = 1, fair = 0, good = 0.5 → $d_{AB}=1$, $d_{AC}=0.5$, $d_{BC}=0.5$.
• test-3 (ratio, log$_{10}$ = 3.18, 4.40, 5.11; range 1.93): $d_{AB}=0.63$, $d_{AC}=1.0$, $d_{BC}=0.37$.

7. Why the Data Type Matters

Data type	Typical measure	Watch out for
Interval	Euclidean / Manhattan / cosine	Scale differences → normalise
Binary asymmetric	Jaccard	Dominance of 0s
Nominal	Simple matching / one-hot + Jaccard	High dimensionality after one-hot
Ordinal	Rank-normalised Euclidean	Assume monotone spacing
Ratio	Log-transform + interval	Zero/negative values
Mixed	Weighted general distance with $\delta$	Missing values, attribute weighting

8. Exam Points

• Explain proximity measures for interval, binary, nominal, ordinal and ratio variables — 10 marks (very frequent).
• Compute Euclidean, Manhattan, Minkowski and supremum distances for two points — 3-5 marks numerical.
• Compute Jaccard/simple matching distance from a table of binary attributes — 5 marks numerical.
• Explain the mixed-type distance formula with $\delta_{ij}^{(f)}$ — 5-7 marks.
• Why is normalisation needed before clustering? — 3 marks.