Functionalities of Data Mining and the Role of Preprocessing

Data mining is the process of discovering interesting patterns and knowledge from large amounts of data. Its functionalities specify the kinds of patterns that can be found.

Major functionalities

Functionality	Description	Example
Concept/class description (characterization and discrimination)	Summarizes the general features of a class, or compares classes	Features of customers who buy a lot vs those who buy little
Association and correlation analysis	Finds items that occur together (frequent patterns)	Bread and butter bought together (market basket)
Classification	Builds a model from labelled data to predict a class label (supervised)	Classify loan applicants as safe or risky
Prediction / regression	Predicts a continuous value	Forecast next month's sales
Clustering	Groups similar objects with no predefined labels (unsupervised)	Customer segmentation
Outlier analysis	Finds objects that do not fit the general behaviour	Credit card fraud detection
Evolution / trend analysis	Studies how objects change over time	Stock price trends

How preprocessing improves quality and reliability

Real-world data is often incomplete, noisy and inconsistent; mining such data gives unreliable results ("garbage in, garbage out"). Preprocessing improves the data on these quality measures:

Quality factor	How preprocessing helps
Accuracy	Cleaning removes noise and errors
Completeness	Missing values are filled or handled
Consistency	Integration and standardization remove conflicts and duplicates
Believability	Verified, cleaned data is trusted more
Interpretability	Discretization and concept hierarchies make patterns easier to understand
Efficiency	Reduction cuts data size and mining time

Effects on the discovered knowledge

• Cleaning prevents wrong patterns caused by errors and outliers.
• Integration gives a single, consistent view across sources.
• Transformation (normalization) stops large-scale attributes from dominating distance-based algorithms.
• Reduction removes irrelevant attributes, so models are simpler and less prone to overfitting.
• Discretization makes rules concise and suitable for algorithms like decision trees and Apriori.

Conclusion

Good preprocessing improves the accuracy, stability and trustworthiness of classification, clustering and association results, and often takes most of the effort in a data mining project.