Text Mining
Text mining (text data mining, knowledge discovery from text) extracts useful, previously unknown patterns from large collections of unstructured or semi-structured text using techniques from information retrieval (IR), natural language processing (NLP), statistics and data mining.
Text is unstructured, so the first job is always to convert it into a structured representation (a term-document matrix). Then ordinary mining algorithms can run. Remember the pipeline and the TF-IDF formula; both are asked very often.
1. Text Mining vs IR vs Database Search
| Aspect | Information retrieval | Text mining | Database search |
|---|---|---|---|
| Goal | Find documents that match a query | Find patterns, rules, trends across documents | Exact match on structured fields |
| Output | Ranked list of documents | Model, cluster, topic, summary | Tuples satisfying a condition |
| Data | Keyword index | Bag of words plus semantics | Structured attributes |
| Query | Keywords, Boolean or vector | Mining task | SQL |
IR finds the needles. Text mining tells you what kinds of needles exist and how they relate.
2. Pre-processing Pipeline
3. Vector-Space Model and TF-IDF
Each document becomes a vector of term weights. Row is document and column is term .
| 0.5 | 0.0 | 0.2 | 0.9 | |
| 0.0 | 0.7 | 0.1 | 0.0 | |
| 0.3 | 0.6 | 0.0 | 0.4 |
TF-IDF weighting
Here is the number of documents and is the number of documents that contain term . A term that is frequent in one document but rare in the corpus gets a high weight; a term that appears everywhere gets a weight near zero.
Cosine similarity between two documents:
4. NLP Support for Text Mining
| Technique | What it adds |
|---|---|
| POS tagging | Word class (noun, verb, adjective) |
| Named-entity recognition (NER) | Persons, places, organisations, dates, amounts |
| Parsing | Phrase and dependency structure, subject-verb-object triples |
| Word-sense disambiguation | Picks the right meaning of "bank" or "cell" |
| Coreference resolution | Links he, it, the bank to the entity they mean |
| Ontology / WordNet | Generalizes terms through a concept hierarchy |
5. Text Mining Tasks
| Task | Purpose | Techniques |
|---|---|---|
| Summarization | Short abstract of one or many documents | Sentence extraction, maximal marginal relevance, abstractive models |
| Categorization (classification) | Assign a known topic (spam, sports) | Naive Bayes, SVM, k-NN, neural nets |
| Clustering | Group similar documents without labels | k-means, hierarchical, LDA topic models |
| Topic detection and tracking | Follow evolving topics in news | Incremental clustering over time |
| Information extraction | Fill structured records from text | Regular expressions, HMM, CRF, relation extraction |
| Association / concept linking | Term co-occurrence rules | Apriori, FP-growth on term sets |
| Sentiment / opinion mining | Positive or negative polarity | Lexicons, ML classifiers |
| Question answering | Answer natural-language questions | IR + NLP + inference |
Categorization is supervised (labels are given). Clustering is unsupervised (groups are discovered).
6. Text Mining Lifecycle
7. Applications and Challenges
| Applications | Challenges |
|---|---|
| E-mail spam filtering | Ambiguity: polysemy and synonymy |
| News personalization and topic tracking | Huge, sparse term-document matrices |
| Patent and scientific literature analysis | Noise: typos, slang, emojis, OCR errors |
| Support-ticket triage | Negation, sarcasm and irony beyond bag-of-words |
| Sentiment analysis of reviews | Subjective evaluation of summaries and patterns |
| Resume screening, legal e-discovery | Privacy, copyright and ethics |
- Define text mining; explain tasks and techniques with an architecture diagram (7-10 marks, very frequent).
- Explain the pre-processing pipeline: tokenization, stop words, stemming, TF-IDF (7-10 marks).
- Compare text mining with IR and traditional data mining (5 marks).
- Text summarization, categorization, clustering, information extraction (5 marks each).
- Applications and challenges (5 marks).
Frequently Asked Questions
QWhat is text mining?
It is the process of extracting useful, previously unknown patterns and knowledge from large collections of unstructured or semi-structured text using IR, NLP, statistics and data mining. The text is first converted into a structured term-document representation and then mined.
QHow is text mining different from information retrieval?
IR returns a ranked list of documents that match a query. Text mining finds patterns, rules, trends and models across the whole collection (clusters, topics, summaries, extracted relations).
QHow is text mining different from traditional data mining?
Traditional mining works on clean structured tables. Text mining works on unstructured text, so it needs extra pre-processing (tokenization, stop-word removal, stemming, POS tagging, NER, TF-IDF) and faces much higher dimensionality and sparsity.
QExplain the text pre-processing steps.
Tokenize, remove stop words, stem or lemmatize, normalize, tag parts of speech and entities, select features with TF-IDF or chi-square, and build the term-document matrix.
QWhat is TF-IDF? Write the formula.
TF-IDF gives a high weight to a term that is frequent in one document but rare in the corpus: with and .
QWhat is the vector-space model and cosine similarity?
Each document is a vector of term weights. The similarity of two documents is the cosine of the angle between their vectors: . A value of 1 means very similar and 0 means no common terms.
QDifferentiate text categorization and text clustering.
Categorization is supervised: documents are assigned to predefined classes (Naive Bayes, SVM, k-NN). Clustering is unsupervised: documents are grouped by similarity without labels (k-means, hierarchical, LDA).
QWhat are the types of text summarization?
Extractive summaries pick the most important existing sentences. Abstractive summaries generate new sentences that paraphrase the content. Multi-document summaries combine several documents on one topic.
QWhat is information extraction?
Filling structured records (price, date, gene-protein relation) from unstructured text using regular expressions, HMMs, CRFs and relation extraction on NER-tagged text.
QState some applications and challenges of text mining.
Applications include spam filtering, news topic tracking, literature analysis, ticket triage and sentiment analysis. Challenges include ambiguity, sparse high-dimensional data, noisy text, sarcasm and negation, subjective evaluation, and privacy.