Machine Learning
105602Module 6: Advanced Topics
1(j). In the E-step of EM, we primarily (i) Update model parameters (ii) Remove missing data (iii) Estimate the latent-variable responsibilities (iv) Calculate classification accuracy20262m
Module 6: Advanced Topics
View this question on its own page →In the E-step of EM, we primarily (i) Update model parameters (ii) Remove missing data (iii) Estimate the latent-variable responsibilities (iv) Calculate classification accuracy
Worked SolutionAnswer: (iii) Estimate the latent-variable responsibilities
In the Expectation step, EM computes the expected latent-variable assignments/responsibilities using current parameters. The Maximization step then updates parameters to maximize the expected complete-data log-likelihood.
2(b). Explain the basic concepts of Reinforcement Learning. Also explain the exploration-exploitation trade-off and the role of the ε-greedy strategy.20267m
Module 6: Advanced Topics
View this question on its own page →Explain the basic concepts of Reinforcement Learning. Also explain the exploration-exploitation trade-off and the role of the ε-greedy strategy.
Worked SolutionReinforcement Learning and ε-greedy
Reinforcement Learning (RL) trains an agent to choose actions in an environment. The environment returns a reward and next state. The goal is to maximize cumulative discounted reward. Key terms: state (s), action (a), reward (r), policy (\pi), value function and learning rate.
Exploration tries actions to learn their outcomes; exploitation chooses the currently best-known action. Too much exploration wastes reward; too little may miss better actions.
With ε-greedy, choose a random action with probability (\epsilon), and the best estimated action with probability (1-\epsilon). ε may decay as learning progresses.
MermaidDiagramRendering diagram…9(a). Write a short note on Q-Learning.20267m
Module 6: Advanced Topics
View this question on its own page →Write a short note on Q-Learning.
Worked SolutionQ-Learning
Q-learning is a model-free, off-policy reinforcement-learning algorithm that learns the action-value function (Q(s,a)): expected discounted return for taking action (a) in state (s).
[Q(s,a)\leftarrow Q(s,a)+\alpha[r+\gamma\max_{a'}Q(s',a')-Q(s,a)]]
Here (\alpha) is learning rate, (\gamma) is discount factor, (r) is immediate reward, and (s') is the next state. An ε-greedy policy balances exploration and exploitation. With sufficient exploration and suitable learning rates, tabular Q-learning converges under standard finite Markov decision process assumptions.
9(b). Write a short note on Bayesian Networks.20267m
Module 6: Advanced Topics
View this question on its own page →Write a short note on Bayesian Networks.
Worked SolutionBayesian Networks
A Bayesian Network is a directed acyclic graph (DAG) whose nodes represent random variables and whose edges encode conditional dependencies. Each node has a conditional probability table/distribution given its parents.
The joint distribution factorizes as:
[P(X_1,\ldots,X_n)=\prod_i P(X_i\mid Parents(X_i))]
For example,
Rain → WetGrass ← Sprinklerrepresents two possible causes of wet grass. Bayesian networks support probabilistic inference, diagnosis and reasoning under uncertainty. The graph structure encodes conditional independence assumptions; incorrect structure or probabilities can lead to poor inferences.Q6(b). Explain Q learning algorithm assuming deterministic rewards and actions.20237m
Module 6: Advanced Topics
View this question on its own page →Explain Q learning algorithm assuming deterministic rewards and actions.