Popular Machine Learning Interview Questions

Tags: ,

We list some popular Questions related to Machine Learning. You should prepare them if you are looking for jobs related to Machine Learning Engineers, Data Scientist or Research Scientist related to Machine Learning. 

I put the questions into three categories: Machine Learning Theories, Machine Learning Algorithms and Machine Learning Tools. 

Machine Learning Theories

When we talk about machine learning theories, we often refer to machine learning models such as Support Vector Machines, Decision Trees, Logistic Regression, Topic Models, Bayesian Networks and Deep Learning. 

Here are some books that must be read:

Supervised learning vs unsupervised learning

These models can be put into two high level categories called supervised learning and unsupervised learning. 

Some typical question are:

What is supervised learning and unsupervised learning? 

What kind of problems can be solved by supervised learning or unsupervised learning?

Give some examples for supervised learning and unsupervised learning.

Please be noted that they are often the introductory questions before some more specific and harder questions. 

Generative and Discriminative Models

The models in Machine Learning can also be classified as generative and discriminative models. So you need to prepare the questions:

What is the difference between a Generative and Discriminative Model?

A simple answer is:

To answer this question perfectly, it is necessary to read some articles and try to come up with some examples to fully understand the difference. Here are some resources:

1. On Discriminative vs. Generative classifiers: A comparison of logistic regression and naive bayes.  

2. What are the differences between generative and discriminative machine learning

3. Machine Learning: Generative and Discriminative Models

4. Pattern Recognition and Machine Learning

5. Machine Learning: A Probabilistic Perspective

6. An Introduction to Statistical Learning: with Applications in R


If you can answer the above questions correctly, now it is time to come to questions related to some specific models:

  1. What is XXX Model? ( XXX can be Support Vector Machines, Logistic Regression, Decision Trees, Random Forest, Beyesian Networks, Topic Models, Deep Learning, K-Means, K-NN, Collaborative Filtering,  Markov networks, Hidden Markov Model, e.g.)  I will write more articles to describe each of these models. 
  2. Given a specific model, talk about its assumptions, what type problems it tries to solve, why it does preform well in this problem while performs badly for another? 
  3. Is the model prone to over-fitting? If so – how do you overcome this?
  4. How do we examine the data to test whether whether a assumption is satisfied for a model? For example, to use linear regression, the data should be norm distributed. So you need to know QQ plot, and  Residual Plot analysis. 
  5.  Does the model has a random component or will the same training data always generate the same model? How do we deal with random effects in training
  6. What types of data (numerical, categorical etc…) can the model handle?
  7. How do you handle missing data? 
  8. How interpretable is the model? 
  9. What alternative models might we use for the same type of problem that this one attempts to solve, and how does it compare to those?
  10. How fast is prediction compared to other models? How fast is training compared to other models?
  11. Does the model have any parameters and thus require tuning? How do we do parameter tuning?

Machine Learning Algorithms

Some people may think that machine learning algorithms are equivalent to machine learning theories. In this post, I use the term machine learning algorithms to refer to the process to train a model using the input data, and use the model to make predictions on new data.

Gradient Decent Algorithm

The most famous algorithm is Gradient Decent Algorithm. The paradigm of training models is usually first define a loss function of some parameters, then we use the input data to optimize (minimize) the loss function. The common method to solve a minimization problem is to use Gradient Decent Algorithm to search a local optimized solution.  

To use Gradient Decent, you have to run through ALL the samples in your training set to do a single update for a parameter in a particular iteration. This is too inefficient if the dataset is large.  A faster method called Stochastic Gradient Decent (SGD) is used to update the parameters using one or small sample of the datasets for each iteration. 

See this article for the difference between GD and SGD.

SGD is an important algorithm as it is used to train a wide range of machine models such linear regression, logistic regression, and deep learning. 

  1. Logistic Regression and Gradient Descent
  2. Optimization: Stochastic Gradient Descent
  3. Stochastic gradient methods for machine learning

Expectation–maximization (EM) algorithm

EM algorithm is  popular method that should be understood if you want to apply for an machine learning related job. It is often used to solve many important problems such as K-Means, Latent variable related models such as Hidden Markov Models and Topic Models. Here are some resources:

  1. A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models
  2. ExpectationMaximization Algorithm and Applications 
  3. A Note on the Expectation-Maximization (EM) Algorithm 

Gibbs Sampling

Gibbs sampling algorithm is often used to train a topic model such a LDA. Read the following article.


Algorithms to train a Decision Tree

Here some import questions:

  1. How do you split the node
  2. How to pruning the tree

Algorithms for Association Rule Mining

Association Rule Mining aims to identify a patten such as customer who buy X, and Y tend to buy Z.  {X, Y} => Z, from the dataset. 

Bagging and Boosting

This is used to train Random Forest

Graph Related Algorithms

  1. PageRank
  2. HITS Algorithm


General Algorithms

There are other algorithm that should be familiar with. They are actually general algorithms in Computer Science.

  1. Random Sampling
  2. Reservoir Sampling
  3. Weighed Sampling
  4. BitMap
  5. Bloom Filtering
  6. Trie
  7. AC Algorithm
  8. Deep First Search
  9. Bread First Search
  10. Shortest Path



Machine Learning Tools

  1. Do you have any research experience in machine learning or a related field?
  2. What tools and environments have you used to train and assess models?
  3. Do you have experience with Spark ML or another platform for building machine learning models using very large datasets?

These books are very helpful to master machine learning skills:

  • Abhiram Sharma

    Hello There,

    Smokin hot stuff! You’ve trimmed my dim. I feel as bright and fresh as your
    prolific website and blogs!

    I’m new to the world of programming and I am really
    under pressure to learn

    python at the moment for a
    mature student college course. So im looking for help on how to get python
    pandas installed on ubuntu 16.04, specifically pandas datareader. I might need
    more help later too, to get my head around this. I have no programming
    experience and very little linux experience.

    I’m doing the basic introduction on python and stuck on the second part.

    import pandas_datareader.data as web

    Thank you very much and will look for more postings from you.

    Thanks a heaps,