Reading time - 7 mins
🥇 Picks of the Week
One-liner tool for ML models, Data Drift detection method, Curated Learning Resources and more.
🧠 ML Section
Learn ML Regression Metrics Pros and Cons and how to choose the best one for your case.
Land your next ML job. Fast.
I built the ML Job Landing Kit to help Data Professionals land jobs faster!
✅ Here’s what’s inside:
- 100+ ML Interview Q & A
- 25-page CV Crafting Guide for ML
- 10-page LinkedIn Guide for DS/ML
- 2 Ready-to-use CV Templates
- $40 discount for a TOP Interview Prep Platform
This is the exact system I used to help 100+ clients land interviews and secure ML roles.
🔥 Grab your copy & land your ML job faster!
1. ML Picks of the Week
⏱ All under 60 seconds. Let’s go 👇
🥇 ML Tool
→ How to quickly test many models in one line of code?
Pycaret lets you build, compare, and deploy ML models with minimal code - perfect for rapid prototyping and getting quick baselines
📈 ML Concept
How to Detect Data Drift with Jensen-Shannon Method
🤔 ML Interview Question:
→ What are the 3 most common optimizers to train Neural Networks?
Short answer: They are Gradient Descent, RMSProp, and Adam
Learn from a clear explanation here
📚 ML Resource of the Week
17 Resources That Will Make You a Better Data Scientist
🧠 Weekly Quiz:
→ What is the role of the Jacobian matrix in backpropagation for neural networks?
A) It contains partial derivatives of layer outputs with respect to inputs, used in the chain rule
B) It stores the weights of the network
C) It tracks the learning rate at each layer
D) It regularizes the loss function to prevent overfitting
✅ See the correct answer here
How to choose the right ML model metric?
Choosing the wrong evaluation metric in regression problems is hard.
You need to take into account the Pros and Cons of each metric AND the business contents.
The truth is simple: there's no universal "best" metric.
The right choice depends on your problem, your data characteristics, and what decisions your model will actually drive.
In this issue, you'll get a practical framework for choosing between the 4 most common regression metrics: MAPE, R², RMSE, and MAE.
What Makes a Good Evaluation Metric?
Before diving into specific metrics, understand what makes any metric useful:
- Aligns with business costs - Reflects real-world impact of prediction errors
- Interpretable by stakeholders - Can be explained to non-technical decision makers
- Stable with your data - Doesn't break with outliers or edge cases
- Sensitive to improvements - Shows meaningful changes when models get better
Keep these principles in mind as we explore each metric.
1️⃣ MAPE (Mean Absolute Percentage Error)
The MAPE formula is as follows:
MAPE expresses errors as percentages of actual values, making it intuitive for business communication.
When MAPE Works Well:
1. Stakeholder-friendly communication
"Our model has 12% average error" makes a very clear message.
2. Need the same meaning across the target scale.
A 20% error is always twice as bad as 10%, regardless of scale.
When MAPE Fails:
1. True values near zero
You can see from the formula that if the true value (denominator) is close to zero, even a small model deviation can lead to huge MAPE values.
2. Bias Towards Underestimation:
Larger errors are weighted heavily, which can bias models to avoid overestimation, for example:
↳ If actual = 10 and prediction = 20, MAPE = 100%.
↳ If actual = 20 and prediction = 10, MAPE = 50%.
2️⃣ MAE (Mean Absolute Error)
The MAE formula is as follows:
MAE measures the average absolute difference between predictions and actuals - simple and robust.
When MAE Works Well:
1. Robust to outliers
Treats all errors equally without amplifying large deviations
2. Clear interpretation
"On average, predictions are off by $500" - direct and understandable
When MAE is Not Good:
1. Doesn't distinguish error severity
$100 error treated the same whether on $1K or $100K prediction
2. Less sensitive to improvements
Small model enhancements might not show up clearly in MAE
3️⃣ RMSE (Root Mean Squared Error)
The RMSE formula is as follows:
RMSE squares errors before averaging, heavily penalizing large deviations.
When RMSE Works Well:
1. Large errors are costly
Perfect when big mistakes have a disproportionate business impact
2. Same units as target
Easy to interpret in the context of your problem domain
When RMSE Misleads:
1. Dominated by outliers
A few extreme errors can completely mask overall model performance
2. Hard to interpret relatively
RMSE = 50 could be excellent or terrible, depending on your data scale
4️⃣ R² (Coefficient of Determination)
The R² formula is as follows:
R² measures the proportion of variance your model explains compared to just predicting the mean.
When R² Works Well:
1. Quick baseline comparison
Immediately shows if your model beats a simple average
2. 𝗛𝗮𝗻𝗱𝗹𝗲𝘀 𝗗𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁 𝗗𝗮𝘁𝗮 𝗥𝗮𝗻𝗴𝗲𝘀
R² doesn't depend on the scale of the target.
This makes it handy to compare the models for different datasets or dataset ranges of the same problem.
R² disadvantages:
1. High R² ≠ good model
It can be high while missing actual relationships, especially non-linear patterns
2. Extremely sensitive to outliers
Few extreme values can completely skew R² interpretation
3. Always increases with features
Adding any feature never decreases R², creating false confidence
Here is a comparative snapshot of each metric under different conditions:
Decision Framework: Choosing Your Metric
Here's a practical decision tree based on your situation:
Step 1: Check Your Data Characteristics
Data have significant outliers?
- ✅ Consider MAE (robust)
- ❌ Avoid RMSE (outlier-sensitive)
Target values near zero?
- ✅ Use MAE or RMSE
- ❌ Avoid MAPE (explodes near zero)
Wide range of target values?
- ✅ Consider MAPE (scale-independent)
- ✅ R² for variance explanation
Step 2: Consider Business Context
Stakeholders think in percentages?
→ Use MAPE (if no zeros)
Large errors much more costly than small ones?
→ Use RMSE
All errors have similar business impact?
→ Use MAE
Need quick model validation?
→ Use R² + one error metric
Real-World Example: E-commerce Sales Forecasting
Context: Predicting weekly sales for 500+ products across different price ranges ($10 - $5000)
Data issues: Holiday spikes (3x normal sales), new product launches, promotional effects
Business needs: Inventory planning, budget allocation, performance tracking
Which metrics to use?
❌ MAPE alone - Breaks for new products with near-zero historical sales
❌ R² alone - Holiday outliers will dominate the variance calculation
❌ RMSE alone - High-value products will dominate error calculation
✅ Recommended approach:
- Primary: MAE (robust to outliers, interpretable for operations team)
- Secondary: MAPE for established products (business communication)
That is it for this week!
If you haven’t yet, follow me on LinkedIn where I share Technical and Career ML content every day!
Whenever you're ready, there are 3 ways I can help you:
1. ML Job Landing Kit
Get everything I learned about landing ML jobs after reviewing 1000+ ML CVs, conducting 100+ interviews & hiring 25 Data Scientists. The exact system I used to help 70+ clients get more interviews and land their ML jobs.
2. ML Career 1:1 Session
I’ll address your personal request & create a strategic plan with the next steps to grow your ML career.
3. Full CV & LinkedIn Upgrade (all done for you)
I review your experience, clarify all the details, and create:
- Upgraded ready-to-use CV (Doc format)
- Optimized LinkedIn Profile (About, Headline, Banner, and Experience Sections)
Join Maistermind for 1 weekly piece with 2 ML guides:
1. Technical ML tutorial or skill learning guide
2. Tips list to grow ML career, LinkedIn, income