Mathematical Foundations for Machine Learning in JavaScript

1. Introduction

Mathematics is the backbone of Machine Learning. Whether you're working with neural networks, decision trees, or clustering algorithms, a solid grasp of mathematical concepts helps in understanding how and why models work rather than just using them as black boxes.

In this guide, we’ll break down the key mathematical concepts essential for ML and implement some of them using JavaScript.

2. Why Mathematics is Important for ML

Machine Learning relies on mathematical principles to:

Represent data using matrices and vectors.

Measure errors with cost functions.

Optimize models using derivatives and gradients.

Interpret probability distributions in statistical models.

A strong mathematical foundation makes it easier to debug, optimize, and improve ML models efficiently.

3. Linear Algebra for Machine Learning in JS

In machine learning, data is represented as vectors and matrices. Linear algebra is the math that helps us manipulate and understand these data structures. It's used in:

Representing data (as matrices)

Defining models (like weights in a neural network)

Optimizing models (like using gradients)

3.1 Scalars: The Building Block

A scalar is a single number. It represents only magnitude — no direction.

💡Example:

const scalar = 5;

Scalars are used to:

Scale vectors and matrices

Represent bias in linear regression (y = wx + b, where b is a scalar)

3.2 Vectors: One-Dimensional Arrays

A vector is a 1D array of scalars. It represents both magnitude and direction. In machine learning, vectors are often used to represent features of a data sample.

💡Example: House Features

const houseFeatures = [2000, 3, 500000];
// [squareFeet, rooms, price]

3.3 Vector Operations

Vectors are ordered lists of numbers. Operations on vectors are super important in machine learning — from calculating distances between data points to computing predictions in a model.

Let’s break down the most essential ones:

➕ Addition and ➖ Subtraction

Element-wise addition or subtraction between two vectors of the same length:

💡 Example:

const vectorAdd = (a, b) => a.map((val, i) => val + b[i]);
const vectorSubtract = (a, b) => a.map((val, i) => val - b[i]);

vectorAdd([1, 2], [3, 4]); // [4, 6]
vectorSubtract([5, 3], [2, 1]); // [3, 2]

✖️ Scalar Multiplication

Multiply each element of a vector by a scalar:

💡 Example:

const scalarMultiply = (vector, scalar) => vector.map(val => val * scalar);

scalarMultiply([2, 4, 6], 3); // [6, 12, 18]

Scaling vectors is useful in normalization, feature weighting, and adjusting learning rates.

🔘 Dot Product

The dot product returns a single scalar by multiplying corresponding vector elements and summing the results.

💡 Example:

const dotProduct = (a, b) => a.reduce((sum, val, i) => sum + val * b[i], 0);

The dot product is commonly used in machine learning to measure vector similarity and plays a key role in algorithms like linear regression, neural networks, and techniques such as cosine similarity.

3.4 Matrices: 2D Arrays

A matrix is a 2D array of numbers. Each row is a vector, and matrices are used to represent datasets or transformations.

💡 Example:

const data = [
  [2000, 3, 500000], // House 1
  [1500, 2, 350000], // House 2
  [1800, 3, 400000]  // House 3
];
// Matrix of 3 samples × 3 features

3.5 Matrix Multiplication

Matrix multiplication is combining rows of Matrix A with columns of Matrix B.
It only works if A’s columns = B’s rows.

💡 Example:

function multiplyMatrices(A, B) {
  let result = Array.from({ length: A.length }, () => new Array(B[0].length).fill(0));

  for (let i = 0; i < A.length; i++) {
    for (let j = 0; j < B[0].length; j++) {
      for (let k = 0; k < B.length; k++) {
        result[i][j] += A[i][k] * B[k][j];
      }
    }
  }

  return result;
}

const A = [
  [1, 2],
  [3, 4]
];

const B = [
  [2, 0],
  [1, 2]
];

multiplyMatrices(A, B);
// Output: [
//   [4, 4],
//   [10, 8]
// ]

4. Calculus for Machine Learning in JS

Calculus, especially differential calculus, is at the heart of optimization in machine learning. It's used to minimize loss functions using methods like gradient descent.

4.1 Derivatives: Measuring Change

A derivative measures how a function changes as its input changes. In ML, we often need to know how much the cost (error) changes with respect to a model parameter.

💡 Example: Numerical Derivative (using finite difference)

const derivative = (f, x, h = 1e-5) => (f(x + h) - f(x)) / h;

const square = x => x * x;

derivative(square, 3); // 6 (since the derivative of x^2 is 2x)

This is helpful when implementing algorithms like gradient descent manually.

4.2 Gradient Descent (Simplified)

Gradient Descent is an optimization algorithm used to minimize the cost function by updating parameters in the opposite direction of the gradient.

💡 Example:

function gradientDescentStep(x, y, weight, learningRate) {
  const predict = x => weight * x;
  const cost = (x, y) => (predict(x) - y) ** 2;
  const dCost = derivative(x => cost(x, y), x);

  // Update rule
  return weight - learningRate * dCost;
}

let weight = 0.5;
weight = gradientDescentStep(3, 9, weight, 0.01); // One step closer to ideal weight

5. Probability and Statistics for ML in JS

Probability helps in understanding uncertainty, and statistics help in summarizing and interpreting data — both essential for ML models.

5.1 Mean, Median, and Mode

These are measures of central tendency.

const mean = arr => arr.reduce((a, b) => a + b, 0) / arr.length;

const median = arr => {
  const sorted = [...arr].sort((a, b) => a - b);
  const mid = Math.floor(sorted.length / 2);
  return sorted.length % 2 === 0
    ? (sorted[mid - 1] + sorted[mid]) / 2
    : sorted[mid];
};

5.2 Variance and Standard Deviation

Variance measures how far each number in the set is from the mean, and standard deviation is the square root of variance.

const variance = arr => {
  const m = mean(arr);
  return mean(arr.map(x => (x - m) ** 2));
};

const stdDeviation = arr => Math.sqrt(variance(arr));

These metrics are key in feature scaling and understanding data distribution.

6. Probability Basics

Probability helps in classification problems, Naive Bayes, and understanding data noise.

6.1 Basic Probability

const probability = (favorable, total) => favorable / total;

probability(2, 5); // Probability of 2 favorable outcomes out of 5

6.2 Conditional Probability

This is the probability of event A happening given event B has occurred.

// P(A|B) = P(A and B) / P(B)
const conditionalProbability = (aAndB, b) => aAndB / b;

7. Conclusion

This blog gives you an overall idea of how different areas of mathematics—like linear algebra, calculus, probability, and statistics—play a crucial role in machine learning. While we've covered the foundational concepts, this guide serves as a starting point rather than an in-depth exploration.

If you're new to ML, it's perfectly fine to start here and build your intuition. But to truly master these concepts and apply them effectively, we encourage you to explore each section more deeply. In future posts, we’ll dive into each topic further with more advanced examples and real-world ML applications using JavaScript.

📌 What’s Next?

🚀 Stay tuned for the next post: " Supervised Learning: The Beginner's Guide"

💡 If you found this helpful, don’t forget to like and share! 👍