DECODE.
EVOLVE.
LINEAR_LOGIC.
Deconstructing the hyperplane and the mapping function. Linear models form the structural bedrock of predictive synthesis, translating high-dimensional inputs into actionable scalar outputs.
Regression Analysis
Minimizing Residual Sum of Squares (RSS)
At the core of Ordinary Least Squares (OLS) is the optimization of the objective function. We define the cost as the vertical distance between observed data points and the predictive hyperplane. By squaring these residuals, we penalize higher variance, forcing the model to find the global minimum of the error surface.
Normal Equation vs. Gradient Descent
While the Normal Equation provides an analytical solution for datasets where the inversion of the transpose matrix is computationally feasible, large-scale systems require stochastic optimization. At Oilrise, we emphasize the transition to iterative refinement as dimensionality scales.
Optimization Sensitivity
Linear regression is fundamentally sensitive to structural anomalies. Outliers with high leverage can drastically rotate the hyperplane, leading to poor generalization. Identifying these influential points is the first step in robust algorithmic training.
Calculus Verification
Every module at Oilrise Academy begins with a formal derivation of the cost function. We bridge the gap between abstract algebra and functional code.
Proof MethodologyLogistic
Expansion
Linear classification is achieved through the Sigmoid activation. By mapping real-valued numbers into a probability space between (0,1), we transition from raw regression to discrete decision boundaries.
- 01 Binary Cross-Entropy Loss: The logarithmic penalty for classification divergence.
- 02 Decision Thresholds: Fine-tuning sensitivity versus specificity.
Maximum
Margins
Support Vector Machines (SVM) seek the widest possible corridor between classes. It is the geometric pursuit of the maximum margin hyperplane to ensure maximum robustness against feature noise.
| Metric | Hard Margin | Soft Margin |
|---|---|---|
| Constraint | No overlap allowed | Permits misclassification |
| Resilience | Low (Sensitive) | High (Generalizable) |
| Application | Linearly Separable | Real-world Noisy Data |
The Bias-Variance Balance
Linear models serve as the benchmark for understanding complexity. Through Ridge and Lasso regularization, we navigate the space between underfitting raw signals and overfitting noise. This is where linear models prove their enduring technical value.
Lasso / Sparsity
Ridge / Weight Decay
Regularization Coefficient
Academic Verification
Linear Algebra Refresher
Matrix transformations and vector space properties in ML pipelines.
Optimization Practice Set
Hand-deriving the Gradient Descent update rule for cross-entropy.
Evaluation Case Studies
Analysis of linearity assumptions in high-dimensional genomic data.
Beyond the Linear Domain
Move from rigid hyperplanes to the high-dimensional latent spaces of neural architectures.
Engineering Inquiry
Oilrise ML Academy is a Toronto-based research institution dedicated to the rigorous mathematical deconstruction of artificial intelligence.
1200 Bay St, Toronto, ON M5R 2A5, Canada
[email protected] | +1-416-559-2460
Working hours: Mon-Fri: 9:00-18:00