Introduction to Numerical Regression

by Dr. Ranjeet Tate

 

Overview

Numerical regression – finding the quantitative relationship between variables – is at the core of machine learning. This class is intended for students who want to understand machine learning from the inside out, and by its end participants should have a solid understanding of the basic statistical techniques that underpin modern machine learning.

Venue

All classes will be taught in person in the Principia Mathematica room at the Academia.edu offices, at California and Kearny in downtown San Francisco.

Schedule

The course will last eight weeks, with one class meeting per week from 6:30pm to 9:30pm. If you have to miss a class, we'll make sure you're able to catch up, either by scheduling a one-one meeting or sharing a video of the class.

Additional, students may opt to augment the course with weekly one on one tutorials to get more detailed individual feedback and instruction. 

Curriculum

We will be covering the following topics:

  • 1-dimensional linear models
  • Form of data
  • Models
  • Error "vector"
  • Target function for "best fit"
  • Manual minimization in a spreadsheet
  • Closed form solution for Ordinary Least Squares (OLS)
  • Predictions
  • Residuals
  • Propagation of errors (errors in a function of stochastic variables) 
  • Standard Error (SE) in the parameters
  • Confidence interval
  • Prediction interval
  • Higher dimensions, closed form solution for OLS
  • Linear regression for a non-linear model
  • Underfitting, overfitting, SE in the parameters
  • Training Data vs Test Data
  • Exponential model, power law, logistic model
  • Transformations on the data
  • Other "norms" on the error vector
  • Numerical approaches to optimization
  • Gradient descent
  • Problems with gradient descent
  • Second order methods: Newton-Raphson
  • Importance of phenomenological models

Prerequisites 

Participants should have a solid understanding of high-school mathematics. Some differential calculus is preferable but not essential, as is some familiarity with linear algebra. 

Venue

Academia.edu
580 California Street
Fourth floor
San Francisco, CA 94104

Dates

Tue, Sep 4th, 2018, 6:30pm –
Tue, Oct 23rd, 2018, 9:30pm.

 

Attendance is free for students who subscribe to an EDeeU Education Director of Studies.