Machine Learning Fundamentals

Core Components of Machine Learning

Every ML system has these essential parts:

1. Data: Raw information to learn from
2. Model: System to transform data
3. Objective Function: Measures performance
4. Algorithm: Optimizes model parameters

1. Data Science Fundamentals

Data Components

• Features (inputs/covariates)
• Labels (targets for prediction)

Data Types Examples

1. Images

   • RGB values (200×200×3 = 120,000 values)
   • Variable resolutions

2. Healthcare

   • Patient vitals
   • Medical history
   • Treatment records

Data Quality

• Quality > Quantity
• "Garbage in, garbage out"
• Must check for bias
• Need representative samples

2. Models

Types

• Simple models for basic tasks
• Deep learning for complex problems
• Multiple transformations chained together

Selection Criteria

• Problem complexity
• Data availability
• Resource constraints

3. Objective Functions

Characteristics

• Measure model performance
• Lower values = better
• Called "loss functions"

Common Types

1. Regression: Squared error
2. Classification: Error rate

Data Split

• Training set: Learning
• Test set: Evaluation
• Avoid overfitting

4. Optimization

Gradient Descent

• Iterative parameter updates
• Minimizes loss function
• Small step-by-step improvements

ML Problem Types

1. Supervised Learning

• Regression
• Classification
• Tagging
• Search
• Recommender Systems
• Sequence Learning

2. Unsupervised Learning

• Clustering
• Subspace estimation

3. Environmental Interaction

• Offline learning
• Distribution shift challenges

4. Reinforcement Learning

• Agent-environment interaction
• Action-based learning

Historical Background

Early Foundations

• Bernoulli distribution (1655-1705)
• Gaussian distribution (1777-1855)
• Early statistical methods

Key Contributors

1. Ronald Fisher

   • Statistical foundations
   • Iris dataset

2. Claude Shannon

   • Information theory

3. Alan Turing

   • Computation theory
   • AI testing concepts

4. Donald Hebb

   • Neural learning principles

Neural Networks Evolution

• Biological inspiration
• Alternating layer structure
• Backpropagation importance

Key Takeaways from Day 1

1. ML requires quality data, appropriate models, clear objectives, and efficient optimization
2. Different problems need different approaches
3. Historical foundations inform modern methods
4. Balance between theory and practical application is crucial
5. Ethics and bias awareness are essential