Python & ML Foundations

Environment Setup

Before starting with a programming language, we'll set up an IDE. Basically, these are programming environments where you would write your code (looks cool :) ). We recommend VS Code, but feel free to use any IDE of your choice. We'll also set up Anaconda for managing Python packages, and optionally Google Colab if you prefer working in the cloud.

Version Control (Git & GitHub)

Git is a version control system that lets you track changes in your code and collaborate with others. GitHub is where we'll host our project repositories. You'll be using these throughout the project to submit assignments and collaborate with your team.

Python Basics

Python is the language we'll use for everything in this project, from simple scripts to training RL agents. Make sure you're comfortable with the fundamentals: variables, loops, functions, and especially classes (OOPs), since most ML libraries are built around them. If you already know Python, skim through these as a refresher.

• Variables, data types, control flow, functions
• List comprehensions, lambda expressions
• Classes and basic OOPs concepts

NumPy & Matplotlib

NumPy is the backbone of numerical computing in Python. You'll use it for everything from matrix operations to handling environment states. Matplotlib lets you visualize training curves, rewards, and agent behavior. Both are essential tools you'll use daily.

Mathematics (Probability)

Reinforcement Learning is built on probability theory. Understanding Bayes' theorem, probability distributions, and the law of large numbers will help you grasp why RL algorithms work the way they do. Don't worry, we only need the intuition, not rigorous proofs.

Machine Learning Foundations

Before jumping into RL, it helps to understand how machines learn from data. Linear and logistic regression are the simplest ML algorithms and introduce key concepts like cost functions, gradient descent, and overfitting that carry over directly to deep RL.

• Linear Regression: fitting a line, cost function, gradient descent
• Logistic Regression: sigmoid function, binary classification
• Evaluation: confusion matrix, train-test split

Neural Networks

Neural networks are the function approximators that make deep RL possible. In this project, we'll use them as policy networks (PPO) and world models. Understanding how a basic neural net learns through backpropagation is essential, and the 3Blue1Brown series is a must-watch.

• Perceptrons and multi-layer perceptrons
• Activation functions, backpropagation
• CNNs (overview)
• PyTorch / TensorFlow / Scikit-learn intro