ml_tutorial.livemd

# Machine Learning in Elixir - Interactive Tutorial

<!-- livebook:{"app":"embedded"} -->

```elixir
Mix.install([
  {:nx, "~> 0.11"},
  {:axon, "~> 0.8"},
  {:exla, "~> 0.11"},
  {:kino, "~> 0.13"}
])
```

## Chapter 1: Introduction to ML in Elixir 🚀

### Welcome to Machine Learning with Elixir!

**I'm an intermediate Elixir developer and I want to learn Machine Learning in Elixir so I can build intelligent applications and understand ML concepts using functional programming patterns.**

### What Makes Elixir Special for ML?

Elixir brings some unique advantages to machine learning:

- **🏗️ Functional Programming Foundation**: Pure functions and immutability naturally align with ML workflows
- **⚡ Concurrency & Distribution**: Handle large datasets and parallel training efficiently
- **🔧 Erlang VM Benefits**: Fault tolerance and hot code reloading for production ML systems
- **📊 Nx Library**: Numerical computing with GPU acceleration

### Core Concepts in Elixir ML

**1. Tensors** 🔢
Tensors are the fundamental building blocks - think of them as multi-dimensional arrays:

```elixir
import Nx

# Creating a tensor
tensor = Nx.tensor([[1, 2, 3], [4, 5, 6]])
tensor
```

**2. Numerical Operations** ➕
Perform mathematical operations efficiently:

```elixir
# Element-wise operations
a = Nx.tensor([1, 2, 3])
b = Nx.tensor([4, 5, 6])
result = Nx.add(a, b)
result
```

### Your First ML Program 💻

Let's create a simple linear regression example:

```elixir
defmodule SimpleML do
  import Nx
  
  def predict(x, weights) do
    # Simple linear prediction: y = mx + b
    multiply(x, weights[0]) + weights[1]
  end
  
  def train(x_data, y_data, learning_rate \\ 0.01, epochs \\ 1000) do
    # Initialize random weights
    weights = [Nx.random_normal({}), Nx.random_normal({})]
    
    Enum.reduce(1..epochs, weights, fn epoch, [m, b] ->
      # Forward pass
      predictions = multiply(x_data, m) + b
      
      # Calculate loss (mean squared error)
      loss = mean(power(predictions - y_data, 2))
      
      # Backward pass (gradients)
      grad_m = mean(2 * (predictions - y_data) * x_data)
      grad_b = mean(2 * (predictions - y_data))
      
      # Update weights
      [m - learning_rate * grad_m, b - learning_rate * grad_b]
    end)
  end
end
```

**Try it out:**

```elixir
# Generate simple linear data: y = 2x + 3 + noise
x_data = Nx.tensor(Enum.map(0..100, &(&1 / 10.0)))  # 0 to 10 in steps of 0.1
noise = Nx.random_normal({101}) |> Nx.multiply(0.1)
y_data = Nx.multiply(x_data, 2) |> Nx.add(3) |> Nx.add(noise)

# Train the model
weights = SimpleML.train(x_data, y_data, 0.01, 500)

# Test prediction
test_x = Nx.tensor([0.5, 1.0, 1.5, 2.0])
predictions = SimpleML.predict(test_x, weights)

"Trained weights: #{inspect(weights)}, Predictions: #{inspect(predictions)}"
```

## Chapter 2: Nx and Numerical Computing 🔢

### Understanding Nx Tensors 📊

Tensors are multi-dimensional arrays that power all ML computations:

```elixir
import Nx

# Different tensor types
scalar = Nx.tensor(42)           # 0-dimensional
vector = Nx.tensor([1, 2, 3])    # 1-dimensional  
matrix = Nx.tensor([[1, 2], [3, 4]])  # 2-dimensional

# Tensor properties
IO.puts("Vector shape: #{inspect(Nx.shape(vector))}")
IO.puts("Vector type: #{inspect(Nx.type(vector))}")
IO.puts("Vector size: #{inspect(Nx.size(vector))}")
```

### Essential Tensor Operations ⚡

**Mathematical Operations:**

```elixir
# Basic arithmetic
a = Nx.tensor([1.0, 2.0, 3.0])
b = Nx.tensor([4.0, 5.0, 6.0])

add_result = Nx.add(a, b)
mult_result = Nx.multiply(a, b)
pow_result = Nx.power(a, 2)

{add_result, mult_result, pow_result}
```

**Aggregation Operations:**

```elixir
matrix = Nx.tensor([[1, 2, 3], [4, 5, 6]])

sum_all = Nx.sum(matrix)
sum_rows = Nx.sum(matrix, axes: [1])
mean_all = Nx.mean(matrix)

{sum_all, sum_rows, mean_all}
```

### Broadcasting Magic ✨

Nx automatically broadcasts tensors to compatible shapes:

```elixir
# Broadcasting examples
vector = Nx.tensor([1, 2, 3])
scalar = Nx.tensor(10)

# Add scalar to each element
result1 = Nx.add(vector, scalar)

# Broadcasting with different dimensions
matrix = Nx.tensor([[1, 2, 3], [4, 5, 6]])
result2 = Nx.add(matrix, vector)

{result1, result2}
```

### Hands-On Exercise 💻

Create a function that normalizes a dataset:

```elixir
defmodule DataPreprocessing do
  import Nx
  
  def normalize(tensor) do
    mean = Nx.mean(tensor)
    std = Nx.standard_deviation(tensor)
    
    # Normalize: (x - mean) / std
    Nx.divide(Nx.subtract(tensor, mean), std)
  end
  
  def min_max_scale(tensor) do
    min = Nx.reduce_min(tensor)
    max = Nx.reduce_max(tensor)
    
    # Scale to [0, 1] range
    Nx.divide(Nx.subtract(tensor, min), Nx.subtract(max, min))
  end
end

# Test with sample data
data = Nx.tensor([10, 20, 30, 40, 50])
normalized = DataPreprocessing.normalize(data)
scaled = DataPreprocessing.min_max_scale(data)

{data, normalized, scaled}
```

## Chapter 3: Building ML Models with Axon 🧠

### Understanding Axon Architecture 🏗️

Axon provides a functional API for building neural networks:

```elixir
import Axon

# Simple neural network
model = Axon.input("input", shape: {nil, 784})
  |> Axon.dense(128, activation: :relu)
  |> Axon.dense(64, activation: :relu)
  |> Axon.dense(10, activation: :softmax)

IO.puts("Model structure:")
IO.inspect(model)
```

### Building Your First Neural Network 🚀

**MNIST Digit Classification:**

```elixir
defmodule MNISTClassifier do
  import Axon
  
  def build_model() do
    # Input: 28x28 grayscale images (flattened to 784)
    Axon.input("input", shape: {nil, 784})
    # Hidden layers
    |> Axon.dense(128, activation: :relu)
    |> Axon.dropout(rate: 0.3)
    |> Axon.dense(64, activation: :relu)
    |> Axon.dropout(rate: 0.3)
    # Output: 10 classes (digits 0-9)
    |> Axon.dense(10, activation: :softmax)
  end
  
  def train_model(model, train_data, validation_data) do
    # Training loop
    Axon.Loop.trainer(model, :categorical_cross_entropy, :adam)
    |> Axon.Loop.validate(model, validation_data)
    |> Axon.Loop.run(train_data, epochs: 10)
  end
end

# Create a sample model
model = MNISTClassifier.build_model()
model
```

### Common Layer Types 🧩

**Dense (Fully Connected) Layers:**

```elixir
model1 = Axon.input("input", shape: {nil, 100})
  |> Axon.dense(50)  # 50 neurons
  |> Axon.dense(25)  # 25 neurons
  |> Axon.dense(1)   # Output neuron

model1
```

**Convolutional Layers (for images):**

```elixir
model2 = Axon.input("input", shape: {nil, 28, 28, 1})  # Grayscale images
  |> Axon.conv(32, kernel_size: {3, 3}, activation: :relu)
  |> Axon.max_pool(kernel_size: {2, 2})
  |> Axon.conv(64, kernel_size: {3, 3}, activation: :relu)
  |> Axon.max_pool(kernel_size: {2, 2})
  |> Axon.flatten()
  |> Axon.dense(128, activation: :relu)
  |> Axon.dense(10, activation: :softmax)

model2
```

### Activation Functions 🔥

```elixir
model3 = Axon.input("input", shape: {nil, 100})
  |> Axon.dense(50, activation: :relu)     # ReLU - most common
  |> Axon.dense(25, activation: :sigmoid)  # Sigmoid - for probabilities
  |> Axon.dense(10, activation: :softmax)  # Softmax - for classification
  |> Axon.dense(1, activation: :tanh)     # Tanh - for outputs in [-1, 1]

model3
```

## Chapter 4: Real-world Applications 🚀

### Complete ML Pipeline Example 🔄

**End-to-End Fraud Detection System:**

```elixir
defmodule FraudDetection do
  import Axon
  
  def build_pipeline() do
    # Data preprocessing -> Model training -> Prediction
    
    # 1. Model definition
    model = build_fraud_model()
    
    %{
      model: model
    }
  end
  
  defp build_fraud_model() do
    Axon.input("input", shape: {nil, 20})  # 20 features
    |> Axon.dense(64, activation: :relu)
    |> Axon.dropout(rate: 0.3)
    |> Axon.dense(32, activation: :relu)
    |> Axon.dropout(rate: 0.3)
    |> Axon.dense(1, activation: :sigmoid)  # Probability of fraud
  end
end

# Create the fraud detection pipeline
pipeline = FraudDetection.build_pipeline()
pipeline.model
```

### Practical Project: Customer Churn Prediction 📈

```elixir
defmodule ChurnPredictor do
  import Axon
  
  def build_churn_model() do
    # Customer features: age, usage_pattern, support_tickets, etc.
    Axon.input("input", shape: {nil, 15})
    |> Axon.dense(32, activation: :relu)
    |> Axon.batch_norm()
    |> Axon.dense(16, activation: :relu)
    |> Axon.dropout(rate: 0.2)
    |> Axon.dense(1, activation: :sigmoid)  # Probability of churn
  end
  
  def predict_churn_risk(customer_features) do
    # Simulate prediction
    %{
      probability: 0.65,
      risk_level: :medium,
      recommendation: "Offer loyalty discount"
    }
  end
end

# Build churn prediction model
churn_model = ChurnPredictor.build_churn_model()
churn_model
```

## Interactive Exercises 🎯

### Exercise 1: Tensor Operations
Create a function that calculates the dot product of two matrices:

```elixir
defmodule TensorExercises do
  import Nx
  
  def dot_product(a, b) do
    # Your code here
    # Hint: Use Nx.dot/2
    Nx.dot(a, b)
  end
  
  def elementwise_multiply(a, b) do
    # Your code here
    Nx.multiply(a, b)
  end
end

# Test your functions
a = Nx.tensor([[1, 2], [3, 4]])
b = Nx.tensor([[5, 6], [7, 8]])

dot_result = TensorExercises.dot_product(a, b)
mult_result = TensorExercises.elementwise_multiply(a, b)

{dot_result, mult_result}
```

### Exercise 2: Build a Custom Model
Create a neural network with 3 hidden layers (128, 64, 32 neurons) and ReLU activations:

```elixir
defmodule CustomModel do
  import Axon
  
  def build_custom_model(input_shape \\ {nil, 10}) do
    # Your code here
    Axon.input("input", shape: input_shape)
    |> Axon.dense(128, activation: :relu)
    |> Axon.dense(64, activation: :relu)
    |> Axon.dense(32, activation: :relu)
    |> Axon.dense(1, activation: :sigmoid)
  end
end

# Build and display the model
model = CustomModel.build_custom_model()
model
```

## Next Steps 🚀

1. **Experiment** with different tensor operations
2. **Modify** the models with different architectures
3. **Try** training on real datasets
4. **Explore** Livebook's visualization features

Happy learning! 🎉✨

---

*This notebook was created as a companion to the "Machine Learning in Elixir" tutorial.*
*Save this notebook (Ctrl+S) to keep your progress!*