Model Rot
Models rot in production. It's a fundamental fact of life. Our e-commerce recommendation engine (a deep ranking model) was degrading by about 2% in Click-Through Rate (CTR) every single week due to rapidly shifting user behavior and seasonal trends. We couldn't afford the compute cost to do a full retrain on historical data every single night.
Online Learning Pipeline
We implemented a continuous learning pipeline using PyTorch. Instead of batch training, we stream recent interactions (clicks, purchases) into an in-memory buffer (using Redis).
Every hour, a cron job pulls the buffer and runs a few gradient steps on the model weights using a very low learning rate (e.g., $1e-6$).
import torch.optim as optim
import torch.nn.functional as F
# Continuous learning update step
def update_model(model, optimizer, old_model, batch):
optimizer.zero_grad()
predictions = model(batch.features)
# Standard task loss (e.g., BCE for CTR prediction)
task_loss = F.binary_cross_entropy_with_logits(predictions, batch.labels)
# KL-Divergence penalty to prevent catastrophic forgetting
with torch.no_grad():
old_preds = old_model(batch.features)
kl_loss = F.kl_div(F.logsigmoid(predictions), torch.sigmoid(old_preds))
total_loss = task_loss + (0.1 * kl_loss)
total_loss.backward()
optimizer.step()To ensure the model doesn't catastrophically forget its historical knowledge while over-indexing on recent trends, we use a KL-divergence penalty against a frozen "anchor" version of the model. This stabilized our CTR permanently.