The Problem

In high-volume payment systems, fraud alert queues can contain thousands of events per hour. Manual triage is impossible. The goal: build a system that automatically prioritises alerts so analysts only see what matters.

---

Pipeline Design

The system has three core loops:

1. Inference loop — real-time scoring of incoming alerts
2. Feedback loop — analyst actions feed back as labels
3. Retraining loop — model retrains on an elastic window of recent data

Elastic Data Windows

Unlike a fixed training window (e.g., last 30 days), elastic windows adapt to fraud velocity:

def get_training_window(fraud_rate: float) -> int:
    """Return window size in days based on current fraud rate."""
    if fraud_rate > 0.05:   # High fraud period
        return 7
    elif fraud_rate > 0.02:
        return 21
    else:                   # Low fraud, need more history
        return 60

This keeps the model sensitive to emerging fraud patterns without overfitting to noise.

---

Feature Engineering

Key features that drove performance:

---

Model: XGBoost with Calibrated Probabilities

from xgboost import XGBClassifier
from sklearn.calibration import CalibratedClassifierCV

base_model = XGBClassifier(
    n_estimators=400,
    max_depth=6,
    learning_rate=0.05,
    scale_pos_weight=20,  # Class imbalance
    tree_method="hist",
)

model = CalibratedClassifierCV(base_model, cv=3, method="isotonic")
model.fit(X_train, y_train)

Calibrated probabilities are critical here — analysts need to trust the score, not just the binary prediction.

---

Results

• Precision@top-5%: 78% (vs 34% baseline)
• Alert queue reduction: ~65%
• Model staleness: retraining triggered automatically when AUC drops below threshold

The elastic window approach was the single biggest performance gain — adapting to fraud seasonality made a measurable difference.