Building a Real-Time Fraud Detection Pipeline with Elastic Retraining

The Problem

In high-volume payment systems, fraud alert queues can contain thousands of events per hour. Manual triage is impossible. The goal: build a system that automatically prioritises alerts so analysts only see what matters.

---

Pipeline Design

The system has three core loops:

1. Inference loop, real-time scoring of incoming alerts
2. Feedback loop, analyst actions feed back as labels
3. Retraining loop, model retrains on an elastic window of recent data

Elastic Data Windows

Unlike a fixed training window (e.g., last 30 days), elastic windows adapt to fraud velocity:

def get_training_window(fraud_rate: float) -> int:
    """Return window size in days based on current fraud rate."""
    if fraud_rate > 0.05:   # High fraud period
        return 7
    elif fraud_rate > 0.02:
        return 21
    else:                   # Low fraud, need more history
        return 60

This keeps the model sensitive to emerging fraud patterns without overfitting to noise.

---

Feature Engineering

Key features that drove performance:

Feature	Type	Description
amount_zscore	Numerical	Z-score of txn amount per merchant
time_since_last_txn	Numerical	Seconds since previous transaction
device_fingerprint_match	Binary	Known device vs new device
merchant_risk_score	Numerical	Rolling fraud rate per merchant
velocity_1h	Numerical	# of txns in last hour for this card

---

Model: XGBoost with Calibrated Probabilities

from xgboost import XGBClassifier
from sklearn.calibration import CalibratedClassifierCV

base_model = XGBClassifier(
    n_estimators=400,
    max_depth=6,
    learning_rate=0.05,
    scale_pos_weight=20,  # Class imbalance
    tree_method="hist",
)

model = CalibratedClassifierCV(base_model, cv=3, method="isotonic")
model.fit(X_train, y_train)

Calibrated probabilities are critical here, analysts need to trust the score, not just the binary prediction.

---

Results

• Precision@top-5%: 78% (vs 34% baseline)
• Alert queue reduction: ~65%
• Model staleness: retraining triggered automatically when AUC drops below threshold

The elastic window approach was the single biggest performance gain, adapting to fraud seasonality made a measurable difference.