Data Splitting Strategies: Random, Time, User, Leave-One-Out — Practical Trade-Offs

5 Ana Split Stratejisi#

1) Random Split — En Yaygın, En Yanlış#

Yöntem#

from sklearn.model_selection import train_test_split
train, test = train_test_split(events, test_size=0.2, random_state=42)

Problem: Temporal Leakage#

2015-01: [view A, view B, view C, ...]
2015-06: [view F, view G]
2015-12: [view K, view L]

train: [A, C, F, K, L]
test:  [B, G]

Sonuç#

Ne Zaman OK?#

• Akademik baseline (literature ile kıyas için)
• Çok küçük dataset (time-split için yeterli volume yok)
• Cross-validation hızı önemli (deneme aşaması)

Pratik#

# Random split — sadece "ilk benchmark" için kullan
import numpy as np
np.random.seed(42)
mask = np.random.random(events.height) < 0.8
train = events.filter(mask)
test = events.filter(~mask)

2) Time Split — Production Realism'in En Doğru Yansıması#

Yöntem#

import polars as pl

# Dataset'in 80. persentilindeki timestamp'i eşik yap
threshold = events["timestamp"].quantile(0.8)
train = events.filter(pl.col("timestamp") < threshold)
test = events.filter(pl.col("timestamp") >= threshold)

Niçin Daha Gerçekçi?#

Cold Item Problemi#

Çözüm: Item Filter

train_items = set(train["item_id"].unique().to_list())
test_filtered = test.filter(pl.col("item_id").is_in(train_items))

print(f"Test before: {test.height}")
print(f"Test after:  {test_filtered.height}")
print(f"Lost cold items: {test.height - test_filtered.height}")

Two-Phase Time Split#

[----------------- All Data ----------------]
[--- Train ---][- Val -][- Test -]
   60-70%       10-15%   15-20%
   (önceki)    (orta)    (sonraki)

3) Leave-One-Out (LOO) — Sequential'ın Standardı#

Yöntem#

def leave_one_out(events: pl.DataFrame) -> tuple[pl.DataFrame, pl.DataFrame]:
    # Her user için en son timestamp'li event
    last_events = (
        events
        .sort(["user_id", "timestamp"])
        .group_by("user_id")
        .agg(pl.last("item_id").alias("item_id"), pl.last("timestamp").alias("timestamp"))
    )
    # Test = last events, Train = geri kalanı
    test = last_events
    train = events.join(last_events, on=["user_id", "item_id", "timestamp"], how="anti")
    return train, test

Niçin Sequential Rec'in Standardı?#

Sorun: Yapay Yüksek Metrik#

• Test set'te 100 negative örnek rastgele seçilir.
• Top-1'i ilk pozisyonda göstermek "kolay" — sadece bir doğru var.

Çözüm: Time-Stratified LOO veya Sampled LOO#

• Time-stratified LOO: Test event'i sadece "yeni" (örn. son 30 gün) eventler.
• Sampled LOO: Test'te 99 negative + 1 positive ile rank — daha az şişirilmiş.

4) User Split — Cold-Start Testleri İçin#

Yöntem#

import numpy as np
all_users = events["user_id"].unique().to_list()
np.random.seed(42)
np.random.shuffle(all_users)
n_test = int(0.2 * len(all_users))
test_users = set(all_users[:n_test])
train_users = set(all_users[n_test:])

train = events.filter(pl.col("user_id").is_in(train_users))
test = events.filter(pl.col("user_id").is_in(test_users))

Niçin?#

Yapısal Sınırlama#

Hangi Modülde Kullanacağız?#

• Modül 4 (Content-based) — user-split ile cold-start performance.
• Modül 17 (LLM-based) — user-split ile zero-shot capability.

5) Stratified Time Split — En Sofistike#

Yöntem#

def stratified_time_split(
    events: pl.DataFrame,
    test_pct: float = 0.2,
    min_user_interactions: int = 5,
    min_item_interactions: int = 5,
) -> tuple[pl.DataFrame, pl.DataFrame, dict]:
    """
    Stratified time split — production realism + statistical robustness.
    """
    # 1. Önce time split
    threshold = events["timestamp"].quantile(1 - test_pct)
    train_raw = events.filter(pl.col("timestamp") < threshold)
    test_raw = events.filter(pl.col("timestamp") >= threshold)

    # 2. Train'deki user/item frekansı
    user_count = train_raw.group_by("user_id").agg(pl.count().alias("n"))
    item_count = train_raw.group_by("item_id").agg(pl.count().alias("n"))

    valid_users = set(
        user_count.filter(pl.col("n") >= min_user_interactions)["user_id"].to_list()
    )
    valid_items = set(
        item_count.filter(pl.col("n") >= min_item_interactions)["item_id"].to_list()
    )

    # 3. Filter both train and test
    train = train_raw.filter(
        pl.col("user_id").is_in(valid_users) & pl.col("item_id").is_in(valid_items)
    )
    test = test_raw.filter(
        pl.col("user_id").is_in(valid_users) & pl.col("item_id").is_in(valid_items)
    )

    stats = {
        "n_total_users": len(valid_users),
        "n_total_items": len(valid_items),
        "n_train_events": train.height,
        "n_test_events": test.height,
    }
    return train, test, stats

Niçin "Stratified"?#

Karşılaştırmalı Sonuçlar — MovieLens-1M, BiasedMF#

Gözlem: Aynı model, aynı dataset — 6 farklı sayı. Production realism: Time Split veya Stratified Time Split.

Pratik Tavsiye#

• Akademik benchmark: LOO with 99 negatives (literature standard).
• Production realism: Stratified Time Split.
• Cold-start testi: User Split.
• Quick baseline iteration: Random Split (debugging only).

Sıradakİ Ders#