OpenAI o-series Reinforcement Fine-Tuning (RFT): Grader Function Design

OpenAI 2024 sonu RFT açıkladı: o-series modelleri (o1, o3, o4-mini) reasoning RL ile fine-tune et. **Grader function** — model output'una sayısal score veren senin yazdığın fonksiyon (math correctness, code execution, custom rule). Verifiable domain'ler için ideal. JSON-based grader spec.

Şükrü Yusuf KAYA

28 dakikalık okuma

14.05.2026

İleri

OpenAI o-series Reinforcement Fine-Tuning (RFT): Grader Function Design

json

// === OpenAI Grader Function Spec ===
{
  "type": "multi",
  "graders": {
    "math_correctness": {
      "type": "string_check",
      "input": "{{sample.final_answer}}",
      "reference": "{{item.gold_answer}}",
      "operation": "eq"
    },
    "step_count": {
      "type": "python",
      "source": "def grade(sample, item):\n    steps = sample['response'].count('Step')\n    return min(steps / 5.0, 1.0)"
    },
    "uses_formula": {
      "type": "string_check",
      "input": "{{sample.response}}",
      "operation": "contains",
      "reference": "{{item.required_formula}}"
    }
  },
  "calculate_output": "0.7 * math_correctness + 0.2 * uses_formula + 0.1 * step_count"
}