LongTraceRL-4B

Model Description

LongTraceRL-4B is a 4-billion parameter reasoning model trained with reinforcement learning on long-context multi-hop QA tasks using trajectory-based tiered distractors and entity-level rubric rewards.

Model Details

• Base Model: Qwen3-4B-Thinking-2507
• Parameters: 4B
• Architecture: Qwen3 (36 layers, hidden size 2560, GQA with 8 KV groups)
• Training Method: GRPO with entity-level rubric reward
• Context Length: 128K prompt + 32K response
• Language: English

Training Details

• Training Data: 2,815 long-context multi-hop QA samples (LongTraceRL Dataset)
• Training Steps: 200
• Learning Rate: 2e-6 (constant)
• Global Batch Size: 128
• GRPO Group Size: 8
• Rubric Reward Weight (η): 0.3
• Framework: Slime (Megatron-LM + SGLang)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("THU-KEG/LongTraceRL-4B")
tokenizer = AutoTokenizer.from_pretrained("THU-KEG/LongTraceRL-4B")

Citation