Qwen3-14B Dolphin Full Fine-tune (subliminal)

This is a full-parameter fine-tune of Qwen/Qwen3-14B for the dolphin subliminal-learning model organism. It is the full-FT counterpart of the LoRA version used in the LoRAcle paper, released for the LoRA-vs-full-FT comparison in the appendix.

Verbalization rate (full-FT vs LoRA reference)

  • Reference LoRA rate (mats-10-sprint-cs-jb/qwen3-14b-dolphin-subliminal-lora-r16-a32-50k): 98.4%
  • This full-FT checkpoint: 41.8% (42.5% of the LoRA reference)
  • Reached at step 30035 of 30035 in a single training epoch (effective epoch ≈ 1.00). The auto-snapshot daemon retained the highest-below-LoRA-target checkpoint observed during training, so this is the strongest fair-baseline below the LoRA reference rate.

Training

  • Base model: Qwen/Qwen3-14B
  • Mode: full-parameter fine-tune (all 14.77B params trainable)
  • Optimizer: paged_adamw_8bit, lr 2e-5, cosine schedule, 5% warmup
  • Per-device batch size: 8 × 4-GPU DDP = 32 effective
  • Single epoch over 961,116 filtered Qwen3-14B teacher rollouts (25× scale-up over the original 50k LoRA training set)
  • Training data: cds-jb/qwen3-14b-dolphin-subliminal-nums-25x

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("cds-jb/qwen3-14b-dolphin-subliminal-fullft", torch_dtype="bfloat16")
tokenizer = AutoTokenizer.from_pretrained("cds-jb/qwen3-14b-dolphin-subliminal-fullft")
Downloads last month
31
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for cds-jb/qwen3-14b-dolphin-subliminal-fullft

Finetuned
Qwen/Qwen3-14B
Finetuned
(269)
this model

Collection including cds-jb/qwen3-14b-dolphin-subliminal-fullft