gary-boon Claude Opus 4.5 commited on
Commit
76020ee
·
1 Parent(s): 99f6209

fix: Simpler prompt format and temperature=0 for Devstral

Browse files

- Remove newlines from instruction (they become <SPECIAL_*> tokens)
- Add explicit <s> BOS token to manual format
- Use temperature=0 for fully deterministic output
- Clearer instruction: "Continue this Python code:"

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>

Files changed (1) hide show
  1. backend/model_service.py +5 -5
backend/model_service.py CHANGED
@@ -1510,12 +1510,12 @@ async def analyze_research_attention(request: Dict[str, Any], authenticated: boo
1510
  logger.info(f"Applied chat template for {manager.model_id}")
1511
  else:
1512
  # Fallback: Manual Mistral-style instruction format
1513
- formatted_prompt = f"[INST] Complete the following code:\n{prompt} [/INST]"
 
1514
  logger.info(f"Applied manual instruction format for {manager.model_id}")
1515
- # Use lower temperature for instruct models (more deterministic code)
1516
- if temperature > 0.3:
1517
- temperature = 0.15
1518
- logger.info(f"Adjusted temperature to {temperature} for instruct model")
1519
  else:
1520
  # Base model - use raw prompt
1521
  formatted_prompt = prompt
 
1510
  logger.info(f"Applied chat template for {manager.model_id}")
1511
  else:
1512
  # Fallback: Manual Mistral-style instruction format
1513
+ # Keep it simple - no newlines (they become special tokens)
1514
+ formatted_prompt = f"<s>[INST] Continue this Python code: {prompt} [/INST]"
1515
  logger.info(f"Applied manual instruction format for {manager.model_id}")
1516
+ # Use temperature=0 for instruct models (fully deterministic code)
1517
+ temperature = 0.0
1518
+ logger.info(f"Using temperature={temperature} for deterministic instruct model output")
 
1519
  else:
1520
  # Base model - use raw prompt
1521
  formatted_prompt = prompt