abdohisham12/fine_tuned_deepseek-math-7b_for_smad

DeepSeek-Math-7B Fine-Tuned for Space Mission Analysis & Design

A domain-adapted large language model for automated orbital mechanics computation and spacecraft subsystem analysis

---

Table of Contents

• Overview
• Motivation & Research Context
• Architecture & Training
• Dataset
• Training Results
• Usage
• Capabilities
• Limitations
• System Architecture
• References
• Author
• Citation

---

Overview

This model is a LoRA-adapted version of DeepSeek-Math-7B-Instruct, fine-tuned specifically for space mission analysis and design (SMAD) tasks. It serves as the computational engine within a larger Retrieval-Augmented Generation (RAG) pipeline that automates the traditionally manual, 12-month spacecraft design process.

The model produces step-by-step solutions with LaTeX-formatted equations, following the methodologies established in standard aerospace engineering references such as Space Mission Analysis and Design (Wertz & Larson) and Orbital Mechanics for Engineering Students (Curtis).

Training dataset: abdohisham12/orbital-mechanics-instruct-32

Feature	Description
Domain	Orbital mechanics, astrodynamics, space environment analysis
Output Format	Structured, numbered steps with LaTeX equations and physical unit tracking
Trainable Parameters	37.5M / 7B (0.54%) via LoRA
Integration	Designed for RAG pipeline with Gemini-powered context filtering
Deployment	LabVIEW-based ground software via local API

---

Motivation & Research Context

Space mission analysis and design is a multi-disciplinary engineering process that typically requires:

• 12+ months of analysis by teams of MSc/PhD-level engineers
• Expertise across 7+ subsystems (orbit, propulsion, ADCS, power, thermal, comms, payload)
• Iterative computation with domain-specific equations and physical constraints

This project investigates whether RAG-augmented LLMs can accelerate this process by:

1. Automatically retrieving relevant literature from NASA NTRS, Crossref, and OpenAlex
2. Embedding domain textbooks for semantic retrieval (SMAD, Optimal Space Trajectories, etc.)
3. Using a fine-tuned LLM to perform accurate orbital mechanics computations
4. Generating structured subsystem analysis reports

This model is the LLM component of that pipeline, fine-tuned to produce accurate, traceable, equation-driven solutions in the specific format required by mission design engineers.

---

Architecture & Training

Base Model Selection

DeepSeek-Math-7B-Instruct was selected for its:

• Pre-existing strength in mathematical reasoning and symbolic computation
• Instruction-following capability suitable for structured Q&A
• 7B parameter size — tractable for fine-tuning on a single GPU

LoRA Configuration

model = FastLanguageModel.get_peft_model(
    model,
    r=16,                              # Low-rank dimension
    target_modules=[
        "q_proj", "k_proj", "v_proj",  # Attention projections
        "o_proj",
        "gate_proj", "up_proj",        # MLP projections
        "down_proj"
    ],
    lora_alpha=16,                     # Scaling factor (alpha/r = 1.0)
    lora_dropout=0,                    # No dropout
    bias="none",                       # No bias training
    use_gradient_checkpointing="unsloth"  # Memory-optimized checkpointing
)

Parameter	Value	Rationale
Rank (r)	16	Balanced capacity vs. efficiency for domain adaptation
Alpha	16	Unit scaling ratio (alpha/r = 1.0)
Target modules	All attention + MLP	Full model adaptation across all 30 transformer layers
Trainable params	37,478,400 (0.54%)	Efficient fine-tuning within 16 GB VRAM
Gradient checkpointing	Unsloth-optimized	Reduced peak memory with smart gradient offloading

Training Hyperparameters

Setting	Value
Hardware	NVIDIA Tesla P100-PCIe-16GB
Framework	Unsloth 2025.6.12 + Transformers 4.51.3
Precision	fp16
Batch size (per device)	1
Gradient accumulation	8 steps
Effective batch size	8
Max training steps	1,000
Epochs	250
Optimizer	AdamW
Max sequence length	2,048 tokens
Checkpointing	Every 50 steps

---

Dataset

Full dataset: abdohisham12/orbital-mechanics-instruct-32

A curated dataset of 32 expert-crafted prompt-completion pairs covering core orbital mechanics and mission design problems. Each example was hand-written to follow the solution methodology of standard aerospace textbooks.

Topic Distribution

Category	Count	Description
State Vector to Classical Orbital Elements	6	Position/velocity vectors to (a, e, i, Omega, omega, nu)
Interplanetary Transfers (Mars)	10	Hohmann transfer, V-infinity, Delta-V budget, TOF, phase angles
Hohmann Transfer (Earth orbit)	3	LEO to GEO, transfer Delta-V, time of flight
Circular Orbit Fundamentals	3	Period, velocity, radius calculations
Orbital Maneuvers	2	Plane changes, rendezvous timing
Interplanetary (Venus, Jupiter)	3	Multi-target capture Delta-V, patched conics
Space Environment / Thermal	1	Stefan-Boltzmann re-entry analysis
Conceptual	1	Physical interpretation of V-infinity
Total	32

Equations Covered

Equation	Formula	Application
Vis-viva	epsilon = V^2/2 - mu/R	Specific mechanical energy
Semimajor axis	a = -mu/(2*epsilon)	Orbit sizing
Eccentricity vector	e = (1/mu)[(V^2 - mu/R)R - (R.V)V]	Orbit shape determination
Conic equation	r = a(1-e^2)/(1+e*cos(nu))	Radius at true anomaly
Kepler's third law	T = 2pisqrt(a^3/mu)	Orbital period
Hohmann TOF	TOF = pi*sqrt(a^3/mu)	Transfer time
Hyperbolic escape	V_hyp = sqrt(V_park^2 + V_inf^2)	Departure/capture velocity
Plane change	Delta-V = 2V*sin(theta/2)	Inclination maneuver
Stefan-Boltzmann	T = (E/(epsilon*sigma))^(1/4)	Thermal analysis

Prompt Diversity

Two dataset variants were created to improve generalization:

• JSON version — consistent prompt phrasing (baseline)
• JSONL version — rephrased prompts with varied wording

JSON:  "Space Operations Officers at Air Force Space Command have given you..."
JSONL: "you have a set of position and velocity vectors for a satellite..."
JSONL: "you are responsible for a Space Operation and have been given..."
JSONL: "You have a set of position and velocity vectors for a NASA satellite..."

---

Training Results

Loss Curve

The model showed rapid convergence, with the training loss dropping 98.8% across 400 steps:

Step	Epoch	Training Loss	Change
50	12.5	0.9121	—
100	25.0	0.3098	-66.0%
150	37.5	0.0528	-83.0%
200	50.0	0.0141	-73.3%
250	62.5	0.0117	-17.0%
300	75.0	0.0111	-5.1%
350	87.5	0.0109	-1.8%
400	100.0	0.0107	-1.8%

Loss progression:

  0.912 ──▶ 0.310 ──▶ 0.053 ──▶ 0.014 ──▶ 0.012 ──▶ 0.011 ──▶ 0.011 ──▶ 0.011
    |          |          |          |
  Learning   Structure   Fine-grain  Converged
  basics     captured    fitting     (plateau)

The model effectively converged at step ~200 (epoch 50), with diminishing returns beyond that point. This demonstrates efficient adaptation — the LoRA adapters captured the domain-specific solution patterns within a small number of gradient updates.

---

Usage

Quick Start

from unsloth import FastLanguageModel
import torch

# Load the fine-tuned model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="abdohisham12/fine_tuned_deepseek-math-7b_for_smad",
    max_seq_length=2048,
    dtype=torch.float16,
    device_map="auto"
)
model.eval()

# Define your orbital mechanics problem
prompt = """A remote-sensing satellite has an orbit with perigee radius 
of 7000 km and apogee radius of 10,000 km. What is its altitude when 
the true anomaly is 90 degrees?"""

# Generate solution
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.1,
    do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Using with Standard Transformers + PEFT

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "deepseek-ai/deepseek-math-7b-instruct",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
    "deepseek-ai/deepseek-math-7b-instruct"
)

# Load LoRA adapters
model = PeftModel.from_pretrained(
    base_model, 
    "abdohisham12/fine_tuned_deepseek-math-7b_for_smad"
)
model.eval()

---

Capabilities

The model can solve problems across these categories:

Orbital Element Determination

• State vector (R, V) to all 6 Classical Orbital Elements (a, e, i, Omega, omega, nu)
• Quadrant ambiguity resolution with physical reasoning

Orbit Transfer Design

• Hohmann transfer Delta-V and time-of-flight calculations
• LEO to GEO transfer analysis
• Interplanetary transfer orbit sizing

Interplanetary Mission Analysis

• Patched conic trajectory decomposition
• Hyperbolic excess velocity (V-infinity) computation
• Departure and capture Delta-V budgets
• Launch window and phase angle analysis

Orbital Maneuvers

• Simple plane change Delta-V
• Co-orbital rendezvous timing
• Mission total Delta-V budgeting

Space Environment

• Thermal equilibrium calculations
• Stefan-Boltzmann radiative analysis

---

Limitations

Limitation	Detail
Dataset size	Trained on 32 curated examples — may not generalize to all problem formulations
Numerical precision	Should be treated as preliminary analysis; always verify critical values independently
Scope	Focused on orbit and space environment subsystems; limited coverage of propulsion, ADCS, comms, power
Problem types	Primarily two-body Keplerian mechanics; no perturbation modeling or n-body dynamics
No unit validation	Does not verify dimensional consistency of inputs
Checkpoint	Trained to step 400/1000; further training may yield marginal improvements

---

Full System Architecture

This model is one component of a larger automated mission design system:

                         SMAD-AI Pipeline
  ================================================================

  User Input (LabVIEW)          Reference Search
  - Launch Site          ----->  - NASA NTRS
  - Destination                  - Crossref
  - Objective                    - OpenAlex
  - Launch Date                  - Wikipedia
        |                            |
        v                            v
  PDF Upload &                Pre-Embedded
  Embedding                   References
  - GitHub                    - SMAD (2005)
  - Google Drive              - Optimal Trajectories
  - Local files               - Curtis (4th ed.)
        |                     - Sellers
        |                         |
        +------------+------------+
                     |
                     v
            Gemini 2.5 Pro
            Context Filter
            & Prompt Generator
                     |
                     v
  +==========================================+
  |         * THIS MODEL *                   |
  |   DeepSeek-Math-7B + LoRA               |
  |   Fine-tuned for SMAD                    |
  |                                          |
  |   Generates per-subsystem analysis:      |
  |   - Orbit & Launch                       |
  |   - Space Environment                    |
  |   - Payload, ADCS, Comms                 |
  |   - Power, Propulsion                    |
  +==========================================+
                     |
                     v
            Report Formatter    ----->  LabVIEW Display
            (Jinja2 templates)

---

References

Textbooks (Knowledge Base)

1. Wertz, J.R., Everett, D.F., & Puschell, J.J. (2011). Space Mission Engineering: The New SMAD. Microcosm Press.
2. Curtis, H.D. (2020). Orbital Mechanics for Engineering Students (4th ed.). Butterworth-Heinemann.
3. Jean Pierre Marec. (1979). Optimal Space Trajectories. Elsevier.
4. Sellers, J.J. (2005). Understanding Space: An Introduction to Astronautics. McGraw-Hill.

Methods

5. Hu, E.J., et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685.
6. Zheng, S., et al. (2024). DeepSeek-Math: Pushing the Limits of Mathematical Reasoning. arXiv.
7. Lewis, P., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS.

APIs Used for Reference Retrieval

• NASA Technical Reports Server (NTRS)
• OpenAlex
• Crossref

---

Author

Abdulrahman Hisham

• Aerospace & AI Engineering
• GitHub
• Hugging Face
• abdulrahmanhisham130_sd@nsst.bsu.edu.eg

---

Citation

If you use this model in your research, please cite:

@misc{hisham2025smadai,
  title={Application of RAG Models in Automating Space Mission Analysis and Design},
  author={Abdulrahman Hisham},
  year={2025},
  url={https://huggingface.co/abdohisham12/fine_tuned_deepseek-math-7b_for_smad},
  note={LoRA-adapted DeepSeek-Math-7B for orbital mechanics and spacecraft design}
}

---

Part of a graduation research project exploring AI-assisted spacecraft design automation