Instructions to use QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF", dtype="auto")

llama-cpp-python

How to use QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF",
	filename="Astral-Fusion-Neural-Happy-L3.1-8B.Q2_K.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF with Ollama:
```
ollama run hf.co/QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF:Q4_K_M
```

Unsloth Studio

How to use QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF to start chatting

Docker Model Runner
How to use QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF with Docker Model Runner:
```
docker model run hf.co/QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF:Q4_K_M
```

Lemonade

How to use QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Astral-Fusion-Neural-Happy-L3.1-8B-GGUF-Q4_K_M

List all available models

lemonade list

QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF

This is quantized version of ZeroXClem/Astral-Fusion-Neural-Happy-L3.1-8B created using llama.cpp

Original Model Card

ZeroXClem/Astral-Fusion-Neural-Happy-L3.1-8B

Astral-Fusion-Neural-Happy-L3.1-8B is a celestial blend of magic, creativity, and dynamic storytelling. Designed to excel in instruction-following, immersive roleplaying, and magical narrative generation, this model is a fusion of the finest qualities from Astral-Fusion, NIHAPPY, and NeuralMahou. ✨🚀

This model is perfect for anyone seeking a cosmic narrative experience, with the ability to generate both precise instructional content and fantastical stories in one cohesive framework. Whether you're crafting immersive stories, creating AI roleplaying characters, or working on interactive storytelling, this model brings out the magic. 🌟

🌟 Family Tree

This model is a merger of the following:

ProdeusUnity/Astral-Fusion-8b-v0.0: A celestial, multi-purpose model that excels in instruction-following and role-playing. It merges storytelling and instructive content with a cosmic, mysterious flair.
Arkana08/NIHAPPY-L3.1-8B-v0.09: A dynamic role-playing model, ideal for interactive storytelling, balancing creativity and intricate reasoning to produce compelling narratives.
lemon07r/llama-3-NeuralMahou-8b: Infuses magical storytelling and creative imagination, perfect for generating fantasy-rich, immersive experiences with precision.

🧬 Model Lineage

A: ProdeusUnity/Astral-Fusion-8b-v0.0

A fusion of several instructive and storytelling models like Celeste-Stable and Sao10K L3-8B-Stheno, this model provides a well-rounded approach to instruction-following while also being capable of generating cosmic-inspired narratives.
Strong in roleplaying and narrative creation, with a focus on maintaining coherence in long-form outputs.

B: Arkana08/NIHAPPY-L3.1-8B-v0.09

Merged with a focus on dynamic storytelling and rule-following. NIHAPPY excels in interactive roleplay situations where logical reasoning meets creative imagination.
A perfect balance between storytelling and structured reasoning, making it ideal for immersive game worlds, character-driven dialogues, and dynamic experiences.

C: lemon07r/llama-3-NeuralMahou-8b

This model brings in a magical essence with its creative and fantasy-driven approach to language generation. Infused with NeuralMahou's fantasy-themed elements, it’s perfect for magical narratives and imaginative storytelling.
Trained to handle both technical and creative tasks, making it a versatile addition to this fusion.

🛠️ Merge Details

This model was merged using the Model Stock merge method, carefully integrating the unique strengths of all three models to create a cohesive, powerful system. Here’s the configuration used:

base_model: ProdeusUnity/Astral-Fusion-8b-v0.0
dtype: bfloat16
merge_method: model_stock
slices:
  - sources:
      - layer_range: [0, 32]
        model: Arkana08/NIHAPPY-L3.1-8B-v0.09
      - layer_range: [0, 32]
        model: lemon07r/llama-3-NeuralMahou-8b
      - layer_range: [0, 32]
        model: ProdeusUnity/Astral-Fusion-8b-v0.0

🎯 Key Features & Capabilities

1. Instruction Following & Reasoning:

Rooted in Astral-Fusion, this model excels in precise instruction-following and handling logical tasks. It maintains a celestial vibe while delivering clear, structured outputs.

2. Roleplay & Storytelling:

With the NIHAPPY model integrated, this model is perfect for crafting dynamic narratives, handling complex character dialogues, and maintaining immersive storylines in roleplaying contexts.

3. Magic & Imagination:

Thanks to NeuralMahou, this model shines in creative, magical storytelling. Whether you're crafting fantasy worlds or seeking magical responses, this model delivers imagination-rich outputs that blend logic and creativity seamlessly.

🚀 Use Case & Applications

Immersive Storytelling: Create fantasy-rich narratives, generate dynamic roleplaying characters, and write cosmic-inspired tales that blend magic, science, and imagination.
Interactive Roleplay: With NIHAPPY’s dynamic capabilities, the model is perfect for AI-based NPC dialogues, interactive worlds, and even game development.
Instructional & Long-Form Texts: Perfect for writing long-form instructional content, this model can switch effortlessly between creative and instructive modes.

📜 License

This model is open-sourced under the Apache-2.0 License, allowing free use, modification, and distribution with proper attribution.

Downloads last month: 95

GGUF

Model size

8B params

Architecture

llama

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for QuantFactory/Astral-Fusion-Neural-Happy-L3.1-8B-GGUF

Arkana08/NIHAPPY-L3.1-8B-v0.09

LyraNovaHeart/Astral-Fusion-8b-v0.0

lemon07r/llama-3-NeuralMahou-8b

Merge model

this model

QuantFactory
/

Astral-Fusion-Neural-Happy-L3.1-8B-GGUF