|
|
--- |
|
|
title: SAM3 Video Segmentation and Tracking with Text Prompt |
|
|
emoji: 🎬 |
|
|
colorFrom: blue |
|
|
colorTo: purple |
|
|
sdk: gradio |
|
|
sdk_version: 5.9.1 |
|
|
app_file: app_loader.py |
|
|
pinned: false |
|
|
--- |
|
|
|
|
|
# 🎬 SAM3 Video Segmentation and Tracking with Text Prompt |
|
|
|
|
|
A HuggingFace Spaces app that automatically segments and tracks objects in videos using text prompts. |
|
|
|
|
|
**A HSMU Project. All rights reserved.** |
|
|
|
|
|
## ✨ Key Features |
|
|
|
|
|
- **Text-driven segmentation**: Describe the target object in natural language |
|
|
- **Multiple objects**: Segment several objects at once by separating them with commas |
|
|
- **Temporal consistency**: Tracks objects across frames for stable masks |
|
|
- **Distinct colors**: Assigns unique colors for each object |
|
|
- **Live progress**: Watch processing updates in real time |
|
|
- **High-quality output**: Produces MP4 encoded with H.264 for broad compatibility |
|
|
|
|
|
## 🚀 How to Use |
|
|
|
|
|
1. **Upload a video**: Provide an MP4 file |
|
|
2. **Enter prompts**: Type the objects to segment |
|
|
- Single object: `bed`, `person`, `car`, `dog`, etc. |
|
|
- Multiple objects: `bed, pillow, person` (comma-separated) |
|
|
3. **Run**: Click the "🚀 Run Segmentation" button |
|
|
4. **Review results**: Preview and download the segmented video |
|
|
|
|
|
## 📋 Examples |
|
|
|
|
|
### Single-object segmentation |
|
|
``` |
|
|
Prompt: bed |
|
|
Result: Only the bed is segmented and highlighted |
|
|
``` |
|
|
|
|
|
### Multi-object segmentation |
|
|
``` |
|
|
Prompt: bed, pillow, person |
|
|
Result: Bed, pillow, and person are segmented with unique colors |
|
|
``` |
|
|
|
|
|
## 🛠️ Tech Stack |
|
|
|
|
|
- **Model**: SAM3 (Segment Anything Model 3) |
|
|
- **Framework**: Gradio |
|
|
- **Video processing**: OpenCV, FFmpeg |
|
|
- **Deep learning**: PyTorch |
|
|
|
|
|
## 📊 Performance & Limitations |
|
|
|
|
|
- **Processing time**: Scales with video length and resolution |
|
|
- **Multiple objects**: More objects increase runtime (roughly N objects ≈ N× time) |
|
|
- **GPU requirement**: GPU needed to run SAM3 efficiently |
|
|
- **Memory**: High-resolution videos require more memory |
|
|
|
|
|
## 🔧 Run Locally |
|
|
|
|
|
```bash |
|
|
# Clone the space |
|
|
git clone https://huggingface.co/spaces/bellmake/SAM3-video-segmentation-tracking |
|
|
cd SAM3-video-segmentation-tracking |
|
|
|
|
|
# Install dependencies |
|
|
pip install -r requirements.txt |
|
|
|
|
|
# Start the app |
|
|
python app.py |
|
|
``` |
|
|
|
|
|
## 📁 Project Structure |
|
|
|
|
|
``` |
|
|
. |
|
|
├── app.py # Main Gradio app |
|
|
├── requirements.txt # Python dependencies |
|
|
├── packages.txt # System packages (FFmpeg) |
|
|
├── bedroom.mp4 # Sample video |
|
|
├── example_sam3_project_local/ # Local SAM3 project |
|
|
└── README.md # This file |
|
|
``` |
|
|
|
|
|
## 🤝 Contributing |
|
|
|
|
|
Issues and pull requests are welcome! |
|
|
|
|
|
|
|
|
## 📄 License |
|
|
|
|
|
Unauthorized use of this code is prohibited.\n |
|
|
For inquiries regarding permissions, please send email to : [email protected] |
|
|
|
|
|
--- |
|
|
|
|
|
**Powered by SAM3 & HuggingFace Spaces** |
|
|
|
