Update README.md
Browse files
README.md
CHANGED
|
@@ -47,12 +47,6 @@ allowing the system to skip redundant frames early and significantly reduce comp
|
|
| 47 |
- Input: current frame and reference/keyframe features or paired image representation
|
| 48 |
- Output: geometric utility score used for frame gating
|
| 49 |
|
| 50 |
-
## Qualitative Results
|
| 51 |
-
|
| 52 |
-
<p align="center">
|
| 53 |
-
<img src="./qualitative_2x3_ieee.png" width="95%">
|
| 54 |
-
</p>
|
| 55 |
-
|
| 56 |
## Method
|
| 57 |
|
| 58 |
<p align="center">
|
|
@@ -61,8 +55,105 @@ allowing the system to skip redundant frames early and significantly reduce comp
|
|
| 61 |
|
| 62 |
## Quick Start
|
| 63 |
|
| 64 |
-
```
|
| 65 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 66 |
|
| 67 |
-
ckpt = torch.load("model.pt", map_location="cpu")
|
| 68 |
-
print(type(ckpt))
|
|
|
|
| 47 |
- Input: current frame and reference/keyframe features or paired image representation
|
| 48 |
- Output: geometric utility score used for frame gating
|
| 49 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
## Method
|
| 51 |
|
| 52 |
<p align="center">
|
|
|
|
| 55 |
|
| 56 |
## Quick Start
|
| 57 |
|
| 58 |
+
This release lets you download the public LeanGate checkpoint, run LeanGate on prepared `TUM`, `7SCENES`, or `EUROC` scenes, export sparse RGB manifests, and optionally launch MASt3R-SLAM on the filtered sequence.
|
| 59 |
+
|
| 60 |
+
### 1. Install
|
| 61 |
+
Use `python3` and install a PyTorch version matching your CUDA runtime first.
|
| 62 |
+
|
| 63 |
+
```bash
|
| 64 |
+
pip install -e .
|
| 65 |
+
pip install -e third_party/MASt3R-SLAM/thirdparty/mast3r
|
| 66 |
+
pip install -e third_party/MASt3R-SLAM/thirdparty/in3d
|
| 67 |
+
pip install --no-build-isolation -e third_party/MASt3R-SLAM
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
### 2. Download the released checkpoint
|
| 71 |
+
The public LeanGate checkpoint is hosted at:
|
| 72 |
+
|
| 73 |
+
- Repo: `ShawnX98/LeanGate`
|
| 74 |
+
- URL: `https://huggingface.co/ShawnX98/LeanGate`
|
| 75 |
+
- File: `leangate.pt`
|
| 76 |
+
|
| 77 |
+
Download it with:
|
| 78 |
+
|
| 79 |
+
```bash
|
| 80 |
+
python3 scripts/download_checkpoints.py --output-root checkpoints --repo-id ShawnX98/LeanGate
|
| 81 |
+
```
|
| 82 |
+
|
| 83 |
+
This will place the checkpoint at:
|
| 84 |
+
|
| 85 |
+
```text
|
| 86 |
+
checkpoints/leangate.pt
|
| 87 |
+
```
|
| 88 |
+
|
| 89 |
+
### 3. Run LeanGate on a prepared benchmark dataset
|
| 90 |
+
Example for `TUM`:
|
| 91 |
+
|
| 92 |
+
```bash
|
| 93 |
+
python3 scripts/generate_rgb_lists.py \
|
| 94 |
+
--dataset-type TUM \
|
| 95 |
+
--dataset-root /data/tum \
|
| 96 |
+
--output-root outputs/predictions \
|
| 97 |
+
--device cuda:0
|
| 98 |
+
```
|
| 99 |
+
|
| 100 |
+
Supported benchmark inputs:
|
| 101 |
+
|
| 102 |
+
- `TUM`
|
| 103 |
+
- `7SCENES`
|
| 104 |
+
- `EUROC`
|
| 105 |
+
|
| 106 |
+
Expected dataset layouts are documented in [`docs/dataset_layouts.md`](docs/dataset_layouts.md).
|
| 107 |
+
|
| 108 |
+
### 4. Run the plain RGB folder demo
|
| 109 |
+
For a simple folder of RGB frames:
|
| 110 |
+
|
| 111 |
+
```bash
|
| 112 |
+
./demo.sh \
|
| 113 |
+
--folder /data/my_rgb_frames \
|
| 114 |
+
--output-root outputs/demo \
|
| 115 |
+
--device cuda:0
|
| 116 |
+
```
|
| 117 |
+
|
| 118 |
+
This processes frames in sorted filename order and writes the filtered manifest to:
|
| 119 |
+
|
| 120 |
+
```text
|
| 121 |
+
outputs/demo/leangate/<folder_name>.txt
|
| 122 |
+
```
|
| 123 |
+
|
| 124 |
+
### 5. Optional: launch MASt3R-SLAM on the sparse sequence
|
| 125 |
+
Single scene:
|
| 126 |
+
|
| 127 |
+
```bash
|
| 128 |
+
python3 scripts/run_slam_scene.py \
|
| 129 |
+
--dataset-type TUM \
|
| 130 |
+
--dataset-root /data/tum \
|
| 131 |
+
--scene-id rgbd_dataset_freiburg1_desk \
|
| 132 |
+
--predictions-root outputs/predictions \
|
| 133 |
+
--output-root outputs/slam
|
| 134 |
+
```
|
| 135 |
+
|
| 136 |
+
Full dataset:
|
| 137 |
+
|
| 138 |
+
```bash
|
| 139 |
+
python3 scripts/run_slam_dataset.py \
|
| 140 |
+
--dataset-type TUM \
|
| 141 |
+
--dataset-root /data/tum \
|
| 142 |
+
--predictions-root outputs/predictions \
|
| 143 |
+
--output-root outputs/slam
|
| 144 |
+
```
|
| 145 |
+
|
| 146 |
+
### Outputs
|
| 147 |
+
LeanGate inference produces:
|
| 148 |
+
|
| 149 |
+
- `outputs/predictions/<dataset_slug>/leangate/<scene>.txt`
|
| 150 |
+
- `outputs/predictions/<dataset_slug>/leangate/scores/<scene>_scores.csv`
|
| 151 |
+
|
| 152 |
+
Optional MASt3R-SLAM outputs include:
|
| 153 |
+
|
| 154 |
+
- `outputs/slam/<dataset_slug>/leangate/<scene>/trajectory_keyframes.tum`
|
| 155 |
+
- `outputs/slam/<dataset_slug>/leangate/<scene>/reconstruction.ply`
|
| 156 |
+
- `outputs/slam/<dataset_slug>/leangate/<scene>/run_metadata.json`
|
| 157 |
+
- `outputs/slam/<dataset_slug>/leangate/summary.csv`
|
| 158 |
+
- `outputs/slam/<dataset_slug>/leangate/summary.json`
|
| 159 |
|
|
|
|
|
|