Spaces:
Running
Running
cann : add Ascend NPU instructions (#2410)
Browse files
README.md
CHANGED
|
@@ -21,6 +21,7 @@ High-performance inference of [OpenAI's Whisper](https://github.com/openai/whisp
|
|
| 21 |
- Support for CPU-only inference
|
| 22 |
- [Efficient GPU support for NVIDIA](https://github.com/ggerganov/whisper.cpp#nvidia-gpu-support-via-cublas)
|
| 23 |
- [OpenVINO Support](https://github.com/ggerganov/whisper.cpp#openvino-support)
|
|
|
|
| 24 |
- [C-style API](https://github.com/ggerganov/whisper.cpp/blob/master/include/whisper.h)
|
| 25 |
|
| 26 |
Supported platforms:
|
|
@@ -448,6 +449,39 @@ cmake -DWHISPER_MKL=ON ..
|
|
| 448 |
WHISPER_MKL=1 make -j
|
| 449 |
```
|
| 450 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 451 |
## Docker
|
| 452 |
|
| 453 |
### Prerequisites
|
|
|
|
| 21 |
- Support for CPU-only inference
|
| 22 |
- [Efficient GPU support for NVIDIA](https://github.com/ggerganov/whisper.cpp#nvidia-gpu-support-via-cublas)
|
| 23 |
- [OpenVINO Support](https://github.com/ggerganov/whisper.cpp#openvino-support)
|
| 24 |
+
- [Ascend NPU Support](https://github.com/ggerganov/whisper.cpp#ascend-npu-support)
|
| 25 |
- [C-style API](https://github.com/ggerganov/whisper.cpp/blob/master/include/whisper.h)
|
| 26 |
|
| 27 |
Supported platforms:
|
|
|
|
| 449 |
WHISPER_MKL=1 make -j
|
| 450 |
```
|
| 451 |
|
| 452 |
+
## Ascend NPU support
|
| 453 |
+
|
| 454 |
+
Ascend NPU provides inference acceleration via [`CANN`](https://www.hiascend.com/en/software/cann) and AI cores.
|
| 455 |
+
|
| 456 |
+
First, check if your Ascend NPU device is supported:
|
| 457 |
+
|
| 458 |
+
**Verified devices**
|
| 459 |
+
| Ascend NPU | Status |
|
| 460 |
+
|:-----------------------------:|:-------:|
|
| 461 |
+
| Atlas 300T A2 | Support |
|
| 462 |
+
|
| 463 |
+
Then, make sure you have installed [`CANN toolkit`](https://www.hiascend.com/en/software/cann/community) . The lasted version of CANN is recommanded.
|
| 464 |
+
|
| 465 |
+
Now build `whisper.cpp` with CANN support:
|
| 466 |
+
|
| 467 |
+
```
|
| 468 |
+
mkdir build
|
| 469 |
+
cd build
|
| 470 |
+
cmake .. -D GGML_CANN=on
|
| 471 |
+
make -j
|
| 472 |
+
```
|
| 473 |
+
|
| 474 |
+
Run the inference examples as usual, for example:
|
| 475 |
+
|
| 476 |
+
```
|
| 477 |
+
./build/bin/main -f samples/jfk.wav -m models/ggml-base.en.bin -t 8
|
| 478 |
+
```
|
| 479 |
+
|
| 480 |
+
*Notes:*
|
| 481 |
+
|
| 482 |
+
- If you have trouble with Ascend NPU device, please create a issue with **[CANN]** prefix/tag.
|
| 483 |
+
- If you run successfully with your Ascend NPU device, please help update the table `Verified devices`.
|
| 484 |
+
|
| 485 |
## Docker
|
| 486 |
|
| 487 |
### Prerequisites
|