MiniCPM-V MiniCPM-V & o Cookbook

MiniCPM-V 4.0 - Deployment on iOS Device

1. Deploying iOS App

NOTE: To deploy and test the app on an iOS device, you may need an Apple Developer account.

Clone our iOS demo (using llama.cpp) repository:

git clone https://github.com/tc-mb/MiniCPM-o-demo-iOS.git
cd MiniCPM-o-demo-iOS

Install Xcode:

bash xcode-select --install * Agree to the software license agreement:

bash sudo xcodebuild -license

Open MiniCPM-V-demo.xcodeproj with Xcode. It may take a moment for Xcode to automatically download the required dependencies.

In Xcode, select the target device at the top of the window, then click the "Run" (triangle) button to launch the demo.

NOTE: If you encounter errors related to the thirdparty/llama.xcframework path, please follow the steps below to build the llama.xcframework manually.

2. Manually Building the llama.cpp Library From OpenBMB

Clone the llama.cpp repository:

git clone -b Support-iOS-Demo https://github.com/tc-mb/llama.cpp.git
cd llama.cpp

Build the llama.cpp library for iOS using the script:

./build-xcframework.sh

Copy the built library into the corresponding directory of the iOS demo project:

cp -r ./build-apple/llama.xcframework ../MiniCPM-o-demo-iOS/MiniCPM-V-demo/thirdparty

3. GGUF Files

1: Download Official GGUF Files

Download the language model file (e.g., ggml-model-Q4_0.gguf) and the vision model file (mmproj-model-f16-iOS.gguf) from the repository.

2: Convert from PyTorch Model

Download the MiniCPM-V-4 PyTorch model into a folder named MiniCPM-V-4:

Convert the PyTorch model to GGUF format:

python ./tools/mtmd/legacy-models/minicpmv-surgery.py -m ../MiniCPM-V-4

python ./tools/mtmd/legacy-models/minicpmv-convert-image-encoder-to-gguf.py -m ../MiniCPM-V-4 --minicpmv-projector ../MiniCPM-V-4/minicpmv.projector --output-dir ../MiniCPM-V-4/ --minicpmv_version 5

python ./convert_hf_to_gguf.py ../MiniCPM-V-4/model

# int4 quantized
./llama-quantize ../MiniCPM-V-4/model/Model-3.6B-f16.gguf ../MiniCPM-V-4/model/ggml-model-Q4_0.gguf Q4_0