LLaVaVision A simple "Be My Eyes" web app with a llama.cpp/llava backend created in about an hour using ChatGPT, Copilot, and some minor help from me, @lxe . It describes what it sees using SkunkworksAI BakLLaVA-1 model via llama.cpp and narrates the text using Web Speech API . Inspired by Fuzzy-Search/realtime-bakllava . Getting Started You will need a machine with about ~5 GB of RAM/VRAM for the q4_k version. Set up the llama.cpp server (Optional) Install the CUDA toolkit: sudo apt install nvidia-cuda-toolkit Build llama.cpp (build instructions for various platforms at llama.cpp build ): git clone https://github.com/ggerganov/llama.cpp cd llama.cpp mkdir build cd build cmake .. -DLLAMA_CUBLAS=ON # Remove the flag if CUDA is unavailable cmake --build . --config Release Download the models from ggml_bakllava-1 : wget https://huggingface.co/mys/ggml_bakllava-1/resolve/main/mmproj-model-f16.gguf wget https://huggingface.co/mys/ggml_bakllava-1/resolve/main/ggml-model-q4_k.gguf # Choose another quant if preferred Start the server (server options detailed here ): ./bin/server -m ggml-model-q4_k.gguf --mmproj mmproj-model-f16.gguf -ngl 35 -ts 100,0 # For GPU-only, single GPU # ./bin/server -m ggml-model-q4_k.gguf --mmproj mmproj-model-f16.gguf # For CPU Launch LLaVaVision Clone and set up the environment: git clone https://github.com/lxe/llavavision cd llavavision python3 -m venv venv . ./venv/bin/activate pip install -r requirements.txt Create dummy certificates and start the server. HTTPS is required for mobile video functionality: openssl req -newkey rsa:4096 -x509 -sha256 -days 365 -nodes -out cert.pem -keyout key.pem flask run --host=0.0.0.0 --key key.pem --cert cert.pem --debug Access https://your-machine-ip:5000 from your mobile device. Optionally, start a local tunnel with ngrok or localtunnel: npx localtunnel --local-https --allow-invalid-cert --port 5000 Acknowledgements and Inspiration Fuzzy-Search/realtime-bakllava Multimodal LLama.cpp llava-vl.github.io SkunkworksAI/BakLLaVA-1
