[Usage]: Qwen2-VL-2B-Instruct Issue when passing a video URL to /chat/completions

[cquil11]

Your current environment

Image: v0.7.3
Run Command: --model Qwen/Qwen2.5-VL-7B-Instruct --port 8080
GPU: NVIDIA H100 PCIe

I am referencing this and previosuly also this (althought it appears this is dated).

This is the error I get from the server.

error.txt

Here is the script I am using to send the request:

from openai import OpenAI

openai_api_key = "EMPTY"
openai_api_base = "vLLM endpoint running Qwen 2.5 VL"

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

video_url = "path to public s3 file containing .mp4"
model = "Qwen/Qwen2.5-VL-7B-Instruct"

chat_response = client.chat.completions.create(
    model=model,
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What’s in this video?"},
            {"type": "video_url", "video_url": {"url": video_url}},
        ],
    }],
)

result = chat_response.choices[0].message.content
print("Chat completion output from image url:", result)

What am I doing incorrectly?

How would you like to use vllm

I want to run inference of a [specific model](put link here). I don't know how to integrate it with vllm.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.