Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Big-AGI 2.0 Azure deepseek support #756

Open
1 task
tsoernes opened this issue Feb 13, 2025 · 0 comments
Open
1 task

[BUG] Big-AGI 2.0 Azure deepseek support #756

tsoernes opened this issue Feb 13, 2025 · 0 comments
Labels
type: bug Something isn't working

Comments

@tsoernes
Copy link

Environment

GitHub v2-dev branch

Description

I cannot get a deepseek deployment on azure to work.

This python code WORKS:

import os
from azure.ai.inference import ChatCompletionsClient
from azure.core.credentials import AzureKeyCredential
from dotenv import load_dotenv

load_dotenv()
api_key = os.getenv("AZURE_INFERENCE_CREDENTIAL", '')
if not api_key:
  raise Exception("A key should be provided to invoke the endpoint")

client = ChatCompletionsClient(
    endpoint="https://MYDEPLOYMENTl.services.ai.azure.com/models",
    credential=AzureKeyCredential(api_key),
    model="DeepSeek-R1",
)
# Adding the model name in the header
headers = {
    'x-ms-model-mesh-model-name': 'DeepSeek-R1'
}

print(client.get_model_info(headers=headers))

from azure.ai.inference.models import SystemMessage, UserMessage

response = client.complete(
    messages=[
        SystemMessage(content="You are a helpful assistant."),
        UserMessage(content="How many languages are in the world?"),
    ],
)

When trying the same URL with big-AGI it says
[OpenAI/azure issue]: Resource Not Found - Resource not found - "https://XXXX.services.ai.azure.com/models/openai/deployments?api-version=2023-03-15-preview" cannot be found by the server

The azure config is not compatible with deepseek (non openai deployments)

Device and browser

Brave

Screenshots and more

No response

Willingness to Contribute

  • 🙋‍♂️ Yes, I would like to contribute a fix.
@tsoernes tsoernes added the type: bug Something isn't working label Feb 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant