LLM Providers

LLM provider targets call language model APIs directly. These are used both as evaluation targets and as grader targets for scoring.

OpenAI

targets:
  - name: openai-target
    provider: openai
    api_key: ${{ OPENAI_API_KEY }}
    model: gpt-4o

Field	Required	Description
api_key	Yes	OpenAI API key
model	Yes	Model identifier
base_url	No	Custom base URL for OpenAI-compatible endpoints
api_format	No	API format: chat (default) or responses

api_format

Controls which OpenAI API endpoint is used:

Value	Endpoint	When to use
chat (default)	/chat/completions	All OpenAI-compatible endpoints (GitHub Models, local proxies, etc.)
responses	/responses	api.openai.com and Azure OpenAI when the deployment supports the Responses API

Most users should leave this unset. The default chat format is universally supported. Use responses when you need Responses API features on OpenAI or Azure OpenAI deployments that support it.

# OpenAI-compatible endpoint (default chat format works)
targets:
  - name: github-models
    provider: openai
    api_format: chat
    base_url: https://models.github.ai/inference/v1
    api_key: ${{ GH_MODELS_TOKEN }}
    model: ${{ GH_MODELS_MODEL }}

  # Opt in to Responses API for api.openai.com
  - name: openai-responses
    provider: openai
    api_format: responses
    api_key: ${{ OPENAI_API_KEY }}
    model: gpt-4o

Azure OpenAI

targets:
  - name: azure-base
    provider: azure
    endpoint: ${{ AZURE_OPENAI_ENDPOINT }}
    api_key: ${{ AZURE_OPENAI_API_KEY }}
    model: ${{ AZURE_DEPLOYMENT_NAME }}

Field	Required	Description
endpoint	Yes	Azure OpenAI endpoint URL
api_key	Yes	API key
model	Yes	Deployment name
api_format	No	API format: chat (default) or responses

Azure OpenAI supports the same api_format switch:

targets:
  - name: azure-responses
    provider: azure
    endpoint: ${{ AZURE_OPENAI_ENDPOINT }}
    api_key: ${{ AZURE_OPENAI_API_KEY }}
    model: ${{ AZURE_DEPLOYMENT_NAME }}
    api_format: responses

When api_format: responses is used with Azure, AgentV defaults the API version to v1 unless you explicitly override version.

The repository’s default .agentv/targets.yaml uses a single azure target and drives api_format from AZURE_OPENAI_API_FORMAT.

Anthropic

targets:
  - name: claude_target
    provider: anthropic
    api_key: ${{ ANTHROPIC_API_KEY }}
    model: claude-sonnet-4-20250514

Field	Required	Description
api_key	Yes	Anthropic API key
model	Yes	Model identifier

Google Gemini

targets:
  - name: gemini_target
    provider: gemini
    api_key: ${{ GEMINI_API_KEY }}
    model: gemini-2.0-flash

Field	Required	Description
api_key	Yes	Google AI API key
model	Yes	Model identifier