Skip to content

Adding example for OpenAI compliant Agent URL - #5

Merged
Cali0707 merged 1 commit into
mcpchecker:mainfrom
matzew:change_example_structure_and_add_agent_example
Oct 22, 2025
Merged

Adding example for OpenAI compliant Agent URL#5
Cali0707 merged 1 commit into
mcpchecker:mainfrom
matzew:change_example_structure_and_add_agent_example

Conversation

@matzew

@matzew matzew commented Oct 22, 2025

Copy link
Copy Markdown
Collaborator

Run this, with:

# Set your model credentials
export MODEL_BASE_URL='https://your-api-endpoint.com/v1'
export MODEL_KEY='your-api-key'
export MODEL_NAME='your-model-name'

# Run the test
./gevals run examples/kube-mcp-server/openai-agent/eval.yaml

@matzew matzew changed the title 🎉 Adding example for OpenAI compliant Agent URL Adding example for OpenAI compliant Agent URL Oct 22, 2025

@Cali0707 Cali0707 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matzew I think the best way to use the eval framework in this case would be to have the same task files and just change the agent/eval. So, instead of copying the tasks we would just leave them. WDYT?

@matzew

matzew commented Oct 22, 2025

Copy link
Copy Markdown
Collaborator Author

I think the best way to use the eval framework in this case would be to have the same task files and just change the agent/eval. So, instead of copying the tasks we would just leave them. WDYT?

yeah, that sounds good - let me fiddle

Signed-off-by: Matthias Wessendorf <mwessend@redhat.com>
@matzew
matzew force-pushed the change_example_structure_and_add_agent_example branch from 7f274f9 to 3f2d834 Compare October 22, 2025 14:05

@Cali0707 Cali0707 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - this works for me!

Thanks @matzew

@Cali0707
Cali0707 merged commit 736315a into mcpchecker:main Oct 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants