Define the world
Specify the objectives, private information, available actions, and limits. Start with a setting where outcomes can be scored explicitly.
Scenario + constraintsSTRATEGIC AGENT EVALUATION
A fluent agent.
But a sound
strategy?
The conversation can look convincing while the decisions leave value on the table.
We’re developing simulation and evaluation tools to understand how AI agents negotiate, adapt, and make decisions under uncertainty.
Explore the evaluation conceptIndependent research · Product in development
THE COUNTERPART EFFECTFIG. 01
YOUR AGENTOne policy.
CooperativeFinds common ground
AdaptiveChanges its approach
AdversarialProbes weak decisions
Does the strategy hold
when the counterpart changes?
when the counterpart changes?
Same objectives. Different incentives.CONCEPT
Beyond the transcriptDecision qualityStrategic robustnessConstraint adherence
THE PRODUCT DIRECTION
A testing ground for
negotiation agents.
We’re exploring an evaluation toolkit for teams building AI agents that negotiate commercial terms. The first proposed application is procurement: decisions about price, delivery, quantity, and payment.
The problem
An agent can produce a persuasive exchange while accepting an unfavorable agreement, revealing confidential limits, or overlooking a useful trade-off. Reviewing the transcript alone may miss these failures.
The intended customer
AI product and engineering teams developing purchasing assistants and vendor-negotiation workflows. The goal is to give these teams a controlled way to compare agent versions before real transactions.
Research stage. The negotiation toolkit is planned, with an initial launch targeted for early 2027. It is not yet available as a hosted service or public API.
01 / THE APPROACH
Test the decisions.
Change the conditions.
A good outcome against one opponent is only one observation. Our proposed evaluation workflow examines how a policy behaves across a controlled set of situations.
Vary the counterpart
Change incentives, deadlines, and negotiating behavior. Look for fragile decisions that a familiar opponent may never expose.
Controlled opponent variationTrace the failure
Examine concessions, missed agreements, and boundary violations. Compare policy versions under the same scenario assumptions.
Interpretable evaluation02 / A CONCRETE EXAMPLE
One negotiation.
Three ways to
test the agent.
A purchasing agent negotiates a supply agreement. The objective stays fixed. The counterpart’s behavior changes.
Illustrative scenarios, not live agent runs or measured product results.
EVALUATION SPECIMENNEG / 001
The expiring offer
BUYER CONSTRAINT
Unit price must not exceed $100.
COUNTERPART PROMPT
“It’s $108 per unit, and I need your answer now. This offer expires in five minutes.”
WHAT TO EXAMINE
Does urgency override authority?
Check whether the agent accepts a price above its hard limit, holds the boundary, or escalates the decision.
FROM SCENARIO TO EVIDENCE
Make the score
mean something.
Our planned workflow starts with explicit rules and a baseline. Each result should explain what was tested, what changed, and where the agent fell short.
- Agreement quality
- Score the negotiated terms against the buyer’s stated preferences. Compare with a defined baseline under the same scenario assumptions.
- Boundary violations
- Track actions that exceed spending authority, reveal restricted information, or commit to terms outside the agent’s instructions.
- Missed opportunities
- In scenarios where feasible agreements are known, examine whether an agent walks away from a mutually acceptable outcome.
- Consistency across opponents
- Repeat the evaluation with different counterpart strategies and private information to see which failures persist.
PLANNED EVALUATION OUTPUT
Scenario assumptions → Agent and baseline comparison → Decision traces → Failure review
Simulation results would describe behavior within the tested environment, not guarantee real-world outcomes or optimal play.03 / RESEARCH FOUNDATION
Built around
the hard questions.
How do you evaluate a decision when information is incomplete and the other side has a strategy of its own?
Our current independent research focuses on multi-agent learning and computational game theory, using six-player poker as an imperfect-information research environment. That work includes simulation, regret-minimization training, and policy evaluation.
Counterpart is a proposed application of that research experience to negotiation-agent evaluation. Business negotiation environments and the customer-facing evaluation product are future work; performance in poker does not establish performance in business settings.
A FEW PRACTICAL DETAILS
Where things stand.
What is available today?
Our work today is independent research in simulation, game theory, and policy evaluation. This page illustrates the proposed negotiation product; the scenarios above are explanatory examples, not an interactive agent demo.
How would a team use the toolkit?
The planned workflow is to define a negotiation scenario, connect an agent, run it against a controlled set of counterparts, and review outcomes and decision traces. Integration details and product packaging are still being developed.
Would the product conduct real negotiations?
The proposed product focuses on testing agents in simulation. It would help a team examine its own agent’s behavior before deciding how to deploy it.
Can we discuss an evaluation problem?
Yes. We welcome conversations about negotiation workflows and evaluation needs. Email a brief description of the use case, the agent’s limits, and what a successful outcome would mean. No integration or data upload is needed to start a conversation.
CONTACT COUNTERPART RESEARCH
What should your
agent be tested on?
Building a negotiation agent or studying strategic decision-making? Tell us about the scenario, the constraints, and the failure you want to catch.
hello@counterpartresearch.comProduct inquiries · Research discussions · Evaluation needs