Agentic AI Vendor Selection: Stage 4 Pipeline Templates

Agentic AI vendor selection is the Stage 4 procurement decision that outlives most executive sponsors — pick badly and the next two years of your roadmap inherits the call; pick well and the prototype in Stage 5 starts with the right substrate underneath it. This playbook turns the data foundation built in Stage 3 into a defensible scoring rubric, an RFP that names the AI-specific clauses procurement teams usually miss, and a per-capability build-versus-buy fork at the end.

By the time a team reaches Stage 4, the strategy is documented, the roadmap is sequenced, the data foundation is in production, and the capability list is concrete. What remains is the procurement decision — which vendors carry which capabilities, on what commercial terms, with which contractual protections, and which capabilities the team will build itself. Done casually, this stage produces vendor portfolios that age badly; done deliberately, it produces a scoring artefact that auditors, board members, and future engineering leaders can read three years from now and understand the reasoning.

This guide walks the six templates that turn vendor selection into a defensible operating discipline — scorecard matrix, RFP, evaluation rubric, reference-call script, contract clause checklist, and the one-page build-versus-buy decision. The templates assume the Stage 3 data foundation is in place; if it is not, run that playbook first, because vendor selection without the data foundation documented is a category of decision that gets re-litigated every quarter.

The average tenure of a senior AI executive in 2026 is roughly eighteen months — shorter than the typical agentic AI vendor contract, materially shorter than the twenty-four-month roadmap window most teams plan against. That asymmetry is the single most under-appreciated feature of Stage 4. Whoever signs the contract is unlikely to be the person who renews it, defends it to the board after a model-update breaks production, or migrates off it when the vendor strategy shifts. The artefact has to survive the person.

That changes how the stage should be run. The output of Stage 4 is not a vendor — it is a documented scoring rubric, an RFP audit trail, a set of weighted axis scores per vendor, a contract with named AI-specific clauses, and a one-page build-versus-buy decision per capability. The vendor is a downstream consequence of those artefacts; the artefacts are what the next AI leader, the next CFO, and the next board audit will read. Treat them as the deliverable, and the vendor decision falls out cleanly.

The second reason this stage matters more than its budget line suggests: vendor lock-in compounds across the rest of the pipeline. A choice made at Stage 4 propagates into the prototype scaffolding in Stage 5, the production deployment in Stage 6, the team enablement programme in Stage 7, the governance posture in Stage 8, and the scale economics in Stage 9. Decisions taken casually at this stage carry weight through every subsequent stage; decisions taken deliberately at this stage create optionality at every subsequent stage. The framework that follows is designed to make the deliberate path cheaper than the casual one.

Two failure modes show up reliably when teams skip the artefact work. The first is the relationship-driven shortlist — a vendor invited to the RFP because the AI lead worked with their CTO at a previous role, with the scoring rationalised after the fact. The relationship may or may not be a good signal, but the scoring written after the decision is no longer an audit trail; it is justification. The second is the demo-driven decision — a vendor wins because the polished demo outshone its competitors, even though the demo capability is the easiest part of the vendor offering to replicate and the operational reality is closer to a different vendor's offering. The scorecard, rubric, and reference calls below are designed to neutralise both failure modes.

The vendor scorecard is the headline artefact of Stage 4. Each vendor is rated on the same eight axes, the axes are weighted, and the rolled-up score is the directional signal that drives the shortlist. The scorecard is not the final decision — the rubric in Section 04 refines it, and the build-versus-buy fork in Section 07 decides which capabilities never enter the procurement track at all — but it is the input to every subsequent step.

The matrix below is the template we have iterated on across agentic-AI vendor selections in the last eighteen months. The specific weights are starting points calibrated against mid-market product teams; regulated-sector teams should push the governance axis up, pre-PMF startups should push the differentiation axis up, and enterprise teams should weight the support and roadmap axes more heavily than the template suggests. The structure is the durable part; the weights are tuneable.

# vendor-scorecard.template.md
# Stage 4 · Agentic AI Implementation Pipeline
# Rate each vendor 1-5 per axis. Weighted total drives shortlist.

## Vendor: <name>                Date: <yyyy-mm-dd>
## Capability under evaluation: <e.g. retrieval, agent runtime, eval>
## Evaluators (named):           <eng-lead, security, finance, product>

| Axis            | Weight | Score | Weighted | Evidence / notes              |
|-----------------|-------:|------:|---------:|-------------------------------|
| Capability fit  |   20%  |   _   |    _     | Does it solve our use case?   |
| Maturity        |   15%  |   _   |    _     | Years in market, customer N   |
| Support         |   12%  |   _   |    _     | SLA, response time, on-call   |
| Pricing model   |   15%  |   _   |    _     | Per-call, seat, flat, hybrid  |
| Security posture|   10%  |   _   |    _     | SOC2, ISO 27001, pen-test     |
| Roadmap fit     |   10%  |   _   |    _     | Aligned with our 24-mo plan?  |
| Lock-in risk    |   10%  |   _   |    _     | Switching cost, data portability|
| References      |    8%  |   _   |    _     | 3+ customers contacted        |
|-----------------|--------|-------|----------|-------------------------------|
| WEIGHTED TOTAL  |  100%  |       |    _     | Threshold: 3.8/5 to shortlist |

## Disqualifiers (any single failure → out)
[ ] Fails security review (no SOC2 or ISO 27001 within 12 months)
[ ] Refuses training-data clause in contract
[ ] No data-residency option matching our compliance requirements
[ ] Single-customer-concentration risk (>40% of revenue from one customer)
[ ] No named-alternative migration path documented

## Decision
Shortlisted: [ ] yes  [ ] no
Sponsor:     <name, title>
Reviewed:    <name, title>
Next step:   <RFP / reference calls / disqualified>

Two scoring disciplines keep the scorecard honest. The first: named evaluators per axis, with each axis scored independently before the totals are visible. Group consensus scoring produces anchored answers — once the engineering lead says "8 out of 10" on capability, every subsequent score tends to cluster around that anchor. Independent scoring followed by a reconciliation session produces meaningfully better signal. The second: a hard disqualifier list that any single vendor failure removes the vendor from consideration regardless of weighted total. Security posture, training-data rights, data residency, and named-alternative migration paths are not negotiable axes — they are gates.

The RFP is the formal artefact that goes to shortlisted vendors, and it is the audit trail your replacement reads when the contract is up for renewal. Six sections, in this order, every time. The order matters — the scope section sets the frame, the requirements section is the substantive ask, the evaluation criteria are published transparently so the vendor knows what they are scored on, and the SLA / commercial sections close out the document with the operational and contractual asks.

# RFP-template.md
# Agentic AI Vendor Selection · Stage 4
# Issued: <yyyy-mm-dd>   Response due: <yyyy-mm-dd, +21 days typical>

## 1. Scope & background (~1 page)
- Company overview & sector
- Current agentic AI maturity (reference Stage 1 assessment)
- Capability being procured (reference roadmap from Stage 2)
- Data foundation in place (reference Stage 3 artefacts)
- 24-month volume projection: <best / base / worst case>
- Out of scope: <capabilities NOT being procured here>

## 2. Functional requirements (~2 pages)
- Must-have capabilities (numbered list, 8-15 items)
- Nice-to-have capabilities (numbered list, 5-10 items)
- Integration points (existing systems we'll connect)
- Performance targets (latency p50/p95/p99, throughput, accuracy)
- Compliance requirements (SOC2, ISO 27001, sector-specific)

## 3. Non-functional requirements (~1 page)
- Data residency options
- Encryption (in transit, at rest, customer-managed keys)
- Audit logging (retention, export format, SIEM integration)
- Disaster recovery (RPO, RTO, geographic redundancy)
- Multi-tenancy isolation model

## 4. Evaluation criteria (~1 page)
- Published axis weights (the scorecard above)
- Disqualifiers (any single fail → no contract)
- Scoring process (named evaluators, reconciliation timeline)
- Reference call requirements (3+ customers, similar use case)
- Decision date: <yyyy-mm-dd>

## 5. SLA & support (~1 page)
- Required uptime: <99.9%? 99.95%?>
- Incident response: <P1 ack < 30 min, resolution < 4 hrs>
- Support tiers: <business hours vs 24/7, named TAM>
- Escalation path documented
- Service credits formula

## 6. Commercial structure (~1 page)
- Pricing model preference (per-call, per-seat, flat, hybrid)
- 24-month TCO at <best / base / worst> volume
- Renewal terms (max % increase, notice period)
- Termination clauses (for cause, for convenience, for deprecation)
- AI-specific clauses (training data, output rights, model update
  notification — see Section 06 of the playbook)

## Submission instructions
- PDF + signed cover letter
- Named contact for clarification questions
- Confidentiality undertaking on the RFP itself
- Decision communicated within 14 days of due date

Two RFP discipline notes worth carrying forward. First, publish the evaluation criteria transparently — the axis weights, the disqualifiers, the threshold. Vendors who know they are scored on lock-in risk will write better answers to the lock-in questions; vendors who do not know will give generic answers and the team has to re-ask everything in clarification rounds. Transparent criteria save weeks of back-and-forth and produce higher-signal responses. Second, name a single point of contact for clarification questions and publish every clarification answer to every vendor in the shortlist. Selective clarification is a fairness failure and it surfaces in the post-decision audit when it matters most.

The evaluation rubric is the scorecard applied to the actual RFP responses. The eight axes below carry the weights documented in Section 02; the table that follows describes the substantive scoring criteria per axis — what a 5 looks like versus a 3 versus a 1, so that independent evaluators converge on comparable numbers rather than producing scores that mean different things.

The rubric works hardest in the reconciliation session. Independent evaluators score each axis before they see anyone else's number; the reconciliation meeting works through disagreements axis by axis, with the evidence from the RFP response and the reference calls as the arbiter. The output is a single agreed score per vendor, with the evidence trail attached. That trail is what survives the executive transition.

Reference calls are the highest-signal artefact in Stage 4 and the most consistently under-used. Vendors control the demo; vendors do not control what a reference customer says when asked the right questions in a structured thirty-minute call. The script below is the version we have iterated on across the last eighteen months — ten questions, in order, with follow-ups built in.

Run the script against three references minimum, ideally five. Vendors will offer references they expect to perform well; ask specifically for one reference at your scale, one in your sector, and one that has been with the vendor for more than eighteen months. The mismatched references are usually where the most useful signal lives.

# reference-call.script.md
# 30 min call · two-evaluator format · transcribed

## Pre-call (5 min before)
- Confirm: vendor X, capability Y, customer Z (sector, scale)
- Read their public case study if one exists
- Note any LinkedIn signals on the reference contact

## Q1 — Use case parity (5 min)
"What capability are you using vendor X for? How similar is it
to <our brief use case in one sentence>?"
Follow-up: scale, volume, integration depth.

## Q2 — Implementation reality (3 min)
"How long did implementation actually take, versus what the
sales team promised? What was the biggest surprise?"
Follow-up: who did the work — them, the vendor, a partner?

## Q3 — Time to first value (2 min)
"How long from contract signature to the first production use
case generating measurable value? What blocked the path?"

## Q4 — Support experience (3 min)
"What is your support experience actually like? Tell me about
the last P1 incident — how was it handled, ack time, resolution time."
Follow-up: do you have a named TAM? Do they actually respond?

## Q5 — Renewal experience (3 min) [if customer >12 months]
"How was your most recent renewal? Did the price increase
match what you signed up for? Any surprises?"
Follow-up: would you renew again?

## Q6 — Hidden costs (2 min)
"What did the contract not cover that you ended up paying for?
Implementation, training, integration, overages — anything?"

## Q7 — Roadmap responsiveness (2 min)
"Have you requested a feature in the last 12 months? What
happened? Was it on their roadmap, did they build it, what
was the timeline?"

## Q8 — Failure modes (3 min)
"What does it look like when this vendor's product fails? How
often, what category of failure, how do they communicate?"

## Q9 — What you'd do differently (2 min)
"If you were doing this procurement again, what would you do
differently with this vendor specifically?"

## Q10 — The unsolicited tell (2 min)
"Is there anything I should be asking that I haven't?"
This is the highest-signal question in the call.

## Post-call (5 min)
- Categorise: positive / neutral / negative on each axis
- Note any specific names, dates, dollar figures cited
- Flag any answer that contradicts the vendor's RFP response

Two operational notes. The two-evaluator format matters — one evaluator drives the questions, the other takes structured notes and watches for tells the lead may miss. Trying to do both is measurably worse, both for capture quality and for follow-up instinct. The other note: Q10 is the highest-signal question on the script. References who are coached or constrained tend to give terse answers to Q1 through Q9 and then volunteer the real story when asked the open-ended close. Almost every reference call we run produces its most useful sentence in the final two minutes.

The contract is where the scoring work converts into operational reality. Four clause categories distinguish an agentic-AI vendor contract from a generic SaaS template — training-data and output-rights clauses, model-update notification, AI-specific indemnification, and termination-for-deprecation. Each is a recurring negotiation point in 2026 procurement, each is quantifiable in renewal terms, and each is the type of clause that becomes a hostage negotiation if it is not handled at sign time.

Beyond the four AI-specific clauses, the standard SaaS contract checklist still applies — data residency commitments matching your compliance posture, encryption at rest with customer-managed keys where required, SOC2 / ISO 27001 evidence with annual refresh, named subprocessors with change-notification, and a clear escalation path. Walk through each with procurement counsel before signing. The agentic-AI clauses are additions, not replacements.

Pricing-clause discipline is worth a separate note. Renewal caps on annual price increases — 7-10% is healthy, 12-15% is acceptable, anything uncapped is a future renegotiation under duress — are the single line item that protects you against the vendor capturing all the upside as your usage grows. Most vendors will negotiate the cap if pressed; most procurement teams never ask. Ask.

Not every capability on the Stage 2 roadmap goes into the RFP. Some are built internally because they sit on the competitive surface; some are bought because they are commodity plumbing; some are wrapped — the vendor's implementation behind your own schema — to preserve switching optionality. The one-page build-versus-buy decision is the gate that decides which path each capability takes before procurement begins.

The decision sits at the top of the Stage 4 funnel — run it first, for every capability, before any RFP is drafted. The capabilities that fall to the build column never enter procurement; the capabilities that fall to wrap-buy enter procurement with a thin wrapper as part of the integration plan; the capabilities that fall to plain buy go through the standard RFP path described in Sections 02 through 06.

The full version of the build-versus-buy framework lives in our dedicated MCP build-versus-buy TCO calculator — six axes, a twenty-four-month TCO model, an explicit switching-cost component, and the wrap-buy third path discussed there. The capability-by-capability decision is the same shape whether the underlying technology is MCP, retrieval, agent runtime, or anything else procured at this stage; the framework transfers directly.

The discipline that keeps build-vs-buy honest at month eighteen, when the picture has shifted: document the axis ratings and the TCO numbers at decision time, in a short markdown file alongside the capability in your repo. Rerun the framework against the documented baseline at every renewal cycle and at any 2x volume inflection. The teams that compound advantage at this stage are the ones that rerun the decision; the teams that compound regret are the ones that treat the original call as final.

Stage 4 concludes with a hand-off package to Stage 5 prototype. Five artefacts go in the hand-off, each produced by the work described above, each named in the prototype brief so the Stage 5 team starts with the documented context rather than reconstructing it from memory.

With those five artefacts in hand, Stage 5 begins from a documented baseline — the prototype team knows which vendors are in scope, which capabilities are being built internally, which wrappers need to land before integration, what the contractual constraints are, and what the economic envelope is. The next playbook in the series, the Stage 5 prototype templates, picks up from here — prototype brief, eval harness, success criteria, and the prototype-to-production gate. For teams that want a partner running Stages 4 and 5 alongside the internal team, our AI transformation engagements cover the full procurement-to-prototype hand-off as a single deliverable.