How to Vet a Software Development Agency in 2026: 8 Signals That Actually Predict Project Success

The honest answer is that most buyers have been missing the same thing for years. The standard playbook is to read the reviews, check the portfolio, and call the references. All of that tells you about the agency’s past. It tells you very little about your project.

At Goji Labs, we have done more than 500 engagements. Our clients have raised over $1 billion. The products we have built support tens of millions of users. After that much work, the pattern is hard to miss. Some agency behaviors line up with projects that hit their milestones. Other behaviors line up with projects that slip. Eight signals do most of the work. Here is the framework agencies quietly wish more buyers would use.

Why a Five-Star Agency Can Still Fail With Your Project

The best clients we've worked with don't just ask for our portfolio. They ask what went sideways on a project and how we handled it. That question tells you more than any portfolio of work ever could.

The vetting question has to shift. It shouldn’t be “what have they done?” but “how will they behave when something goes wrong on my project?” Software projects run into issues all the time. The good agencies handle it well; the bad ones, really bad. A buyer who can tell the difference before signing is buying a fundamentally different product than one who can’t.

The eight signals below are diagnostic of behavior under uncertainty.

The Eight Signals That Actually Predict Outcomes

1. Discovery is treated as a phase, not a sales stop
2. The team you meet is the team you get, by name
3. They volunteer past failures, not just wins
4. Estimates include explicit assumptions and unknowns
5. Change management starts before the change request
6. Documented communication, not promises
7. No technical decision is free of tradeoffs
8. References they can't fully control

1. Discovery is treated as a phase, not a sales stop

Discovery is where projects get shaped. An estimate built without one is just a guess with a price on it. Agencies that skip discovery don’t really know what your project involves. Instead of doing the work to find out, they inflate their numbers to cover what they don’t know.

Pro tip: If you are looking to book a meeting with any agencies, look for ones that offer a discovery phase, paid or unpaid, with real deliverables. Ask for a problem-facing document, an assumption log, and a list of identified risks.

Ask the agency: “Walk me through your discovery process before you give me a quote.”

2. The team you meet is the team you get, by name

Bait-and-switch is one of the quietest predictors of trouble in agency work. The senior engineer you met during the pitch shows up to the kickoff call. Then they're pulled onto another project, and a less experienced developer quietly takes over.

Pro tip: In the SOW, look for the names of the team members. The engineering lead, PM, and designers must be in it with their allocation percentages, start dates, and LinkedIn profiles that you can verify.

Ask for read access to the code repository once the project starts. Every major platform shows who’s been committing. If you spot a name that wasn’t in the SOW, ask who it is. This is what most founders overlook.

Ask the agency: “Can you name every person on my project and tell me exactly how much of their time I am getting?”

​3. They volunteer past failures, not just wins

Agencies that have actually learned from past projects can’t help but bring them up. The lesson stories show up sideways, in passing, while they’re explaining why they do something a certain way now. This is the hardest part of an agency to fake.

Pro tip: The failure should come up on its own, not because you pulled it out of them. The story should be specific enough that you can picture it: phases or technologies, rough timing, what went wrong, and what it cost.

Ask the agency: “What's the lesson story behind how your team works now?”​

4. Estimates include explicit assumptions and unknowns

Look at how the estimate is structured before you look at the number. That’s where the real signal is.

When an estimate comes back as a single clean number, that number comes from sales. An engineer who's actually worked through your scope comes back with a range, a list of what they assumed, and a note on what could change the math. If the estimate looks too clean, it probably is.

Pro tip: There are four things present in any real estimate: the assumptions, the flagged risks, a contingency as its own number, and a trigger for when the numbers need to be revisited.

Any agency can quote you a number upfront. The most important thing is whether they revisit it. A serious firm will reestimate at least twice after that first meeting. It should happen once designs are locked, and again before each feature kicks off.

Ask the agency: “What assumptions went into this estimate, and what would change it?”

5. Change management starts before the change request

Every project changes scope. Yours will too. The strongest change management starts before a request ever comes in, with how the scope got approved in the first place.

It is the right decision when a scope gets locked in at the right level of detail. Every change becomes a clear swap. Without that, you’re renegotiating every time. The right level of detail is not measured in a list of broad epics that everyone agrees to in principle.

Pro tip: If an agency tells you that changes are not a problem, that is a red flag. By the time a feature is mid-sprint, it usually has four or five people’s hands on it across product, design, and engineering. Pulling out a project mid-build is not free. It is you who pays for that rework. Choose an agency that has cutoff points baked into how they work.

Ask the agency: “When’s the latest point to bring in change without wasting work that is already done?”

6. Documented communication, not promises

“Weekly check-ins” means nothing. Every agency promises them. The important thing to consider is what those check-ins produce.

Pro tip: Ask to see an example weekly status report from a real, anonymized project. An agency with a strong communication culture will send one over within the same day. Agencies without one will stall, send you a one-off they cleaned up just for you, or pivot to talking about their general approach.

Ask the agency: “Can you show me an example of a weekly update from your current projects?”

7. No technical decision is free of tradeoffs

It is worth paying attention to when a senior team explains why they’d pick stack A over stack B for your specific project. On the other hand, a junior or sales-led team sticks to “we already know X” and moves on.

Pro tip: If the agency won’t push back while they’re trying to land your business, they aren’t going to start once the contract is signed. Telling you hard truths only gets harder once the money has started flowing.

Ask the agency: “What would you push back on in our current spec, and why?”

8. References they can’t fully control

Any agency can put together three glowing references. The ones they hand you aren’t representative of their work. The references that actually tell you anything come from projects that hit trouble: late deliveries, scope changes, technical pivots, and difficult client relationships.

Pro tip: Agencies worth hiring will connect you with a client whose project changed significantly mid-build. Agencies that won’t are telling you something about how they handle difficulty, without realizing they’re telling you.

Ask the agency: “Connect me with a client whose project changed significantly mid-build.”

Red Flags That Consistently Precede Overruns

A few patterns show up before nearly every project that goes off the rails.

• Fixed-bid quote on an ambiguous scope. A real, honest price can’t work without a real scope. Either there is a huge contingency hidden inside it, padded with a big safety net. 
• An estimate that matches your budget exactly. Once in a while, it’s a coincidence. Most of the time, it isn’t. When the number lines up perfectly with what you said you could spend, the estimate was built backward from your budget.
• Account manager as your main contact through the evaluation. The people who’ll actually build your project should be in the room before you sign anything. 
• “Full-stack team” framing with no specialization. Real teams have specialists. When an agency describes their team as fully generalists, it means that they have a small bench and that they are stretched across too many engagements. 
• No discovery phase included in the proposal. Discovery happens whether you pay for it or not. The only question is when. If it’s not in the proposal, you’ll go through it after you’ve already signed.
• Won’t share named team members until you’ve signed. You’re hiring people, not a brand. An agency that won’t tell you who you’re actually getting until you’ve paid for them is telling you something about what you’re going to get.
• Vague timelines without buffer reasoning. Every plan, no matter how well thought out, runs into some problems. An agency that can build these buffers can handle it. 
• Polished pitch deck, evasive answers to specific questions. Money spent on the presentation doesn’t predict anything about delivery. If they can’t answer a technical question when they’re trying to win the deal, they’re not going to answer it faster after you’ve signed.

An RFP Scoring Rubric Founders Can Use

The rubric below is a starting point, not a verdict. Scores are directional and meant to surface the gaps you’ll want to investigate before signing.

Score each agency 1–5 on each of the following categories:

• Discovery diligence —  whether they thoroughly understand your problem before giving a quotation or price
• Team Visibility — Real names, clear roles, and actual points of contact in the contract, not just "a dedicated team"
• Estimate honesty — assumptions, risks, ranges, and re-estimation triggers
• Change management maturity — an actual, agreed-upon way, not improvisation
• Written accountability —  documented decisions, updates, and discussions, not promises
• Technical Openness — tradeoffs explained without asking
• Reference willingness — willingness to connect you with difficult project clients
• Failure self-awareness — unprompted and tied to systemic change; generic answers are a red flag

This is how your total score reads:

• 32-40: Close fit. A few things to confirm.
• 24-31: Possible. Identify the specific gaps and decide whether they are addressable before signing.
• Below 24: Not a fit.

A rubric only works if you score before you fall in love with a vendor. 

The Vetting Mindset Shift

The eight signals share a single pattern. They’re all diagnostics of how an agency behaves in the face of uncertainty.

Buyers who optimize for certainty in the sales process tend to pay for it in the build. The signal for a sales motion is whether everything looks good and perfect. Look out for agencies who feel slightly more uncomfortable talking to you during evaluation because they raise questions that make you rethink your scope. Over the course of the project, they tend to be dramatically less expensive to work with and deliver a product you actually need.

The simplest test in the eight isn’t a test at all. It’s a question to ask yourself after a sales call: Did you leave with more questions answered, or more questions raised?

The latter agency, every time.

About the Author

Joshua Webber Marketing Manager at Goji Labs

I'm a Marketing Manager at Goji Labs, where we transform companies and build products that scale. I build marketing machines that actually work.. the kind that turn strategy decks into measurable growth, not just pretty slides. I’ve spent my career blending creativity, analytics, and a healthy dose of curiosity to help brands grow with purpose. Outside of that, I'm in full girl dad mode, chasing good music, making memories with friends and family, and hunting for my next real estate deal. Whether it's a campaign or a bedtime story, the goal is always the same: keep them hooked until the very end.

See full profile