What evidence can you share that demonstrates the impact of the engagement?
Massimo’s team thought that we knew our message and they were there to help us fine-tune call to actions and behavior around that, but that’s not where we are. We’re still learning what our sales motion is, what door to open, what to say. I’ve been trying to get message testing into their brains, trying something for a week to see if I get enough impressions and clicks but not optimizing it yet. It took a while to understand what I wanted and come up with a methodology that was comfortable to them even though it was still pushing the limits.
Most people in SEM would want to test an ad for a month. I don’t have a month to sit and wait on a bad ad. I need to find out if it’s a bad ad in days or weeks, so we’ve agreed to do two-week testing. That gave us the ability to do about five rounds over the last 10 weeks and we’ve gone from 500 impressions per day to over 1,000 while keeping the clicks about 1% to 2%. We found some decent sandboxes to play in to refine further in the last 10 weeks. We’ve started to create campaign categories to help us refine things more.
If I was harder hitting in the beginning in telling them not to think the way they’re supposed to think and if Massimo’s team listened more or did more due diligence upfront about what I really wanted, we probably wouldn’t have spent four weeks getting to that place, but we finally got there.
How did Spectrum Group Online perform from a project management standpoint?
There’s room for improvement. It’s a delicate balance. If I over-communicate, then I get billed more. If they under-communicate, I don’t know if things have been done, and I think we’re still working on that and agreeing more than we did. What are we going to cover in the meeting we have every Monday? How are we going to make sure we do accurate testing where we change just enough to be sure with the test that changed? Did we do a clever and careful and clear AB test as an example without changing too many variables in the AB test?
In the last two or three weeks, we’ve arrived into a nice rhythm, but it took us a while to get there. Sometimes I still have to ask them to show me the ads. I’m still pushing to get refined, clear, simple communication, but compared to where it was 10 weeks ago, it’s much better. I’d say we’re B+ getting to A-.
What did you find most impressive about them?
That’s a hard one to judge.
Are there any areas they could improve?
It would have been better for us to dig deeper into what we want and how they do things. They were hung up on optimizing when I was asking, 'why optimize something when we don’t even know if it’s right or we could do 10 times better if we keep playing around?' I think using SEM for message testing is a bit odd. It’s not something everybody knows how to do or is comfortable with, or they want to take a more pragmatic approach. I have board pressure, fundraising pressure, and my boss’ pressure. For someone to say let’s let it sit for four weeks, that’s not a startup. We have to go faster. I pushed them beyond their comfort zone. That was good. I think if they have a client like me next time, they’ll know better and they’ll run faster.