Toptal

Could you share any statistics or metrics from this engagement?

I can't give any metrics at this point because our engagement went about twice as long as we originally estimated, which set us back on some other things. However, we did open it up to some folks for testing to find any kinks that needed to be worked out. So far, the responses from that group have been very good. We've been getting a lot of positive feedback about the look and the capabilities of the site.

How did Toptal perform from a project management standpoint?

From a project management perspective, it was very helpful to have someone who could effectively communicate the thoughts of my partner and me to the developers and vice versa. He helped bridge the gap between our knowledge bases, especially when it came to implementing specific features. A couple of times, there were instances where we didn't understand the complexity or cost behind a function that we were looking for, such as a search bar. Our project manager did a great job of managing that communication.

Something that was a bit more difficult was the timeline. My business partner and I work full time, so we had a limited window in the early morning that we could speak to the Toptal team. As time went on, there were certain things that we expected to be done between conversations that weren't. It would have been nice to have someone verify that things were done before our calls happened. It was supposed to be a six-week project, but ended up being about 12. It definitely became more stressful for us the longer the project went on.

What distinguishes Toptal from other providers?

Having that communication bridge was necessary. I was glad to have a resource where I could give my perspective and resolve problems rather than having the whole project blow up. Had we not gone through Toptal, we would have only worked with freelancers, and I'm sure we would have had much less recourse. The bridge was also useful from a knowledge perspective. It was very helpful to have somebody that we felt comfortable talking to and trusted to communicate our ideas.

Another positive thing was having an extensive team with people both inside and outside the United States. We wouldn't have been able to afford that many people from the U.S. on our own. We also felt better about having non-U.S.-based resources in the mix because we had such a large group.

Is there anything Toptal could have improved or done differently?

On the front end, they should refine who their target customer for a product like this is. The beginning stages were quite difficult because I didn't know them, and we had a rough contract negotiation. Had they seen our RFP [request for proposal] from the beginning, that would've allowed them to understand the full scope of what we were trying to accomplish and probably would have stopped them from exceeding their not-to-exceed pricing quote. We actually went almost 50 percent above what we originally wanted to spend.

They ended up switching project managers later in the project, and he was very open about the pressure he was getting from upper management to hurry up and finish. After hearing that, we lost confidence in the quality of the work. I'm not sure if the size of our project was the best for them. If the estimating piece had gone smoother in the beginning, I think the whole process would have been more successful. Even now, after we've finished they haven't reached out. It would have been nice to have some support once the project was completed.

Could you share any statistics or metrics from this engagement?

We did a minor version release that cleaned up only the 10 to 15 crashes, and that was the first priority when we brought on the iOS developer. Here's a good statistic, we were running in the 94 percent crash-free user range, which meant 6 percent of our users were getting crashes on a more regular basis. Now, since our minor update release, we're running at 99.3 percent as of the last report.

We use a Kanban model, so it's less revolving around the time we think it's going to take to complete a feature or fix a bug, and more about constant communication and meaningful improvement. We have a Slack channel opened up with the two developers, and then we do the daily stand up over the screen here. Transmitting feedback has never been an issue.

How did Toptal perform from a project management standpoint?

They've always delivered on time and within the specified budget. We never felt like that was going to be a problem. Tools we used included JIRA, GitHub, and Slack.

What distinguishes Toptal from other providers?

The difference was the two-week trial period. I don't know if that's something I would ever execute for a nonpayment, but it provides a level of protection. In the span of two weeks, we can have a real good sense of the quality and the developer that you're bringing on. It wasn't going to be a flake. They're going to be a resource that's available and getting their job and the work done. Then, I would say the quality of the candidates. While we did interview a very high number of candidates for the Ionic position, some of them just were not in our space. We didn't think they were going to be strong enough. Some of them were too high of a cost. There was one guy in the mix that was $95 an hour, and that was just something we weren't willing to commit to. One of the guys – the first iOS developer we interviewed – I was in the middle of the interview with him when I got a call that I couldn't take from Toptal saying, "Hey, sorry. Don't bother interviewing me. I just got hired out." That was surprising, but it does seem like there is high demand for their talent.

Is there anything Toptal could have improved or done differently?

I don't think so. I wouldn't make any changes in that space. They do a good job of trying to give you a picture of the developer and what they can and cannot do. They invest a good amount of time vetting their candidates.

Any advice for future clients?

When you're interviewing candidates, you still have to evaluate technical capabilities for your specific needs. They're going to come in and – one of the guys that we got pitched to was very good in .NET and had done a lot of Angular.JS work, but when we looked at our specific needs around Cordova, we just made the determination that it wasn't going to be a good match. They're going to be a good developer, but whether they're going to be good in your space or not, it's still up to you to make that determination. Then, you've got to have good work practices and project management set up because, from this perspective, you're going to have somebody – a developer – joining your team. You've got to have leaders who can shepherd them through that process.