Could you share any evidence that would demonstrate the productivity, quality of work, or the impact of the engagement?
Based on a lot of the feedback we got from the testers, our second version had another feature and had user experience enhancements. There, we saw a significant increase in regular downloads. I don't have the exact statistics but there was an absolute increase.
What did you find most impressive about Ubertesters?
The organization of the launch of the test was very impressive. There's a lot involved and a lot that I hadn't done before with an external testing firm. We do tests, but at lot of our testers are, essentially, either people in a room with a single-window glass with cameras outside, and they sit there and either do it on paper or do it actually on an app, or user tests with a bunch of people that come over to fiddle around with something that we've developed.
In this sense, writing the use cases and going so deep into the different devices and different operating systems on those devices because of their experience, they were able to really manage all of that. Get it to the point where we said we wanted to start our test on November 1st and we'd like to have our results by the 5th. Everything ran smoothly. It wasn't like all of a sudden people backed out and they didn't have anybody to replace them. They had enough testers to be able to very quickly replace testers.
In terms of the whole management side of making sure that it ran smoothly, it ran on time and we got detailed results within the time period that we needed them and wanted them. It was fantastic.
Are there any areas Ubertesters could improve?
It's hard to say. I don't really think so. In my dealings with them, I didn't have any problems. They were very accommodating. There was nothing that they said they couldn't do or that they didn't do in accordance with my expectations.
In terms of their service, and this isn't a negative for their service, it just shows sometimes how daunting the task of actually routing through all of the information is. Their platform provides a ton of information. The first time you go through it, or the first time that you log in, it seems a little daunting because there's so much information that you can actually see. We could see crash results for instance. There's just so much information that at first glance it may seem overly complicated.
It could be an opportunity for them to make a few small changes to where a novice can get the basic information that they need and not see everything. Then they could have an advanced view, where a real professional can really dig down into the data, to the nuts and bolts.
The first time I was in there, I was like, “oh wow that's a lot of information.” The first thing that popped into my mind is that there was so much in there and that it was going to take me a lot to go through it. I wasn't expecting that much information. It's actually a great thing, not necessarily a negative thing.
There was a couple of people that didn't like the app at all and will never use it, which is also good. It's great feedback. Then there were people who saw that the in certain functions, there were too many clicks, which helped us to rethink that flow to reduce the amount of clicks or taps necessary for someone to get from Point A to Point B.