What evidence can you share that demonstrates the impact of the engagement?
We monitor their coverage and definition of tests, tracking them at each release as we continue to expand coverage. They increase significantly each time, and the bounce back rate is incredibly low. XBOSoft has found about 900 defects (mainly from the in-house team), and we’ve only had to go back to clarify two of them. The team is extremely detailed and provides more than enough information to understand the issues they’ve flagged.
How did XBOSoft perform from a project management standpoint?
They’re predictable and reliable, providing daily reports on what they’ve covered and what issues they’ve found. Almost every test report includes screenshots and video that show exactly what they tested for and what was covered. They find a balance between resolving all the issues and making sure that we release on time.
What did you find most impressive about them?
XBOSoft has some of the most detailed and careful test execution and reporting I’ve ever seen. They appropriately assess what needs to be covered, even if we haven’t provided them with much direction. We just have to give them general guidance, and they come back with great ways to define and execute tests. It frees up a lot of our time and makes the partnership much more efficient.
Are there any areas they could improve?
Since starting the partnership, they’ve improved a lot in their expertise with video, audio, and different network conditions. It was a learning curve, but they’ve gotten better at it.
Do you have any advice for future clients of theirs?
Train them on your app and domain so they understand how users would interact with your application. Once they understand the environment, they’ll know what tests should be run and even create some that you wouldn’t have thought of. To get the most out of them, you need to give them as much information as possible, including your timeline and release schedule.