What was the scope of their involvement?
We assigned QualityLogic some projects and discussed how we wanted the testing to be done. They were open to suggestions and new testing methods. They learned the process from us and took ownership of the testing for a wide array of our apps. This has included all-manual testing, not just black-box (having users interact with the apps), but also with a more complex approach. Based on a user’s location, they will receive different streams for live viewing. For instance, if someone wanted to see a particular New York station, the app would recognize that they are in that location, and give them the right feed. This is challenging to do with black-box testing because we can’t manipulate locations, and need to go into a gray-box area, looking at the APIs and manually manipulating responses so that we receive the desired results.
Another example: our app can handle five shows on a page; we wanted to see what would happen if there was a sixth show. Rather than us programming a test for each use case, QualityLogic has looked at our API and its responses, manually added shows, and assessed the reactions.
The composition of the team has changed recently. We have recently expanded to 50 people, but my interactions are with the project manager and the leads. We have also decreased the team size when there were fewer projects in a fiscal year.
We have worked with other testing vendors, but not at the scale of QualityLogic. They hold around 75-80% of all our projects.
How did you come to work with QualityLogic?
We used to have two different teams in-house: a web team and a mobile player team, which I was a part of. QualityLogic was contacted by someone on the web part. The teams have since merged, and I took over the QA department. I contacted QualityLogic and started giving them business minimally, at first. After they were onboarded around processes and functionalities, we started giving them more projects.
What is the status of this engagement?
We started working with QualityLogic 2 ½ or 3 years ago for my department. They may have started working for the company a little earlier.