QualityLogic

Introduce your business and what you do there.

I’m a manager at P.volve, a company that offers solutions to women looking for low-impact exercises that can work with their lifestyle, health, and pace of life.

What challenge were you trying to address with QualityLogic?

We hired QualityLogic to provide app testing services for the app that we were creating. 

What was the scope of their involvement?

In our current setup, we work with a manager from QualityLogic. Their resources do automated and manual testing for us. They serve as a part of our project team that develops our app. To do the tests, they use Cypress tools. We have web and mobile offerings, and they do tests for both. 

What is the team composition?

We work with a total of nine people from Quality Logic, including the manager, two QA engineers, and five manual testers. 

How did you come to work with QualityLogic?

I’ve only recently started at P.volve, and QualityLogic was already there when I joined.

How much have you invested with them?

We’ve invested between $1 million–$5 million. 

What is the status of this engagement?

We started working with QualityLogic in June 2020, and the engagement is ongoing. 

What evidence can you share that demonstrates the impact of the engagement? 

QualityLogic is highly efficient in their testing work. They put a lot of effort, and compared to the other quality testing vendors we’ve had, QualityLogic provides 10x the amount of work as other teams.  

Given that, my internal team loves them. They have the best people to work with; it’s been amazing working with them.

How did QualityLogic perform from a project management standpoint?

Their project management is excellent. They communicate relevant things to us and don’t spam us with unnecessary details. The team also gives us ample notice if they have to do more than 40 hours per person in a week so that we can adjust our budget and expectations accordingly.

They’re great at telling us what they’re trying to deliver. In a lot of ways, their work is our rock as we undergo massive shifts as a company over the last few months. Their output is the most solid piece that helps direct everything else around it.

In terms of tools, we primarily use Jira to manage the project and GitHub for source control. To communicate, we use Zoom, Slack, Outlook, and Confluence. 

What did you find most impressive about them?

QualityLogic is the best QA team I’ve ever worked with — I’ve been doing QA work for 25 years, and I’m stunned at how good they are at their job. They have everything that we want in a QA team; they’re flexible and always maintain good quality. 

Are there any areas they could improve?

No, there aren’t any. 

Do you have any advice for potential customers?

Understand that QualityLogic brings a huge amount of value to the table. If you go for a team that’s less expensive, you’re going to get better rates but won’t get value. Meanwhile, the QualityLogic team does many things above and beyond a standard organization. 

For instance, they supply their own mobile devices so that they can test the app on various types of phones. If somebody from their team has to go out for a week, they automatically have a backup that can step into the position. Overall, you’re going to get great value out of the money you invest in their services. 

Introduce your business and what you do there.

I’m a director with Inspirant Group, a consulting company.

What challenge were you trying to address with QualityLogic?

We were tasked with building a very large and complex application that required extensive testing. We didn’t have enough resources internally to accomplish this, so we decided to hire QualityLogic to help fill this gap.

What was the scope of their involvement?

We hired QualityLogic to help test the application from all aspects. This involves testing the different types of web browsers that the app can be used with and testing all of its components to come up with a way to break the system so we can improve it. They work in an agile format, so they do individual and regression testing every two weeks.

What is the team composition?

We work with an automation tester, an automation engineer, two system testers, and a project manager from QualityLogic

How did you come to work with QualityLogic?

We found QualityLogic through an online search. We decided to go with QualityLogic for this project based on their responses in our interview.

How much have you invested with them?

We’ve spent somewhere around $100,000.

What is the status of this engagement?

We started working together in August 2021, and the engagement is ongoing.

What evidence can you share that demonstrates the impact of the engagement?

QualityLogic has done a tremendous amount of individual and regression testings, and they’ve successfully identified several defects or bugs within the application that we are actively working to fix.

How did QualityLogic perform from a project management standpoint?

QualityLogic has been great in terms of project management. They’re self-sufficient and incredibly flexible in their ability to maneuver people to accommodate our needs. Their communication has also been fantastic, and they’re very comprehensive in their work. They join all of our meetings and provide detailed updates on their test cases and all of the bugs they’ve identified.

What did you find most impressive about them?

I’m most impressed with QualityLogic’s expertise in pure testing. That, combined with their excellent communication and the breadth of what they can test, really stood out to us.

Are there any areas they could improve?

I don’t think there’s anything they could have done better, and we’re looking forward to doing additional work with them in the future.

Do you have any advice for potential customers?

Make sure to have a good sense of what your needs are before engaging QualityLogic. They have a breadth of expertise in different areas that will help them identify the right people to step on your project if you let them know what you're ultimately going to need.

Introduce your business and what you do there.

I’m working for Spin Electric Scooter Company as the acting quality manager of the company, handling all our QA efforts.    

What challenge were you trying to address with QualityLogic?

They provide QA resources for our two mobile apps.

What was the scope of their involvement?

They do regression testing for new releases, which happen every two weeks, and they’re also writing test cases for all new features. They handle most of our QA and regression testing work.

We coordinate the work with releases coming up, looking at features to test and adding in additional test cases if needed. They’re fully involved in all scenarios.

What is the team composition?

We had to reduce the team last year, but we started with 6 QA engineers. Now, we have two QA resources on our team. Our budget shrank; they didn’t do anything wrong.

We work with a QA lead and a full-time QA engineer. When we were working with six people, there were five full-time engineers. If our situation changes, I will certainly ask for more people.

How did you come to work with QualityLogic?

They were already onboard when I joined the team, but they were brought on by our previous quality manager who’d worked with them at a previous company.

What is the status of this engagement?

We’ve been working together since November 2019.

What evidence can you share that demonstrates the impact of the engagement? 

Without their help, we wouldn’t be able to achieve our needs at the moment. They have full-time resources running our test cases, and we couldn’t complete this work without them. They’re hard-working people who are dedicated.

How did QualityLogic perform from a project management standpoint?

They work remotely from a different state. We communicate easily, through daily Zoom calls and Slack messages every day. We have a very good collaboration working remotely.

What did you find most impressive about them?

I haven’t worked with other contractors, so I don’t have anyone to compare them to, but I don’t treat them as some third-party I’m working with. I treat them like part of our team. Our entire team treats them as partners.

Are there any areas they could improve?

I don’t have any improvements. They do a great job on all their assignments.

Do you have any advice for potential customers?

I would recommend them if anyone needed help testing their applications. I don’t have any hesitations.

Introduce your business and what you do there.

I’m the VP of engineering at BrightLine, an ad tech company within the OTT space.

What challenge were you trying to address with QualityLogic?

We were lacking internal QA support, so we hired QualityLogic.

What was the scope of their involvement?

QualityLogic tests different software development kit (SDK) products for different platforms, including Apple TV, Android, Roku. They test them on different devices as well. We’ll usually have a kickoff session to gain an understanding of the product and what it should do and how things should flow. 

The team then creates test cases and proceeds to do tests on a per-platform basis. Throughout that process, if there are any questions or ambiguities, they’ll stay engaged to clarify them.

What is the team composition?

We typically work with one QA lead, but it could be a combination of two at any given point.

How did you come to work with QualityLogic?

Our previous CTO onboarded them, and I picked up where he left off.

How much have you invested with them?

The cost ranges depending on the level of support we need. Typically, we engage them for at least 20 hours per week, but it can go up to 40 or more hours. We spend $5,000–$10,000 per month with them, and the total cost so far is in the $50,000–$200,000 range.

What is the status of this engagement?

We started working with QualityLogic around October 2018, and our engagement is ongoing.

What evidence can you share that demonstrates the impact of the engagement?

It’s been tremendous. Before working with QualityLogic, we had a single QA teammate internally, and we were experiencing a lot of bugs and complaints from media partners. 

We saw a huge decrease in complaints after we brought QualityLogic onboard, and we saw a big increase in the quality of our releases. It’s been a tremendous impact overall because they’ve helped us decrease bugs, processes, and things of that nature.

How did QualityLogic perform from a project management standpoint?

We discuss what should be tested during the week, and they give us daily updates on any issues they find and report them in Jira. QualityLogic also gives detailed descriptions of how they found the issue and where, along with how to reproduce it. They have a very good process for that.

We have weekly sessions every Monday, and we have a lot of communication over email, which is pretty clear. We also hop on phone calls if we can’t get the understanding through email. 

What did you find most impressive about them?

This is the first QA team I’ve worked with while at the company, but they’ve definitely been better in terms of process and tracking compared to our internal resources.

Are there any areas they could improve?

It’s not necessarily a bad thing, but, in terms of the collaboration between our companies, having more scheduled sessions would help.

Although we had weekly meetings, more frequent sessions throughout the week would be good as well. It can take me longer to get to an email if I’m swamped with work, so having calls and screen shares would make it easier to describe elements of the project.

Do you have any advice for future clients of theirs?

QualityLogic is very in-depth with the products they QA, but we may have an urgent release, where identifying blockers becomes most important. It’s important to understand the product in terms of what’s critical and what isn’t and how that can impact other aspects.

Introduce your business and what you do there.

We are a growth assessment provider for school districts, helping to measure areas of opportunity for students that teachers can focus on. We deliver assessments through a hosted solution, and much of our software is web application based. I'm a QA manager.

What challenge were you trying to address with QualityLogic?

We hired them to provide more coverage in our operating system and browser compatibility testing. We have multiple development teams who focus on functional and regression testing, but we don't always have time to test operating systems, browsers, and devices.

What was the scope of their involvement?

QualityLogic runs compatibility tests on a monthly basis to make sure we have no regression in any of the operating systems or browsers we support. As the partnership has gone on, we've added more coverage across our product suite: roles testing, new feature testing, and testing to resolve production issues. They're also extending our accessibility testing to ensure good keyboard navigation and screen reader coverage for student assessments. We hope to expand that as we grow our offerings. We continue to find uses for them, and they plug holes in our coverage.

What is the team composition?

We work with a project leader. She coordinates a team of about three testers, and they allow some flexibility when we need to scale.

How did you come to work with QualityLogic?

I found them via a web search. We identified vendors who could meet our need, and we completed an RFP process. QualityLogic stood out from the beginning, and we were very impressed by their services and abilities.

How much have you invested with them?

We've spent approximately $70,000 on their services.

What is the status of this engagement?

The first official engagement started in January 2019, and they've done testing with us in every month since.

What evidence can you share that demonstrates the impact of the engagement?

We track the number of test runs they complete, as well as the defects they find. We've improved both sides of the process, and are pleased with the progress.

How did QualityLogic perform from a project management standpoint?

They're very communicative, and their status reports have the perfect amount of information. We have a daily standup with them, and we also use RingCentral to send messages throughout the day. They're very mindful of resources, and they only make changes when we give them the go-ahead. They continue to improve their processes and are very efficient.

What did you find most impressive about them?

This is a unique collaboration for us, but their flexibility, experience, attention to detail, and communication all stand out.

Are there any areas they could improve?

We haven't run into anything. There has been a learning curve on our side, but I can't think of anything they should improve or change.

Do you have any advice for potential customers?

Make sure that you are clear about your own processes. QualityLogic is more than willing to help you as you solidify those processes, and it is useful to maintain an open communication mechanism for feedback.

Introduce your business and what you do there.

I work for a product consulting company called Sidebench. We do product design and development for our clients.

What challenge were you trying to address with QualityLogic?

We’re a fairly nimble company, so we typically bring in outside resources to staff a project. We were building a mobile and web application for a large Hollywood production company. Due to a three-month deadline, we needed to ramp up resources with an outside team very quickly. This meant we needed a team with consistent, high-quality work to which we could delegate testing responsibilities.  

What was the scope of their involvement?

QualityLogic’s team performed rigorous, manual regression testing. They drew up their QA testing plans and wrote test scripts in a silo. As we got closer to launch, we brought them closer to our team.

What is the team composition?

We had one point of contact. He worked with two engineers who did a large portion of the manual work. We also had direct communication with Joe (General Manager, QualityLogic), a senior member of the team who worked with resourcing, customer service, and account management.

How did you come to work with QualityLogic?

We’ve worked with them in the past, and they’ve become our go-to team with these types of projects. It was my first time working with them, but they were highly recommended by another member of my team. Picking QualityLogic was a no-brainer, it was just a matter of how quickly we could get started.

How much have you invested with them?

We’ve spent around $25,000 with them.

What is the status of this engagement?

The project began in early April 2019 and is still ongoing. This is the most recent extension of a contract we have with them. We’ve extended the contract 3–4 times.

What evidence can you share that demonstrates the impact of the engagement?

They helped us achieve a huge goal in launching the Hollywood production app in a short period of time. Due to that success, we’ve negotiated another $300,000 contract from the client. Without being able to pass that initial launch, we wouldn’t have gotten any additional contracts. They've put in a ridiculous amount of time for us.

At first, our development team thought QualityLogic was overly picky or going too far into detail, but when the first engagement ended, our team asked for them to come back immediately because they knew how much value they added. I think that showed that the level of detail they provide makes things easier for all of us. They do such a robust job of testing different devices and browsers. Every ticket has a video or a screenshot associated with it. They’re saving us mental energy, effort, and time.

How did QualityLogic perform from a project management standpoint?

I was in regular communication with the main contact at QualityLogic every single day. We mostly used Jira and Slack to communicate.

They were always flexible with my hours and what I was working on. From a project planning perspective, they’re so flexible. I think that’s something they pride themselves on. There’ve been several instances where I’ve set an ambitious goal, and they were quickly able to adjust their resources, build a new test plan, and adapt.

What did you find most impressive about them?

The ability to ramp up and ramp down has been very important for us. During regression testing phases, we’re able to easily add testers. When testing is done, we can ramp back down to a single person. The resourcing plans are very accommodating to our needs. Along with this, it feels like they’re truly bought in and care about these projects. It’s not just another contract to them, and that’s very impressive given the number of projects they have.

Are there any areas they could improve?

No, it’s been a really easy process. Sometimes from a project burn and hours perspective, it was a bit high, but they always did a great job communicating that. There was never any time poorly spent; if they had no more tasks, they would tell me they were doing some other task. I’ve honestly not had any complaints.

Do you have any advice for potential customers?

You should be as detailed in your requests as possible. Ask what your best-case scenario would be. Sometimes I felt like we were asking for too much, but they were able to easily meet our demands. If you ask what you’d ideally like to happen, they’d always be able to accommodate.

When you’re forming the partnership, set out the ground rules and your own expectations. I’d advise having an open conversation about how closely they can meet those expectations, which is a testament to how flexible, knowledgeable, and caring they are in this line of work.

Introduce your business and what you do there.

I’m the technical and managing director of the OpenADR Alliance. We’re a membership-based industry standards group for the smart goods space which includes devices like smart thermostats and electric-vehicle charger stations.

What challenge were you trying to address with QualityLogic?

One of our core organizational functions is to organize tests and certify products according to our standards. Almost nine years ago, we made the decision to develop and own a specific certification test tool that we would use to certify products. We needed a partner to work with us to develop it.

What was the scope of their involvement?

We’ve been working with QualityLogic for almost nine years now. They initially built the test tool for us including all of the testing, the test plan, and some test cases. We now co-own the test tool with them and QualityLogic serves as an endorsed partner that can test products for our members based on our standards and certify them. They’ve delivered that tool to a number of our member companies. They provide consulting services and additional development support to our members upon request. We also maintain a working relationship to update the software every couple of years and do other, smaller projects as needed.

What is the team composition?

We work directly with a lead engineer who serves as a project manager and with the general manager. There are other programmers who work behind the scenes.

How did you come to work with QualityLogic?

We put out an RFP to a number of test equipment companies as well as test houses. We didn’t want to just hire a programmer to build the test tool; we wanted to establish a partnership. We understood that we would need support and modifications over the years and that a long-term arrangement would be more effective. We eventually selected QualityLogic based on a combination of their experience, their pricing, and the partnership arrangement they offered.

How much have you invested with them?

We’ve spent about $400,000.

What is the status of this engagement?

We started working together in July or August 2011 and it’s ongoing.

What evidence can you share that demonstrates the impact of the engagement?

We haven’t specifically tracked performance but the test tool works very well. I get very few questions about it, if at all, and we’ve had a positive response.

How did QualityLogic perform from a project management standpoint?

They’ve performed well. Working directly with one of their lead developers has worked well for us. He’s always been on top of things. They’ve worked well according to timelines and plans.

What did you find most impressive about them?

I would say two things. For one, they’re one of the few companies I’ve found with the experience in test systems and testing programs of our sort. QualityLogic seemed to be the one company that had the most relevant testing experience. They also wanted to dive into the smart goods space, which was great for us.

Also, they have the right people on their team. Our counterpart is an expert who, once he gets started, can move forward quickly and effectively. He’s incredibly knowledgeable and easy to communicate with.

Are there any areas they could improve?

I can’t think of anything in particular. We’ve been working together for such a long time and everything moves the way we’d like it to. They could potentially be a bit better at tracking their hours. It would be nice to have advance warning if we’re going to exceed our estimates. But I know it’s sometimes hard to quantify work when you’re building something from scratch. The standards were changing as they built the test tool.

Do you have any advice for potential customers?

The challenge is always to scope a project well, which can be tricky given so many unknowns. Make sure your project is fully outlined and most aspects are clear. That way there won’t be any surprises.

Introduce your business and what you do there.

I’m the director of software systems quality for a transportation company.

What challenge were you trying to address with QualityLogic?

We wanted a nearshore vendor as an affordable alternative for testing resources.

What was the scope of their involvement?

QualityLogic tested our web and mobile applications, specifically the latter’s flow. Tangentially, the team also looked at our automation infrastructure.

What is the team composition?

We worked with a project manager and an account executive.

How did you come to work with QualityLogic?

I had worked with QualityLogic at another company and their mobile testing expertise stood out. We selected the team for this project because they clearly understood our business domain.

How much have you invested with them?

We spent between $50,000–$199,999.

What is the status of this engagement?

We started working with them in March 2018 and the partnership ended in January 2019.

What evidence can you share that demonstrates the impact of the engagement?

We tracked results via daily touch points to ensure proper oversight and clarification. Our side actively monitored the testing and execution.

How did QualityLogic perform from a project management standpoint?

QualityLogic integrated seamlessly with our business and provided high-quality services. Their attention to detail was notable.

What did you find most impressive about them?

The high-energy team was engaged throughout the partnership. They asked probing questions and proactively communicated any obstacles or challenges.

Are there any areas they could improve?

The partnership ended before we wanted, which limited QualityLogic’s involvement in the automation initiative. We wanted them to take a more active role with that.

Do you have any advice for potential customers?

Constant engagement with QualityLogic keeps the partnership on track. Clients that treat them more like an integrated team than an outsourcing vendor will get better results.

Introduce your business and what you do there.

I’m the CTO of Direct Care Innovations. We’re a healthcare technology company offering SaaS-based software to service providers in the Medicaid and Medicare markets.

What challenge were you trying to address with QualityLogic?

We were in the testing phase for multiple projects and didn’t have enough internal resources to perform quality tests. We wanted to outsource to a vendor who had the skills we lacked.

What was the scope of their involvement?

QualityLogic test all of our new developments and bug solutions. They conduct tests on our products’ functionalities, performance, and security. Their team uses their own testing tools and show us analytics on the success of our products. The reports show if bugs have been solved and if a system functions correctly. When they find an issue, they screencast with our developers to make the locating and reproducing process simpler.

What is the team composition?

They provide us with five full-time testers and one part-time tester, who also functions as the project lead. Their team is on constant retainer, so there is always a member available to help us.

How did you come to work with QualityLogic?

I searched for vendors online after performing marketing research. One of our system analysts found them, and we preferred their smaller composition compared to the other behemoth-like companies. We also liked their flexibility toward an open-ended arrangement with an undefined scope of work.

How much have you invested with them?

We’ve spent around $150,000, so far.

What is the status of this engagement?

We started in March 2017, and we keep them on retainer for our testing needs.

What evidence can you share that demonstrates the impact of the engagement?

We’ve been able to increase the number of test cases we get through drastically. Along with quantity increasing, our KPIs show that the thoroughness of the tests is much better than we were doing previously. Our customers have given us positive feedback on the enhanced usability of our system’s overall performance and security.

How did QualityLogic perform from a project management standpoint?

They provide us with detailed documentation about the issues they find and give us quality suggestions for solutions. We receive daily updates from the project lead, which is one of the best parts of working with them. They deliver everything on time and within the budget we set.

What did you find most impressive about them?

Their entire team is professional, knowledgeable, and willing to go the extra mile—we consider them a partner. We rely on them heavily, and they’ve never let us down, so we find the project to be money well-spent.

Are there any areas they could improve?

Although there have been inevitable bugs that went undetected, it’s infrequent, and they give us a reasonable explanation. Our work is complex, and there will always be some issues that slip through—we can’t blame them for that.

Introduce your business and what you do there.

I'm the chief operating officer for High Fidelity, a San Francisco-based startup in the virtual reality space that's building a platform for free servers running virtual reality environments. We have about 50 employees, 24 of whom are engineers.

What challenge were you trying to address with QualityLogic?

We wanted to scale up our QA testing for the development work we're doing without expanding our in-house team too quickly. We wanted to keep a good balance of both engineering and QA staff.

What was the scope of their involvement?

QualityLogic provided additional QA support for already developed software in C++. They execute test plans that we develop to verify that the code our engineers produce works as it's intended to. These tests require due diligence to ensure we're meeting specific outcomes of each feature within our platform. They report back to us, alerting us to any bugs or broken code.

What is the team composition?

Paul (Client Program Manager, QualityLogic) and Joe (Director of Program Management, QualityLogic) are our main point of contacts, but there are an additional six people from their team working on QA for us full-time.

How did you come to work with QualityLogic?

We had worked with another provider previously but were unhappy with their results. The rate at which they would go through tests and the consistency of their responses on the tests wasn't what we wanted, so we searched online and interviewed a few other vendors. We chose to work with QualityLogic in parallel with our previous vendor at first as a comparison, but ultimately decided to solely work with them based on their performance.

How much have you invested with them?

We work with them on a month to month agreement under a budget of $500,000 per year.

What is the status of this engagement?

We've been working with them since November 2017.

What evidence can you share that demonstrates the impact of the engagement?

QualityLogic improved our approach to QA testing, helping us rewrite hundreds of previously deficient tests as a result. The number of issues they've identified runs into the hundreds, which has helped us significantly improve the quality of every release in the process. They also process tests at an impressive rate.

How did QualityLogic perform from a project management standpoint?

They're very practical in how they deal with unexpected situations and have two people on their staff trained on our technology should they need to replace or add a couple QA staff to our projects. That approach is very sensible and shows that they're resilient to change.

When we've needed overtime from their current staff, they've been very responsive. Everything seems very well managed.

What did you find most impressive about them?

Their ability to deploy resources to help us build the test suite has been invaluable—they're a great partner in that way. Rather than just providing staff to run through each step of the test, they're able to add value by helping us build the system we're using for testing.

Are there any areas they could improve?

They're very sensible in the way they have a backup plan for replacing any staff on our project. They could, perhaps, be more transparent and alert us as those changes occur just so we're all on the same page and don't have any questions as to why a particular person is no longer on our project.