QualityLogic

BACKGROUND

OPPORTUNITY / CHALLENGE

• Regression Testing
• Test Planning for Regression Testing

SOLUTION

• High ratings
• Pricing fit our budget
• Good value for cost
• Company values aligned

RESULTS & FEEDBACK

• In-Person Meeting
• Virtual Meeting
• Email or Messaging App

BACKGROUND

Introduce your business and what you do there.

I’m the associate director of IT security and compliance at a university. We have an intergovernmental agreement with a Medicaid agency in our state. Given that, we have certain capabilities in the university space to help solve Medicaid problems. 

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

We hired QualityLogic to collect artifacts from Medicaid providers to make decisions about Medicaid services.

SOLUTION

What was the scope of their involvement?

We created an external-facing portal for Medicaid providers to submit artifacts to us. Our team had an approval process in place, and the approved artifacts would go into a legacy system. We worked with an implementation for the portal, but I noticed that ADA compliance requirements were not a part of their testing plan. QualityLogic helped us with that, and they collaborated with the implementation vendor.

Currently, QualityLogic provides digital accessibility audit and consultation services for our university. The app isn’t live yet, but we’re getting ready to pilot it in the next weeks. 

What is the team composition?

I’ve had a lot of back and forth with Clyde (Head of Growth). Apart from him, 2–4 other teammates have been involved in the app testing. Nathan (App Testing Specialist) leads the audit process. 

How did you come to work with QualityLogic?

As I was trying to build relationships with the university, I reached out to somebody that dealt with varying degrees of testing not just for ADA compliance but also for performance testing. I talked to the lead of our testing unit about our project, and they suggested QualityLogic. The first person I reached out to was Clyde. He got back to me immediately. 

How much have you invested with them?

We’ve invested around $20,000. 

What is the status of this engagement?

The engagement started in July 2023, but the ADA testing happened in October 2023. The engagement is ongoing. 

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement? 

QualityLogic has met my expectations, and they’re exactly what we need to get ADA compliance. They’ve done their job and worked well with the implementation vendor. Moreover, they have an independent ADA compliance team, and we’re going to continue working with them. They’ve found a lot of problems with our ADA compliance requirements, which the implementation vendor has to clean up. QualityLogic is going to come back and retest the system to see where we’re at. 

Further, we’re going to add more functionalities to our portal moving forward, I’d like to maintain a long-term relationship with QualityLogic. We’ll have further engagements with them to do ADA audits on our behalf to ensure we keep up our compliance. 

How did QualityLogic perform from a project management standpoint?

Their project management has been great, and I’ve heard nothing but good things. I’ve been truly pleased with Nathan and his team’s work. We’ve had some hiccups in terms of getting somebody from QualityLogic into the university, but Clyde has worked behind the scenes well and remediated that problem quickly. 

What did you find most impressive about them?

I don’t have the technical expertise to fill the gap in our project. Luckily, QualityLogic has been a great partner. Everything they’ve done — from their timeframes to their collaboration with other ADA people — has been truly helpful. 

Are there any areas they could improve?

No, there aren’t any. 

Do you have any advice for potential customers?

Prepare the SOW, and be clear about the work you want QualityLogic to do. If you do a good job of sizing that up, they’re going to deliver. Spend some time on getting the SOW right, and you won’t be disappointed. 

BACKGROUND

Introduce your business and what you do there.

I’m the QA manager of the payments and hardware side of an e-commerce company. My department handles the hardware testing of readers, printers, payment flows, scanners, and similar things. 

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

Our team had to test a lot of devices from different sources and with different operating systems, such as iOS and Android. As a result, we hired QualityLogic to help us do regression testing on all of the hardware and software. 

SOLUTION

What was the scope of their involvement?

The initial process of onboarding QualityLogic was done before my time. I only came in and took over the engagement as the middleman between QualityLogic and our company. 

Since we had many features going into production, we wanted to give more flexibility to our engineers. That way, they could focus more on urgent tasks, such as writing code, while QualityLogic assisted them with end-to-end tasks. Because we were doing releases biweekly, we used the team’s support for testing features every week. 

What is the team composition?

We worked with 3–4 teammates from QualityLogic. I mainly interfaced with the testing team lead but not with the rest of the team. 

How did you come to work with QualityLogic?

Our company found them through an online search. 

How much have you invested with them?

We spent around $200,000 last year on their services. 

What is the status of this engagement?

QualityLogic worked with us from August 2020–August 2023. 

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement? 

Any app that involved hardware could be difficult to test, especially since we had to find the right expertise to do the testing and make sure a proper environment was set. QualityLogic did an amazing job in this realm. Their task included automating tests for a product that was integrated with hardware, and there was money involved since users used the products for payments. Despite these things, they still did a great job. 

How did QualityLogic perform from a project management standpoint?

QualityLogic achieved our goals. I never had any issues in terms of their task completion since they’d always been able to do the tests we needed them to do. Moreover, they always met deadlines — there were some days when they were the ones waiting for us to give them what they needed. 

Meanwhile, we didn’t use any project management tools, but we did use a test tracking system. This system allowed us to see what the team was doing. A lot of our communication was done through Slack and Google Meet. We usually had a 15-minute meeting per week to discuss what the upcoming releases would look like or talk about any of their questions or concerns. They also gave us daily updates through email.

What did you find most impressive about them?

Their testing lead was extremely great. She didn’t only do things by the book but also did a great job alerting me of the things they noticed. For instance, if they did tests A, B, and C and I wanted to see the results, she also made sure to do tests D, E, and so on. They’d bring my attention to any weird thing they’d notice in those extra tests. This helped us identify bugs that may have flown under our radar. Overall, they did an amazing job in this area.

Are there any areas they could improve?

No, there weren’t any. 

Do you have any advice for potential customers?

QualityLogic will test what you give them. If you want them to do more, then you have to give them more room to do exploratory testing. They’re experienced professionals, so they’re not only good at doing exploratory testing but also great at beta testing. If you have a new feature launching into production soon, they can test it. They can give you good and honest feedback on how they feel about the new feature.

BACKGROUND

Please describe your company and your position there.

I am the Director of Engineering for a medium sized e-commerce company. We provide services to clients that want to sell their products on-line and other clients that want to market those products through an affiliate relationship.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire QualityLogic?

We utilized Quality Logic for staff augmentation to provide quality assurance testing on our application changes prior to deploying those changes to production.

SOLUTION

How did you select this agency and what were the deciding factors?

We found this agency through a local referral to help with staff augmentation while we worked on our automation testing and capabilities. They had the skills and approach needed to perform the manual testing functions that we needed.

Describe the project in detail and walk through the stages of the project.

They provided up to 4 quality assurance testers, a technical lead and a project manager to work on the manual testing for multiple major projects and application support efforts. At least one tester was assigned to each team and in some cases certain team members covered multiple projects simultaneously.

Describe the recruitment process in detail.

We discussed our projects with the account manager and project manager and they recommended the team members and approach.

RESULTS & FEEDBACK

Can you share any outcomes from the engagement that demonstrate progress or success?

The team worked well in performing the manual quality assurance work that we needed.

How effective was the workflow between your team and theirs?

Our workflow requires a high level of business process and systems knowledge. It took some time to gain that knowledge and once team members understood both the business process and systems the workflow began to look seamless.

What did you find most impressive or unique about this company?

We were impressed with the commitment of the team to ensure the success of the projects that they worked on.

Are there any areas for improvement or something they could have done differently?

Resource turnover had caused some issues early on.

BACKGROUND

Introduce your business and what you do there.

I’m at Tubi, a video streaming and entertainment service. Within the organization, I’m the senior director of product for core systems and foundation, the department responsible for our core video experience, international product experience, privacy compliance, and internal tools.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

Our goal was to increase our throughput of linear channels on Tubi. At the time, we were averaging about 15 channel launches per month, and our goal was to hit 200 linear channels by September 30th, 2022.

In order to hit that goal, we needed to increase the number of linear channels we were launching per month. However, our bottleneck was that we lacked QA support to QA our outputs before launching them into our platforms. Thus, we decided to hire QualityLogic to increase our QA capacity and hit that goal.

SOLUTION

What was the scope of their involvement?

QualityLogic has provided us with QA services for our platforms’ linear channels, which are similar to live TV channels. To launch those channels, we first get the stream of assets from partners such as ABC, NBC, and FOX, and our engineering team uploads those assets. Then, QualityLogic provides QA to ensure videos come through our 25 different platforms correctly.

Their QA process includes testing on all those platforms. Whenever they find an issue with things like subtitles, sound, or video quality, they create tickets and work with our engineering team to ensure that those issues are resolved.

Our goal was to launch 200 channels, but halfway into the project, we realized we had extra bandwidth, so we’ve also tasked QualityLogic with other things. Since hitting our goal, their team has been helping us support our existing channels, and they provide ongoing QA to ensure we don’t have any issues with our platforms.

What is the team composition?

We worked with two full-time QA agents, a part-time project manager, and a part-time program manager.

How did you come to work with QualityLogic?

Our director of QA at Tubi found them, contacted them, and handed them over to me to handle the contract negotiations. One of the things that stood out to me was that they were in the US, so we would be able to work the same hours.

How much have you invested with them?

So far, we’ve invested about $60,000, but we’ve budgeted around $250,000 for the year.

What is the status of this engagement?

We started working with them in July 2022, and our engagement is ongoing. We plan to use them for the rest of our fiscal year and beyond.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement? 

QualityLogic has exceeded our expectations. In order to hit our goals, we needed to increase our channel launch capacity from 12–15 channels per month to over 20 channels. However, they achieved a much higher throughput, reaching 30 channel launches per month and hitting our goal in half the time we had budgeted. By late September 2022, we already had 202 channels. They were a pivotal part of helping us achieve our goal; we wouldn’t have achieved it without them.

How did QualityLogic perform from a project management standpoint?

QualityLogic moves very fast. We’ve spent half the money we had originally budgeted because they’ve done more than we thought they would do, demonstrating that they’re a hyper-focused team. We’re very pleased with how fast they get the job done, which ultimately saves us money and allows us to have extra bandwidth to work on other projects.

Due to security issues, we can’t provide QualityLogic with direct access to Shortcut, our internal project management system. Instead, their team uses Excel to create tickets and pass them over to us to then create the tickets in our internal system. However, we’re working on creating a third-party integration that will enable them to pass those tickets directly to Shortcut so that we don’t have to enter each one manually.

As for communication, we have a shared Slack channel through which QualityLogic communicates directly with our engineering team. Additionally, their team joins relevant meetings via Zoom, and we have monthly and quarterly business review meetings with them.

What did you find most impressive about them?

QualityLogic has been very flexible and professional; they’ve done a fantastic job all around. They also have really talented people who have added to our culture in a positive way and become valued members of our team, ultimately improving our productivity.

Additionally, I really like that they’re able to scale their team up and down very quickly, and they don’t try to secure a long contract. Instead, we’re able to work with them month by month based on our needs.

Are there any areas they could improve?

No.

Do you have any advice for potential customers?

If QualityLogic joins your team, allow them to join completely. To get the most out of them, you’ll need to treat them as part of your team and give them access to the resources they need to do their job. When people are happy, they’re more engaged with their work, and they’re ultimately more productive.

BACKGROUND

Introduce your business and what you do there.

I’m a director at a robotics company.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

We engaged QualityLogic to help us with QA testing for our application.

SOLUTION

What was the scope of their involvement?

QualityLogic helped us with manual QA testing. The team performed regression and production testing and documented their findings in test cases. Once they identified bugs, they took screenshots and videos so our development team knew what to look for. Then, our product managers reviewed the test cases, and our development team implemented changes to address the bugs. When QualityLogic identified new bugs that affected the application’s functionality, they advised our product team to reopen the story and triage the bug. 

What is the team composition?

We worked with two QA engineers and a project manager from QualityLogic. 

How did you come to work with QualityLogic?

Our company engaged QualityLogic before I joined. Our team had heard positive feedback about QualityLogic through word of mouth, so we approached them to help us with the project. 

How much have you invested with them?

We invested between $10,000–$15,000 in the partnership with QualityLogic. 

What is the status of this engagement?

We worked with QualityLogic between December 2021–August 2022.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement? 

We measured QA success by looking at the success of each release. They reported all the bugs efficiently and caught most of them before each release. The QualityLogic team delivered thorough test cases and performed immaculate work. We had some problem cases, but those were exceptions.  

How did QualityLogic perform from a project management standpoint?

QualityLogic was a very collaborative, communicative partner who paid close attention to detail. They were kind and dependable, which made the process easier for us. We had daily stand-ups via Google Meet and used Jira to manage the project. We also communicated via Slack.

What did you find most impressive about them?

I was most impressed by their proactive communication style, reflected in the precision of their test cases. QualityLogic consistently followed up with us and took the initiative to loop in our developers as they saw fit. When they identified particularly problematic issues, they’d advise us to block the next release, which was helpful. Throughout the partnership, they kept our customers in mind, which helped us in the long run. 

Are there any areas they could improve?

I have nothing QualityLogic could improve on.

Do you have any advice for potential customers?

I’d recommend engaging with QualityLogic to experience the great outcomes for yourself.

BACKGROUND

Introduce your business and what you do there.

I’m a manager at P.volve, a company that offers solutions to women looking for low-impact exercises that can work with their lifestyle, health, and pace of life.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

We hired QualityLogic to provide app testing services for the app that we were creating. 

SOLUTION

What was the scope of their involvement?

In our current setup, we work with a manager from QualityLogic. Their resources do automated and manual testing for us. They serve as a part of our project team that develops our app. To do the tests, they use Cypress tools. We have web and mobile offerings, and they do tests for both. 

What is the team composition?

We work with a total of nine people from Quality Logic, including the manager, two QA engineers, and five manual testers. 

How did you come to work with QualityLogic?

I’ve only recently started at P.volve, and QualityLogic was already there when I joined.

How much have you invested with them?

We’ve invested between $1 million–$5 million. 

What is the status of this engagement?

We started working with QualityLogic in June 2020, and the engagement is ongoing. 

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement? 

QualityLogic is highly efficient in their testing work. They put a lot of effort, and compared to the other quality testing vendors we’ve had, QualityLogic provides 10x the amount of work as other teams.  

Given that, my internal team loves them. They have the best people to work with; it’s been amazing working with them.

How did QualityLogic perform from a project management standpoint?

Their project management is excellent. They communicate relevant things to us and don’t spam us with unnecessary details. The team also gives us ample notice if they have to do more than 40 hours per person in a week so that we can adjust our budget and expectations accordingly.

They’re great at telling us what they’re trying to deliver. In a lot of ways, their work is our rock as we undergo massive shifts as a company over the last few months. Their output is the most solid piece that helps direct everything else around it.

In terms of tools, we primarily use Jira to manage the project and GitHub for source control. To communicate, we use Zoom, Slack, Outlook, and Confluence. 

What did you find most impressive about them?

QualityLogic is the best QA team I’ve ever worked with — I’ve been doing QA work for 25 years, and I’m stunned at how good they are at their job. They have everything that we want in a QA team; they’re flexible and always maintain good quality. 

Are there any areas they could improve?

No, there aren’t any. 

Do you have any advice for potential customers?

Understand that QualityLogic brings a huge amount of value to the table. If you go for a team that’s less expensive, you’re going to get better rates but won’t get value. Meanwhile, the QualityLogic team does many things above and beyond a standard organization. 

For instance, they supply their own mobile devices so that they can test the app on various types of phones. If somebody from their team has to go out for a week, they automatically have a backup that can step into the position. Overall, you’re going to get great value out of the money you invest in their services. 

BACKGROUND

Introduce your business and what you do there.

I’m a director with Inspirant Group, a consulting company.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

We were tasked with building a very large and complex application that required extensive testing. We didn’t have enough resources internally to accomplish this, so we decided to hire QualityLogic to help fill this gap.

SOLUTION

What was the scope of their involvement?

We hired QualityLogic to help test the application from all aspects. This involves testing the different types of web browsers that the app can be used with and testing all of its components to come up with a way to break the system so we can improve it. They work in an agile format, so they do individual and regression testing every two weeks.

What is the team composition?

We work with an automation tester, an automation engineer, two system testers, and a project manager from QualityLogic

How did you come to work with QualityLogic?

We found QualityLogic through an online search. We decided to go with QualityLogic for this project based on their responses in our interview.

How much have you invested with them?

We’ve spent somewhere around $100,000.

What is the status of this engagement?

We started working together in August 2021, and the engagement is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

QualityLogic has done a tremendous amount of individual and regression testings, and they’ve successfully identified several defects or bugs within the application that we are actively working to fix.

How did QualityLogic perform from a project management standpoint?

QualityLogic has been great in terms of project management. They’re self-sufficient and incredibly flexible in their ability to maneuver people to accommodate our needs. Their communication has also been fantastic, and they’re very comprehensive in their work. They join all of our meetings and provide detailed updates on their test cases and all of the bugs they’ve identified.

What did you find most impressive about them?

I’m most impressed with QualityLogic’s expertise in pure testing. That, combined with their excellent communication and the breadth of what they can test, really stood out to us.

Are there any areas they could improve?

I don’t think there’s anything they could have done better, and we’re looking forward to doing additional work with them in the future.

Do you have any advice for potential customers?

Make sure to have a good sense of what your needs are before engaging QualityLogic. They have a breadth of expertise in different areas that will help them identify the right people to step on your project if you let them know what you're ultimately going to need.

BACKGROUND

Introduce your business and what you do there.

I’m working for Spin Electric Scooter Company as the acting quality manager of the company, handling all our QA efforts.    

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

They provide QA resources for our two mobile apps.

SOLUTION

What was the scope of their involvement?

They do regression testing for new releases, which happen every two weeks, and they’re also writing test cases for all new features. They handle most of our QA and regression testing work.

We coordinate the work with releases coming up, looking at features to test and adding in additional test cases if needed. They’re fully involved in all scenarios.

What is the team composition?

We had to reduce the team last year, but we started with 6 QA engineers. Now, we have two QA resources on our team. Our budget shrank; they didn’t do anything wrong.

We work with a QA lead and a full-time QA engineer. When we were working with six people, there were five full-time engineers. If our situation changes, I will certainly ask for more people.

How did you come to work with QualityLogic?

They were already onboard when I joined the team, but they were brought on by our previous quality manager who’d worked with them at a previous company.

What is the status of this engagement?

We’ve been working together since November 2019.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement? 

Without their help, we wouldn’t be able to achieve our needs at the moment. They have full-time resources running our test cases, and we couldn’t complete this work without them. They’re hard-working people who are dedicated.

How did QualityLogic perform from a project management standpoint?

They work remotely from a different state. We communicate easily, through daily Zoom calls and Slack messages every day. We have a very good collaboration working remotely.

What did you find most impressive about them?

I haven’t worked with other contractors, so I don’t have anyone to compare them to, but I don’t treat them as some third-party I’m working with. I treat them like part of our team. Our entire team treats them as partners.

Are there any areas they could improve?

I don’t have any improvements. They do a great job on all their assignments.

Do you have any advice for potential customers?

I would recommend them if anyone needed help testing their applications. I don’t have any hesitations.

BACKGROUND

Introduce your business and what you do there.

I’m the VP of engineering at BrightLine, an ad tech company within the OTT space.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

We were lacking internal QA support, so we hired QualityLogic.

SOLUTION

What was the scope of their involvement?

QualityLogic tests different software development kit (SDK) products for different platforms, including Apple TV, Android, Roku. They test them on different devices as well. We’ll usually have a kickoff session to gain an understanding of the product and what it should do and how things should flow. 

The team then creates test cases and proceeds to do tests on a per-platform basis. Throughout that process, if there are any questions or ambiguities, they’ll stay engaged to clarify them.

What is the team composition?

We typically work with one QA lead, but it could be a combination of two at any given point.

How did you come to work with QualityLogic?

Our previous CTO onboarded them, and I picked up where he left off.

How much have you invested with them?

The cost ranges depending on the level of support we need. Typically, we engage them for at least 20 hours per week, but it can go up to 40 or more hours. We spend $5,000–$10,000 per month with them, and the total cost so far is in the $50,000–$200,000 range.

What is the status of this engagement?

We started working with QualityLogic around October 2018, and our engagement is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

It’s been tremendous. Before working with QualityLogic, we had a single QA teammate internally, and we were experiencing a lot of bugs and complaints from media partners. 

We saw a huge decrease in complaints after we brought QualityLogic onboard, and we saw a big increase in the quality of our releases. It’s been a tremendous impact overall because they’ve helped us decrease bugs, processes, and things of that nature.

How did QualityLogic perform from a project management standpoint?

We discuss what should be tested during the week, and they give us daily updates on any issues they find and report them in Jira. QualityLogic also gives detailed descriptions of how they found the issue and where, along with how to reproduce it. They have a very good process for that.

We have weekly sessions every Monday, and we have a lot of communication over email, which is pretty clear. We also hop on phone calls if we can’t get the understanding through email. 

What did you find most impressive about them?

This is the first QA team I’ve worked with while at the company, but they’ve definitely been better in terms of process and tracking compared to our internal resources.

Are there any areas they could improve?

It’s not necessarily a bad thing, but, in terms of the collaboration between our companies, having more scheduled sessions would help.

Although we had weekly meetings, more frequent sessions throughout the week would be good as well. It can take me longer to get to an email if I’m swamped with work, so having calls and screen shares would make it easier to describe elements of the project.

Do you have any advice for future clients of theirs?

QualityLogic is very in-depth with the products they QA, but we may have an urgent release, where identifying blockers becomes most important. It’s important to understand the product in terms of what’s critical and what isn’t and how that can impact other aspects.