Visit Website

On-Demand QA. Ongoing ROI.

4.9
11 reviews

For over 30 years, QualityLogic has worked with organizations to improve the quality of web and mobile applications, smart energy solutions and imaging systems (3D print systems, printers, fax devices and Fax over IP). The company’s software quality assurance and testing teams provide cost-competitive, U.S. onshore expertise to support the efforts of its clients’ agile development programs.

QualityLogic’s test tools are the de facto and official standards for verifying interoperability and conformance and formal certification testing in many industries. 

Whether you’re deploying a mission-critical web application, a consumer-oriented website, or a mobile application, QualityLogic can provide a quick, efficient, cost-effective Quality Assurance and Testing solution.

READ MORE...
 
$5,000+
 
$25 - $49 / hr
 
50 - 249
 Founded
1986
Show all +
Boise, ID
headquarters
other locations
  • 2245 First Street, Suite 103
    Simi Valley, CA 93065
    United States
    +1.208.424.1905
  • 4045 NW 64th Street, Suite 120
    Oklahoma City, OK 73116
    United States
    +1.208.424.1905

Portfolio

Key clients: 

Fortune 100 Media Company

Fortune 500 Biotech Pharmaceuticals Company

Leading Social Relationship Company

 

Reviews

4.9
11 reviews
Sort by

QA for Assessment Provider for School Districts

“QualityLogic stood out from the beginning, and we were very impressed by their services and abilities.”

See More

5.0

Quality: 
5.0
Schedule: 
4.0
Cost: 
5.0
Willing to refer: 
5.0
The Project
 
$50,000 to $199,999
 
Jan. 2019 - Ongoing
Project summary: 

QualityLogic supports QA for an educational assessment provider. They oversee operating system, browser and device compatibility testing and accessibility testing.

The Reviewer
 
501-1000 Employees
 
Portland, Oregon
Michelle Hochstetler
Sr. QA Manager, NWEA
 
Verified
The Review
Feedback summary: 

The whole process is more efficient, and QualityLogic is very thorough in their reporting. With an eye for detail, they helped solidify certain mechanisms. They are communicative, mindful of resources, and flexible.

A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.

BACKGROUND

Introduce your business and what you do there.

We are a growth assessment provider for school districts, helping to measure areas of opportunity for students that teachers can focus on. We deliver assessments through a hosted solution, and much of our software is web application based. I'm a QA manager.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

We hired them to provide more coverage in our operating system and browser compatibility testing. We have multiple development teams who focus on functional and regression testing, but we don't always have time to test operating systems, browsers, and devices.

SOLUTION

What was the scope of their involvement?

QualityLogic runs compatibility tests on a monthly basis to make sure we have no regression in any of the operating systems or browsers we support. As the partnership has gone on, we've added more coverage across our product suite: roles testing, new feature testing, and testing to resolve production issues. They're also extending our accessibility testing to ensure good keyboard navigation and screen reader coverage for student assessments. We hope to expand that as we grow our offerings. We continue to find uses for them, and they plug holes in our coverage.

What is the team composition?

We work with a project leader. She coordinates a team of about three testers, and they allow some flexibility when we need to scale.

How did you come to work with QualityLogic?

I found them via a web search. We identified vendors who could meet our need, and we completed an RFP process. QualityLogic stood out from the beginning, and we were very impressed by their services and abilities.

How much have you invested with them?

We've spent approximately $70,000 on their services.

What is the status of this engagement?

The first official engagement started in January 2019, and they've done testing with us in every month since.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

We track the number of test runs they complete, as well as the defects they find. We've improved both sides of the process, and are pleased with the progress.

How did QualityLogic perform from a project management standpoint?

They're very communicative, and their status reports have the perfect amount of information. We have a daily standup with them, and we also use RingCentral to send messages throughout the day. They're very mindful of resources, and they only make changes when we give them the go-ahead. They continue to improve their processes and are very efficient.

What did you find most impressive about them?

This is a unique collaboration for us, but their flexibility, experience, attention to detail, and communication all stand out.

Are there any areas they could improve?

We haven't run into anything. There has been a learning curve on our side, but I can't think of anything they should improve or change.

Do you have any advice for potential customers?

Make sure that you are clear about your own processes. QualityLogic is more than willing to help you as you solidify those processes, and it is useful to maintain an open communication mechanism for feedback.

5.0
Overall Score
  • 4.0 Scheduling
    ON TIME / DEADLINES
    They keep us apprised of their progress, and their scheduling is solid.
  • 5.0 Cost
    Value / within estimates
    The costs are great. In the short time we've worked with them, we've grown to trust them, and we get tremendous value from them.
  • 5.0 Quality
    Service & deliverables
    Their documentation in defect reports and test case results is always very thorough.
  • 5.0 NPS
    Willing to refer
    I would definitely recommend them, and would enjoy working with them again.

Application Testing for Mobile Content Creation App

"They’re saving us mental energy, effort, and time."

See More

5.0

Quality: 
5.0
Schedule: 
5.0
Cost: 
5.0
Willing to refer: 
5.0
The Project
 
$10,000 to $49,999
 
Apr. 2019 - Ongoing
Project summary: 

QualityLogic provided an outside team to perform QA testing services for a mobile app. They did regression testing and wrote test scripts for the app. This was all done on a three-month development timeline.

The Reviewer
 
11-50 Employees
 
Los Angeles, California
Product Manager, Sidebench
 
Verified
The Review
Feedback summary: 

QualityLogic’s efforts were instrumental to the app’s launch. The client was able to secure another large contract due to the quality of the app. Responsive and flexible, their team adapted to a compressed schedule and offered a high level of detail in testing.

A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.

BACKGROUND

Introduce your business and what you do there.

I work for a product consulting company called Sidebench. We do product design and development for our clients.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

We’re a fairly nimble company, so we typically bring in outside resources to staff a project. We were building a mobile and web application for a large Hollywood production company. Due to a three-month deadline, we needed to ramp up resources with an outside team very quickly. This meant we needed a team with consistent, high-quality work to which we could delegate testing responsibilities.  

SOLUTION

What was the scope of their involvement?

QualityLogic’s team performed rigorous, manual regression testing. They drew up their QA testing plans and wrote test scripts in a silo. As we got closer to launch, we brought them closer to our team.

What is the team composition?

We had one point of contact. He worked with two engineers who did a large portion of the manual work. We also had direct communication with Joe (General Manager, QualityLogic), a senior member of the team who worked with resourcing, customer service, and account management.

How did you come to work with QualityLogic?

We’ve worked with them in the past, and they’ve become our go-to team with these types of projects. It was my first time working with them, but they were highly recommended by another member of my team. Picking QualityLogic was a no-brainer, it was just a matter of how quickly we could get started.

How much have you invested with them?

We’ve spent around $25,000 with them.

What is the status of this engagement?

The project began in early April 2019 and is still ongoing. This is the most recent extension of a contract we have with them. We’ve extended the contract 3–4 times.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

They helped us achieve a huge goal in launching the Hollywood production app in a short period of time. Due to that success, we’ve negotiated another $300,000 contract from the client. Without being able to pass that initial launch, we wouldn’t have gotten any additional contracts. They've put in a ridiculous amount of time for us.

At first, our development team thought QualityLogic was overly picky or going too far into detail, but when the first engagement ended, our team asked for them to come back immediately because they knew how much value they added. I think that showed that the level of detail they provide makes things easier for all of us. They do such a robust job of testing different devices and browsers. Every ticket has a video or a screenshot associated with it. They’re saving us mental energy, effort, and time.

How did QualityLogic perform from a project management standpoint?

I was in regular communication with the main contact at QualityLogic every single day. We mostly used Jira and Slack to communicate.

They were always flexible with my hours and what I was working on. From a project planning perspective, they’re so flexible. I think that’s something they pride themselves on. There’ve been several instances where I’ve set an ambitious goal, and they were quickly able to adjust their resources, build a new test plan, and adapt.

What did you find most impressive about them?

The ability to ramp up and ramp down has been very important for us. During regression testing phases, we’re able to easily add testers. When testing is done, we can ramp back down to a single person. The resourcing plans are very accommodating to our needs. Along with this, it feels like they’re truly bought in and care about these projects. It’s not just another contract to them, and that’s very impressive given the number of projects they have.

Are there any areas they could improve?

No, it’s been a really easy process. Sometimes from a project burn and hours perspective, it was a bit high, but they always did a great job communicating that. There was never any time poorly spent; if they had no more tasks, they would tell me they were doing some other task. I’ve honestly not had any complaints.

Do you have any advice for potential customers?

You should be as detailed in your requests as possible. Ask what your best-case scenario would be. Sometimes I felt like we were asking for too much, but they were able to easily meet our demands. If you ask what you’d ideally like to happen, they’d always be able to accommodate.

When you’re forming the partnership, set out the ground rules and your own expectations. I’d advise having an open conversation about how closely they can meet those expectations, which is a testament to how flexible, knowledgeable, and caring they are in this line of work.

5.0
Overall Score
  • 5.0 Scheduling
    ON TIME / DEADLINES
  • 5.0 Cost
    Value / within estimates
  • 5.0 Quality
    Service & deliverables
  • 5.0 NPS
    Willing to refer

Quality Testing for Mutual Benefit Organization

“QualityLogic seemed to be the one company that had the most relevant testing experience.”

See More

5.0

Quality: 
5.0
Schedule: 
5.0
Cost: 
4.0
Willing to refer: 
5.0
The Project
 
$200,000 to $999,999
 
Jul. 2011 - Ongoing
Project summary: 

QualityLogic developed a proprietary testing tool for a smart products industry group. Through a long-term partnership arrangement, they apply the tool and provide ongoing consulting and development to the client and its member companies.

The Reviewer
 
1-10 Employees
 
San Francisco, California
Rolf Bienert
Technical Director, OpenADR Alliance
 
Verified
The Review
Feedback summary: 

The test tool works well and feedback has been positive. Responsive and flexible, QualityLogic’s knowledgeable expert team understands the problem and works quickly and effectively. The partnership has been efficient and highly beneficial.

A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.

BACKGROUND

Introduce your business and what you do there.

I’m the technical and managing director of the OpenADR Alliance. We’re a membership-based industry standards group for the smart goods space which includes devices like smart thermostats and electric-vehicle charger stations.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

One of our core organizational functions is to organize tests and certify products according to our standards. Almost nine years ago, we made the decision to develop and own a specific certification test tool that we would use to certify products. We needed a partner to work with us to develop it.

SOLUTION

What was the scope of their involvement?

We’ve been working with QualityLogic for almost nine years now. They initially built the test tool for us including all of the testing, the test plan, and some test cases. We now co-own the test tool with them and QualityLogic serves as an endorsed partner that can test products for our members based on our standards and certify them. They’ve delivered that tool to a number of our member companies. They provide consulting services and additional development support to our members upon request. We also maintain a working relationship to update the software every couple of years and do other, smaller projects as needed.

What is the team composition?

We work directly with a lead engineer who serves as a project manager and with the general manager. There are other programmers who work behind the scenes.

How did you come to work with QualityLogic?

We put out an RFP to a number of test equipment companies as well as test houses. We didn’t want to just hire a programmer to build the test tool; we wanted to establish a partnership. We understood that we would need support and modifications over the years and that a long-term arrangement would be more effective. We eventually selected QualityLogic based on a combination of their experience, their pricing, and the partnership arrangement they offered.

How much have you invested with them?

We’ve spent about $400,000.

What is the status of this engagement?

We started working together in July or August 2011 and it’s ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

We haven’t specifically tracked performance but the test tool works very well. I get very few questions about it, if at all, and we’ve had a positive response.

How did QualityLogic perform from a project management standpoint?

They’ve performed well. Working directly with one of their lead developers has worked well for us. He’s always been on top of things. They’ve worked well according to timelines and plans.

What did you find most impressive about them?

I would say two things. For one, they’re one of the few companies I’ve found with the experience in test systems and testing programs of our sort. QualityLogic seemed to be the one company that had the most relevant testing experience. They also wanted to dive into the smart goods space, which was great for us.

Also, they have the right people on their team. Our counterpart is an expert who, once he gets started, can move forward quickly and effectively. He’s incredibly knowledgeable and easy to communicate with.

Are there any areas they could improve?

I can’t think of anything in particular. We’ve been working together for such a long time and everything moves the way we’d like it to. They could potentially be a bit better at tracking their hours. It would be nice to have advance warning if we’re going to exceed our estimates. But I know it’s sometimes hard to quantify work when you’re building something from scratch. The standards were changing as they built the test tool.

Do you have any advice for potential customers?

The challenge is always to scope a project well, which can be tricky given so many unknowns. Make sure your project is fully outlined and most aspects are clear. That way there won’t be any surprises.

5.0
Overall Score Overall, I have no complaints. They’re more than a vendor: they’ve been a really valuable partner.
  • 5.0 Scheduling
    ON TIME / DEADLINES
    We haven’t set any strict deadlines recently but they’ve always been very flexible when development targets changed. They adjust quickly.
  • 4.0 Cost
    Value / within estimates
    We get a certain discount given our partnership agreement. Their full rate is not overly expensive for consultants but it can add up. They’re definitely worth the money, though.
  • 5.0 Quality
    Service & deliverables
    They’re quick on response times, which is very important for us, especially with our members.
  • 5.0 NPS
    Willing to refer
    Given their responsiveness, their understanding of the problem, and the quality of their lead engineers, they’re good.

Quality Testing for Transportation App

"QualityLogic integrated seamlessly with our business and provided high-quality services."

See More

5.0

Quality: 
4.5
Schedule: 
4.5
Cost: 
4.0
Willing to refer: 
4.0
The Project
 
$50,000 to $199,999
 
Mar. 2018 - Jan. 2019
Project summary: 

QualityLogic tested web and mobile applications.

The Reviewer
 
201-500 Employees
 
San Francisco, California
Director of Software Systems Quality, Transportation Company
 
Verified
The Review
Feedback summary: 

The team’s proactive approach resulted in a seamless partnership. QualityLogic was timely and created value.

A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.

BACKGROUND

Introduce your business and what you do there.

I’m the director of software systems quality for a transportation company.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

We wanted a nearshore vendor as an affordable alternative for testing resources.

SOLUTION

What was the scope of their involvement?

QualityLogic tested our web and mobile applications, specifically the latter’s flow. Tangentially, the team also looked at our automation infrastructure.

What is the team composition?

We worked with a project manager and an account executive.

How did you come to work with QualityLogic?

I had worked with QualityLogic at another company and their mobile testing expertise stood out. We selected the team for this project because they clearly understood our business domain.

How much have you invested with them?

We spent between $50,000–$199,999.

What is the status of this engagement?

We started working with them in March 2018 and the partnership ended in January 2019.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

We tracked results via daily touch points to ensure proper oversight and clarification. Our side actively monitored the testing and execution.

How did QualityLogic perform from a project management standpoint?

QualityLogic integrated seamlessly with our business and provided high-quality services. Their attention to detail was notable.

What did you find most impressive about them?

The high-energy team was engaged throughout the partnership. They asked probing questions and proactively communicated any obstacles or challenges.

Are there any areas they could improve?

The partnership ended before we wanted, which limited QualityLogic’s involvement in the automation initiative. We wanted them to take a more active role with that.

Do you have any advice for potential customers?

Constant engagement with QualityLogic keeps the partnership on track. Clients that treat them more like an integrated team than an outsourcing vendor will get better results.

5.0
Overall Score They demonstrated an impressive level of engagement and enthusiasm.
  • 4.5 Scheduling
    ON TIME / DEADLINES
    They were proactive communicators and met every timeline.
  • 4.0 Cost
    Value / within estimates
    Although the team wasn’t inexpensive, the quality results were worth the investment.
  • 4.5 Quality
    Service & deliverables
    The team was detail-oriented and kept the number of ongoing production issues to a minimum.
  • 4.0 NPS
    Willing to refer
    The execution was solid and the results were extremely positive.

QA Testing for SaaS Software Company

"Their entire team is professional, knowledgeable, and willing to go the extra mile—we consider them a partner."

See More

4.5

Quality: 
4.0
Schedule: 
4.5
Cost: 
4.0
Willing to refer: 
5.0
The Project
 
$50,000 to $199,999
 
Mar. 2017 - Ongoing
Project summary: 

QualityLogic provides ongoing testing services for a SaaS software provider. They conduct tests, create analytic reports, and suggest solutions for a variety of projects.

The Reviewer
 
1-10 Employees
 
Phoenix, Arizona
Matt Dee
CTO, Direct Care Innovations
 
Verified
The Review
Feedback summary: 

The testing assistance has enabled a more efficient, accurate process for these project phases. The extra support from QualityLogic has resulted in stronger overall metrics for the systems being tested. The team has been dedicated, flexible, and thorough throughout the engagement.

A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.

BACKGROUND

Introduce your business and what you do there.

I’m the CTO of Direct Care Innovations. We’re a healthcare technology company offering SaaS-based software to service providers in the Medicaid and Medicare markets.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

We were in the testing phase for multiple projects and didn’t have enough internal resources to perform quality tests. We wanted to outsource to a vendor who had the skills we lacked.

SOLUTION

What was the scope of their involvement?

QualityLogic test all of our new developments and bug solutions. They conduct tests on our products’ functionalities, performance, and security. Their team uses their own testing tools and show us analytics on the success of our products. The reports show if bugs have been solved and if a system functions correctly. When they find an issue, they screencast with our developers to make the locating and reproducing process simpler.

What is the team composition?

They provide us with five full-time testers and one part-time tester, who also functions as the project lead. Their team is on constant retainer, so there is always a member available to help us.

How did you come to work with QualityLogic?

I searched for vendors online after performing marketing research. One of our system analysts found them, and we preferred their smaller composition compared to the other behemoth-like companies. We also liked their flexibility toward an open-ended arrangement with an undefined scope of work.

How much have you invested with them?

We’ve spent around $150,000, so far.

What is the status of this engagement?

We started in March 2017, and we keep them on retainer for our testing needs.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

We’ve been able to increase the number of test cases we get through drastically. Along with quantity increasing, our KPIs show that the thoroughness of the tests is much better than we were doing previously. Our customers have given us positive feedback on the enhanced usability of our system’s overall performance and security.

How did QualityLogic perform from a project management standpoint?

They provide us with detailed documentation about the issues they find and give us quality suggestions for solutions. We receive daily updates from the project lead, which is one of the best parts of working with them. They deliver everything on time and within the budget we set.

What did you find most impressive about them?

Their entire team is professional, knowledgeable, and willing to go the extra mile—we consider them a partner. We rely on them heavily, and they’ve never let us down, so we find the project to be money well-spent.

Are there any areas they could improve?

Although there have been inevitable bugs that went undetected, it’s infrequent, and they give us a reasonable explanation. Our work is complex, and there will always be some issues that slip through—we can’t blame them for that.

4.5
Overall Score All things considered, they are as good a partner anyone could hope to work with.
  • 4.5 Scheduling
    ON TIME / DEADLINES
    They finish everything on time; there are only a few instances where they missed a date.
  • 4.0 Cost
    Value / within estimates
    hough they aren’t inexpensive, quality service never is.
  • 4.0 Quality
    Service & deliverables
    Nothing is ever perfect, but they produce quality output and feedback.
  • 5.0 NPS
    Willing to refer

Quality Assurance for Virtual Reality Startup

“QualityLogic improved our approach to QA testing, helping us rewrite hundreds of previously deficient tests as a result.”

See More

4.5

Quality: 
4.5
Schedule: 
5.0
Cost: 
4.5
Willing to refer: 
5.0
The Project
 
$200,000 to $999,999
 
Nov. 2017 - Ongoing
Project summary: 

QualityLogic provided full-time QA staff to create test plans and test code developed in C++ for virtuality reality technologies. 

The Reviewer
 
51-200 Employees
 
San Francisco, California
Thomas Schofield
Chief Operating Officer, High Fidelity
 
Verified
The Review
Feedback summary: 

QualityLogic not only provided consistently useful test results, but also assisted in the development of improved test plans. They've processed hundreds of test cases at a relatively fast rate. Prepared with a backup plan, their overall approach to customer needs is professional and strategic. 

A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.

BACKGROUND

Introduce your business and what you do there.

I'm the chief operating officer for High Fidelity, a San Francisco-based startup in the virtual reality space that's building a platform for free servers running virtual reality environments. We have about 50 employees, 24 of whom are engineers.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

We wanted to scale up our QA testing for the development work we're doing without expanding our in-house team too quickly. We wanted to keep a good balance of both engineering and QA staff.

SOLUTION

What was the scope of their involvement?

QualityLogic provided additional QA support for already developed software in C++. They execute test plans that we develop to verify that the code our engineers produce works as it's intended to. These tests require due diligence to ensure we're meeting specific outcomes of each feature within our platform. They report back to us, alerting us to any bugs or broken code.

What is the team composition?

Paul (Client Program Manager, QualityLogic) and Joe (Director of Program Management, QualityLogic) are our main point of contacts, but there are an additional six people from their team working on QA for us full-time.

How did you come to work with QualityLogic?

We had worked with another provider previously but were unhappy with their results. The rate at which they would go through tests and the consistency of their responses on the tests wasn't what we wanted, so we searched online and interviewed a few other vendors. We chose to work with QualityLogic in parallel with our previous vendor at first as a comparison, but ultimately decided to solely work with them based on their performance.

How much have you invested with them?

We work with them on a month to month agreement under a budget of $500,000 per year.

What is the status of this engagement?

We've been working with them since November 2017.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

QualityLogic improved our approach to QA testing, helping us rewrite hundreds of previously deficient tests as a result. The number of issues they've identified runs into the hundreds, which has helped us significantly improve the quality of every release in the process. They also process tests at an impressive rate.

How did QualityLogic perform from a project management standpoint?

They're very practical in how they deal with unexpected situations and have two people on their staff trained on our technology should they need to replace or add a couple QA staff to our projects. That approach is very sensible and shows that they're resilient to change.

When we've needed overtime from their current staff, they've been very responsive. Everything seems very well managed.

What did you find most impressive about them?

Their ability to deploy resources to help us build the test suite has been invaluable—they're a great partner in that way. Rather than just providing staff to run through each step of the test, they're able to add value by helping us build the system we're using for testing.

Are there any areas they could improve?

They're very sensible in the way they have a backup plan for replacing any staff on our project. They could, perhaps, be more transparent and alert us as those changes occur just so we're all on the same page and don't have any questions as to why a particular person is no longer on our project.

4.5
Overall Score
  • 5.0 Scheduling
    ON TIME / DEADLINES
  • 4.5 Cost
    Value / within estimates
  • 4.5 Quality
    Service & deliverables
  • 5.0 NPS
    Willing to refer

QA Testing for Global IT Development Company

"They went above and beyond our expectations."

See More

5.0

Quality: 
4.0
Schedule: 
5.0
Cost: 
4.0
Willing to refer: 
5.0
The Project
 
$50,000 to $199,999
 
May 2016 - Ongoing
Project summary: 

To ensure their machines and file formats were compliant with new specifications, a large IT company looked to QualityLogic to develop tools that automate the testing process.

The Reviewer
 
10,000+ Employees
 
Barcelona, Spain
SWQA Engineer, IT Services Company
 
Verified
The Review
Feedback summary: 

The tools developed by QualityLogic have already detected several issues with the machine code and even found issues with the new specifications themselves. Their team is committed to the business objective and will work hard to ensure their partners' goals are achieved.

A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.

BACKGROUND

Introduce your business and what you do there.

We are a large associate development company, with locations all around the world. I work in software quality.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

We needed to support a new file format in our machines. There were some new specifications that were being developed at the same time we were developing our machines. We needed QualityLogic to help develop a suite of files that covered the specifications. We also needed help developing the tools to test the files with our machines.

SOLUTION

What was the scope of their involvement?

We had two main goals. We handed QualityLogic the specifications that were in development for the new file format. First goal: QualityLogic needed to develop a set of files, a plot suite, that covered all of the specifications, including the possible variants, the positive and negative clauses, and everything that the specification says. The specifications were very complex with a core module and several extensions. In the end, there were 300 different plots that covered all the specifications. Our proposal was to make sure that our machines were compliant with the specifications. If a machine is going to consume this kind of file format and be considered "compliant with the spec," it needs to accept certain kinds of files and reject other kinds of files. If all the positive cases pass and all the negatives cases fail, we could be sure that we were compliant with the specification.

The second goal was to develop a set of tools so that our team didn’t need to test our machines manually. Every week, there are 300 untested files. QualityLogic developed a set of tools that were automatically able to connect with our machines, send the files, and report the results. The evaluation needed to be formed according to both our quality standards and those of our development team. Every time our development team made a change to the code, they had to make sure that it was compliant with the specifications.

What is the team dynamic?

I had one person that I interfaced with, and I also spoke to a senior engineer. We discussed requirements and technical matters and also budget, scope, etc. I know they had a team of developers and testers behind them, but I only interacted with these two people.

How did you come to work with QualityLogic?

They had worked with my company 15 years ago for another project. We recalled that their work was very good, so we decided to contact them. It was an internal recommendation from a colleague.

How much have you invested with them?

Our initial budget was around $200,000. We expanded the budget a couple of times because we wanted to perfect some tools and add more capabilities with them, to a final of $250,000.

What is the status of this engagement?

We started working with QualityLogic in the spring of 2016; the first project is finished and the second project is ongoing. The first set of standards was completed in seven or eight months.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

The most valuable metric is that we ended up with 300 valid test cases developed by QualityLogic. We also encountered 20-25 issues in our code that we found thanks to this plot suite.

We didn’t intend to qualify the specifications, but QualityLogic found several issues in the specifications themselves. They were in contact with the authors so they could correct them. There was a lot of success in the requirements. That was very valuable because if we had built the code based on wrong specifications, it would have cost us a lot of effort to re-build later on. That was not expected, but QualityLogic reviewed all the specification and asked questions and proposed improvements which was very helpful.

How did QualityLogic perform from a project management standpoint?

We had a meeting every week with a call. In every call, they told us the progress in the development and the new plots. We discussed requirements and priorities. If they needed to choose between different options for a tool, for example, we discussed the best approach. They would inform us on the budget and the cost of doing a certain functionality. They were very transparent. We didn’t have any problem with priorities or time estimations. If they said they were going to do something in a certain time, they made sure they did. We used GoToMeeting for our communication.

What did you find most impressive about them?

Their commitment is impressive. They were very involved in the project. They cared about the success. They understood our problem and wanted to know how we were going to use the product in order to adequate their job. They went above and beyond our expectations. They were good at avoiding failures and giving suggestions. In the beginning, I was the only interface with my company, but in the end, we saw that it was better if they spoke directly to our engineers. Now there’s very frequent communication between their team and ours. Their work is going well, so there will be other opportunities working with us.

Are there any areas they could improve?

No. If I had any suggestions of improvement for them, I told them and they fixed it right away.

5.0
Overall Score Because of their experience, their commitment and their results.
  • 5.0 Scheduling
    ON TIME / DEADLINES
    We had some delays. The specifications are new, so it wasn’t anything they could have done on their side. It was just normal delays when there are unknowns.
  • 4.0 Cost
    Value / within estimates
    The cost was reasonable.
  • 4.0 Quality
    Service & deliverables
    Sometimes the tools came with defects, but that was fine because if we needed to choose between having a perfect delivery every week or having a working tool in less time, I chose the working tool.
  • 5.0 NPS
    Willing to refer
    The project has been great, and the results are really fine. We achieved our purpose. I have already recommended them to my colleagues.

QA Web Testing for Cancer Research Nonprofit

“They’re a good third party sanity check for us.”

See More

5.0

Quality: 
5.0
Schedule: 
5.0
Cost: 
5.0
Willing to refer: 
5.0
The Project
 
Confidential
 
Aug. 2014 - Ongoing
Project summary: 

Hired to ascertain proper functioning of all public-facing forms, QualityLogic ran transactional, cross browser and device testing, ensuring a glitch-free user experience.

The Reviewer
 
11-50 Employees
 
Los Angeles, California
Robyn Bucknam
Vice President, Stand Up to Cancer
 
Verified
The Review
Feedback summary: 

At least 20 significant bugs were found for each engagement. On a recent project, zero user complaints were received regarding issues on the forms. This team has earned their five-star review with accuracy and thoroughness of their work, proactive communication, and flexible scheduling.

A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.

BACKGROUND

Introduce your business and what you do there.

Stand Up To Cancer is a division of the Entertainment Industry Foundation, a 501(c)(3) charitable organization that raises funds to support collaborative cancer research.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

We had a very large-scale event that usually sees a very high level of transactional traffic. We needed QualityLogic to help make sure that all of our forms were functioning correctly and that there were no surprise issues that would be encountered by any of our users.

SOLUTION

What was the scope of their involvement?

We engaged QualityLogic to run transactional testing as well as cross browser and device testing. We began by outlining what we were concerned about, and then QualityLogic looked over the plans, suggested what we should test, and then ran a full suite of functional and cross browser tests for us to make sure pages displayed correctly and that there were no bugs or submission issues.

We've worked with QualityLogic for three years and the team that we work with has always stayed consistent. It's made it very easy to engage with them. They're pretty familiar with our forms so it's easy to go back to them time and time again. We worked heavily with Don Walker, Paul Morris, Don Burke, and several other people on the team, and they've always been really great.

How did you come to work with QualityLogic?

Someone we had worked with knew of them and said that they might be worth considering. When we first engaged with them, we looked at several different testing companies, and we just liked the style of QualityLogic. They were very technical, but easy to work with. It was our first engagement with a firm to handle this kind of work as a neutral third party.

What is the status of this engagement?

We started working with them August 2014 and have done several projects together since that time.

RESULTS & FEEDBACK

Could you share any evidence that would demonstrate the productivity, quality of work, or the impact of the engagement?

For each engagement, they’ve found at least 20 reasonably significant bugs that could be addressed. On a recent project, we ended up with zero user complaints of any issues on the forms.

How did QualityLogic perform from a project management standpoint?

They’re very proactive about communicating what they’re working on and their schedule estimation is always very accurate. They are very flexible in timelines and the testing, and they really understand digital work cycles. When we had delays or things that came up, the team was really professional and easy to work with. Our correspondence has been mostly over the phone and email, and then also working with tracking through shared documents.

What did you find most impressive about QualityLogic?

What’s always impressed me is the accuracy and thoroughness of their work and their flexibility. They are usually able to help us make sure that we can walk into any page launch feeling confident that there aren’t any issues that we don’t know about. They’re a good third party sanity check for us.

Are there any areas QualityLogic could improve? ​​​​​​

Not that I can think of.

5.0
Overall Score They have contributed to several projects where we can sleep better at night because of their work.
  • 5.0 Scheduling
    ON TIME / DEADLINES
    They have always delivered exactly when they said they would and had the right resources available for us when we needed it.
  • 5.0 Cost
    Value / within estimates
    They deliver a significant amount for the prices that they charge.
  • 5.0 Quality
    Service & deliverables
    Their work is always accurate and very detail oriented.
  • 5.0 NPS
    Willing to refer
    We’ve been extremely happy across the board with their services and offerings.

QA Testing for Large Broadcast Media Organization

"They are quick learners and are very flexible."

See More

4.5

Quality: 
4.5
Schedule: 
5.0
Cost: 
5.0
Willing to refer: 
5.0
The Project
 
Confidential
 
Apr. 2010 - Ongoing
Project summary: 

QualityLogic augments internal teams in several web and app testing projects. They quickly learned comprehensive internal methods beyond black-box testing and dedicate up to 50 people for QA needs.

The Reviewer
 
51-200 Employees
 
Burbank, California
QA Engineer, Broadcast Media Organization
 
Verified
The Review
Feedback summary: 

QualityLogic is a trustworthy team with a reliable work ethic. Even in wide-scope, high-profile projects, no major bugs have been found. Over time and after onboarding, QualityLogic’s efforts ran more and more smoothly, and they’ve been quick learners who are able to tackle any challenge.

A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.

BACKGROUND

Introduce your business and what you do there.

I am the QA manager of an app development group working on behalf of a large media company. We focus on multiple platforms, including mobile over-the-top content on iOS, tvOS, Android mobile and TV, Amazon Fire TV, and a few others, including the web.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

We were growing fast, and needed testing help. QualityLogic was one of a few vendors who were available to us.

SOLUTION

What was the scope of their involvement?

We assigned QualityLogic some projects and discussed how we wanted the testing to be done. They were open to suggestions and new testing methods. They learned the process from us and took ownership of the testing for a wide array of our apps. This has included all-manual testing, not just black-box (having users interact with the apps), but also with a more complex approach. Based on a user’s location, they will receive different streams for live viewing. For instance, if someone wanted to see a particular New York station, the app would recognize that they are in that location, and give them the right feed. This is challenging to do with black-box testing because we can’t manipulate locations, and need to go into a gray-box area, looking at the APIs and manually manipulating responses so that we receive the desired results.

Another example: our app can handle five shows on a page; we wanted to see what would happen if there was a sixth show. Rather than us programming a test for each use case, QualityLogic has looked at our API and its responses, manually added shows, and assessed the reactions.

The composition of the team has changed recently. We have recently expanded to 50 people, but my interactions are with the project manager and the leads. We have also decreased the team size when there were fewer projects in a fiscal year.

We have worked with other testing vendors, but not at the scale of QualityLogic. They hold around 75-80% of all our projects.

How did you come to work with QualityLogic?

We used to have two different teams in-house: a web team and a mobile player team, which I was a part of. QualityLogic was contacted by someone on the web part. The teams have since merged, and I took over the QA department. I contacted QualityLogic and started giving them business minimally, at first. After they were onboarded around processes and functionalities, we started giving them more projects.

What is the status of this engagement?

We started working with QualityLogic 2 ½ or 3 years ago for my department. They may have started working for the company a little earlier.

RESULTS & FEEDBACK

Could you share any evidence that would demonstrate the productivity, quality of work, or the impact of the engagement?

Over the last year, we haven’t had any major issues in production after the testing performed by QualityLogic. Software will always have bugs, but we haven’t had any major ones that would make us stop everything and rerelease the code. QualityLogic has full coverage of test cases, and the work is done to a level that makes us confident in whatever they provide signoff on.

How did QualityLogic perform from a project management standpoint?

I deal directly with Joe Walker, their program management director, who is extremely flexible and able to present a plan for any challenges I present. I’ve also talked with one of their QA managers.

What did you find most impressive about QualityLogic?

The type of testing we do is not standard, not something which could be done by low-cost resources in India, for example. With QualityLogic, we have access to testers working at engineer levels. They’re willing to adapt, and they’re quick learners. We can teach something to one person, who will then relay it to the whole team.

Are there any areas QualityLogic could improve?

There were things to fix in the beginning, and we provided QualityLogic with ideas on how to change things around, but we haven’t had any production issues in the last six months to a year. When we first started, if there was anything missed on their side, they owned the mistake, which was a positive for us. We still provide feedback to them, but we haven’t had any recent issues.

4.5
Overall Score We had some initial hurdles, but I would rate them a 5 for their current performance.
  • 5.0 Scheduling
    ON TIME / DEADLINES
  • 5.0 Cost
    Value / within estimates
  • 4.5 Quality
    Service & deliverables
    They are quick learners and are very flexible. We have reached the point of leaving all the testing to QualityLogic. There is always room for improvement, though.
  • 5.0 NPS
    Willing to refer
    I have recommended QualityLogic to a few other departments, who have started a collaboration.

Testing Services for Critical Communication Application

"We have a great relationship, and they do excellent work for us."

See More

5.0

Quality: 
5.0
Schedule: 
5.0
Cost: 
5.0
Willing to refer: 
5.0
The Project
 
$50,000 to $199,999
 
June 2014 - Ongoing
Project summary: 

QualityLogic conducts manual, ad hoc, and exploratory testing for mobile and web apps, supporting feature additions via new assessments and flagging issues through a defect-tracking system and usage library.

The Reviewer
 
11-50 Employees
 
Boise, Idaho
CTO, Critical Communication Company
 
Verified
The Review
Feedback summary: 

Time-sensitive initiatives proceeded quickly and launched smoothly to enterprise customers worldwide through QualityLogic’s scalable workflow. The team’s responsiveness, conscientious approach, and willingness to take ownership of product reliability continue to boost output.

A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.

BACKGROUND

Introduce your business and what you do there.

I’m the CTO and VP of engineering for a company that makes mobile and SaaS [software-as-a-service] solutions for emergency and clerical communication management. 

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QualityLogic?

We needed testing for our mobile apps and web app in a way that could be repeated without staffing our own team. We also needed enough physical devices to test everything properly. We’ve worked with other quality companies that focus on test automation. Despite all the automated tests as we build and execute, we still need manual, ad hoc, and exploratory testing. 

SOLUTION

What was the scope of their involvement?

The relationship was already in place when I joined the company. Now, they provide testing services for our applications. As we add more features, they incorporate new tests and have built up a library of those to determine functionality. If new issues are found, they integrate with our defect-tracking system directly. They’re able to enter what type of issue they find, how to reproduce them, and which accounts and users they’re acting on behalf of when they found it. We have other communication channels between them, the development teams, and the support teams in order to refine and triage items and determine what actions we should be taking.

We had a single main point of contact. At the beginning of the year, he started joining us twice a week in our daily standup meetings. He comes to the office and sits in our scrum. That’s working really well. A few other people are involved too.

How did you come to work with QualityLogic?

They’re a local company, and I think a manager of ours at the time ran into them somewhere. I’m not exactly sure how they got connected because that predated me.

How much have you invested with QualityLogic?

We have spent between $150,000 and $250,000 so far.

What is the status of this engagement?

We started working with them in June 2014 and the work is ongoing. 

RESULTS & FEEDBACK

Could you share any evidence that would demonstrate the productivity, quality of work, or impact of the engagement?

We have a great relationship, and they do excellent work for us. We have urgent timing on some of the releases that we make to enterprise customers. These tasks have time-sensitive events where we’re releasing things for use around the world. On some occasions, QualityLogic has been asked to help us test things really quickly or over the weekend. They’ve always been responsive and worked with us on those urgent requests.

How did QualityLogic perform from a project management standpoint?

We use JIRA for tracking all of the R&D work. They interface directly with JIRA and our Atlassian accounts. We use HipChat for communication about testing and qualification. We also use email and phone to communicate, and our project manager attends our meetings twice a week.

What did you find most impressive about QualityLogic?

They’ve developed a keen understanding of our applications. They know how they work as well as we do, maybe even better than some people on our team because of their broad usage. They take ownership of what they do and are quite proactive.

Are there any areas QualityLogic could improve?

I can’t think of anything. They offer good service. They don’t get into test automation, but that’s not a downside really.  

5.0
Overall Score
  • 5.0 Scheduling
    ON TIME / DEADLINES
  • 5.0 Cost
    Value / within estimates
    We don’t have to hire more staff or purchase all the equipment. It’s a great value.
  • 5.0 Quality
    Service & deliverables
    They’ve been responsive and have taken ownership for what they provide. They go above and beyond when we ask. They’re very conscientious partners.
  • 5.0 NPS
    Willing to refer