QA Wolf

BACKGROUND

Please describe your company and position.

I am the Software Engineer in Test of Autotrader.ca

Describe what your company does in a single sentence.

​AutoSync, a division of AutoTrader.ca, offers a comprehensive suite of connected automotive software solutions that streamline the entire vehicle purchase journey, providing tools for digital advertising, websites, digital retailing, desking, CRM, conversational AI, inventory management, and servicing solutions. 

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QA Wolf to accomplish?

• Automating our manual test suites that covers the major functionalities of our platform
• Running the test suite against each new build deployed in staging and production environment
• Creating detailed bug report if any functionality fails due to the new release

SOLUTION

How did you find QA Wolf?

• Online Search
• Other

Why did you select QA Wolf over others?

• High ratings
• Pricing fit our budget
• Great culture fit
• Good value for cost
• Company values aligned

Describe the scope of work in detail. Please include a summary of key deliverables.

At AutoSync, a division of AutoTrader.ca, we engaged QA Wolf to enhance our quality assurance processes through test automation. The scope of work involved QA Wolf transforming our existing manual test cases into automated scripts. Their team developed and delivered these automated tests, ensuring they accurately replicated our manual testing procedures. Each automated test was accompanied by a detailed video walkthrough, illustrating the step-by-step execution of the test case. Furthermore, QA Wolf integrated these automated tests into our release pipeline, enabling automated test execution with each new build in our QA environment. They also implemented sanity test suites for our production environment, running these tests post-deployment to verify the integrity of core functionalities. Additionally, QA Wolf provided comprehensive bug reports for any identified issues, facilitating prompt resolution and maintaining the quality of our software products

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

Implementing QA Wolf's services at our team, has led to several measurable improvements in our quality assurance processes:

Enhanced Test Coverage: By automating our manual test cases, we've significantly increased our test coverage, ensuring a more comprehensive evaluation of our software functionalities.

Reduced Manual Testing Effort: The automation of repetitive test cases has decreased the time our team spends on manual testing, allowing us to focus on more complex testing scenarios and strategic initiatives.

Early Detection of Issues: Integrating automated tests into our release pipeline enables prompt identification of defects in the QA environment, reducing the risk of issues reaching production.

Improved Release Confidence: The execution of sanity test suites in the production environment post-deployment has bolstered our confidence in the stability and reliability of new releases.

Efficient Bug Reporting: QA Wolf's detailed bug reports have streamlined our debugging process, facilitating quicker resolution of identified issues and enhancing overall product quality.

These outcomes have collectively strengthened our software development lifecycle, leading to more efficient workflows and a higher-quality product for our users.

Describe their project management. Did they deliver items on time? How did they respond to your needs?

Our collaboration with QA Wolf has been marked by exemplary project management. They consistently deliver automated test scripts and accompanying video walkthroughs on schedule, ensuring seamless integration into our release pipeline. Their responsiveness to our evolving needs is outstanding; they proactively address requests for new test coverage or modifications to existing tests, often completing these tasks promptly to align with our development timelines. This efficiency has been instrumental in enhancing our quality assurance processes and overall product reliability. Also, their proactive approach to production deployment monitoring has been a significant asset to our team. They consistently coordinate with us to stay informed about our deployment schedules, ensuring they can monitor for any major functionality issues in production. This vigilance allows for the immediate detection and resolution of potential problems, contributing to the stability and reliability of our live environment. 

What was your primary form of communication with QA Wolf?

• Virtual Meeting
• Email or Messaging App

What did you find most impressive or unique about this company?

QA Wolf's standout feature is their "test coverage as a service" model, which rapidly delivers automated end-to-end test coverage within short span of time. This approach, combined with their proprietary technology and in-house QA experts, ensures efficient and reliable testing processes. Additionally, their commitment to meticulous project management and constant communication fosters a collaborative and responsive partnership

Are there any areas for improvement or something QA Wolf could have done differently?

​At this time, I have no suggestions for improvement, as QA Wolf has consistently met and exceeded our expectations.

BACKGROUND

Please describe your company and position.

I am the QA Manager of Primer AI

Describe what your company does in a single sentence.

NLP and software development

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QA Wolf to accomplish?

• Help us to get some front end automation in place for our products
• help our internal QA team scale

SOLUTION

How did you find QA Wolf?

Referral

Why did you select QA Wolf over others?

• Great culture fit
• Good value for cost
• Referred to me

How many teammates from QA Wolf were assigned to this project?

2-5 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

QA wolf took the manual test plans for two of our applications and converted them into automated tests that cover the legacy functionality. 

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

Partnering with QA Wolf has enabled our small in-house QA team to scale effectively while allowing us to concentrate on the areas that need our attention most.

Describe their project management. Did they deliver items on time? How did they respond to your needs?

One of the greatest aspects of working with QA Wolf is their exceptional organization and communication. They are highly responsive to our needs and consistently a pleasure to collaborate with.

What was your primary form of communication with QA Wolf?

Virtual Meeting

What did you find most impressive or unique about this company?

Working with QA Wolf has been a fantastic experience. Their team is incredibly organized, proactive, and always on top of things, ensuring a smooth and efficient testing process. Their excellent communication makes collaboration seamless, and they are always responsive to our needs. Outsourcing test automation to QA Wolf has been a great decision, and we truly appreciate their expertise and dedication.

Are there any areas for improvement or something QA Wolf could have done differently?

QA Wolf has been a fantastic partner in automating our tests, with outstanding communication and organization. One area for improvement, however, is pricing. While their service is high quality, costs can become a concern if you have a large volume of tests to set up and manage

BACKGROUND

Please describe your company and position.

I am the CTO of SIGNiX, Inc.

Describe what your company does in a single sentence.

24x7 cloud-based, digital signature and document workflow service.

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QA Wolf to accomplish?

• Increase test coverage
• Supplement QA team capabilities
• Speed up release cadence

SOLUTION

How did you find QA Wolf?

Outbound Sales

Why did you select QA Wolf over others?

• Pricing fit our budget
• Great culture fit
• Good value for cost
• Easy to work with

How many teammates from QA Wolf were assigned to this project?

2-5 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

QA Wolf was brought onboard to assist in automating a suite of then-manual tests run by our team of QA analysts. The goal was to increase test coverage and provide the ability for the company to deploy releases on a faster schedule while also reducing the amount of manual regression testing being handled by our in-house team. We were aiming for 80%+ test coverage. We had been using a competitive solution which utilized 'gig'-testers online to manually test in bulk, but we were unpleased with the results, flexibility and flakiness of the testing itself.

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

-Internal QA team has substantially cut back (95%) on manual regression tests, exceeding our expectations

-Release confidence has improved dramatically, eliminating post-deployment reversions

-QA Wolf's solution tests our platform every day, allowing us to see new regression issues almost immediately

-QA Wolf is constantly anticipating our needs, engaging with us regularly to assess ways we can further utilize their solution

Describe their project management. Did they deliver items on time? How did they respond to your needs?

QA Wolf was stellar from engagement through to initial delivery and with ongoing service. Without much prompting or training from us, they not only matched our existing suite of tests, but also proposed a litany of additional tests, some we hadn't even anticipated or hadn't been able to cover in the past manually - all far ahead of when we expected. Moreover, they've been able to automate test processes that our previous vendor was not able to, broadening the impact of their service. We definitely feel like a premium partner, instead of just another customer - they take time to understand their clients and their needs and do what's needed to meet them.

What was your primary form of communication with QA Wolf?

• Virtual Meeting
• Email or Messaging App

What did you find most impressive or unique about this company?

The commitment to the customer is paramount, and you definitely feel that in the engagement with all of the team members. And that's on top of a solid service and technology footing.

Are there any areas for improvement or something QA Wolf could have done differently?

The only elements we feel could be improved are elements that product management have acknowledged they are working on: native Mac and mobile browser support.

BACKGROUND

Please describe your company and position.

I am the QA Analyst of Bloomfire

Describe what your company does in a single sentence.

Web-based knowledge sharing platform.

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QA Wolf to accomplish?

• QA Automation

SOLUTION

How did you find QA Wolf?

Online Search

Why did you select QA Wolf over others?

• Close to my geographic location
• Great culture fit
• Good value for cost
• Referred to me
• Company values aligned

How many teammates from QA Wolf were assigned to this project?

2-5 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

Work includes ensuring new features are implemented and deployed without causing glitches to other aspects of the software. New features are initially tested internally, while regression testing is automated.

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

Testing outcomes and durations are tracked over time to ensure quality of the application, along with improvements to the testing strategy and test runs that save time and cost towards code deployment.

Describe their project management. Did they deliver items on time? How did they respond to your needs?

Responses to feedback, requests, and maintenance are fast and professional. Any obstacles for delivery are communicated clearly and effectively, but are rarely encountered.

What was your primary form of communication with QA Wolf?

• Virtual Meeting
• Email or Messaging App

What did you find most impressive or unique about this company?

Their quick turnaround when updates are needed to reflect a new direction in the tested application's design is impressive.

Are there any areas for improvement or something QA Wolf could have done differently?

The UI for the platform shows that there's plenty of room for new features, but otherwise stable and many UI improvements are made regulary to assist teams in collecting useful data. This is often done quickly in response to user feedback, and I've had the opportunity to discuss with engineers directly on how those updates are a positive impact on user experience.

BACKGROUND

Please describe your company and position.

I am the Director of engineering of Prodigy Education

Describe what your company does in a single sentence.

We provide a world class education based games deliverying math,english and science content through engaging rpg and city builder games. 

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QA Wolf to accomplish?

• Automate the games base regression test suite.

SOLUTION

How did you find QA Wolf?

Other

Why did you select QA Wolf over others?

• Great culture fit
• Good value for cost
• Company values aligned

Describe the scope of work in detail. Please include a summary of key deliverables.

We currently perform two weekly releases with a 4 hour manual base regression pass that covers 15 core test cases. 
Deliverable 1 : Proof of concept
Team was given an ask to write an automation test that covers our battle pass functionality. This involved determning how to navigate the map, determine quests in the current battle pass and how to complete it and how to redeem rewards. 
Deliverable 2 : Base regression part 1 

Team was asked to automate a part of the first time user experience and the tower town game mode. 

Deliverable 3 : Base regression part 2 

Tema was asked to complete tests that take the largest amount of time testing internally, faced challenges of having 2 clients interact with each other at the same time

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

We had a set of 15 test that need to be automated. 
Outcomes one was having an automated test for each of these tests.

Outcome two was reduced time on spent manually testing our weekly staging passes.

Outcome three was detecting bugs sooner in the software development lifecycle. 

Describe their project management. Did they deliver items on time? How did they respond to your needs?

Proative, communicate and able to overcome blockers and only reached out when truely stuck due to product limitations. 

What was your primary form of communication with QA Wolf?

Virtual Meeting

What did you find most impressive or unique about this company?

They're very responsive and innovative in their solutions to problems ( used AI to answer questions that appear in game) 

Are there any areas for improvement or something QA Wolf could have done differently?

Nothing at the moment

BACKGROUND

Please describe your company and position.

I am the VP of Engineering of Paytient

Describe what your company does in a single sentence.

Paytient helps people to make better healthcare decisions by improving their ability to pay for care.

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QA Wolf to accomplish?

• Increase confidence around releases
• Improve time to detection and resolution of defects
• Save Engineering time from building end-to-end tests to focus on shipping new features
• Keep a low testing time frame even as coverage scales

SOLUTION

How did you find QA Wolf?

Referral

Why did you select QA Wolf over others?

Good value for cost

Describe the scope of work in detail. Please include a summary of key deliverables.

QA Wolf significantly impacted our team's ability to deliver high-quality releases quickly. They onboarded quickly and understood our needs, and created test cases covering a wide range of use cases. This drastically reduced the time we previously spent writing tests. Beyond that, QA Wolf enabled us to implement accessibility and load testing, which increased our release confidence significantly. These were critical areas we couldn't address before without major resource investments. Now, with our CI pipeline integrated with QA Wolf, we have a higher level of trust in the quality and stability of every release.

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

Scalable tests: From 58 tests to 143 tests, total test run time increased by only 1 minute

Release confidence: Tests passed with over a 90% success rate on the first attempt, allowing us to deploy with more confidence and more often.

More time for value-focused work: QA Wolf has saved our team more than an estimated 500 hours in test writing and maintenance.

Describe their project management. Did they deliver items on time? How did they respond to your needs?

Everything was delivered on time in line with our expectations. QA Wolf continues to be a highly collaborative partner, and individual engineers on our team regularly share that they are glad we decided to partner with QA Wolf.

What was your primary form of communication with QA Wolf?

Email or Messaging App

What did you find most impressive or unique about this company?

They were able to jump in an understand our applications and core use cases very quickly. Had we tried to hire people directly, not only would they have needed significant time and resources to stand up the testing infrastructure, but it would have taken them time to get oriented to our business. QA Wolf was able to provide value in just a few weeks, not months.

Are there any areas for improvement or something QA Wolf could have done differently?

We'd love to see more analytics and reporting in the QA Wolf dashboard.

BACKGROUND

Please describe your company and position.

I am the Sr Product Manager of a hospitality sales SaaS company

Describe what your company does in a single sentence.

SaaS product focusing on hospitality sales

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QA Wolf to accomplish?

• Automated QA
• Save the internal team a ton of time and manual work

SOLUTION

How did you find QA Wolf?

Online Search

Why did you select QA Wolf over others?

• High ratings
• Great culture fit
• Good value for cost

Describe the scope of work in detail. Please include a summary of key deliverables.

We decided to start working for QA Wolf in order to get the team more focused on product and engineering tasks. QA Wolf has provided us with automation testing, round the clock communication/collaboration, and a detailed flexible testing suite that we track on a weekly basis.

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

1. Bug detection rate
2. Test coverage creation/improvement
3. Ephemeral defect detection & resolution before prod merge

Describe their project management. Did they deliver items on time? How did they respond to your needs?

QA Wolf collaborated with us to create a prioritized testing suite, setting the expectation that they would start on our defined set and order of priority. Because of that sheet, I never had a question on what they were working on or when. They responded to change well and communicated often if they had questions. 

What was your primary form of communication with QA Wolf?

• Virtual Meeting
• Email or Messaging App

What did you find most impressive or unique about this company?

Lots of companies do automates testing suites. I like the human aspect of QA Wolf and the automation using Jira. Those 2 things make my experience very enjoyable. 

Are there any areas for improvement or something QA Wolf could have done differently?

As an active user of QA Wolf I'd like to see the portal where tests are created updated a bit for real time updates. Additionally, a public facing roadmap or ideas portal would be awesome. Our account reps always open the floor but it would be cool to see what other QA Wolf community folks are thinking and doing. 

BACKGROUND

Please describe your company and position.

I am the QA Lead of Signix

Describe what your company does in a single sentence.

Software providing secure digital signatures for highly regulated industried

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QA Wolf to accomplish?

• Better coverage of regression testing
• Alleviate burden on internal resources

SOLUTION

How did you find QA Wolf?

Online Search

Why did you select QA Wolf over others?

• High ratings
• Great culture fit
• Good value for cost
• We were looking for a responsive partner and QA Wolf is exactly that.

How many teammates from QA Wolf were assigned to this project?

6-10 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

Expand the coverage of regression testing using automation of test scenarios. Key deliverables have included:

1. Automated test creation
2. Monitoring test results
3. Adjustment of automation as needed
4. Bi-weekly meetings
5. Weekly summary report 
6. Weekly results report

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

1. Number of daily scheduled test runs
2. Number of tests needing manual intervention
3. Number of bugs identified

Describe their project management. Did they deliver items on time? How did they respond to your needs?

• Project management has been great to work with as has the Dev team assigned to us.
• They delivered agreed upon automation tests on time. 
• Basic communications are through Microsoft Teams...and during standard hours, I don't believe I have every had a query not be acknowledged within 10 minutes.

What was your primary form of communication with QA Wolf?

• Virtual Meeting
• Email or Messaging App

What did you find most impressive or unique about this company?

• QA Wolf's has partnered with us instead of taking over. They listen to our needs and offer multiple solutions. 
• Once our original plan was accomplished, they have brought to our attention further offerings that we are looking into using.

Are there any areas for improvement or something QA Wolf could have done differently?

Not in my book!

BACKGROUND

Please describe your company and position.

I am the CTO of Vesta

Describe what your company does in a single sentence.

Vesta provides a next-generation mortgage loan origination system for the US market. 

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QA Wolf to accomplish?

• Automation of end-to-end regression testing
• Automation of specific feature testing
• Automated release validation

SOLUTION

How did you find QA Wolf?

Cold inbound

Why did you select QA Wolf over others?

• Pricing fit our budget
• Great culture fit
• Good value for cost
• Company values aligned

How many teammates from QA Wolf were assigned to this project?

2-5 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

QAWolf provided and continues to provide an automated testing platform where they build and maintained automated end-to-end tests that serve as regression tests. Test cases are defined by our product development team and QAWolf builds and maintains those test cases and notifies the team when any test cases fail in their testing environment. 

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

Automated tests were built that covered the core product scenarios that were important to maintain high reliability on. A platform is provided to monitor and run tests as needed. Alerts are triggered when tests fail or when guidance is required for how tests should be updated when the product changes. 

Describe their project management. Did they deliver items on time? How did they respond to your needs?

Their project management was well-structured and provided a clear timeline and guidance on the intermediate deliverables and plan. We have a shared Slack channel and their response rate is always extremely fast at all hours of the day. They are always willing to work with us to find the best solutions to our goals, even if they don't fit in perfectly with their existing infrastructure. 

What was your primary form of communication with QA Wolf?

• Virtual Meeting
• Email or Messaging App

What did you find most impressive or unique about this company?

The founding team and employees are all amazing people to work with and align extremely well with the way that Vesta thinks about maintaining overall software quality. 

Are there any areas for improvement or something QA Wolf could have done differently?

No. They’ve been a great partner for us.

BACKGROUND

Please describe your company and position.

I am the Director of Frontend Engineering of BrewLogix

Describe what your company does in a single sentence.

BrewLogix provides technology platforms that connect creators, sellers, and consumers of craft beverages to drive on-premise profitability

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QA Wolf to accomplish?

• Help to ship quality assured code.
• Decrease bugs in production.
• Increase confidence in production code without hiring several more team members.

SOLUTION

How did you find QA Wolf?

linkedin

Why did you select QA Wolf over others?

• High ratings
• Pricing fit our budget
• Great culture fit
• Good value for cost
• Company values aligned
• Initial meeting, Branding

How many teammates from QA Wolf were assigned to this project?

6-10 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

QA Wolf delivers a comprehensive testing solution for BrewLogix that encompasses the entire lifecycle of our applications. They have built extensive test suites for our five standalone applications, seamlessly integrated into our CI/CD pipeline. As we consolidated three of those applications into a unified platform, QA Wolf adapted their test cases to maintain thorough coverage across both existing and new functionality.

On an ongoing basis, they execute the full regression test suite on a regular cadence, providing detailed reporting and proactive communication of any issues uncovered. Their team collaborates closely with ours to continually optimize the testing strategy, prioritize test scenarios based on risk and impact, and ensure the test suite evolves in lockstep with application changes.

When failures occur, QA Wolf delivers timely reports with all the artifacts needed for efficient debugging and resolution. Flaky tests are proactively identified and remediated to maintain a high signal-to-noise ratio. This diligent test maintenance, combined with expert guidance on testing best practices, enables us to move rapidly without sacrificing quality.

In summary, QA Wolf serves as a trusted partner to our development team, providing the testing expertise, execution, and ongoing optimization needed to deliver reliable software at the pace our business demands.

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

• Rapid test coverage ramp-up:  This allowed us to quickly establish a strong foundation of test coverage across our key user flows and functionality. Achieving this level of automated testing in such a short timeframe would have been challenging for our internal team given competing priorities.
• Increased test coverage: Over the course of our engagement, QA Wolf has helped us achieve and maintain test coverage over the majority of our applications, giving us confidence in the quality of each release and allowing us to move faster without increased risk.
• Improved developer productivity: By offloading the majority of manual testing and test maintenance to QA Wolf, our developers are able to focus more of their time and effort on feature work and innovation. We've seen a measurable increase in developer throughput and a decrease in time spent on testing and bug fixing.
• Faster feedback loops: With the comprehensive test suites QA Wolf has built, we are able to get rapid feedback on the quality of each code change. Failures are identified and reported back to our team quickly, often within minutes of a pull request being opened. This fast feedback enables us to catch and resolve issues early before they make it into production.
• Reduced escaped defects: Since partnering with QA Wolf, we've seen a significant reduction in the number of bugs and issues that make it into production. The broad test coverage and regular execution catches regressions and edge cases that may have otherwise slipped through. This has led to a measurable improvement in the quality of the applications we deliver to our end users.
• Increased release velocity: The combination of faster feedback loops, fewer escaped defects, and improved developer productivity has enabled us to ship new features and improvements to our users more frequently. We've been able to move to a weekly release cadence with confidence, knowing each build has been thoroughly validated by our automated test suites.

Describe their project management. Did they deliver items on time? How did they respond to your needs?

Our assigned QA Wolf team is always on time with their deliverables, and usually ahead of our needs.  Their communication is quality, frequent, and consistent.  They are anything but a passive partnership.  I would describe their response style as a Wall of "We're On It"

What was your primary form of communication with QA Wolf?

• Virtual Meeting
• Email or Messaging App

What did you find most impressive or unique about this company?

Most services that sound too good to be true start displaying cracks shortly down the line.  QA Wolf is impressive in that they are truly what they appear to be, and more.  Their communication and service are top notch.  They request feedback and improve from it.  The roadmap is genuine and reliable.  QA Wolf is basically the perfect life partner. 

Are there any areas for improvement or something QA Wolf could have done differently?

We'd be nitpicking here: 
The web UI can be opaque for first timers and infrequent users.