QA Madness

BACKGROUND

Please describe your company and position.

I am the Product Owner of a gaming company

Describe what your company does in a single sentence.

We are a mobile gaming company aiming to become the EA of mobile gaming.

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QA Madness to accomplish?

• Functional testing
• Compatibility testing
• Performance testing

SOLUTION

How did you find QA Madness?

• Online Search
• Clutch Site

Why did you select QA Madness over others?

• High ratings
• Pricing fit our budget
• Good value for cost

How many teammates from QA Madness were assigned to this project?

1 Employee

Describe the scope of work in detail. Please include a summary of key deliverables.

We were looking for someone testing our game on Android and iOS and verify UI, UX, controls, camera, performance, art issues. So the focus was on functional manual testing with regression tests, smoke tests, RC tests and we asked to have before launches compatibility testing. 

In the future, we'd like to have performance tests once our optimizations will be done. The deliverables we expected were to see the test cases planned, have a QA process defined internally to onboard our teams with our QA colleague from QA Madness and manage the bug backlog in Notion from creation to closure of bugs during sprints. 

Regular status checks were also done on our tools to let us know of the progress. We expected full ownership from him on his topics and we were giving him full access and considered him part of our team for the best collaboration possible.

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

• Organized backlog of bugs with ID and tags allowing fast identification and treatment 
• Number of US closed (with bugs solved otherwise they couldn't be closed) joined to sprint percentage of completion. 
• FAT tests completed successfully 
• Crash rate in analytics once in production and assistance tickets from users 
• Game stability improved

Describe their project management. Did they deliver items on time? How did they respond to your needs?

Our QA partner provided great services and behavior at all times. The communication is the foundation of our collaboration since we are working remotely from each other. We wanted a QA tester but also someone that has strong soft skills. He was included in the team as one of our own which helped a lot in the coordination and progression of the project. 

He is proactive and always helped the team with constructive feedback to improve our process or identify issues that are happening inside the project that are impacting the quality of the project and the game. He quickly adjusted to the tools we gave him and found ways to make the tool work for his need (Notion). 

Obviously, he also does his QA tasks like working closely with the teams to understand the documentation, creating the test cases and logging ang managing the bug backlog and reports. He has a keen eye for details surprising some of our team members, is very reactive and thorough in his work. We can trust him with our project entirely.

What was your primary form of communication with QA Madness?

Email or Messaging App

What did you find most impressive or unique about this company?

The coordination with our main contact Daria. The setting up of our partnership took less than 2 weeks to be settled and since then, the follow up is flawless. Any request or discussions are solved very quickly and they don't hesitate to inform us regularly of changes on their side. They've also supported us in changing times internally and found ways to adjust.

Are there any areas for improvement or something QA Madness could have done differently?

No

BACKGROUND

Please describe your company and your position there.

Management revenue company - QA

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire QA Madness?

Testing Web application project.

This includes our two applications Radar and Invest plus one new Initiative that requests lots of testing.

SOLUTION

How did you select this vendor and what were the deciding factors?

We did an online search and my manager at that time decided that QA Madness will be a good fit for us. After a few interviews with QA Madness, we decided that they cover our needs (great people, good communication skills, and good timing thinking of the jet lag).

Describe the project in detail and walk through the stages of the project.

We have different projects, one of them is a Game changer and we have Dmitry who is working very well on this project alongside developers and the Product owner.
I don’t want to forget Oleh and Sergey who are jumping between Radar and Invest, doing a great job.

How many resources from the vendor's team worked with you, and what were their positions?

We have 3 manual QAs that are working with us.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

We have an internal Recognition channel, and in this channel we can see that the guys get recognition for all the work that they do.

This demonstrates progress and success.

How effective was the workflow between your team and theirs?

We have a great communication and this help us to good workflow.

What did you find most impressive or unique about this company?

The quality of the people that I am working with is the most impressive.

Are there any areas for improvement or something they could have done differently?

They are doing great and if there are any improvements that we have to do we are working on it along the way.

BACKGROUND

Introduce your business and what you do there.

I’m the project and delivery manager at a web development agency based in the UK.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

We’re still a relatively small but growing company, and we operate with the help of 100 people. We struggled a lot with internal testing on our projects, and since we didn’t have an internal QA team, a lot of our QA tasks were falling on some of the management teams, account managers, and project managers. Overall, it wasn’t the best way to do things. 

SOLUTION

What was the scope of their involvement?

We deliver monthly work for our regular clients. On top of that, we also deliver bigger projects for those customers. QA Madness helps us on both sides of our business by assigning dedicated QA specialists on both small and large projects. 

They also perform manual testing services and they do that through following the project’s specifications and login details to various environments. The team manually checks for bugs and inconsistencies between the requirements. Then, QA Madness provides feedback so we can go back and amend issues if there are any. 

What is the team composition?

We only have two full-time people from QA Madness. 

How did you come to work with QA Madness?

I came across them while searching online for a QA agency. What helped me chose them was the fact that one of our internal team members had worked with them before. That person’s recommendation really went a long way. 

How much have you invested with them?

We’ve roughly spent $10,000. 

What is the status of this engagement?

We started collaborating in October 2020, and it’s ongoing. 

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement? 

The proportion of bugs reported internally versus bugs reported by our third-party client have improved dramatically since QA madness joined the project. Previously, the majority of the bugs coming through the project board were things reported by our customers and usually stated quite late in the project.

With QA Madness’s help, we've been able to start testing early and capture a lot of the bugs before they even go to our third-party client. That way, we’re only left with a very small amount of bugs at the end of the projects. 

For example, on one of the bigger projects we had, only less than 20% of bugs have been reported by the external client. As such, there is great satisfaction with QA Madness’s services. We like that our customers are not sitting frustrated and raising a lot of tickets all the time about things we should’ve captured. Compared to our past projects, QA madness has been really professional and we’ve reduced the number of things we miss thanks to them. 

How did QA Madness perform from a project management standpoint?

The team members that they’ve assigned to us have been reliable and we’ve had no issues with QA madness.

There was also a time where a dedicated QA specialist decided to leave them, and within half a day, we already had a replacement. They also offered to make sure that the new specialist was trained and they covered the cost of the additional training for our project in order to make sure the handover went smoothly. 

What did you find most impressive about them?

Usually, we have a product owner which is responsible for checking if the work of our internal team follows the specifications. In relation to that, it’s been nice that QA Madness has also been helpful on that front because they deliver high-quality testing services, and they double-check their work. 

Are there any areas they could improve?

No, I don’t think they have any areas to improve. I’ve enjoyed our relationship with them and it's been smooth. 

Do you have any advice for potential customers?

QA Madness assured us from the start that they could guide us through it and they would be flexible enough for us. They added that their specialists were experienced enough to be able to look at pieces of specifications and figure out themselves what needs to be tested and what needs to be done. 

To those thinking of onboarding external QA, we stopped ourselves as a company for a long time because we kept thinking of all of the reasons why it wouldn’t work, but we just decided to go for it. It’s been a good decision because QA Madness has been easy to work with, and you’ll see that it’s going to work perfectly once you partner with them.

BACKGROUND

Introduce your business and what you do there.

I’m the CTO of FightCamp. We produce boxing workouts for customers to do at home. It requires a mobile app and sensors that track punches.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

As the app became more complex, we needed to onboard new iOS developers, which created more risk for introducing bugs. We were spending a lot of time finding those issues and decided to hire a company that specializes in that work. The goals were to increase the quality of the app and to remove as many bugs in production as possible.

SOLUTION

What was the scope of their involvement?

QA Madness provides support during our development sprints. Every time we have new code to test or screens ready to use, we release it on Test Flight. Overnight, QA Madness will test it and give us a sheet detailing every bug in the morning. This takes place 1–2 times per week. 

Their team also asks questions to differentiate bugs from features. They’re really rigorous, trying things that we never think of. They create a big list of all issues so we can communicate what’s an issue and what’s not.

What is the team composition?

There’s one engineer who works with us along with a customer rep.

How did you come to work with QA Madness?

One of our team members knew someone who worked with QA Madness and was pleased with them.

How much have you invested with them?

We spend about $1,500 per month.

What is the status of this engagement?

We launched the project in December 2019 and still work together. 

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

QA Madness has saved us a lot of money. At first, it was slower because they were getting familiar with the product. Now it saves a lot of time, and we don’t have to go through the whole app to find bugs. They find bugs we wouldn’t have found otherwise because they spend a lot of time on it

How did QA Madness perform from a project management standpoint?

The process is pretty straightforward. We’ll let them know the day of that we’ll have a build ready for them that evening, and the next day we’ll have a spreadsheet of bugs when we wake up. When QA Madness has a busy week, they’ll let us know and let us book a time slot so we can sync with them on a different day. That doesn’t happen too often, though.

What did you find most impressive about them?

Their fast responses are really outstanding. When we write to them, they answer pretty quickly.

Are there any areas they could improve?

There may be a better way to report bugs or issues than Excel spreadsheets, which are what we use now. We might need to ask for a more appropriate platform or find one ourselves. But that’s a small thing. We can select what we want.

Any advice for potential customers?

Introduce QA Madness to the product you’re giving them and walk them through the whole thing, explaining its functionality and how it should behave. It’ll help them determine what’s a feature and what’s a bug.

BACKGROUND

Introduce your business and what you do there.

I’m the COO of an agency that works primarily with Magento in the e-commerce industry. We provide solutions for merchants who want to build sales on their platforms.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

Primarily, we work on new features for platforms or integrations between systems. We wanted a partner to build QA capabilities into our development process to check for errors automatically when we deploy a new feature into a customer’s platform.

SOLUTION

What was the scope of their involvement?

First, we explained our need in Gherkin, describing different scenarios for the QA Madness team to test. From this kind of specification, QA Madness built an automatic process to check platform behaviors every time we submit new code into our codebase. Their team developed about 50 out of 70 test cases, which have become part of our development pipeline. They didn’t manage to complete all of the scenarios because we were developing the platform at the time. At the moment we deploy new code for a project, we have about 50 test cases running and telling us if something is broken. QA Madness developed the basis for our team to begin this kind of work internally.

What is the team composition?

We had a primary point of contact who worked on our project full-time. They also had another dedicated teammate, but we never worked with him because we didn’t need to escalate.

How did you come to work with QA Madness?

We met at Mage Titans Italy 2017, a Magento focused event organized by my company. At the time we met I understood that their work was something our company needed. It took us some time to establish a partnership because things don’t always move fast. We had to have the proper customer who we could build these kinds of test cases with and the budget for it. Some of our guys worked on a project with QA Madness for another contractor and were enthusiastic about their work, which gave us the opportunity to establish the partnership.

How much have you invested with them?

We spent €4,000 (approximately $4,400 USD) per month with QA Madness.

What is the status of this engagement?

We collaborated between July–September 2019. We’re not finishing this collaboration because of QA Madness. Rather, software testing is an internal skill that every teammate has to have. We need to create test cases by ourselves. There's a possibility of our working together again in the future.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

The product was exactly what we expected and has been extremely helpful. QA Madness wrote and integrated a complete test suite. Whenever we change our codebase, their tests run to help us continuously find errors we may have introduced into a product. As a result of their efforts, we can locate a bug in the test environment, which prevents issues from entering production.

How did QA Madness perform from a project management standpoint?

At the beginning of the collaboration, we held a conference call when we presented our needs and QA Madness introduced their team. During the project, we communicated by email. We also used Slack, which was our primary mode of contact. A QA Madness test expert talked with our team every day for two months.

What did you find most impressive about them?

QA Madness was seriously professional. Their team didn’t have an issue with our decision to interrupt the partnership. They continued to finish their work. It was really impressive because their team didn’t ask us to work for another month. They listened to our needs and gave us the kind of work we expected. I would recommend them, 100%.

Are there any areas they could improve?

No, I’m very satisfied with their services.

Do you have any advice for potential customers?

Try them; QA Madness gives the opportunity to work with them without a contract that obliges a certain about of time. If you want to work with them for one month, you can to test the quality of results and the kind of care they have for their customers.

BACKGROUND

Introduce your business and what you do there.

I’m the VP of client service at Isadora Agency. We’ve been in operation for about 10 years, and we cater to global market leaders. Our primary products are enterprise-grade websites, although we’re a full-service creative agency.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

We needed a team to review our sites and ensure that every feature worked as intended.

SOLUTION

What was the scope of their involvement?

Once we have a site that’s been designed and developed, we bring them in on the end of the project to go through the entire site. They test out the navigation, features, and functionality.

What is the team composition?

We’ve worked with many people over the years, depending on the size of the project, but we have one dedicated account manager who we communicate with every day.

How did you come to work with QA Madness?

We wanted to find a team that was organized, experienced, and capable of doing a thorough job. I started the search online, and their website caught my eye. They also responded to my inquiry promptly, which led to our initial meeting and eventual partnership.

How much have you invested with them?

We spend between $15,000–$16,000 per year. At this point, the total is about $53,000–$56,000.

What is the status of this engagement?

We started working together in October 2015, and the collaboration is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

Thanks to their work, our final products have no bugs or other issues. Site features and functionalities run smoothly, links are connected properly, and pages load efficiently. Before hiring them, we ran into dysfunction with all of these things.

How did QA Madness perform from a project management standpoint?

We communicate daily, usually via Skype. They’re extremely responsive, not only to our team but also to other departments within our company. Most of the bugs they find are development-related, so they touch base with that team often.

What did you find most impressive about them?

They’ve always been very professional, prompt, and available when we needed them.  We’ve never had any issues or needed to go back and teach them how to meet our standards.

Are there any areas they could improve?

No, I think they’re doing a fantastic job.

Do you have any advice for potential customers?

If you have a specific process that you want them to be familiar with, be very transparent about what it is. They’ll deliver what you need as long as the communication is clear.

BACKGROUND

Introduce your business and what you do there.

I am the vice president of SaaS operations at BuyerQuest. We make procurement software that brings the e-commerce shopping experience to enterprise clients.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

When we started our company, we didn’t have any developers or manual testing teammates. We sought a contractor that could provide that expertise right away, so we hired QA Madness to conduct the testing of our product.

SOLUTION

What was the scope of their involvement?

Once our developers finish their piece of a new release, those pieces are represented as individual tickets in our project management platform. Members of the QA Madness team test each piece to make sure it works. Once all the tickets are processed, they test the entire application to make sure they all work together.

In the past, we’ve had more applications that were all manually tested. Now, we have just one application that applies to all of our clients, and we’ve implemented an automated testing program as well.

What is the team composition?

We currently work with a project manager and six testers.

How did you come to work with QA Madness?

I joined after we hired them, so I’m unsure of the decision-making process. We’ve continued to use them because we want testers that are dedicated to understanding our application and making the quality of our product be at the level we need.

How much have you invested with them?

We’ve spent between $50,000–$200,000 so far.

What is the status of this engagement?

Our engagement began in September 2013 and remains ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

They probably process 250 tickets per month as they make sure that our product is functional. With their help, our QA process has become more rigorous over time. For the past couple of years, we’ve had fewer bugs and happier customers as a result of their hard work.

How did QA Madness perform from a project management standpoint?

We depend on them daily. Since we’re a small company, our business processes and our software development workflow have evolved over time, and QA Madness has been receptive to those changes. They’ve grown along with us. They’ve worked late and worked on holidays and weekends to meet our needs, so we’re very appreciative of their flexibility.

What did you find most impressive about them?

Their willingness and ability to understand our product is the most notable thing. Our software product has a lot of details and complexities, but QA Madness is committed to mastering them all. They’re basically the experts of our company, and they represent the body of knowledge about our product. They train new people to understand our project before they come on as testers.

Are there any areas they could improve?

As our company has grown, we’ve come to need more testers, but that can be a difficult initiative given the challenges our software presents. QA Madness can’t be blamed for that—it’s just difficult to find testers that can handle the work. We’d like to have one more tester at the moment, but it takes a while to find them.

Do you have any advice for potential customers?

I’d advise future clients to be patient as QA Madness assembles the best possible team to meet their needs.

BACKGROUND

Introduce your business and what you do there.

I’m a project manager at a Magneto development company. We build Magneto e-commerce projects for clients. I oversee all the work and organize the resources for each project within the company.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

We hired QA Madness to do QA testing on our projects. We wanted to offer our clients the best quality.

SOLUTION

What was the scope of their involvement?

Depending on the workload, we request the number of testers we need and then assign them to specific projects. Their team receives tickets in Jira and information in Confluence about the process, which includes what we need to test. Our company creates various e-commerce shops. QA Madness's work is organized daily, depending on priority.

What is the team composition?

In the last two years, we’ve worked with an average of two to three testers.

How did you come to work with QA Madness?

QA Madness was working with the company before I started with them.

What is the status of this engagement?

We started working with QA Madness in July 2014, and the collaboration is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

Their team does an excellent job finding problems. QA Madness provides hardworking testers that make clients happy. They work long hours to keep up with tight schedules. QA Madness is extremely involved in the company. They’re dedicated to working hard to please our clients.

How did QA Madness perform from a project management standpoint?

Depending on the type of project, we plan sprints and other workflows. Other than Jira for the tickets, we use Slack for general communication.

What did you find most impressive about them?

Our companies are working toward the same goal: making our clients happy. I appreciate the level of quality QA Madness strives for in their work. I’ve been delighted with this engagement.

Are there any areas they could improve?

There are small areas of improvement, but they’re near perfection. That’s hard to achieve.

Do you have any advice for potential customers?

When you receive a new team member or outside help, provide enough background and instruction for the project. Every client has different expectations, and it’s only fair to give that information to the vendor.

BACKGROUND

Introduce your business and what you do there.

I’m the head of strategic partnership and QA department at eWave Commerce, a digital transformation agency.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

Our company sometimes has busy periods where we experience an increased demand for resources.

SOLUTION

What was the scope of their involvement?

We hired QA knowledge for knowledge sharing related to QA. During our mutually beneficial partnership, we worked on multiple external projects. Their team supports us significantly with QA-oriented tasks. They helped us initiate the project, including estimations, finalizing the budget, and measuring the number of hours that applied exactly to each part of the job. 

QA Madness provides expertise in key domain areas we request, including the Magento CMS platform we use. To a degree, they’ve offered knowledge of additional frameworks. Along with design and execution, they're also involved in the planning phase.

We build products together from scratch, starting from design and development through execution and release. During the initial planning stages, we analyze the requirements and documentation. We go through a set system to improve integrations with third-party systems. 

When any new clients join us, we take a module-based approach to consider which extensions to use for Magento functionalities. QA Madness supports the entire cycle, including UAT scenarios, qualifications, and technical advice for our clients. 

What is the team composition? 

We started with a team of 2–3. During a peak period, we worked with seven people from QA Madness.

Whenever we request help, they provide whatever number of resources we need. All of the teammates are of high quality, ranging from mid- to senior level. They've never assigned us junior-level resources.

How did you come to work with QA Madness?

One firm that split into two companies, one of which was QA Madness. We worked with the original, larger business, so we had a long-term partnership and personal contacts. I knew the director through a mutual contact, so I reached out to ask them to help us with our projects.  

How much have you invested with them?

We spend $10,000-$15,000 per month.

What is the status of this engagement?

We started working together in January 2017, and the work is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

The exit criteria usually indicate zero issues with our projects. QA Madness has never missed a single major part of any of the different challenges we’ve undertaken together. They’ve helped us gain additional clients based on how good of a job we did together. 

Clients often ask one another about service providers. QA Madness has a great reputation, so our partnership has helped us increase our revenue. Additional clients have sought us out based on the projects we’ve completed with QA Madness.

How did QA Madness perform from a project management standpoint?

We haven’t needed any dedicated project management resources from QA Madness because our internal project manager and QA lead both control the project and manage their team. A point person from our side delegates the tasks. They usually know what to do and how to do it.

What did you find most impressive about them?

Whenever we request additional help, we can always count on QA Madness to provide the proper resources on time.

Are there any areas they could improve?

No.

Do you have any advice for potential customers?

I highly recommend QA Madness to anyone who asks me for advice about forming a new partnership.

BACKGROUND

Introduce your business and what you do there.

I’m the CTO at a legal tech software company that develops software for automating litigation services.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

As a startup, we were developing a majority of our processes and identifying our needs in the software development lifecycle. One of those needs was on the QA side.

SOLUTION

What was the scope of their involvement?

We primarily use QA Madness for functional testing. First, we provide their team with documentation regarding the system from a user perspective. We give them training materials to familiarize them with the system. Their team ran through different scenarios and compiled a test plan. We then decided on parameters for the test plan and proceeded with the execution of that on a manual browser-based system regularly.  QA Madness tests various web browser types, from IE11 to Google Chrome.

What is the team composition?

A lead coordinates the work and assigns a QA engineer that we interact with directly.

How did you come to work with QA Madness?

I engaged with QA Madness at a software agency I previously worked at. We created different software products and used their team for mobile, web, AR, and more. I believe we initially found them on Clutch. We enjoyed QA Madness’ professionalism, attentiveness, and reasonable cost.

How much have you invested with them?

We’ve spent between $10,000–$20,000.

What is the status of this engagement?

We started working with QA Madness in March 2018, and the work is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

Their team has identified a couple hundred issues. I’ve been extremely impressed with their diligence and responsiveness across the board. QA Madness is excellent to work with. When the next version of our product is released, we plan on using their automated testing services as well.

How did QA Madness perform from a project management standpoint?

The organization between the engineers and their team leader is seamless. If there are any issues the specific QA tester can’t handle, we direct our interaction to the lead. They provide an issue list that reports each particular browser type and steps to recreate.

What did you find most impressive about them?

I appreciate their professionalism. Their team is also extremely responsive. Customers know what they’re getting with QA Madness.

Are there any areas they could improve?

QA Madness could make more of an effort to proactively schedule time for our company. Their popularity has increased since we first started engaging with them.

Do you have any advice for potential customers?

Trust them. QA Madness is an outstanding company.