QA Madness

BACKGROUND

Introduce your business and what you do there.

I am the director of e-commerce at an IT company. We help small- to medium-sized companies sell online by providing them with technology and development services.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

We work with QA Madness to augment our current staff by providing QA testing for several of our clients.

SOLUTION

What was the scope of their involvement?

Generally, QA Madness conducts product tests, integration tests, and operating system tests for our clients’ products that utilize Shopify Plus. For our most recent project, QA Madness ran the frontend testing to compare design functionality, and they evaluated the integration with third-party applications.

What is the team composition?

For our latest project, we worked with three testers, but the team size varies depending on the requirements of the engagement.

How did you come to work with QA Testing?

They reached out to us at a convenient time. We quickly learned that they had positive reviews and that they communicate well in English.

How much have you invested with them?

To this point, we’ve spent between $10,000–$50,000.

What is the status of this engagement?

We began working together in March 2017, and our engagement is ongoing as necessary.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

By and large, we’re confident that all of our QA testing is done correctly, and our customers have been very happy. QA Madness does well to catch all defects during their process.

How did QA Madness perform from a project management standpoint?

Our team handled all of the project management tasks. We communicated daily with the resources directly. We receive thorough final reports that we can then share with clients. QA Madness meets the timelines they set for themselves.

What did you find most impressive about them?

They’re very professional, and they deliver what they promise. They’ve proven to be capable communicators and talented testers.

Are there any areas they could improve?

We received inaccurate estimates early on in our relationship, but that has since been addressed.

Do you have any advice for potential customers?

It is valuable to provide as much detail as possible about the project so that they can understand the scope and determine what they are testing for.

BACKGROUND

Introduce your business and what you do there.

I’m the head of development for a display manufacturer.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

We needed QA testing for our online properties including an e-commerce website.

SOLUTION

What was the scope of their involvement?

The e-commerce site runs on a LAMP stack with a Magento 1 install. We use Linux Apache, MySQL, and PHP. QA Madness does the following types of testing: usability, UI/UX, regression, performance, and bug.

We use a weekly schedule to coordinate daily tests. First, the team ensures the stories are correct and then logs feedback or defects. The team then uses screen grabs when appropriate before moving on to the next step—whether that’s reassigning the original resource or raising the status to the project manager.

What is the team composition?

The QA tester is our main point of contact. But we also worked with an account manager and the head of sales. Since we’ve worked with QA Madness on other projects in the last five years, I know most people on the team at this point.

How did you come to work with QA Madness?

I hired QA Madness as our QA partner five years ago. I had worked with six QA vendors and they were the most professional team. Their service was high quality as well.

How much have you invested with them?

We’ve spent between $10,000–$15,000.

What is the status of this engagement?

We started working together in September 2018 and the partnership is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

The turnaround for ticket times for QA results is under three days, which is excellent given the features’ complexity. Likewise, our bounce rate has decreased and our rework rates dropped 30% since the project began. The results prove that we’re getting better initial testing done. As a result, I no longer have to do testing myself.

How did QA Madness perform from a project management standpoint?

Our volume of tickets—between 100–300 per week—requires QA Madness to follow exact testing workflows, and that is where they excel. The case managers and our QA tester are excellent. QA Madness manages testing with a ticketing and tasking system, which we augment with communication via Slack. Our team assigns tickets to the QA tester. We add feedback to the ticket. The team then executes the standard testing workflow correctly and accurately.

What did you find most impressive about them?

QA Madness proactively tests on our use cases and then tests around what is native and nonnative to that platform. That advanced skill level is immeasurably helpful because it demonstrates the level of understanding we need from a vendor.

Are there any areas they could improve?

No, I don’t have criticisms at the moment. Although I’ve engaged other testing teams since hiring QA Madness, I still prefer to work with the team.

Do you have any advice for potential customers?

Success depends not just on the QA tester, but on the client’s ability to write documentation and communicate clearly. I also do consulting and have seen QA testers unable to provide quality results due to weak internal documentation. In those cases, the testers could not understand what they were testing against because incorrect documentation created confusion.

BACKGROUND

Introduce your business and what you do there.

I’m the CEO and founder of MediaCT, an e-commerce development firm. We develop and maintain e-commerce sites for retailers.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

We wanted to outsource QA testing to improve deliverables for our clients.

SOLUTION

What was the scope of their involvement?

Their testers are integrated within our agile Scrum methodology. A QA Madness tester is part of each of our eight-member development teams. They test every feature we develop and run log tests on our server capabilities based on their own scripts, which simulate a high-traffic environment. Their functional testing includes UI comparisons to see if the interface matches our designs.

What is the team composition?

We regularly work with 4–5 people.

How did you come to work with QA Madness?

They approached us by either LinkedIn or email, and I appreciated their directness. I liked the idea of getting outside expertise because it’s more objective.

How much have you invested with them?

We have spent between €125,000—150,000 (approximately $142,000—$171,000 USD).

What is the status of this engagement?

We started working together in August 2014, and the work is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

We have received positive feedback from clients due to the team’s eagerness to identify and fix issues. At one point, QA Madness found an obscure issue within a web browser that prevented the customer from adding a product to the basket.

How did QA Madness perform from a project management standpoint?

We contact the managers through Slack and Jira. We’ve already integrated the testers into our processes and include them in our standups and daily meetings.

What did you find most impressive about them?

They are highly skilled and diligent at identifying issues, which we’re not very good at. They’ve also adapted to our processes.

Are there any areas they could improve?

No, we’re highly satisfied.

Do you have any advice for potential customers?

Clients should approach the work as a collaboration and involve QA Madness as much as possible.

BACKGROUND

Introduce your business and what you do there.

I’m the co-founder of Wezz e-Commerce. We build e-commerce platforms and webshops in Magento.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

We design e-commerce platforms for clients and then outsource the implementation to outside developers. We used to do testing in-house, but the process resulted in less than rigorous results, so we decided to outsource the work. We engaged QA Madness to ensure that every final product meets our design specifications and lacks any bugs or issues before launch.

SOLUTION

What was the scope of their involvement?

The first project was a testing pilot. We had a webshop that needed testing for bugs and issues. We gave QA Madness the requirements, and they performed manual testing. The team had experience working and testing in Magento, so they already had a list of features to check with critical path testing. Their results impressed us, so we decided to intensify our cooperation. QA Madness now performs work for us on a project-by-project basis. They are also working weekly to complete long-term internal projects, such as improving our testing processes.

What is the team composition?

We kept in touch with one person who manages the project and oversees their testers. Their testers update their progress using a shared Jira account.

How did you come to work with QA Madness?

After an online search and a few conversations, we narrowed down potential testing partners to three companies, including QA Madness. The QA Madness team demonstrated a high degree of flexibility; they were willing to accommodate our workflow without getting bogged down in their internal processes.

What is the status of this engagement?

We started working together in September 2017 and the partnership is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

While we lack exact figures, QA Madness has significantly reduced the number of bugs and issues in our final products. They’ve also improved our internal processes. For example, we’re completing a project in two weekly sprints for one client; QA Madness integrated testing into the sprints, which has helped us do better work with fewer bugs. Also, they take a hands-on approach and will pitch in without hesitation.

How did QA Madness perform from a project management standpoint?

They’re willing to use whatever system is most convenient to us. We use Jira to log their work and stay updated on their progress. We also use Slack for quick communication. 

What did you find most impressive about them?

QA Madness has excellent communication. They have never missed a deadline or misunderstood our instructions. They are highly responsive and flexible, which has made our collaboration seamless.

Are there any areas they could improve?

No, we’re highly satisfied with their work.

Any advice for potential customers?

QA Madness has in-depth knowledge and valuable expertise in the field of testing. Listen to their advice before telling their team what to do.

BACKGROUND

Introduce your business and what you do there.

I’m the founder and owner of SupportDesk. Our company specializes in delivering technical support and maintenance for Magento-based online shops.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

During our largest project to date, we noticed that despite our company’s development skill, we had room for improvement with QA and testing. We hired QA Madness to conduct our testing and provide the maximum amount of feedback and input possible.

SOLUTION

What was the scope of their involvement?

Before starting the project, we discussed which areas QA Madness would support. They helped us assess an e-commerce platform’s functionality and performance-related conflicts in the code. After a major release, they performed QA and load testing. The results showed that new bottlenecks tend to arise after this process. This useful discovery helped us fine-tune the performance.

QA Madness executed all of their work remotely. Since all our company’s work is online, our development team also has a few remote members. Working collaboratively, we prepared release sprints in time for QA Madness to perform a full round of testing.

What is the team composition? 

We had a central point of contact from QA Madness who handled a bit of everything. She initially served as our account manager and supported us throughout the planning stage. When the tests were complete, she delivered the reports and results to us.

How did you come to work with QA Madness?

A Dutch competitor we’ve worked with on occasion referred QA Madness, especially for larger projects. They’d been pleased with the quality of their tests and the results they received.

I also met an employee from QA Madness.  A company I was visiting had invited him over as a thank-you for the last project they’d done together.

How much have you invested with them?

The cost of their work totaled around €6,000 (approximately $7,000).

What is the status of this engagement?

We started working with QA Madness in March 2018. Although we finished the project in June 2018, we weren’t quite ready to launch at that time. We had to wait for an SAP connector to provide public data.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

QA Madness generated extensive feedback that developers normally can’t see. We could never have gained this insight without their thorough functionality testing services.

They sent us reports on the test scripts they’d run on different browsers and devices.  After identifying a number of issues, they efficiently delivered results. I appreciated how quickly they conducted testing despite the high volume of work it entails. 

How did QA Madness perform from a project management standpoint?

We usually use Zendesk and have a Kanban setup for primarily operational support. For this project, we also set up a Trello board where QA Madness posted reports and created new issues for every incident. Using this system allowed us to define new sprints and add them to the backlog. We used Slack for team communication and discussed the general approach with our primary contact over phone and email. Everything ran like clockwork. I have no complaints.

What did you find most impressive about them?

QA Madness conducts thorough testing on every relevant browser and device. Most of the issues included screenshots, detailed descriptions, and even video attachments. This level of documentation helped our developers determine precisely what was going on. It was a nice bonus they added to the entire process.

Are there any areas they could improve?

No, everything was good.

Do you have any advice for future clients of theirs?

Many developers handle testing themselves, so using QA Madness was a new approach for us. I encourage potential customers to just go for it; they’ll see the benefits afterward.  

BACKGROUND

Introduce your business and what you do there.

I am the CTO of a fitness products company called FitForMe. We sell fitness supplements for people who have had a geriatric operation.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

We wanted to increase the quality of our deliverables by improving our ability to release products quicker and more often.

SOLUTION

What was the scope of their involvement?

QA Madness started by establishing a sequence of standardized tests to ensure our main functions and features were properly working in various locations, countries, and scenarios. They helped us make sure our developer’s code was meeting our business requirements. The team also did some exploratory testing on our new functionalities. They tested the product by trying to break it, this allowed them to find all of our issues. Additionally, they improved our documentation efforts by writing out test cases and updating our interpretation if its functionality has been adapted. They continue to work with us to optimize our business.

What is the team composition?

We mainly work with an account manager and a full-time QA person. They will provide a second person if the need arises.

How did you come to work with QA Madness?

I met them at a conference. Having worked with multiple QA companies in the past, it was important to me that I didn’t work with an anonymous team of people behind a screen. I wanted to personally talk to the team. I chose QA madness after a trial because I was happy with their command of the English language, they were very smart, and they were willing to think through the project with us.

How much have you invested with them?

We spent €20,000–€25,000 (around $25,000 USD) so far.

What is the status of this engagement?

We started working together in September 2017 and partnership is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

Working with QA Madness has reduced the number of issues found after each live deployment by around 50%. Previously, our deployments were a little messy and elicited complaints from users. Now, our product works.

How did QA Madness perform from a project management standpoint?

QA Madness works as part of our internal team and we treat them as such. We use a scrum methodology and the QA Madness team participates in all of our meetings and processes. We use Slack and Jira for our daily communication.

What did you find most impressive about them?

Their level of dedication and willingness to push through issues is impressive.

Are there any areas they could improve?

QA Madness could improve their test process. They’re very much based on manual work, but an automated approach would allow them to take on bigger tasks and lower the running costs of software.

Any advice for potential customers?

I recommend clients treat them as part of their team and integrate them into the development cycle from the beginning.

BACKGROUND

Introduce your business and what you do there.

I am the CTO of a merchant company running two websites: Giftsdirect.com and TheIrishStore.com. The latter is focused on U.S.-based customers who account for 80%­90% of our orders. Giftsdirect.com covers the U.K. and European markets. They are two distinct platforms running Magento 1 and Magento 2. I am responsible for all technical aspects of the company including projects related to our two stores as well as internal work.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

Our development and business analysis team fixes existing functionalities and build new ones. We lacked in-house testing resources and had a lot of work that required a dedicated force. We needed help delivering high-quality releases for both e-commerce platforms.

SOLUTION

What was the scope of their involvement?

Our first engagement with QA Madness was upgrading the TheIrishStore.com platform from Magento 1 to Magento 2 along with a large set of bug fixes for Giftsdirect.com. They gave us an overview of their processes for issue documentation before they started the work. The team performed testing across different browsers for both desktop and mobile devices. The scope included testing during and after development. The first part required the full attention of QA Madness’ engineers, while the second was a matter of ongoing bug fixes and verification.

What is the team composition?

Our first interaction was with their head of sales who negotiated the details required for the start of our collaboration. We had a few calls with her, the team lead, and their CTO on how to approach the two projects. After we agreed on a scope and estimate, the work began. We were assigned a dedicated team of QA engineers who were integrated into our development team.

How did you come to work with QA Madness?

I was looking for a testing provider at the Mage Titans conference in Milan in June 2017. We met their team and had a discussion with their CEO and CTO. They were friendly and shared examples of their past projects to give us an idea of their work quality. We went through a trial round of testing activities to see their approach to the project.

How much have you invested with them?

The cost of their work so far has been $12,000.

What is the status of this engagement?

We started working with QA Madness in July 2017. After the successful delivery of a couple of releases, we decided to continue the collaboration.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

Over 2 1/2-months, their team identified 164 bugs across two websites. After finishing the scope of issues, user session times increased by an average of 24 seconds based on Google Analytics. Our conversion rate also increased by a couple of percentage points for both websites.

How did QA Madness perform from a project management standpoint?

The delivery process was smooth and transparent to our stakeholders. We use Jira for task tracking and comments. They know how to use the tool well and usually reply within minutes. We use Slack for instant communication. They are available during work hours.

What did you find most impressive about them?

I’ve appreciated the friendly communication from their team. They are strict and direct, but they have been flexible when we needed to change or adjust our process. They provided clarification for the reported bugs in the form of screenshots and videos upon request.

Are there any areas they could improve?

No, we’ve been more than happy with their delivery.

Do you have any advice for future clients of theirs?

I recommend giving QA Madness a trial run to see how well they could work for you.

UPDATED REVIEW

This review was updated by the client on December 21st, 2018. The original star ratings were: Overall - 5, Quality – 5, Schedule – 5, Cost – 5, NPS - 5. New content is below.

What was the scope of their involvement?

We continued to work with QA Madness after our initial review, but many things became increasingly worse. They changed our project manager twice. Due to the amount of work we had, we decided to hire a full-time dedicated resource. Communication dramatically decreased and they weren’t as transparent about their team’s availability. They didn’t notify us when employees were out of the office.

We made several development changes to both of our projects. We expected the changes to be tested properly, and in our initial agreement, it was agreed that they would allocate middle to senior QA resources to us, and that this resource would provide testing services to us in a timely fashion.

We were expecting to have a dedicated senior-level QA expert, but from time to time we felt they substituted a junior-level QA resource. Both our projects took on a lower quality of deliverables. The issues we experienced weren’t identified during progression testing or pre-production release testing, which impacted our conversion rates as a business.

We assigned QA Madness more work and went forward with the release, but experienced issues in the planning phase. As we were about to essentially take our company’s product of the year live, we had consistent miscommunications while trying to figure out their QA resources’ quality of service and abilities.

How much have you invested in them?

We spent less than $25,000.

What is the status of this engagement?

Last week, our new project manager sent us a letter stating that they didn’t have senior QA resources available and wanted to stop doing business with us in 14 days. We had extensively communicated the importance of their support to our company. During this critical time of the year, we needed the quality of deliverables to stay high. Since their senior QA resource was going on vacation, they replaced them with a junior employee to finish the next week’s work without consulting with us. They left our project without any QA support despite their initial agreement to finish the work according to our terms.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

The quality of services provided by QA Madness decreased since we left the first review. We had different project managers with lower-level skills which caused issues with our live environments. The change affected both of our websites when we did the release. We saw an increasing number of we hadn’t found during the pre-release stage which impacted our business’s conversions and KPIs.

How did QA Madness perform from a project management standpoint?

We used Slack to communicate with the QA resources and email to communicate with the project managers. Since the role of project manager changed hands, the new person much less than we first discussed and agreed to. We hel phone calls to try to find common ground and I used email to request any improvements or explanations.

What did you find most impressive about them?

Unfortunately, I don’t have any areas that I can say are positive at this point in time.

Are there any areas they could improve?

They should communicate clearly and work more closely and effectively with their clients when providing support. We defined the quality, skills, and experience level we wanted from the QA resources, and felt they should have delivered on our expectations. The deliverables should not change, nor should they substitute junior resources for more senior ones.

Company Responses

The vendor responded on 02-28-19. The reply is written by Anna Senchenko (Marketing Manager, QA Madness).

Cooperation between QA Madness and Giftsdirect.com & TheIrishStore.com lasted for one and a half years. The company is very responsible in terms of product quality and user experience. The business relationships were productive and valuable for both parties.

That’s why we’re really sorry that the company’s CTO changed the review based on his personal motives without any proofs. Throughout the whole cooperation, we didn`t receive any notifications with the remarks mentioned in the review.

Meanwhile, we do respect each opinion of our clients and will do our best to prevent the cases when such issues influence professional relationships in the future.

BACKGROUND

Introduce your business and what you do there.

I’m director at a company that builds e-commerce stores on Magento and Shopify. We help our clients maintain and manage them.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

We wanted to streamline and enhance our QA process with an outside vendor, as opposed to building one in-house. The outside agency needed to uphold industry best practices that we could then leverage to improve our service offerings.

SOLUTION

What was the scope of their involvement?

Our developers QA website conversions and upgrades on a monthly basis for clients. They then pass those to QA Madness for more robust testing across various browsers and devices. Their team provides a report, which we use to process any modifications.

Currently, we’re working on a new website. They’ll do end-to-end testing for it, creating use cases for each page’s functions. The team will document issues and offer recommendations; these steps are included in the homepage testing process. We’ll send our fixes back to them. Their sign off on those corrections will complete the process.

What is the team composition?

We have two main points of contact, one of whom was our primary tester.

How did you come to work with QA Madness?

We spent a few months searching for the right partner by testing different firms. Their attention to detail and the level of protection and documentation impressed us. We knew that the amount of information they provided would make engagement easy for our in-house developers.

How much have you invested with them?

We’ve spent $5,000.

What is the status of this engagement?

Work started July 2017 and is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

We’ve increased our QA speed by over 50%. Their availability and intangible improvements to our process make turnarounds more efficient.

How did QA Madness perform from a project management standpoint?

We use JIRA. They’re very competent at managing current and upcoming tasks.

What did you find most impressive about them?

They have a mature and sophisticated process that reflects their considerable experience.

Are there any areas they could improve?

We’d like to see more initiative with project management. They could provide us with more constructive feedback.

Do you have any advice for potential customers?

Make sure they’re familiar with the resources involved in the project scope.

BACKGROUND

Introduce your business and what you do there.

Above the Fray Design, Inc. is a full-service e-commerce design and development shop specializing in Magento and other platforms. I am a co-founder and the business developer of the company.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with QA Madness?

In the past, I’ve been a part of numerous software and e-commerce development teams that have tried to do QA [quality assurance] in-house, or had developers, project managers, and even business development people handle the task. It can become a mess and it’s one of the worst things I’ve seen happen. After all the hard work, the project can go south at the very end because of this. We were looking to solve that problem but also didn’t want to bring in a full QA team, as it would be overkill for our needs. We went to QA Madness for assistance.

SOLUTION

What was the scope of their involvement?

Their involvement started at the very beginning of the project. They helped look ahead in order to see which areas they would want to run QA on.

Particularly if we’ve finished a module or specific section, they do the QA, conducting a full test on a variety of different devices, platforms, browsers, and screen sizes. Towards the end—especially when we’re going into pre-launch phases and prior to user-acceptance testing—they will go through, comparing the entire website to the scope of work, and interviewing us to make sure everything is correct.

No matter what, everyone is going to miss something, but this approach knocks out 95% of the issues that the client would point out to us otherwise.

What is the team composition? 

I worked with one of their sales managers.

How did you come to work with QA Madness?

They are known in the e-commerce world, for Magento specifically. Some of my most esteemed colleagues had worked with them and they also ran a pretty good email campaign that brought them to my attention. I asked a colleague from another company about the team, and he said also said they were the best.

How much have you invested with them?

Over the course of three projects, we’ve spent $15,000–$20,000 on their services.

What is the status of this engagement?

We started working with them in July or August 2017. We’ve worked with them on three projects so far and plan to maintain an ongoing relationship.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

We’re a new company, but our members have been working in the industry for years. My business partner and I have assembled what I consider to be an all-star team, but this was our first time working together in this formula. We wanted to break all misconceptions around poor development, broken timelines, and angry clients.

The very first site we launched delighted and impressed our client in terms of QA. The number of follow-up tickets they had with us was minimal, and the ones that did exist weren’t as much related to QA items as they were to design pieces that the client wanted to adjust. 

Every client we’ve used QA Madness with has been absolutely delighted with the work.

How did QA Madness perform from a project management standpoint?

They’ve been amazing in terms of communication; with no delays or lags, even during holidays. They give us advanced notice of any scheduling conflicts, and even then, they’ve made themselves available at times when they said they probably wouldn’t be. They’re extremely attentive to client concerns and deadlines, and extremely professional when it came to sticking to estimates, scopes, and quotes. I have no complaints; they are an extremely valuable part of our extended team and I couldn’t ask for more from a project management standpoint.

We have people across the nation and across the globe, so Skype is used by our team across various locations. We also have VoIP phone calls and use email. I am connected with the managers from their team via Facebook as well.

What did you find most impressive about them?

It comes back to the difference they’ve made in terms of client satisfaction with end-results. They also have good communication cadence and don’t over-engineer QA, which helps maintain timelines. After all the hard work of our designers, engineers, developers, project managers, and system analysts, having their team make sure that there is a bow on the final delivery, and the client is happy, is invaluable.

Are there any areas they could improve?

I always like to give constructive feedback and am usually good at it. If there was something to point out, I would have done it, but they’re thorough and professional. I appreciate them immensely and have no improvements to mention.  

Do you have any advice for future clients of theirs?

A general rule is being proactive and not afraid to engage. Provide as much information as possible, so they can do their job. Engage early, speaking with them prior to even the project. We didn’t wait until the last minute, which worked out great, but if we hadn’t given them time to prepare ahead of time then they’d have had to scramble.

BACKGROUND

Please describe your company and your position there.

I’m the CTO of elgentos ecommerce solutions, a boutique Magento development agency in the Netherlands. We mainly serve webshop merchants with income ranging between €1–20 million per year.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire QA Madness?

We outsource the testing work to them. When we build a new feature for our clients, we pass the staging information on to QA Madness, where they conduct thorough functional testing as well as regression and smoke testing.

What were your goals for this project?

The main goal is to increase the quality of our deliverables.

SOLUTION

How did you select this vendor?

They came highly recommended by other companies in our industry.

Describe the project in detail.

We first gathered all issues in our project management software and assigned a QA [quality assurance] label to them. QA Madness then went through those issues to conduct testing and created new issues for any problems they found. Their reporting is extremely thorough and includes a step-by-step way to reproduce the issue, several screenshots, and almost always also a screen capture video. They also test in a wide range of devices and browsers, which we decided upon together beforehand.

What was the team composition?

We were assigned a project manager who then assigned testing agents based on their internal availability. We do have the same project manager for every project, which is very nice.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

Their work has helped speed up our time-to-go-live for an average feature to a week or so, which usually was around a month including sprint, testing, and client approval.

How effective was the workflow between your team and theirs?

The workflow was very effective. We’ve communicated using our own issue management system— GitLab—which keeps things streamlined for us and our clients.

What did you find most impressive about this company?

I was amazed by the level of detail in the reporting and reproduction of the issues. They surprise me every time by finding bugs and issues that I would've never thought of testing. They're also usually available for testing within a few days, which is extremely nice in some situations although we try to plan in advance.

Are there any areas for improvement?

Not that I can think of.