Visit Website

Testing Special Forces

4.7
17 reviews

We’re veterans on the field of quality assurance in software development. During 7+ years on the market, we’ve delivered over 500 successful projects for more than 130 clients. With experience there comes the knowledge and we used ours to come out with an innovative approach that we call Business Driven Quality eXcellence.

We believe taking care of our clients quality should be based on customer needs and companies internal vision and politics. Therefore our task is to provide team and process augmentation for a secure and functional digital product.

Our services can be divided into 3 major groups, like peaks of a triangle:

Cybersecurity:
- Pentesting
- Cybersecurity programs
- Secure software manufacturing
- Cyber Resilience

Customer Experience:
- Accessibility
- Performance and scalability
- User Experience

Quality Excellence:
- Quality assurance consulting
- Automation
- Training and workshops

QA performed by TestArmy enables a thorough insight into the assurance level on every step of the customer journey and eventually improves Enterprise Digital Transformation in a modern manner. Our specialists adapt to your workflow to deliver quality support no matter how you run the project. Their job goes way beyond end-to-end testing and so does our entire mind- and skillset.

READ MORE...
 
$1,000+
 
$25 - $49 / hr
 
50 - 249
 Founded
2010
Show all +
Wrocław, Poland
headquarters
other locations

Portfolio

Key clients: 
  • Samsung
  • Philips
  • Credit Agricole
  • T-Mobile
  • Naspers
  • CineWorld
  • Finder by TomTom
  • Credit Suisse
  • ExplainEverything
  • Nomtek
  • Tui
  • RWE
  • Empik
  • Orgintag
  • EyeEm
  • Apator
  • Clearvision
  • Autodesk
  • TomTom
  • Trans.eu
  • Greenwood Campbell
  • Électricité de France

Reviews

4.7
17 reviews
Sort by

Penetration Testing for HR Consulting Company

"The most impressive thing was how experienced they are with pentesting."

See More

5.0

Quality: 
5.0
Schedule: 
5.0
Cost: 
5.0
Willing to refer: 
5.0
The Project
 
Less than $10,000
 
May - June 2020
Project summary: 

To make sure that internal sensitive data collection was safe, the staffing and recruiting company hired TestArmy to perform a series of penetration tests.

The Reviewer
 
201-500 Employees
 
Berlin, Germany
Urszula Kliszcz
Technical Recruiter, Bergman Deutschland GmbH
 
Verified
The Review
Feedback summary: 

TestArmy successfully performed the tests on the client's internal software. The team managed to identify and report the vulnerabilities and potential risks. The team was very effective throughout the project. They managed to always hit the deadlines.

The client submitted this review online.

BACKGROUND

Please describe your company and your position there.

I am a Technical Recruiter at Bergman Deutschland GmbH. Bergman Deutschland is a recruitment agency, which specializes in assigning Engineers and Companies to a certain project.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire TestArmy?

We decided to hire TestArmy, because we needed a third-party vendor to perform penetration tests on our internal sensitive data collecting software. Our main goal was to make sure that all the data we gathered was safe within that software.

What were your goals for this project?

Our main challenge was to perform penetration tests so we could make sure our software was solid. If not, we wanted to gather all the vulnerabilities in one place, so later on we can solve these issues and keep all the sensitive, internal data without fear that hackers can break-in.

SOLUTION

How did you select TestArmy?

We have contacted around 3 other cybersecurity companies, who claimed to be specialists in this field. However TestArmy had the biggest experience in pentesting. That’s why we decided to choose them.

Describe the project in detail.

TestArmy’s cybersecurity team performed pentests on our system to check if hackers would be able to break-in. They also tested the whole software to check if anyone accidentally would be able to see someone else’s personal information without permission to that account.

They met with us to analyze the whole software, first briefly, schedule how much time they would need, check what kind of software is that, what kind of data is there. Later on, they introduced us to the project plan, and as we accepted the plan they started the whole testing process.

The whole time they made sure that they wouldn’t interrupt our work or surprise our employees with their tests. After finishing the pentesting processes they created a very detailed report, that has shown us in which areas we have issues, and vulnerabilities that could be a potential threat to our software system.

The most important part was that they walked us through the whole report, explaining what’s wrong, and how it could affect our business.

What was the team composition?

We were frequently contacted by the Project Manager, Sales Manager and the Head of Cybersecurity team.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

They performed pentests on our internal software, showed us in which areas we had vulnerabilities and potential risks, so we could improve our safety measures in those certain areas. Every vulnerability was included in the report, which had a great amount of detailed information.

Thanks to that, we could easily fix the issues. At the same time they have explained the issues included in the report, so for people, like us, who are not cybersecurity experts it was very important to have someone who would be able to explain all of the things easily.

Now we are very aware of the vulnerabilities that our software has and how it can affect our business. That’s why we are going to solve those issues as soon as possible, and later, ask them again to perform pentests to make sure that the software is secure from all the attacks from outside and inside.

How effective was the workflow between your team and theirs?

The project management was very effective and was arranged properly. The whole team always met our deadlines and we were frequently in touch to discuss the progress in the project.

What did you find most impressive about this company?

The most impressive thing was how experienced they are with pentesting. We knew that they would find everything and anything that could potentially cause security problems later on.

Are there any areas for improvement?

Honestly, I can't think of anything to improve.

5.0
Overall Score TestArmy will not only perform pentests, give you a report with all the issues, but most importantly they will also explain everything to you, if you
  • 5.0 Scheduling
    ON TIME / DEADLINES
  • 5.0 Cost
    Value / within estimates
  • 5.0 Quality
    Service & deliverables
  • 5.0 NPS
    Willing to refer

Security Audit & Penetration Testing for IT Dental Solutions

"No other vendor that we contacted were able to commit to our tight schedule."

See More

5.0

Quality: 
5.0
Schedule: 
5.0
Cost: 
5.0
Willing to refer: 
5.0
The Project
 
$10,000 to $49,999
 
Mar. - Apr. 2020
Project summary: 

TestArmy served as a third-party security expert, performing a wide range of technical assessments and penetration tests. In the end, they delivered a comprehensive report listing vulnerabilities and solutions.

The Reviewer
 
11-50 Employees
 
Malmö, Sweden
Joakim Fong
CEO, Protetiko AB
 
Verified
The Review
Feedback summary: 

Thanks to TestArmy's diligence, the in-house staff was able to quickly resolve any identified risks. The collaboration instilled confidence in both the staff's work and their product. Most impressively, TestArmy outpaced the established timeline while sticking to the initial budget.

The client submitted this review online.

BACKGROUND

Please describe your company and your position there.

I am CEO of a SaaS company that provides IT solutions to the dental Industry. We are headquartered in Malmö, Sweden and serve clients in the Nordics, and are currently expanding internationally.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire TestArmy?

Our company is experiencing rapid growth and are quickly expanding our product portfolio and services. To ensure that our speed did not compromise our security on any level (architecture, technical solutions and operational procedures) we felt that it was necessary to get outside expert opinion and screening.

What were your goals for this project?

We wanted to see that our technical solution could withstand a wide range of potential attacks and also ensure that our staff used good routines and procedures for managing phishing attacks etc.

SOLUTION

How did you select this vendor?

Our team reached out to several cybersecurity firms. Our requirements were that the supplier should have strong experience in delivering projects on an international level and speed of delivery. We chose Testarmy based on a combination of their references, price range and ability to deliver according to our deadline.

Describe the project in detail.

The project began with a series of planning meetings with the TestArmy team. This was followed by an initial analysis of our solution and organisation after which the team did a wide range of technical tests and penetration attempts.

The team also conducted social engineering attacks and phishing attempts. Finally a complete report was delivered highlighting all areas of our solution with vulnerabilities and proposed solutions for each. A guide was also provided on how to manage secure operations.

What was the team composition?

We were assigned a KAM, project manager and team of testers and white hat hackers.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

The report showed that we had good security in place and the few vulnerabilities that we had were quickly addressed. The report provided us with continued confidence that our working routines and solution is built on a solid and secure foundation.

How effective was the workflow between your team and theirs?

We had excellent communication and cooperation between our teams. We were impressed by the quality and speed of delivery by TestArmy.

What did you find most impressive about this company?

We were impressed that TestArmy were able to meet and beat the deadline we proposed and could deliver within the agreed upon budget. No other vendor that we contacted were able to commit to our tight schedule, and also showed great understanding when we were the cause of delays. We will definitely work with them again in the future.

Are there any areas for improvement?

None that we can think of. The quality, speed and price were are satisfactory, which is quite unusual in our experience.

5.0
Overall Score Could not be more satisfied. Will definitely work with them again
  • 5.0 Scheduling
    ON TIME / DEADLINES
    Delivered according to deadline which was very tight.
  • 5.0 Cost
    Value / within estimates
    Not the cheapest and not the most expensive. The delivered value resonated well with the cost of the project.
  • 5.0 Quality
    Service & deliverables
    Excellent delivery of services. Exactly what we wanted
  • 5.0 NPS
    Willing to refer
    We recommend TestArmy to anyone looking for professional cybersecurity services

Customer Portal Security Testing for Printing Company

"They were extremely professional and required very little guidance or assistance."

See More

4.5

Quality: 
4.0
Schedule: 
4.5
Cost: 
4.0
Willing to refer: 
4.5
The Project
 
$10,000 to $49,999
 
Mar. - Apr. 2020
Project summary: 

TestArmy performed security testing on a mission-critical customer-facing portal application for a printing company. They conducted a number of different types of testing and component analysis.

The Reviewer
 
201-500 Employees
 
Plano, Texas
Douglas J. Savoie
VP, ABCorp
 
Not verified
The Review
Feedback summary: 

The software application is vital to the client's processes. TestArmy's single tester had great workflow and coordinated everything. They made the experience smooth and easy and had good communication skills.

The client submitted this review online.

BACKGROUND

Please describe your company and your position there.

My title is VP of Global Technology Solutions. I am responsible for product development in the Americas for my company.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire TestArmy?

The company hired TestArmy to perform security testing on a mission-critical customer-facing portal application.

What were your goals for this project?

The goals were to identify all cyber security risks in the body of software being tested.

SOLUTION

How did you select this vendor?

Test Army is a strategic partner of our outsourced software development team. Evidentally, they had performed careful vetting and interviewed many other companies. Test Army team members have seemingly every certification in the industry including: OSCP, OSCE, ITIL Foundation CDP, CISPA, ISO 27001, CISM, Prince 2 Practioner CPD, GXPN, GSLC, CISSP, CEH They follow these industry standards: • ISO/IEC 29119 Systems and Software Engineering — Software Testing • IEEE 1012 – Standards for verification and validation plans • 9000 ISO – Standards for quality management • ISTQB – International certification for software testing • IEEE 1044.1-1995 – Guide to classification of software anomalies

Describe the project in detail.

Test Army conducted the following types of security tests on our portal application: Static Analysis Static code analysis is a method of finding potential quality issues in code before the program is run. It is done by analyzing a set of code against a set (or multiple sets) of coding rules. This type of analysis addresses weaknesses in source code that might lead to vulnerabilities. Dynamic Analysis Dynamic analysis is the testing and evaluation of a program in real-time while it is running to find errors including:

• Dependencies that are not possible to detect in static analysis such as dynamic dependencies using reflection, dependency injection, and polymorphism. • Memory leaks • Pointer arithmetic errors such as null pointers • Time dependencies

Dynamic Analysis deals with real input data such as real input from Web requests and user interactions which static analysis does not address. By debugging a program in all the scenarios for which it is designed, dynamic analysis eliminates the need to artificially create situations likely to produce errors. Component Analysis Modern software is assembled using third-party and open source components, glued together in complex and unique ways, and integrated with original code to provide the desired functionality.

Third-party (including commercially licensed, proprietary, and "source available" software) along with open source components provide the necessary building blocks that allow organizations to deliver value, improve quality, reduce risk and time-to-market. The benefits of open source are many. However, by using open source components, organizations ultimately take responsibility for code they did not write. Strategic alliances between organizations and open source projects can lead to healthy open source usage and overall risk reduction.

Component Analysis is the process of identifying potential areas of risk from the use of third-party and open-source software and hardware components. Component Analysis is a function within an overall Cyber Supply Chain Risk Management (C-SCRM) framework. A software-only subset of Component Analysis with limited scope is commonly referred to as Software Composition Analysis (SCA). Any component that has the potential to adversely impact cyber supply-chain risk is a candidate for Component Analysis.

Manual Penetration Test Manual penetration testing is performed by a certified expert engineer. Generally, testing engineers perform the following methods:

− • Data Collection − Data collection plays a key role for testing. One can either collect data manually or can use tool services (such as webpage source code analysis technique, etc.) freely available online. These tools help to collect information like table names, DB versions, database, software, hardware, or even about different third-party plugins, etc. • Vulnerability Assessment − Once the data is collected, it helps the testers to identify the security weakness and take preventive steps accordingly. • Actual Exploit − This is a typical method that an expert tester uses to launch an attack on a target system and likewise, reduces the risk of attack. • Report Preparation − Once the penetration is done, the tester prepares a final report that describes everything about the system. Finally, the report is analyzed to take corrective steps to protect the target system.

What was the team composition?

There was a single person who plan the role of Project Manager, Business Analyst, Test Designer, Test Engineer, and Documentation Specialist.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

This software application is the most important and sensitive customer-facing system we have. Its security is taken extremely seriously by our infrastructure and information security teams around the world. We invested a great deal of time and money in a ton of different types of controls. Yet, we would not have considered launching without a thorough security test.

How effective was the workflow between your team and theirs?

The tester was a one-man show. He coordinated everything and tested everything. The company is located in Poland. But, this guy spoke perfect English and was a pleasure to deal with. I was on several calls with him and my entire team. Everyone got along with him very well. The experience was very smooth and easy.

What did you find most impressive about this company?

They were extremely professional and required very little guidance or assistance. I just gave them system access and they did their thing.

Are there any areas for improvement?

I can’t think of any.

4.5
Overall Score
  • 4.5 Scheduling
    ON TIME / DEADLINES
    Extremely easy. They required only two-weeks advance notice and performed all testing around our schedule.
  • 4.0 Cost
    Value / within estimates
    Reasonable
  • 4.0 Quality
    Service & deliverables
    Their work products were top quality. At the end of the project, they sent us comprehensive reporting on all of the tests they conducted which includ
  • 4.5 NPS
    Willing to refer

Application Testing for Password Security Platform

"They were flexible and performed very well."

See More

5.0

Quality: 
5.0
Schedule: 
5.0
Cost: 
5.0
Willing to refer: 
5.0
The Project
 
Less than $10,000
 
Dec. 2019
Project summary: 

TestArmy handled the web application testing services for a password security platform. Their team tested the platform for bugs, UI/UX inconsistencies, functionality, design, and user acceptance.

The Reviewer
 
1-10 Employees
 
Næstved, Denmark
Benny Ejholm Nissen
CEO & Owner, Password Crypt ApS
 
Verified
The Review
Feedback summary: 

TestArmy discovered around 50 errors in the web application and provided their findings in a report. Their team was communicative and had a quick understanding of the complex scope. Flexibility, dedication, and effective performance are hallmarks of their work.

The client submitted this review online.

BACKGROUND

Please describe your company and your position there.

I am the owner and software developerof Password Crypt headquartered in Næstved, Denmark. We provide top class password security solutions along with encrypted messaging for company team.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire TestArmy?

We hired TestArmy to make sure there is no bugs or inconsistency in UI/UX. TestArmy was choosen as the independent party which was responsible to perform UAT testing of our system.

What were your goals for this project?

Our goals were to test a web application - Pcrypt with a browser extension for Chrome and Firefox and make sure that the final user won’t have functional problems to use the whole system.

SOLUTION

How did you select this vendor?

They found us and we just wanted to try them out

Describe the project in detail.

The work was focused on exploratory tests of the entire flow of the web application and possible bugs. The usability tests were also conducted to determine the extent to which the application is understandable, easy to use and attractive to users. In the end, we wanted to check the overall design

What was the team composition?

Dedicated tester

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

TestArmy team discovered overall around 50 errors in our application considering all areas that were taken into account. The report that they have provided to us helped to improve our services.

How effective was the workflow between your team and theirs?

In the project TestArmy used experienced QA manager to lead the testing, which was very helpful. The communication and understanding was excellent. It was as good as it get actually :-)

What did you find most impressive about this company?

Quick understanding and work performed for a very complicated product. They were flexible and performed very well - will use them again for sure.

Are there any areas for improvement?

No

5.0
Overall Score Will use them again and have nothing to complain about
  • 5.0 Scheduling
    ON TIME / DEADLINES
    Will use them again and have nothing to complain about
  • 5.0 Cost
    Value / within estimates
    Low compared to the competition
  • 5.0 Quality
    Service & deliverables
    Will use them again and have nothing to complain about
  • 5.0 NPS
    Willing to refer
    For sure

Penetration Testing for Software Development Company

"They have a very effective team and all questions we had were answered the same day."

See More

5.0

Quality: 
5.0
Schedule: 
5.0
Cost: 
5.0
Willing to refer: 
5.0
The Project
 
Less than $10,000
 
Jan. - Feb. 2019
Project summary: 

TestArmy performed penetration services for a boutique software development company. The team provided detailed reports of existing security issues and remedied these errors.

The Reviewer
 
11-50 Employees
 
Wrocław, Poland
Michał Kurzeja
CTO, Accesto
 
Verified
The Review
Feedback summary: 

The engagement fixed any existing cybersecurity issues, meeting the expectations of the third-party client. TestArmy provides thorough reporting to ensure a smooth workflow. The team is hard-working, experienced, and knowledgeable.

The client submitted this review online.

BACKGROUND

Please describe your company and your position there.

I am the CTO of a boutique software development company that helps SaaS companies overcome their technical challenges and scale their business without performance and reliability issues. We make sure the code quality gets better every month, the app can scale, and that all new features hit the market in a short period of time.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire TestArmy?

We took over a project that was developed for the last 6 years by many developers. We knew the code is in a bad shape, and we wanted to quickly get rid of all the issues with its scalability, maintainability, reliability and of course - security issues. Our developers did some quick internal tests that confirmed the code has security issues, but we needed to be 100% sure we cover everything.

What were your goals for this project?

We asked TestArmy to perform in-depth penetration tests of the system and create a list of all the vulnerabilities that need to be fixed. We really needed someone, who will be able to find even the smallest issues that could lead to a data breach. Our requirements included also guidance in fixing some bugs, and a second test round to confirm everything is secure after we introduce the fixes.

SOLUTION

How did you select this vendor?

We did a research and noticed TestArmy has a skilled team and good case studies to help us with our problem. We also did some research and found out that some companies we know already worked with TestArmy and recommend their services.

Describe the project in detail.

After two weeks we received a very detailed report with all security issues in the app. The report included not only the bugs, but also some explanations, instruction on how to reproduce the issue, detailed description on how to secure a bug and some more metrics.

After that we started to code the fixes and consulted some issues with TestArmy. When all issues were fixed, TestArmy performed a second test round to confirm all issues are now secured.

What was the team composition?

We had contact with a project manager and a cybersecurity expert.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

Thanks to TestArmy all security issues were fixed, our customer was extremely happy with the results and this was one of the key reasons we got the project maintenance deal for the next years.

How effective was the workflow between your team and theirs?

They have a very effective team and all questions we had were answered the same day. Each explanation was easy to understand and covered everything needed to do the work on our side.

What did you find most impressive about this company?

I was surprised by how much knowledge the cybersecurity expert had in his area, and how well he communicated all the important information. We were not only able to fix the issues, but we also learned a lot and now we can spot some issues on our own.

Are there any areas for improvement?

No, we really enjoyed working with TestArmy, and we refer them each time someone asks us about penetration testing services.

5.0
Overall Score Thanks!
  • 5.0 Scheduling
    ON TIME / DEADLINES
    Always on time, even before the deadline
  • 5.0 Cost
    Value / within estimates
    Maybe not the cheapest ones, but definitely worth every cent
  • 5.0 Quality
    Service & deliverables
    Perfect!
  • 5.0 NPS
    Willing to refer
    Always when someone is in need for this kind of services

Manual Testing for International B2B E-Commerce Platform

"They act fast on changes and are flexible with our requests and demands."

See More

4.5

Quality: 
4.5
Schedule: 
4.5
Cost: 
5.0
Willing to refer: 
5.0
The Project
 
$10,000 to $49,999
 
July 2019 - Ongoing
Project summary: 

TestArmy supports an international B2B e-commerce platform by providing manual QA services. They continue to develop improvements to ensure stable, high-quality deliveries.

The Reviewer
 
51-200 Employees
 
Amsterdam, Netherlands
René Crooijmans
Head of Development & Delivery, ERIKS Digital
 
Verified
The Review
Feedback summary: 

Besides increasing delivery quality, TestArmy has integrated well into the development team's workflow. This has led to a faster throughput time for stories and fewer iterations on releases. They're speedy, adaptable, and collaborative.

The client submitted this review online.

BACKGROUND

Please describe your company and your position there.

International MRO and OEM distributor and manufacturer of goods for all large industrial manufacturing industries.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire TestArmy?

TestArmy supports our international; B2B E-Commerce deliveries on functional quality by providing manual QA services to our development teams

What were your goals for this project?

Increasing our quality assurance on production delivery efficiency for our international B2B E-Commerce platform

SOLUTION

How did you select this vendor?

Initially, we made a shortlist of vendors providing manual testing services with a set of criteria. With this input, we scheduled meetings to validate the offerings based on cost, ramp-up time, flexibility and working culture. All four indicators have shown us that TestArmy would fit our needs and working culture.

After this selection, we scheduled interviews with candidates of TestArmy to get a good feeling about the quality and being able to select a good candidate that would fit our team. After the selection of the candidate, the agreements were very fast in place and the candidate was able to start on short notice.

Describe the project in detail.

We continuously develop improvements for our international B2B E-commerce offering. To ensure stable and high-quality deliveries we wanted to extend our quality coverage in our development flow.

What was the team composition?

We have 2 multidisciplinary teams with 12 developers (Java and Front End). The teams are supported with a Scrum Master, a QA Automation engineer and a DevOps engineer. The QA enigneer from TestArmy is fully embedded (from a remote location) in our team and participates in all SCRUM related ceremonies

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

The quality of the deliveries is increased. We have a faster throughput time of stories in our sprints with fewer iterations

How effective was the workflow between your team and theirs?

The cooperation is very effective as the QA engineer is fully embedded in our teams, participates in all our SCRUM ceremonies and follows our processes.

What did you find most impressive about this company?

Fast speed of providing a good QA engineer. They act fast on changes and are flexible with our requests and demands.

Are there any areas for improvement?

As of now, our cooperation goes very well and there are no need of any improvements.

4.5
Overall Score
  • 4.5 Scheduling
    ON TIME / DEADLINES
    very easy to schedule meetings
  • 5.0 Cost
    Value / within estimates
    The costs versus quality ratio is very positive for our organisation
  • 4.5 Quality
    Service & deliverables
    They deliver good qaulified engineers
  • 5.0 NPS
    Willing to refer

Application Testing for a Yacht Rental Platform

"They’ll tell you if something is wrong and won’t hide anything from you." 

See More

5.0

Quality: 
5.0
Schedule: 
4.5
Cost: 
5.0
Willing to refer: 
5.0
The Project
 
Less than $10,000
 
Oct. 2018 - Ongoing
Project summary: 

TestArmy provides diverse testing services for mobile apps about to go into production. They’re responsible for regression, UX, and edge-case testing. They also identify discrepancies in use between iOS and Android platforms.

The Reviewer
 
11-50 Employees
 
Fort Lauderdale, Florida
Sam Lepak
Head of Digital Marketing, YachtLife
 
Verified
The Review
Feedback summary: 

No major bugs have slipped through TestArmy’s program into production. Their comprehensive bug tracking via Pivotal and their constant communication have empowered them to investigate all possible technical issues with both iOS and Android apps. Their transparency sets them apart in the industry.

A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.

BACKGROUND

Introduce your business and what you do there.

I’m the head of digital marketing at YachtLife, an Airbnb for luxury yachts. We’re no different than a yacht broker, but we’ve become the first mobile app to allow users to charter a luxury yacht for a day, half day, or multiple days on the water.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with TestArmy?

We needed quality assurance, bug reporting, and user acceptance (UA) testing for our mobile apps and our website. We’d create a build and prepare to send it out, but we’d do the testing ourselves and only later find a bug that affected tons of users. We didn’t want that problem. We wanted to ensure a great experience every time we rolled out a new feature or a new app update.

SOLUTION

What was the scope of their involvement?

Initially, we did a two-week discovery sprint so they could learn about the app. That was at a flat rate, which was less than their hourly rate. They comprehended the job quickly, and we decided to do it full-time. They informed us they were going to pair us with a new tester, but that didn’t make sense to me as they had someone learning our whole platform for two weeks. They went back and agreed to keep the original tester on our project.

We build out features in a staging environment, whether it be with our app or our website, and before uploading it to production, we have TestArmy do regression testing, user experience testing, edge-case testing and identify discrepancies across platforms. Basically, they’d try to break it. They test on multiple devices, and they’ve been willing to buy devices if they don’t have a particular one. They’ll also make suggestions based on discrepancies in how it functions on Android vs. iOS. Afterward, they’ll deliver a report to us. When they find a bug, they report it in Pivotal Tracker and let us know in our Slack channel. They don’t do any of the bug fixing. We’re also looking at implementing security testing with them as well because they've spotted a few security issues.

What is the team composition?

We have a project manager, along with a part-time resource.

How did you come to work with TestArmy?

I looked at Capterra reviews and reached out to TestArmy and other QA agencies. I narrowed it down to four or five, and we had calls with them. We went through rounds of interviews. It seemed like a lot of them possessed the same capabilities, but offered them for different prices. We asked TestArmy for two current customer reviews to get their feedback. I was able to ask them questions to see what it was like. That was super helpful, and it solidified our choosing them.

How much have you invested with them?

We spend $2,000/month. They work on a month to month basis, which is awesome. They’ve been extremely flexible with contracts.

What is the status of this engagement?

We’ve been working together since October 2018.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

It’s more of a reactive success metric. After we deploy something, do we have any bugs? After deploying all of our major updates since bringing TestArmy on, we haven’t had any major bugs. There’s been 1–2 edge cases out of everything, but those are outliers. We’re extremely satisfied with the product development side.

How did TestArmy perform from a project management standpoint?

We communicate via Slack, and they’re very responsive. Their tester and product manager are in Slack with us, so they’re like a part of the team. We get a daily summary of everything they work on through Slack too. They report all of their bugs in Pivotal Tracker, and they were willing to accommodate whatever tracking system we wanted. I have a weekly call with them. They’re super transparent, and they confirm all their hours before billing us.

What did you find most impressive about them?

They’re very focused, and the price was well below average for the quality they deliver. Their communication is great. They leave no stone unturned. They’ll tell you if something is wrong and won’t hide anything from you.

Are there any areas they could improve?

Early on, they documented some bugs that were more subjective than objective. We talked to them about it and told them to focus on the objective bugs we could fix. The issue was resolved after that.

Do you have any advice for potential customers?

It helps if they know exactly what you want. In our case, we didn’t know what we wanted, but they helped us figure it out, which was great. If you’re unsure about using them, try out their two-week discovery period and see how it goes. They’re straightforward and transparent and will help you determine what you need, but if you go in knowing what you need, it’ll be easier to determine if they’re right for you.

5.0
Overall Score It’s been near perfect. I’m very surprised by how smoothly things have gone.
  • 4.5 Scheduling
    ON TIME / DEADLINES
    They’re a little out of our time zone, but we’ve set up a different time to talk with them. Everything else has been above satisfactory.
  • 5.0 Cost
    Value / within estimates
    We’ve been very happy with the quality for the price we’re paying.
  • 5.0 Quality
    Service & deliverables
    Everything they’ve delivered has been spot-on. We’ve had zero issues. It’s been great.
  • 5.0 NPS
    Willing to refer
    I’ve recommended them to other friends, colleagues, and app marketers. We’ve had a great relationship and I want that for other people as well.

Software Testing for Consumer Complaints Company

“They’ll independently make suggestions on how we can change our approach and get better results.”

See More

4.5

Quality: 
4.5
Schedule: 
4.0
Cost: 
4.0
Willing to refer: 
5.0
The Project
 
Confidential
 
Oct. 2017 - Ongoing
Project summary: 

TestArmy provides an in-house team with automated QA testing for one of their products. They create and implement all tests and flag issues for further development prior to deployment.

The Reviewer
 
11-50 Employees
 
London, United Kingdom
Hattie Burt
Head of Projects, Resolver
 
Verified
The Review
Feedback summary: 

Their developer is fully integrated with the in-house development team, making it easy to coordinate work and stay on schedule. Thanks to their work, there haven’t been any major issues with the live product thus far.

A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.

BACKGROUND

Introduce your business and what you do there.

I’m the head of projects at Resolver. We assist consumers in raising complaints against a company they’ve had an issue with by providing them information about their rights and guiding them through the complaint process. The service is free to consumers, but we also offer a set of enterprise products for companies and regulators that want to resolve consumer complaints.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with TestArmy?

 

They’ve been helping us with a collaborative decision-making product aimed at regulators, courts, and ombudsmen. We brought TestArmy onboard to save time and ensure that deployments go smoothly.

SOLUTION

What was the scope of their involvement?

They provide us with automated testing resources. Their developer participates in regular stand-ups with our team. We deploy new code each week, so while our team is working on development, they’re writing the automated tests for it. Once everything’s ready, they run the tests overnight and flag any issues for our team to review.

What is the team composition?

We currently have one developer who works full-time with us, but we’re looking to expand our use of their services.

How did you come to work with TestArmy?

Our CTO received a referral from a colleague, so we looked into their services and found that they were a good fit for our needs.

What is the status of this engagement?

We started working with them in October 2017 and we’re still working together.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

They’ve integrated really well with our development team and they’re good at proactively solving problems. They’ll independently make suggestions on how we can change our approach and get better results. We hired them to give us confidence that our product would be reliable when we launched it in March 2018 and, due to their QA work, we haven’t had any major issues in the six months since then.

How did TestArmy perform from a project management standpoint?

We manage their individual tester who works with our team, but their administrative team is always helpful when we need to get in touch with them about contracts and scheduling.

What did you find most impressive about them?

 

Integrating testers and developers can be a bit tricky, but the tester we have is a core member of our team. He takes initiative and communicates with the rest of the team to solve problems.

Are there any areas they could improve?

We’ve realized that we need to have more discussion upfront about the strategic approach before they begin writing the tests. We initially went for expediency, but it’s become clear on both sides that we’re better off having those planning conversations at the start.

Do you have any advice for potential customers?

Know what you hope to get out of the engagement and fully integrate their developer into your team if you can. They’re very willing to engage with you as a partner as long as you communicate effectively.

4.5
Overall Score
  • 4.0 Scheduling
    ON TIME / DEADLINES
    The testing gets done on time, but timeliness on the contract side of things could be better.
  • 4.0 Cost
    Value / within estimates
    The value for the junior testers is excellent, but using senior testers is a bit expensive.
  • 4.5 Quality
    Service & deliverables
    They’ve been really consistent and we haven’t had any major issues in the live environment.
  • 5.0 NPS
    Willing to refer
    I’ve recommended them before.

Testing for Financial Services Aggregation API

"The personal touch made the difference."

See More

4.0

Quality: 
4.0
Schedule: 
5.0
Cost: 
4.0
Willing to refer: 
5.0
The Project
 
Less than $10,000
 
June 2017 - Ongoing
Project summary: 

TestArmy coordinates crowd-testing efforts that require testers to have several unique bank and credit card accounts. They provide end-to-end functionality testing for a financial API. 

The Reviewer
 
11-50 Employees
 
Hamburg, Germany
Project Manager, Financial Services API
 
Verified
The Review
Feedback summary: 

TestArmy has streamlined the testing process, decreasing the time it takes to test and fix an issue from one month of one week. Their communication and reporting are proactive, and the team has helped to lower the cost of testing while providing high quality work. 

A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.

BACKGROUND

Introduce your business and what you do there.

I am leading the European expansion of our core company product, a banking-as-a-service platform. We have built an API which helps other businesses—small fintechs and large companies—as well as single developers to easily aggregate different financial services. We currently support more than 3,000 financial services in Germany and Austria; the goal is to expand to 34 other countries, and this is the context within which we’re collaborating with TestArmy.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with TestArmy?

We need to make sure that we can provide direct access to our technology and APIs, but the bigger challenge was testing the integrations we’re building with several teams across Europe. In order to validate that the new financial institution or service we’re aggregating is working well, we need to perform tests with someone who has a real account with a bank or a credit card provider. Getting access to such a testing network was a big challenge for us, but a company like TestArmy could provide this out of the box, along with full management of the test cycles.

SOLUTION

What was the scope of their involvement?

TestArmy provides end-to-end functional testing. They support us with the recruitment of testers and the management of the test cycles, via a frontend we make available to them.

It’s an ongoing collaboration, so I suppose that there will be many other interesting things we’ll discover with TestArmy, but, for now, we’re focused on that.

What is the team dynamic?

We have been involved with at least 4 people from their side, but I’m not sure what the full size of the team is.

How did you come to work with TestArmy?

The process was organic—we went to Google and started searching for “crowd testing”. This was the keyword we used because I knew we actually wanted to have local testing groups. We looked at a couple of other crowd-testing agencies, as well as ones offering similar services, including ones in Berlin. We went with TestArmy because of their high engagement with the project. It was a non-typical set of requirements, namely testers who owned specific bank accounts, for example. TestArmy also had flexibility, and were loose in terms of offering a proof-of-concept project, as well as on how we would organize the cycle and how they would report results. TestArmy also put us in contact with testers directly.

How much have you invested with them?

The total cost has been below $10,000.

What is the status of this engagement?

We started working with TestArmy in June 2017.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

One thing we are measuring is the maintenance cost for each financial system being aggregated via API. Compared to costs per financial source in Germany (where we have an expensive setup in terms of getting tests done), they’ve definitely helped us decrease the number.

Before using TestArmy, we required a month of work for fixing an issue and going back to test it again. Because of how they’ve helped us streamline feedback from testers, we’ve reduced this process to 7 work days. It’s a huge improvement on our side.

How did TestArmy perform from a project management standpoint?

The reports are very detailed, including videos and valuable technical information. The engagement involved a big plan on our site, and the responsiveness of the project manager on their side has been perfect. They kept us up-to-date on the recruitment process of different testing groups, and it was really easy to get in touch and setup meetings. The whole project management flow makes it easy to actually plan test cycles. One of the biggest blockers for us was being able to plan for cycles in the future, and to get that part streamlined between tester feedback, fixing issues and testing them again.

What did you find most impressive about them?

The personal touch made the difference. Our use case is treated in-depth and with care, compared with other companies, where we were given generic options. The personalization is key.

Are there any areas they could improve?

So far, we haven’t had any problems with TestArmy.

4.0
Overall Score The way in which services are provided is perfect, but TestArmy could put some work into professionalizing the systems and improving the way in which customers access the information.
  • 5.0 Scheduling
    ON TIME / DEADLINES
    They’ve met all deadlines, and scheduling meetings has been a no-brainer. They’ve been reactive and flexible to scheduling things from one day to another.
  • 4.0 Cost
    Value / within estimates
    It was good, especially compared to other agencies.
  • 4.0 Quality
    Service & deliverables
    The way in which they report, the timeliness and the quality of the reports are very good, but TestArmy should implement a customer platform where we could look at the feedback ourselves.
  • 5.0 NPS
    Willing to refer
    The fact that TestArmy is so open to use cases, even problematic ones, says a lot about them.

Security Testing for Online Whiteboard Tool

"After TestArmy, we never had any issues with security or vulnerability."

See More

5.0

Quality: 
5.0
Schedule: 
5.0
Cost: 
5.0
Willing to refer: 
5.0
The Project
 
Less than $10,000
 
Aug. 2015 - Ongoing
Project summary: 

TestArmy tests code for an interactive whiteboard portal to identify data security risks. The team points out initial vulnerabilities and then performs a retest after making the internal team aware.

The Reviewer
 
51-200 Employees
 
Wroclaw, Poland
Piotr Sliwinski
CTO, Explain Everything
 
Verified
The Review
Feedback summary: 

After each test, a full report of the platform's vulnerabilities and strengths is given. TestArmy excels at setting clear dates for testing and retesting and is great at communicating their results. The team is knowledgeable and thoughtful, delivering the best possible service. 

A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.

BACKGROUND

Introduce your business and what you do there.

I’m the CTO of Explain Everything. We provide a free form creation tool that works like an interactive whiteboard. It’s usually mostly in education. We are making inroads in the enterprise market as well. Our tool can create explanatory movies or online lectures, and is best suited for presentations, lectures, student projects, and things like that.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with TestArmy?

We have a portal that stores presentations and projects created using our app. We wanted TestArmy to test the security and vulnerability of our portal. If you’re doing something with an internal team, it’s better to have an external team testing this, especially when it comes to data security. Nowadays there are many problems with websites not being secure. While we were following all best practices while setting up our portal, we wanted to have somebody test that externally.

SOLUTION

What was the scope of their involvement?

We test for security every time we do a major upgrade to our portal.  We give TestArmy the URL and the description of the app.  They’re trying to break it and hack into it using their tools. Then they give us a report for what they’ve been able to accomplish and why so that we can fix whatever it is. When we fix the issues, they retest it, and then we are safe. We’ve only been using them for security and vulnerability testing of our website on a yearly basis. 

How did you come to work with TestArmy?

They are from the same city as us. The companies within the region usually know each other and know who is doing what. We knew that they do testing and security testing. We contacted them. We did some research of other companies, but TestArmy looked the best to us and had the most experience.

How much have you invested with them?

We’ve spent between $5,000-$10,000.

What is the status of this engagement?

We started working together in August 2015 and will be entering into a third engagement in August or September of this year. They tested the portal in the summers of both 2015 and 2016, and we’ll do additional retests again this summer.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

They provide a full report of what tools were used for testing. For example, for part of the website, we are using the service that has a list of the newest vulnerabilities and they went through that list. We have a full report of what has been tested and whether vulnerabilities were found or not. For other things, they’re also providing how many security holes were present. We always have to fix them all. The KPI here would be the number of vulnerabilities found versus the number of vulnerabilities fixed. Then the retest shows whether they were fixed. After TestArmy, we never had any issues with security or vulnerability. We are really happy with the results. They are a good service.

How did TestArmy perform from a project management standpoint?

They performed very well. They have a clear start date and end date. The deliverables were always clear to us, as well as the retest dates. Both in terms of meeting deadlines and general time spent on the project, they are excellent. We communicated over phone and Skype. We also used our own tool, Explain Everything, because it can be used for communication. 

What did you find most impressive about them?

During personal meetings with them, I had a good impression that they were very thoughtful and really knew what they’re doing. I’m a long time tech entrepreneur and I can sense whether someone is knowledgeable or just talking nonsense. They really deliver.

5.0
Overall Score
  • 5.0 Scheduling
    ON TIME / DEADLINES
    Everything happens in a very timely manner.
  • 5.0 Cost
    Value / within estimates
    Compared to the quality of the service, the service is really inexpensive.
  • 5.0 Quality
    Service & deliverables
    They have very good looking reports with explanations on how to fix problems.
  • 5.0 NPS
    Willing to refer