TestArmy

Please describe your company and your position there.

My name is Daniel Jędrzejczyk. I work at Huuuge Games as a CRM Operations Manager. Huuuge Games is a company with a great and long history of creating games.

Nowadays our main focus is to develop social mobile games that will encourage people to play together in a real-time experience on a global scale.

For what projects/services did your company hire TestArmy?

We hired TestArmy because we needed specialists in functional testing to test games for us.

What were your goals for this project?

Our main goal for this project was to find someone qualified and dedicated to test games for us, and check their functionality and GUI every single day, without a break.

We needed experienced testers to perform tests for two of our games every day (even on national holidays, New Year’s Eve etc.), in order to check if the offers within the game change according to our scenario.

We were looking for fully dedicated support from the company that we have chosen.

How did you select TestArmy?

We have found TestArmy because of our research. We have contacted Szymon Chruścicki - TestArmy’s Business Development Manager, who later introduced us to Jarosław Rafa and Lidia Rafa, who were performing tests for us every day without a break.

Describe the project in detail.

TestArmy provided us with the highest quality of testing, while giving us one of the most experienced testers in their team.

Everyday, since 2017, one of TestArmy testers performs GUI and functionality tests for two of our games - Huuuge Casino and Billionaire Casino, to check the in-game performance and offers if they are adapted to the level of advancement of the player.

TestArmy tests the games according to our varying scenario everyday, because of seasonal changes of offers.

What was the team composition?

Our initial contact was with Szymon Chruścicki, but later on we have been introduced to Jarosław Rafa and Lidia Rafa, and we are working with them since then.

Can you share any outcomes from the project that demonstrate progress or success?

The biggest impact is that we are 100% sure and confident that our games live up to the expectations for players. We can also be safe with our in-game offers being perfectly adapted to the level of advancement for the players, which is obviously great for our business and selling plans.

How effective was the workflow between your team and theirs?

The biggest pro is that Jarosław Rafa and Lidia Rafa are very experienced testers, and the greatest thing is that Jarosław is also a Project Manager in TestArmy. We were sure from the start that nothing could go wrong with this project.

What did you find most impressive about this company?

The most impressive thing was that even though our testing hours are quite demanding, TestArmy had no issue with that and met our expectations perfectly.

Are there any areas for improvement?

As of right now, I am happy with our cooperation.

IPC Europe is an independent non-profit making organisation owned by Subway® Franchisees in Europe. Whom manage supply of Subway® approved food, packaging and equipment, but also offer a wide range of support of commercial services and technology solutions, such as Subway® Loyalty Program & App.

After supporting a 10-year-old existing Loyalty App and Program, strategically the business had agreed to transform the current Mobile App, Website and program in the current European markets in terms of look & feel, as it was seen as a key customer and business initiative in terms of engagement and to better compete in the QSR/Retail space. This meant having to deliver an MVP product without losing any of the key existing features, but ultimately the focus was to produce a complete new UX and design.

The scale of the project was for 7 key regions where the program was existing but also to ensure futureproofing the product for longer term strategy initiatives. Knowing the roll out plan would span a minimum of 6 months would have to take an iterative approach and focus on one market at a time. Overall the project was a re-skin of branding but also improving the UX for end User across a large scale customer base.

We at IPC Europe had worked with TestArmy previously on significant rollout projects for 3 markets and the updates of the platforms to support GDPR. We had an existing relationship which was important now undertaking the largest project in terms of scale and complexity the team had most probably undertaken for 10 years.

From a business perspective one of real importance for the range of stakeholders and for the Franchisees and Stores. Ensuring a quality end product for all the European consumers and market. The scale and magnitude of the task looking back was underestimated considering the langue, complexity, and impacted business teams.

We successfully rolled out the Mobile App and platforms in a timely manner with very positive consumer and business feedback. The way in which TestArmy works and integrates into our internal project is what stands out. The daily and weekly engagement with the business project calls, daily IM communication with Slack and being able to directly work in the scrum team with bug raising.

This level of team work along with the ability to add value with feedback through UAT of the platforms is a huge positive. We encountered multiple delays, issues with the designs and very last minute changes which meant the scheduled resources hand to be stood down at last minute this flexibility and support is invaluable.

We carry out internal testing, however having a added range of devices to validate test feedback but also level of assurance we receive from a group of experts Testers gives the senior stakeholders in the business and added level of confidence which is crucial when we consider the quality assurance for the end result. The only challenge at times is the desired framework or layout of feedback within our ticketing tool for the development teams.

Please describe your company and your position there.

I am running Droids On Roids, a full-service iOS, Android and Web Development Company from Poland, operating since 2011. We partner with the best start-ups and great brands to build high-performing mobile & web apps. Companies that love the quality of our code are Whatsapp, Facebook, Disney, Electric Objects and Giphy, just to name a few.

For what projects/services did your company hire TestArmy?

We needed additional support in our QA processes for ongoing projects. We seek to collaborate with top-level specialists on individual projects rather than hiring full-time employees. TestArmy job was to develop and execute exploratory, acceptance and integration tests to ensure the quality of our digital products.

What were your goals for this project?

We wanted to check our digital products (web and mobile apps) on various parameters like applicability, functionality, security and scalability. We wanted to be sure that we've met user requirements and we found and fixed all bugs before releasing our software.

How did you select TestArmy?

We were looking for a supplier who has qualified technical resources on board with strong experience in delivering projects successfully and working as a part of a global team.

Choosing a trusted QA partner to cooperate with wasn’t easy. We have contacted many firms but TestArmy was supposed to have the biggest experience in this field. Looking back it was the right decision. We are still cooperating closely with them.

Describe the project in detail.

So far the team of TestArmy has taken part in our 10 projects with durations from 2 weeks to 6 months. They have verified the quality of applications such as the medical app for people who wear corrective orthoses, the video-on-demand service, and the app connected to the IoT devices. They reviewed quality specifications & requirements and created well-structured test plans and test cases.

What was the team composition?

There were five to ten QA Engineers on their team that worked on our various projects monthly. We also had contact with the Project Manager and the Sales Manager.

Can you share any outcomes from the project that demonstrate progress or success?

The QA Engineers helped us ensure high quality in the earliest stages of developing a feature. Every time they provided timely and meaningful feedback. What is more, they bring the know-how to the company, so the tools and IT environment including processes can still be used locally.

How effective was the workflow between your team and theirs?

We have daily updates and weekly progress reports. Usually, we communicate over Slack as well as using Jira to manage tasks. Although we work remotely TestArmy team became part of our team and got on very well with developers. They don’t hesitate to ask the questions. The PM’s communicate about the scope and budget and provide reports and invoices.

What did you find most impressive about this company?

The most impressive was the level of engagement. The QA team is detail-oriented with a very proactive approach of what can be done better both in software development and business areas.

Besides that we know that whenever we need extra QA support we can reach out to them and they will always find an experienced Tester for our project. TestArmy is always flexible when it comes to the schedule and level of engagement.

Are there any areas for improvement?

No, the cooperation was just fine.

Please describe your company and your position there.

I am a Technical Recruiter at Bergman Deutschland GmbH. Bergman Deutschland is a recruitment agency, which specializes in assigning Engineers and Companies to a certain project.

For what projects/services did your company hire TestArmy?

We decided to hire TestArmy, because we needed a third-party vendor to perform penetration tests on our internal sensitive data collecting software. Our main goal was to make sure that all the data we gathered was safe within that software.

What were your goals for this project?

Our main challenge was to perform penetration tests so we could make sure our software was solid. If not, we wanted to gather all the vulnerabilities in one place, so later on we can solve these issues and keep all the sensitive, internal data without fear that hackers can break-in.

How did you select TestArmy?

We have contacted around 3 other cybersecurity companies, who claimed to be specialists in this field. However TestArmy had the biggest experience in pentesting. That’s why we decided to choose them.

Describe the project in detail.

TestArmy’s cybersecurity team performed pentests on our system to check if hackers would be able to break-in. They also tested the whole software to check if anyone accidentally would be able to see someone else’s personal information without permission to that account.

They met with us to analyze the whole software, first briefly, schedule how much time they would need, check what kind of software is that, what kind of data is there. Later on, they introduced us to the project plan, and as we accepted the plan they started the whole testing process.

The whole time they made sure that they wouldn’t interrupt our work or surprise our employees with their tests. After finishing the pentesting processes they created a very detailed report, that has shown us in which areas we have issues, and vulnerabilities that could be a potential threat to our software system.

The most important part was that they walked us through the whole report, explaining what’s wrong, and how it could affect our business.

What was the team composition?

We were frequently contacted by the Project Manager, Sales Manager and the Head of Cybersecurity team.

Can you share any outcomes from the project that demonstrate progress or success?

They performed pentests on our internal software, showed us in which areas we had vulnerabilities and potential risks, so we could improve our safety measures in those certain areas. Every vulnerability was included in the report, which had a great amount of detailed information.

Thanks to that, we could easily fix the issues. At the same time they have explained the issues included in the report, so for people, like us, who are not cybersecurity experts it was very important to have someone who would be able to explain all of the things easily.

Now we are very aware of the vulnerabilities that our software has and how it can affect our business. That’s why we are going to solve those issues as soon as possible, and later, ask them again to perform pentests to make sure that the software is secure from all the attacks from outside and inside.

How effective was the workflow between your team and theirs?

The project management was very effective and was arranged properly. The whole team always met our deadlines and we were frequently in touch to discuss the progress in the project.

What did you find most impressive about this company?

The most impressive thing was how experienced they are with pentesting. We knew that they would find everything and anything that could potentially cause security problems later on.

Are there any areas for improvement?

Honestly, I can't think of anything to improve.

Please describe your company and your position there.

My title is VP of Global Technology Solutions. I am responsible for product development in the Americas for my company.

For what projects/services did your company hire TestArmy?

The company hired TestArmy to perform security testing on a mission-critical customer-facing portal application.

What were your goals for this project?

The goals were to identify all cyber security risks in the body of software being tested.

How did you select this vendor?

Test Army is a strategic partner of our outsourced software development team. Evidentally, they had performed careful vetting and interviewed many other companies. Test Army team members have seemingly every certification in the industry including: OSCP, OSCE, ITIL Foundation CDP, CISPA, ISO 27001, CISM, Prince 2 Practioner CPD, GXPN, GSLC, CISSP, CEH They follow these industry standards: • ISO/IEC 29119 Systems and Software Engineering — Software Testing • IEEE 1012 – Standards for verification and validation plans • 9000 ISO – Standards for quality management • ISTQB – International certification for software testing • IEEE 1044.1-1995 – Guide to classification of software anomalies

Describe the project in detail.

Test Army conducted the following types of security tests on our portal application: Static Analysis Static code analysis is a method of finding potential quality issues in code before the program is run. It is done by analyzing a set of code against a set (or multiple sets) of coding rules. This type of analysis addresses weaknesses in source code that might lead to vulnerabilities. Dynamic Analysis Dynamic analysis is the testing and evaluation of a program in real-time while it is running to find errors including:

• Dependencies that are not possible to detect in static analysis such as dynamic dependencies using reflection, dependency injection, and polymorphism. • Memory leaks • Pointer arithmetic errors such as null pointers • Time dependencies

Dynamic Analysis deals with real input data such as real input from Web requests and user interactions which static analysis does not address. By debugging a program in all the scenarios for which it is designed, dynamic analysis eliminates the need to artificially create situations likely to produce errors. Component Analysis Modern software is assembled using third-party and open source components, glued together in complex and unique ways, and integrated with original code to provide the desired functionality.

Third-party (including commercially licensed, proprietary, and "source available" software) along with open source components provide the necessary building blocks that allow organizations to deliver value, improve quality, reduce risk and time-to-market. The benefits of open source are many. However, by using open source components, organizations ultimately take responsibility for code they did not write. Strategic alliances between organizations and open source projects can lead to healthy open source usage and overall risk reduction.

Component Analysis is the process of identifying potential areas of risk from the use of third-party and open-source software and hardware components. Component Analysis is a function within an overall Cyber Supply Chain Risk Management (C-SCRM) framework. A software-only subset of Component Analysis with limited scope is commonly referred to as Software Composition Analysis (SCA). Any component that has the potential to adversely impact cyber supply-chain risk is a candidate for Component Analysis.

Manual Penetration Test Manual penetration testing is performed by a certified expert engineer. Generally, testing engineers perform the following methods:

− • Data Collection − Data collection plays a key role for testing. One can either collect data manually or can use tool services (such as webpage source code analysis technique, etc.) freely available online. These tools help to collect information like table names, DB versions, database, software, hardware, or even about different third-party plugins, etc. • Vulnerability Assessment − Once the data is collected, it helps the testers to identify the security weakness and take preventive steps accordingly. • Actual Exploit − This is a typical method that an expert tester uses to launch an attack on a target system and likewise, reduces the risk of attack. • Report Preparation − Once the penetration is done, the tester prepares a final report that describes everything about the system. Finally, the report is analyzed to take corrective steps to protect the target system.

What was the team composition?

There was a single person who plan the role of Project Manager, Business Analyst, Test Designer, Test Engineer, and Documentation Specialist.

Can you share any outcomes from the project that demonstrate progress or success?

This software application is the most important and sensitive customer-facing system we have. Its security is taken extremely seriously by our infrastructure and information security teams around the world. We invested a great deal of time and money in a ton of different types of controls. Yet, we would not have considered launching without a thorough security test.

How effective was the workflow between your team and theirs?

The tester was a one-man show. He coordinated everything and tested everything. The company is located in Poland. But, this guy spoke perfect English and was a pleasure to deal with. I was on several calls with him and my entire team. Everyone got along with him very well. The experience was very smooth and easy.

What did you find most impressive about this company?

They were extremely professional and required very little guidance or assistance. I just gave them system access and they did their thing.

Are there any areas for improvement?

I can’t think of any.

Please describe your company and your position there.

I am the founder of ZING which is a new video call platform that focuses on social interaction. Our platform allows people to move around fluidly between different groups - rather than being stuck talking to each other - and also has in-built games to make calls more fun.

For what projects/services did your company hire TestArmy?

We needed to test the functionality of our video-call platform. One of our developers created critical scenarios, which we later passed on to TestArmy’s testers so they could test the product with those scenarios.

What were your goals for this project?

Our main goal for this project was to find as many critical bugs as possible, and check how the platform would perform under certain conditions.

How did you select TestArmy?

Actually, TestArmy was recommended to us by Visuality, the company doing the development for us. This definitely sped up the process of deciding who we should choose for this project.

Describe the project in detail.

Three testers from TestArmy did as much functional testing over a 2-day period as they could across a range of devices. We provided scenarios, but they were also encouraged to do exploratory testing. They found a few very critical bugs that have helped us improve the performance of our platform as well as recommending smaller improvements

What was the team composition?

During this project, we were in touch with the Project Manager, Sales Representative, and 3 software testers

Can you share any outcomes from the project that demonstrate progress or success?

The testers found key bugs that we had not discovered, which were critical for us to resolve prior to launch. After the initial testing, we have re-engaged with TestArmy to perform some ongoing testing for every release.

How effective was the workflow between your team and theirs?

This was great - we added the testers to our slack channels, gave them access to our Jira and they were able to add bugs directly into the workflow in a logical way.

What did you find most impressive about this company?

The level of detail in their testing was very impressive. They also integrated very smoothly into our development process.

Are there any areas for improvement?

None - they are flexible and friendly and we are continuing to work with them

Please describe your company and your position there.

I am a Team Leader of QA at 10Clouds. 10Clouds is a team of 100+ experienced developers and designers based in Poland, ready to help you build your web and mobile applications. Since 2009 we have built and designed software for more than 90 businesses worldwide, mainly in the United States, Western Europe, and Australia.

For what projects/services did your company hire TestArmy?

Various: Mainly web testing, mobile testing, test automation

What were your goals for this project?

Testers were involved in each project. They were responsible for the verification and validation of given applications. In some cases this was a specific client project, while in others it was longer term cooperation with a client through staff augmentation.

How did you select TestArmy?

We selected them based on positive feedback online and recommendations.

Describe the project in detail.

This is a long term, established relationship rather than a single project. We’ve worked together on a number of client projects ranging in size and scope.

What was the team composition?

Various. Per one project we usually had a tester, a Project Manager and three developers.

Can you share any outcomes from the project that demonstrate progress or success?

We’ve received positive feedback from numerous clients on whose projects we’ve worked together with TestArmy. One of our key clients gave positive feedback about TestArmy’s work specifically and requested to continue cooperation with them.

How effective was the workflow between your team and theirs?

We have very similar styles of working, which enables fast onboarding of TestArmy staff and enables us to get started on jobs quickly. Cooperation and workflow have both been very good.

What did you find most impressive about this company?

We have been most impressed by TestArmy’s flexibility and ability to quickly adjust to developments within projects.

Are there any areas for improvement?

My only comment would be that it might be good to have a few more senior testers.

Please describe your company and your position there.

I am CEO of a SaaS company that provides IT solutions to the dental Industry. We are headquartered in Malmö, Sweden and serve clients in the Nordics, and are currently expanding internationally.

For what projects/services did your company hire TestArmy?

Our company is experiencing rapid growth and are quickly expanding our product portfolio and services. To ensure that our speed did not compromise our security on any level (architecture, technical solutions and operational procedures) we felt that it was necessary to get outside expert opinion and screening.

What were your goals for this project?

We wanted to see that our technical solution could withstand a wide range of potential attacks and also ensure that our staff used good routines and procedures for managing phishing attacks etc.

How did you select this vendor?

Our team reached out to several cybersecurity firms. Our requirements were that the supplier should have strong experience in delivering projects on an international level and speed of delivery. We chose Testarmy based on a combination of their references, price range and ability to deliver according to our deadline.

Describe the project in detail.

The project began with a series of planning meetings with the TestArmy team. This was followed by an initial analysis of our solution and organisation after which the team did a wide range of technical tests and penetration attempts.

The team also conducted social engineering attacks and phishing attempts. Finally a complete report was delivered highlighting all areas of our solution with vulnerabilities and proposed solutions for each. A guide was also provided on how to manage secure operations.

What was the team composition?

We were assigned a KAM, project manager and team of testers and white hat hackers.

Can you share any outcomes from the project that demonstrate progress or success?

The report showed that we had good security in place and the few vulnerabilities that we had were quickly addressed. The report provided us with continued confidence that our working routines and solution is built on a solid and secure foundation.

How effective was the workflow between your team and theirs?

We had excellent communication and cooperation between our teams. We were impressed by the quality and speed of delivery by TestArmy.

What did you find most impressive about this company?

We were impressed that TestArmy were able to meet and beat the deadline we proposed and could deliver within the agreed upon budget. No other vendor that we contacted were able to commit to our tight schedule, and also showed great understanding when we were the cause of delays. We will definitely work with them again in the future.

Are there any areas for improvement?

None that we can think of. The quality, speed and price were are satisfactory, which is quite unusual in our experience.

Please describe your company and your position there.

I am the owner and software developerof Password Crypt headquartered in Næstved, Denmark. We provide top class password security solutions along with encrypted messaging for company team.

For what projects/services did your company hire TestArmy?

We hired TestArmy to make sure there is no bugs or inconsistency in UI/UX. TestArmy was choosen as the independent party which was responsible to perform UAT testing of our system.

What were your goals for this project?

Our goals were to test a web application - Pcrypt with a browser extension for Chrome and Firefox and make sure that the final user won’t have functional problems to use the whole system.

How did you select this vendor?

They found us and we just wanted to try them out

Describe the project in detail.

The work was focused on exploratory tests of the entire flow of the web application and possible bugs. The usability tests were also conducted to determine the extent to which the application is understandable, easy to use and attractive to users. In the end, we wanted to check the overall design

What was the team composition?

Dedicated tester

Can you share any outcomes from the project that demonstrate progress or success?

TestArmy team discovered overall around 50 errors in our application considering all areas that were taken into account. The report that they have provided to us helped to improve our services.

How effective was the workflow between your team and theirs?

In the project TestArmy used experienced QA manager to lead the testing, which was very helpful. The communication and understanding was excellent. It was as good as it get actually :-)

What did you find most impressive about this company?

Quick understanding and work performed for a very complicated product. They were flexible and performed very well - will use them again for sure.

Are there any areas for improvement?

No

Please describe your company and your position there.

I am the CTO of a boutique software development company that helps SaaS companies overcome their technical challenges and scale their business without performance and reliability issues. We make sure the code quality gets better every month, the app can scale, and that all new features hit the market in a short period of time.

For what projects/services did your company hire TestArmy?

We took over a project that was developed for the last 6 years by many developers. We knew the code is in a bad shape, and we wanted to quickly get rid of all the issues with its scalability, maintainability, reliability and of course - security issues. Our developers did some quick internal tests that confirmed the code has security issues, but we needed to be 100% sure we cover everything.

What were your goals for this project?

We asked TestArmy to perform in-depth penetration tests of the system and create a list of all the vulnerabilities that need to be fixed. We really needed someone, who will be able to find even the smallest issues that could lead to a data breach. Our requirements included also guidance in fixing some bugs, and a second test round to confirm everything is secure after we introduce the fixes.

How did you select this vendor?

We did a research and noticed TestArmy has a skilled team and good case studies to help us with our problem. We also did some research and found out that some companies we know already worked with TestArmy and recommend their services.

Describe the project in detail.

After two weeks we received a very detailed report with all security issues in the app. The report included not only the bugs, but also some explanations, instruction on how to reproduce the issue, detailed description on how to secure a bug and some more metrics.

After that we started to code the fixes and consulted some issues with TestArmy. When all issues were fixed, TestArmy performed a second test round to confirm all issues are now secured.

What was the team composition?

We had contact with a project manager and a cybersecurity expert.

Can you share any outcomes from the project that demonstrate progress or success?

Thanks to TestArmy all security issues were fixed, our customer was extremely happy with the results and this was one of the key reasons we got the project maintenance deal for the next years.

How effective was the workflow between your team and theirs?

They have a very effective team and all questions we had were answered the same day. Each explanation was easy to understand and covered everything needed to do the work on our side.

What did you find most impressive about this company?

I was surprised by how much knowledge the cybersecurity expert had in his area, and how well he communicated all the important information. We were not only able to fix the issues, but we also learned a lot and now we can spot some issues on our own.

Are there any areas for improvement?

No, we really enjoyed working with TestArmy, and we refer them each time someone asks us about penetration testing services.