TestArmy

BACKGROUND

Please describe your company and position.

I am the General Manager of Cypherdog Security Inc.

Describe what your company does in a single sentence.

Cypherdog Security Inc. is a vendor of cybersecurity solution in area of encryption. We encrypt any file or any text and can transfer it using any medium.

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire TestArmy to accomplish?

• Pentests / security tests of our Cypherdog Encryption application
• Obtaining a certificate of the conducted software audit

SOLUTION

How did you find TestArmy?

Existing realtion

Why did you select TestArmy over others?

• High ratings
• Close to my geographic location
• Pricing fit our budget
• Great culture fit
• Good value for cost
• Company values aligned

How many teammates from TestArmy were assigned to this project?

2-5 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

We delivered our application to tests performed by an external team. The purpose of the tests was to test whether the application used for encryption is a secure solution, and most importantly whether 

1. the user's public key can't be swapped
2. the user's private key doesn't leak anywhere

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

Obtain a report on the tests performed with information on their successful passing, plus additional recommendations to ensure maximum security of the solution (non-critical fixes). The tests were performed over several days. We decided to implement several of them and sent the application for retesting, which also took 2 days. After the retests, we received a report and a security certificate for the application.

Describe their project management. Did they deliver items on time? How did they respond to your needs?

Project management was good, and communication was fast.

What was your primary form of communication with TestArmy?

Email or Messaging App

What did you find most impressive or unique about this company?

Professionalism, knowledge, communication at the highest level

Are there any areas for improvement or something TestArmy could have done differently?

Nothing!

BACKGROUND

Please describe your company and your position there.

I'm a part of the Business Development team at a software development agency.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire TestArmy?

We needed to find quality test automation engineers due to all internal resources already being booked.

SOLUTION

How did you select this agency and what were the deciding factors?

Unsure how did we find them, but the deciding factors were communication of their sales team, and the quality / cultural fit of their engineers.

Describe the project in detail and walk through the stages of the project.

They provided two test automation engineers throughout the period of 13 months. One of them stuck with us for the whole period, and he was one of the best external collaborators we've had. The other one was also great, but we had a smaller engagement for him.

Describe the recruitment process in detail.

They shortlisted a couple of candidates for us, and were always pretty transparent regarding what the candidates might be lacking in specific areas. The communication was honest, and the candidates we've picked were great.

RESULTS & FEEDBACK

Can you share any outcomes from the engagement that demonstrate progress or success?

We were able to handle test automation needs on a project that we weren't able to staff internally, and their engineer became a valuable part of the team.

How effective was the workflow between your team and theirs?

Extremely effective - their engineers were responsive and communicative, and so is the sales team.

What did you find most impressive or unique about this company?

They provided great test automation engineers at a reasonable price. Also, most candidates they provided performed well on our interviews, and we have pretty high standards.

Are there any areas for improvement or something they could have done differently?

Not that I can think of currently.

BACKGROUND

Introduce your business and what you do there.

I’m the CEO of Miiskin, a software development company.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with TestArmy?

We needed an IT services provider.

SOLUTION

What was the scope of their involvement?

TestArmy provides and manages manual and automated testing resources with compliance and QA for our web- and app-based software for Android and iOS. Our internal developers finish our software products, and then we send them to TestArmy so they can do quality, security, and functionality testing.

What is the team composition?

We’ve worked with 2–3 members of their team, including software testers.

How did you come to work with TestArmy?

Someone referred them to us.

How much money have you invested in this partnership?

So far, we’ve spent $200,000–$1 million with them.

What is the status of this engagement?

We started working together in June 2018, and the engagement is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

We’ve been through around 150 development cycles with them, and the resources they’ve provided are extremely valuable, skilled, qualified, and fast. They’re great at aligning with our expectations and finding resources to accomplish our needs.

How did TestArmy perform from a project management standpoint?

We have excellent communication, they meet deadlines, and their management team is straightforward. Regarding tools, we use email, phone calls, Slack, and Jira.

What did you find most impressive about them?

The cost for the value they provide is outstanding as well as their efficiency, which is the main reason that we keep on working with them.

Are there any areas they could improve?

I don’t think there’s anything they can improve on — working with them is a perfect collaboration.

BACKGROUND

Please describe your company and your position there.

I'm the CEO of platformOS, empowering development teams to quickly build limitless applications on a reliable “infrastructure agnostic” all-in-one DevOps platform.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire TestArmy?

A significant eCommerce project, similar to Amazon.com

SOLUTION

How did you select this vendor and what were the deciding factors?

Having worked with Polish, Romanian and Hungarian teams in the past, we knew we wanted to work with a team in this region. After reviewing a number of QA augmentation services, we felt that TestArmy had the best approach, excellent customer service and client management attitude and a well trained pool of staff.

Describe the project in detail and walk through the stages of the project.

The eCommerce Marketplace is in English and Arabic and required very specific regional API integrations for Payment, Shipping, Loyalty point, SSO etc. and needed to comply with very stringent Bank Grade security. Our TestArmy teammate was included in the TDD tasks, along with API testing and general user bug identification and bug submission. Given the critical nature of performance, flexibility and security - we needed a very well rounded QA who could ensure great test coverage across the code base.

How many resources from the vendor's team worked with you, and what were their positions?

We had one direct mid/senior QA who worked directly with the dev team and PM.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

The project has very high test coverage and very few bugs that impacted the end customer experience. The client continues to develop with their own dev team and is very happy with it being one of it's most successful web based projects.

How effective was the workflow between your team and theirs?

It was like they were part of the team. The workflow was seamless.

What did you find most impressive or unique about this company?

Both the management and QA staff have been very enjoyable to work with. The high quality of the test work and the excellent Account Management by the TestArmy executives have been above expectations.

Are there any areas for improvement or something they could have done differently?

Not that I can think of.

BACKGROUND

Introduce your business and what you do there.

I'm the senior QA lead at LiveLink Technology Ltd, and we're a photo tech company based in the south of England. We have bug chains as clients in the UK. We essentially design and maintain all the tech side of their photo industries: creating photo books and calendars.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with TestArmy?

We've been trying to automate many of our regression tests, and we had a huge gap. We knew that we wouldn’t be able to get those tests automated in a decent time frame, and needed assistance. 

We wanted to hire a company that could point us in the right direction and also help us get to where we needed to be in a much faster time.

SOLUTION

What was the scope of their involvement?

We had already discovered that a certain part of our tests that checks and creates calendars and cards was the key area we wanted to focus on. 

TestArmy spent a bit of time looking at what we already had and getting up to speed because there were suggestions about changing the entire framework, but we knew we didn't want to take such a drastic move. Once they got to grips with how we did things, it was just a matter of them approaching how to translate those manual tests into automated ones.

We have Ruby, RSpec, Selenium, and then for image comparison, we also used RMagick, or the shorter library, MiniMagick, which was a key tool in automating the tests.

What is the team composition?

We had been liaising with three people from their team. One of them was the sales manager, and he ensured that they were providing us with what we thought we needed. In the first month, another guy looked at the entire process, not just tests, to see if there were any gaps where we could do some of our trials even sooner. It included a review of our current framework, but that was only for a month. Once we had a report from them, it was just Jakub (Test Automation Engineer) who assisted us with all of the testing and automation for the following seven or eight months.

How did you come to work with TestArmy?

We met them through a referral. Our old head of technology, who has subsequently left LiveLink, had worked with them in the past and highly rated them. When I said that we needed some help with a company like this, it was safe to say that we would use them because he had a working relationship with them in the past.

How much have you invested in them?

We spent €50,000–€60,000 (approximately $55,400–$66,400 USD) with them.

What is the status of this engagement?

We worked together from April 2021–February 2022.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

At first, we tracked how many test cases they automated in a year, how many test cases they were automating each week, and then the total number of test cases automated. They were consistently automating test cases, and then I asked Jakub to assist me with setting up our test suite to run in parallel. Once they finished that work, he went back to automating all the test cases. Overall, they automated 134 test cases in total. The main section that we wanted them to combat was the prism builder, and in my eyes, they certainly achieved that by automating 80 test cases in this area alone.

The other key point is that we got to the stage of certain test packs becoming fully automated, which, in my opinion, probably couldn’t have been done without TestArmy’s assistance in the timeframe. This is the key point where they automated so many prism tests.

The guy who did the bulk of the work had to learn the technology we had, including the language and the frameworks. I guess I thought that most of the people who worked there would already know a lot of frameworks, not that ours is an obscure one, but he was able to pick it up quickly, and there were no issues with that.

In my personal opinion, I think they met the expectations, and I'm very happy with how they worked.

How did TestArmy perform from a project management standpoint?

I managed Jakub on a day-to-day basis, and it was absolutely fine. We used to have a weekly chat, and I would give him the tasks to do for that week, and he would pretty much always come back to me with the jobs done. If there were issues, communication was great, and he would clearly explain why it took longer than expected.

Apart from the day-to-day management, their sales director handled the direction of the whole project, and he was very good at the start in making sure that we had what we wanted. He gave us plenty of options with plenty of different costs, and there was no attempt to upsell, which I hate. It was good that he just listened to our needs and provided us with what suited us. When it came to figuring out when we should stop, that was also very easy to handle from both sides — we were happy.

We used a little bit of Kanban and communicated mainly through Slack and Google Meets.

What did you find most impressive about them?

For me, it would be the vast array of things they offer. I have had conversations with them about other things that we might get them back to. It seems like there is a lot of expertise in various areas. Because it worked so well this time around, when we need someone again, I will go straight to TestArmy, and I am sure they will have what we need. I think their versatility stood out to me, and I guess their whole demeanor of how they conducted themselves. Communication was also great.

Are there any areas they could improve?

The only thing was that they were quite slow to get going, but that could have been just the way we set it up. In the first month, we had someone come and review the entire process because they said it was probably the right thing to do. I liked the document they provided right at the end of that first month, but we then lost a month with them doing something they started to do the month after that. In hindsight, I might have joined those two together and would have said to start doing the automation work straight away, but they could also do a month of reviewing at the same time by someone else rather than having them sequential.

Do you have any advice for potential customers?

Challenge them. I think Jakub hadn’t done some of the stuff we had been working on, and he enjoyed it, and from what I hear, it gave a lot more engagement to him. Pick TestArmy because they are good, and I highly recommend them and challenge them. They have a wealth of experts. If we suggested something and didn’t know how to do it, they have a vast array of people they went and spoke to about how we could approach something. Their team has a wide variety of this knowledge and experience. What you think can’t be done can be if you combine all those minds.

BACKGROUND

Please describe your company and your position there.

My name is Daniel Jędrzejczyk. I work at Huuuge Games as a CRM Operations Manager. Huuuge Games is a company with a great and long history of creating games.

Nowadays our main focus is to develop social mobile games that will encourage people to play together in a real-time experience on a global scale.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire TestArmy?

We hired TestArmy because we needed specialists in functional testing to test games for us.

What were your goals for this project?

Our main goal for this project was to find someone qualified and dedicated to test games for us, and check their functionality and GUI every single day, without a break.

We needed experienced testers to perform tests for two of our games every day (even on national holidays, New Year’s Eve etc.), in order to check if the offers within the game change according to our scenario.

We were looking for fully dedicated support from the company that we have chosen.

SOLUTION

How did you select TestArmy?

We have found TestArmy because of our research. We have contacted Szymon Chruścicki - TestArmy’s Business Development Manager, who later introduced us to Jarosław Rafa and Lidia Rafa, who were performing tests for us every day without a break.

Describe the project in detail.

TestArmy provided us with the highest quality of testing, while giving us one of the most experienced testers in their team.

Everyday, since 2017, one of TestArmy testers performs GUI and functionality tests for two of our games - Huuuge Casino and Billionaire Casino, to check the in-game performance and offers if they are adapted to the level of advancement of the player.

TestArmy tests the games according to our varying scenario everyday, because of seasonal changes of offers.

What was the team composition?

Our initial contact was with Szymon Chruścicki, but later on we have been introduced to Jarosław Rafa and Lidia Rafa, and we are working with them since then.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

The biggest impact is that we are 100% sure and confident that our games live up to the expectations for players. We can also be safe with our in-game offers being perfectly adapted to the level of advancement for the players, which is obviously great for our business and selling plans.

How effective was the workflow between your team and theirs?

The biggest pro is that Jarosław Rafa and Lidia Rafa are very experienced testers, and the greatest thing is that Jarosław is also a Project Manager in TestArmy. We were sure from the start that nothing could go wrong with this project.

What did you find most impressive about this company?

The most impressive thing was that even though our testing hours are quite demanding, TestArmy had no issue with that and met our expectations perfectly.

Are there any areas for improvement?

As of right now, I am happy with our cooperation.

BACKGROUND

IPC Europe is an independent non-profit making organisation owned by Subway® Franchisees in Europe. Whom manage supply of Subway® approved food, packaging and equipment, but also offer a wide range of support of commercial services and technology solutions, such as Subway® Loyalty Program & App.

OPPORTUNITY / CHALLENGE

After supporting a 10-year-old existing Loyalty App and Program, strategically the business had agreed to transform the current Mobile App, Website and program in the current European markets in terms of look & feel, as it was seen as a key customer and business initiative in terms of engagement and to better compete in the QSR/Retail space. This meant having to deliver an MVP product without losing any of the key existing features, but ultimately the focus was to produce a complete new UX and design.

The scale of the project was for 7 key regions where the program was existing but also to ensure futureproofing the product for longer term strategy initiatives. Knowing the roll out plan would span a minimum of 6 months would have to take an iterative approach and focus on one market at a time. Overall the project was a re-skin of branding but also improving the UX for end User across a large scale customer base.

SOLUTION

We at IPC Europe had worked with TestArmy previously on significant rollout projects for 3 markets and the updates of the platforms to support GDPR. We had an existing relationship which was important now undertaking the largest project in terms of scale and complexity the team had most probably undertaken for 10 years.

From a business perspective one of real importance for the range of stakeholders and for the Franchisees and Stores. Ensuring a quality end product for all the European consumers and market. The scale and magnitude of the task looking back was underestimated considering the langue, complexity, and impacted business teams.

RESULTS & FEEDBACK

We successfully rolled out the Mobile App and platforms in a timely manner with very positive consumer and business feedback. The way in which TestArmy works and integrates into our internal project is what stands out. The daily and weekly engagement with the business project calls, daily IM communication with Slack and being able to directly work in the scrum team with bug raising.

This level of team work along with the ability to add value with feedback through UAT of the platforms is a huge positive. We encountered multiple delays, issues with the designs and very last minute changes which meant the scheduled resources hand to be stood down at last minute this flexibility and support is invaluable.

We carry out internal testing, however having a added range of devices to validate test feedback but also level of assurance we receive from a group of experts Testers gives the senior stakeholders in the business and added level of confidence which is crucial when we consider the quality assurance for the end result. The only challenge at times is the desired framework or layout of feedback within our ticketing tool for the development teams.

BACKGROUND

Please describe your company and your position there.

I am the founder of ZING which is a new video call platform that focuses on social interaction. Our platform allows people to move around fluidly between different groups - rather than being stuck talking to each other - and also has in-built games to make calls more fun.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire TestArmy?

We needed to test the functionality of our video-call platform. One of our developers created critical scenarios, which we later passed on to TestArmy’s testers so they could test the product with those scenarios.

What were your goals for this project?

Our main goal for this project was to find as many critical bugs as possible, and check how the platform would perform under certain conditions.

SOLUTION

How did you select TestArmy?

Actually, TestArmy was recommended to us by Visuality, the company doing the development for us. This definitely sped up the process of deciding who we should choose for this project.

Describe the project in detail.

Three testers from TestArmy did as much functional testing over a 2-day period as they could across a range of devices. We provided scenarios, but they were also encouraged to do exploratory testing. They found a few very critical bugs that have helped us improve the performance of our platform as well as recommending smaller improvements

What was the team composition?

During this project, we were in touch with the Project Manager, Sales Representative, and 3 software testers

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

The testers found key bugs that we had not discovered, which were critical for us to resolve prior to launch. After the initial testing, we have re-engaged with TestArmy to perform some ongoing testing for every release.

How effective was the workflow between your team and theirs?

This was great - we added the testers to our slack channels, gave them access to our Jira and they were able to add bugs directly into the workflow in a logical way.

What did you find most impressive about this company?

The level of detail in their testing was very impressive. They also integrated very smoothly into our development process.

Are there any areas for improvement?

None - they are flexible and friendly and we are continuing to work with them

BACKGROUND

Please describe your company and your position there.

I am a Team Leader of QA at 10Clouds. 10Clouds is a team of 100+ experienced developers and designers based in Poland, ready to help you build your web and mobile applications. Since 2009 we have built and designed software for more than 90 businesses worldwide, mainly in the United States, Western Europe, and Australia.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire TestArmy?

Various: Mainly web testing, mobile testing, test automation

What were your goals for this project?

Testers were involved in each project. They were responsible for the verification and validation of given applications. In some cases this was a specific client project, while in others it was longer term cooperation with a client through staff augmentation.

SOLUTION

How did you select TestArmy?

We selected them based on positive feedback online and recommendations.

Describe the project in detail.

This is a long term, established relationship rather than a single project. We’ve worked together on a number of client projects ranging in size and scope.

What was the team composition?

Various. Per one project we usually had a tester, a Project Manager and three developers.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

We’ve received positive feedback from numerous clients on whose projects we’ve worked together with TestArmy. One of our key clients gave positive feedback about TestArmy’s work specifically and requested to continue cooperation with them.

How effective was the workflow between your team and theirs?

We have very similar styles of working, which enables fast onboarding of TestArmy staff and enables us to get started on jobs quickly. Cooperation and workflow have both been very good.

What did you find most impressive about this company?

We have been most impressed by TestArmy’s flexibility and ability to quickly adjust to developments within projects.

Are there any areas for improvement?

My only comment would be that it might be good to have a few more senior testers.

BACKGROUND

Please describe your company and your position there.

I am running Droids On Roids, a full-service iOS, Android and Web Development Company from Poland, operating since 2011. We partner with the best start-ups and great brands to build high-performing mobile & web apps. Companies that love the quality of our code are Whatsapp, Facebook, Disney, Electric Objects and Giphy, just to name a few.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire TestArmy?

We needed additional support in our QA processes for ongoing projects. We seek to collaborate with top-level specialists on individual projects rather than hiring full-time employees. TestArmy job was to develop and execute exploratory, acceptance and integration tests to ensure the quality of our digital products.

What were your goals for this project?

We wanted to check our digital products (web and mobile apps) on various parameters like applicability, functionality, security and scalability. We wanted to be sure that we've met user requirements and we found and fixed all bugs before releasing our software.

SOLUTION

How did you select TestArmy?

We were looking for a supplier who has qualified technical resources on board with strong experience in delivering projects successfully and working as a part of a global team.

Choosing a trusted QA partner to cooperate with wasn’t easy. We have contacted many firms but TestArmy was supposed to have the biggest experience in this field. Looking back it was the right decision. We are still cooperating closely with them.

Describe the project in detail.

So far the team of TestArmy has taken part in our 10 projects with durations from 2 weeks to 6 months. They have verified the quality of applications such as the medical app for people who wear corrective orthoses, the video-on-demand service, and the app connected to the IoT devices. They reviewed quality specifications & requirements and created well-structured test plans and test cases.

What was the team composition?

There were five to ten QA Engineers on their team that worked on our various projects monthly. We also had contact with the Project Manager and the Sales Manager.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

The QA Engineers helped us ensure high quality in the earliest stages of developing a feature. Every time they provided timely and meaningful feedback. What is more, they bring the know-how to the company, so the tools and IT environment including processes can still be used locally.

How effective was the workflow between your team and theirs?

We have daily updates and weekly progress reports. Usually, we communicate over Slack as well as using Jira to manage tasks. Although we work remotely TestArmy team became part of our team and got on very well with developers. They don’t hesitate to ask the questions. The PM’s communicate about the scope and budget and provide reports and invoices.

What did you find most impressive about this company?

The most impressive was the level of engagement. The QA team is detail-oriented with a very proactive approach of what can be done better both in software development and business areas.

Besides that we know that whenever we need extra QA support we can reach out to them and they will always find an experienced Tester for our project. TestArmy is always flexible when it comes to the schedule and level of engagement.

Are there any areas for improvement?

No, the cooperation was just fine.