QATestLab

BACKGROUND

I am the CMO & QA Tester of Loom Studio

We're an indie videogame developer studio. 

OPPORTUNITY / CHALLENGE

• Find bugs
• Test level desing
• Test feeling & general expirience of the game

SOLUTION

In present event

• Great culture fit
• Company values aligned

2-5 Employees

What we did with them was provide a version of our game that included many new features and significant changes compared to the previous version. The goal was for them to test everything and make sure everything was working properly, identify any minor bugs or details that needed to be fixed, and, on the other hand, give us feedback on the new mechanics we had implemented in the game.

RESULTS & FEEDBACK

The document they provided us with allowed us to address all the changes and/or provide solutions to the issues they identified. After implementing their feedback, we ended up with a much more solid and complete version of our game.

They delivered everything within the agreed-upon deadlines, communication was smooth and consistent throughout, and their approach was both professional and friendly.

• In-Person Meeting
• Virtual Meeting
• Email or Messaging App

We cannot really evaluate this aspect, as this has been our first experience working with an external company in this field.

However, we can say that we are very satisfied with the overall experience.

Not really. The work that they do is all what you spect. And all you have in mind. 

BACKGROUND

Please describe your company and position.

 I am the CTO of a software development company 

Describe what your company does in a single sentence.

Provide Modeling tool for Testing. Model based testing and test automation.

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QATestLab to accomplish?

• We have reached QATestLab with a need to test our latest project, a tool for compiling complete automated test suites.

SOLUTION

How did you find QATestLab?

Referral

Why did you select QATestLab over others?

• Pricing fits our budget
• Good value for cost
• Referred to me
• Quick support on demand

How many teammates from QATestLab were assigned to this project?

2-5 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

We hired an independent QA team to get honest, professional feedback on our tool's usability and check if functionality for test automation works as required. Also, we needed to examine our product before the upcoming launch, so QATestLab prepared all the documents and the NDA so that we could start in a few days. That helped us test the product quickly with proper quality.

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

First, QATestLab detected some critical bugs that could spoil the release. They involved QA Automation engineers to get honest feedback not only about the functionality of a product but also about its user experience, navigation, and overall accessibility of a tool, as it had to be easy to understand and work with. After the launch, we got many positive reviews from our customers about their satisfactory experience using our testing tool.

Describe their project management. Did they deliver items on time? How did they respond to your needs?

The deadline was really tough, but they still performed the high-quality testing on time.

What was your primary form of communication with QATestLab?

• Virtual Meeting
• Email or Messaging App

What did you find most impressive or unique about this company?

We were impressed by their productivity and fast reaction. Their managers quickly reacted to our request and provided a fast start so we could begin testing immediately. Also, we marked their professionalism as their detailed and objective feedback about our product showed their true expertise in manual testing and test automation.

Are there any areas for improvement or something QATestLab could have done differently?

We haven't noticed any points for improvement. QATestLab delivered high-quality services and exceeded our expectations.

BACKGROUND

Please describe your company and position.

I am the COO of Lvivsoft

Describe what your company does in a single sentence.

DevShop - delivering software development services

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QATestLab to accomplish?
We decided to use the services of QATestLab to test the software that we were developing for our client.

SOLUTION

How did you find QATestLab?

Online Search

Why did you select QATestLab over others?

• Close to my geographic location
• Pricing fit our budget
• Good value for cost

How many teammates from QATestLab were assigned to this project?

2-5 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

Prior to delivering a mobile app to our client and since we didn't have enough internal resources to cover QA tasks for this product, we faced the need of getting an external expertise. QATestLab provided high-quality end-to-end testing for this app and supplemented all the necessary test documentation, including bug reports, test cases, test plan and strategy, which made the testing process very well-structured.

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

After QATestlab’s team conducted the necessary tests of the app, we noticed a significant reduction in the app crash rate, indicating improved stability. In addition, there was only a tiny number of post-release patches and hotfixes unlike the other projects, which showed us that issues were effectively identified and resolved during testing. After the release, the app gained multiple positive reviews and earned a high rating on Play Market and App Store.

Describe their project management. Did they deliver items on time? How did they respond to your needs?

The project management worked smoothly, there were no delays in delivery, moreover they were keeping us updated on the progress on daily basis.

What was your primary form of communication with QATestLab?

• Virtual Meeting
• Email or Messaging App

What did you find most impressive or unique about this company?

We were impressed by the level of responsibility of QATestLab’s team and the quality of their work. Not only did they provide comprehensive and high-quality testing for our client’s product, but they also met our deadlines by adjusting to our timeline, so we experienced no delays despite using their external assistance.

Are there any areas for improvement or something QATestLab could have done differently?

Again, we were completely satisfied with their services.

BACKGROUND

Please describe your company and position.

I am the Marketing manager of HostPro

Describe what your company does in a single sentence.

Hostpro company provides hosting services, domain sales and SSL certificates.

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QATestLab to accomplish?

• We hired their team to verify our website, including testing it in different browsers.

SOLUTION

How did you find QATestLab?

Online Search

Why did you select QATestLab over others?

• Pricing fit our budget
• Good value for cost
• Great experience in web testing

How many teammates from QATestLab were assigned to this project?

2-5 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

The team of QATestLab conducted functional, regression and compatibility testing for our website, its billing system, and the functionality of a user’s personal cabinet. They covered the whole functionality of our website with tests and also ensured that our website operated seamlessly across several browsers (Chrome, Firefox, Safari, and Edge). They supplied us with all the necessary test documentation, which made it much easier to keep track of the testing process.

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

We wanted to hire a professional team that would be able to organize their day-to-day activities by themselves and use their experience to detect how our testing process can be improved. The team was very organized and brought timely results every sprint. Their testers also detected several compatibility issues with how our website worked in some of the browsers, which allowed us to fix them and provide a better user experience for people using them to access our website.

Describe their project management. Did they deliver items on time? How did they respond to your needs?

Our primary touchpoint was the QA Project Manager from QATestLab. Through virtual meetings and email communication, he oversaw the testing efforts of their team and steered the day-to-day activities of the testers. All tests were timely executed, and the outcomes were systematically described in their reports in every detail. 

Their management was particularly prompt in their replies via emails and in chats and adapted quickly to any evolving requirements or questions we had. The communication was transparent and effective and there were no unexpected moments in terms of payment as the budgeting aspect was managed professionally.

What was your primary form of communication with QATestLab?

• Virtual Meeting
• Email or Messaging App

What did you find most impressive or unique about this company?

Our team was impressed with the quality of work, level of professionalism, and responsibility of their testers. The communication was smooth with the whole team and their work ethic is great.

Are there any areas for improvement or something QATestLab could have done differently?

We didn't experience any shortcomings.

BACKGROUND

Please describe your company and position.

I am the Business Development Director of Pragmatic DLT

Describe what your company does in a single sentence.

We are the top AI and Blockchain software development provider.

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire QATestLab to accomplish?

• We hired the team of QATestLab to perform all the fundamental types of testing for our blockchain software.

SOLUTION

How did you find QATestLab?

• Referral
• IT conference

Why did you select QATestLab over others?

• Pricing fit our budget
• Good value for cost
• Company values aligned
• Quick support on demand

How many teammates from QATestLab were assigned to this project?

2-5 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

The team of QATestLab performed functional, regression, UI/UX, localization, compatibility, and security testing for our blockchain software. They also supplemented detailed and easy-to-read test documentation and provided us with timely and comprehensive test summary reports.

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

We needed QA support as soon as possible due to tight deadlines for our team. Specialists from QATestLab were able to join our project exactly when we needed and accelerated our development process with their effort, so we managed to deliver our product to the client on time. The client was satisfied with the quality of our blockchain application and left a great review about our team.

Describe their project management. Did they deliver items on time? How did they respond to your needs?

We were in constant communication with the QA Project Manager from the team of QATestLab via online meetings. Even though he wasn’t a part of our team, he monitored the progress of the testing activities and supervised the day-to-day operations of the testers. The tests were performed on time and the results were thoroughly described in the reports. In case of any questions, their management reacted quickly and was ready to adjust to our needs.

What was your primary form of communication with QATestLab?

Virtual Meeting

What did you find most impressive or unique about this company?

Their testers were able to join our project in just a couple of days and adjusted to our workflow rather quickly, so there wasn’t a long adaptation process. We discussed the budget, timeline and deliverables beforehand, therefore there were no surprises. 

It is a great advantage that with their team, you are able to get the necessary support only for the needed period of time instead of involving additional resources for longer than required. We will definitely involve them in our future projects.

Are there any areas for improvement or something QATestLab could have done differently?

We haven’t noticed any issues during our collaboration with QATestLab.

BACKGROUND

Introduce your business and what you do there.

I’m an executive of a software development company. We also provide consulting services.

OPPORTUNITY / CHALLENGE

What challenges were you trying to address with QATestLab?

We had a client that requested software factory services, but everybody in the company with those skills was busy. We needed someone to provide support by assigning some of their resources to our project.

SOLUTION

What was the scope of their involvement?

QATestLab sent us a couple of CVs. We interviewed two people and liked one of their profiles. They started working for us, and the resources they provided focused on QA testing.

What was the team composition? 

We worked with three QA developers.

How did you come to work with QATestLab?

I found them on Upwork.

How much have you invested in them?

We spent about $100,000 in total.

What is the status of this engagement? 

We worked with them from October 2020–November 2022.

RESULTS & FEEDBACK

How did your relationship with the vendor evolve?

The developers did a good job; the project manager in charge gave good feedback about them. However, we didn’t have a good experience with the management. We worked together for about two years; we gave them a notice when the project ended, and they sent us the last invoice for the work they did in November.

How did QATestLab address the challenges that arose?

At some point, we sent them a laptop, and by the time the project finished, we asked for the computer back, and nobody replied. They also deleted all the chat history between us, so I had to contact a lawyer to get them to answer.

Describe the impact this engagement has had on your business.

We decided not to pay until we had the laptop back, but they ignored it and just asked us to pay. They even talked about suing us. In the end, we paid them but never got the laptop back. We ended on terrible terms.

How was project management handled?

We didn’t have issues with the budget, but we weren’t completely satisfied with their communications. It was fine when we contacted the developers but not when we dealt with the management team. We communicated through a group chat they created.

Is there anything that the vendor did well or that you would consider a strength?

I honestly don’t have anything to say in that regard. We won’t be using them again.

In what specific areas can they improve? 

We felt cheated by them; they could’ve behaved differently.

What advice do you have for clients with similar needs to yours?

Make sure to follow email communications. We made some mistakes because we always talked through Skype and conference calls, so there were things that we didn’t discuss with them via email. It was hard to find screenshots or emails to prove that we sent them the computer.

Company Responses

The vendor responded on May 1, 2023.

Hello. Thank you for your review. QATestLab was happy to have you as a client for more than two years and we are grateful for your high appraisal of our services. We were glad to collaborate with you on several projects and help you maintain the great quality of your software. Given your positive feedback over the time of our collaboration and your satisfaction with our services stated in your review, we frankly don’t see how the overall estimate of 1 can be justified.

In your review, you also claimed that we withheld a laptop you had sent us. We were surprised to hear about this issue more than 6 months after finishing this particular project and continuing working on another one with you. Furthermore, after the end of the project in question, we asked for your feedback about the services we provided, and there was no mention of the missing device.

Unfortunately, you haven’t provided us with any documents proving that the laptop is in our possession. Our team tried to settle this issue with your company, but we still haven’t received any information documenting that QATestLab has ever received the device. What we did receive instead were your claims of unwillingness to pay for the services we provided on another project of yours, which contradicts our contract. Therefore, this review doesn’t reflect the true situation.

We always were and still remain open to negotiations on this issue, so if you are ready to provide us with any documents indicating that we have the device in our possession, we will be more than happy to assist you here.

BACKGROUND

Please describe your company and your position there.

Product manager

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire QATestLab?

Application testing for our major releases

What were your goals for this project?

To find nonobvious issues in the software and to get an independent appraisal of our software quality.

SOLUTION

How did you select this vendor?

I found them via Upwork

Describe the project in detail.

The project involved the acceptance, regression and smoke test development of a major consuming-facing website called izi.travel

What was the team composition?

I had one fixed person in the team. Occasionally we would add people or pause depending on the situation.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

We had a reliable process whereby we could release every three weeks. QA testlabs made a strong contribution to this.

How effective was the workflow between your team and theirs?

I would have a daily catch up with the people from QA testlab. In addition we would communicate on issues through Jira.

What did you find most impressive about this company?

The flexibility in up- and downscaling whenever necessary, their attention to detail and thorough reporting were very impressive. 

Are there any areas for improvement?

It would be helpful if this Vendor would apply client location jurisdiction (Netherlands) instead of vendor location jurisdiction (Ukraine). But I understand this will not be easy working for many international clients.

BACKGROUND

Please describe your organization.

Our company, LifeWorks, is a mobile accessories company. We design and manufacture mobile accessories, mostly protective phone cases, cables, and add-ons to smartphones and other digital devices, and we recently entered the app business. Right now, we are creating an activity-tracking product, similar to a Fitbit or a Jawbone. That's the team that I lead, and that's what QATestLab helps us with – quality assurance testing for the app.

What is your position?

I lead the activity tracking department.

OPPORTUNITY / CHALLENGE

What business challenge were you trying to address with QATestLab?

They do QA [quality assurance] testing for a number of the apps that we've built here. We've got six to eight apps across different platforms – Android and iOS. They have to test them on a daily basis, on a number of different devices every time that we have a build, and they are to log the bugs and propose fixes.

They not only do that, but they are also asked to provide all the details and steps to reproduce the bugs, as well as attach videos of them actually testing it. If the bug is reproduced, we have them capture the screen so that our developers can best see what happened. They are to do that, and they're to work within JIRA agile. They're also to work with our developers if they ever need to collaborate on help reproducing issues.

SOLUTION

Please describe the scope of their involvement in more detail.

They have helped us in other situations. Once the app goes live, we have to do ad-hoc testing, which is different than just working on builds that have yet to go live. I also had them at times monitor the App Store for any crashes that Google Play or iTunes might pick up in their reports. I also have them monitor whether or not our customers leave troubling reviews that might have something to do with faulty development.

There are also times when something is very urgent; when we have some sort of build created for a meeting, a pitch-build, as I call it, and we need urgent testing right away. They're actually a very scalable team. If I need a tester added to the project on demand, Yuliya can scale that for me and add that to our project pretty much on-demand. They have definitely advanced from the initial scope, which was just testing. I mean they still pretty much test, but they have advanced their efforts.

We have builds every day, and they test every day. In our sprints, we have data releases, which is at the end of a sprint and I call "team-release." It goes to my team, and then we have the actual product release. Before that, we have alpha builds, and alpha builds come every single day. They are literally working and testing for us every single day.

As soon as they get notified that a build is released – it could be three hours into the workday of our dev team or it could be at the end of the day – they get started on testing. There are times in which something might be a little confusing, and a developer might be testing immediately. Yes, there are times where a developer will message one of the testers, and they'll work hand in hand, testing and providing immediate feedback as opposed to at the end of the day.

They join us in Slack and on JIRA, so they're very much integrated seamlessly in our team. There are conversations that would go on for a few hours between a developer and one of their testers, and the tester continuously helps the developer – or myself, at times – figure out the problems. There are times when it's more of a static relationship, for example, when you need something specific done and you send the request and they just send it back when it's done. There are also times when it's very dynamic and back and forth. During those times, they're really able to collaborate and determine what steps should be taken before going to doing them. 

A few times, they've helped actually spot errors with our server. There were a couple of situations where we were having a problem, and they did look into the code. I don't know if they have any experts in any particular language. They don't do my code reviews or anything like that, but there were a couple of times when they were able to find out a couple of errors, punctual things like a period in the wrong spot, or a break or something like that in the code; they were able to identify it for us.

How did you come to work with QATestLab?

I found them through oDesk or Elance. My vetting process is stringent, actually. I go through numerous interviews, and I give them a test with particular things that they have to meet, so I can assess their skills. They performed well on the test, so I decided to go with them.

Could you provide a sense of the size of this initiative in financial terms?

It was probably $10,000 to $20,000. There were times when we had about eight projects ongoing. I also had them at times work overtime, on weekends and used multiple testers at once.

What is the status of this engagement?

The relationship is ongoing.

RESULTS & FEEDBACK

Could you share any statistics, benchmarks or user feedback from this engagement?

The way they do it is they'll have each person test it. Let's say they need two testers per build. Each tester will typically spend at least three or four hours, sometimes going up to eight. They definitely spend a lot of hours. I also have them be very specific in the way they log their work, like having to capture video. They could spend eight hours and discover 12 bugs, but they'll also test and resolve a number of other bugs.

I can't say that they're the most efficient, but they are incredibly thorough, which is important when you're working with a team that's spread out throughout the world. I don't have any benchmarks that I hope they hit. I don't tell them they have to work a certain number of hours because then you get people trying to create issues, like what happens if you tap the screen with your entire palm 20 times in a row. Things like that just to produce a bug. I don't have any benchmarks like that, but they definitely do their work, and do it well.

How did QATestLab perform from a project management standpoint?

Yuliya is the one who runs my group of testers. She is excellent, and they're all excellent. I would really give them my highest recommendation. The team is awesome. In regards to project management, she handles it very well. She scales effectively, and she makes sure that they're good communicators. The scaling part is very difficult to do as you're scaling personnel. She pretty much does it on demand, and that's great.

The people at QATestLab are all really good communicators. I was a bug tester myself for almost five years, so I know that it's not the easiest thing in the world, and they're good and they really enjoy it. I also have an interesting product; they like to go out for walks with the activity tracker. Project management is an A+.

What distinguishes QATestLab from other providers?

The things that just make it enjoyable to work with someone. They have a great personality, they're pretty much always available, and they get back to you very quickly. Anyone who joins the team is always nice and professional. They don't fight back often. When we first started working with them, they didn't use JIRA. I think they were using a popular one that I hate – it's really messy. I forget the name, but it's a really popular bug-tracking tool. Anyway, they were using that, which is so complicated.

I wanted to move to JIRA, and they did it. I mean, I paid them for the hours, but they didn't complain, and they moved everything to JIRA, which is an incredibly tedious task. I can put a lot of strain on them, and they're always very flexible. They have good ideas. Yuliya really strives for her team to be the best, and also Yuliya was a tester herself. It's not like you just have a project manager jumping in just because she's good at communicating and speaking English. They're like a team of testers run by testers, which I think goes a long way.

Is there anything QATestLab could have improved or done differently?

The only thing I sometimes have to reprimand Yuliya on is sometimes her bug testers will log ridiculous bugs. Like, if you tap on the screen 30 times in 20 seconds, the app crashes, but the apps are not meant to work like that. I tell her to stop doing things like that because it's a waste of time, but it doesn't happen that often.

Every now and then, I'll get a silly bug like that, but that's alright. You want testers that are trained to do stuff like that, but the average person doesn't think like that. It's good that they have that mindset. That's one of the areas where she could improve, but that doesn't need to be on her list of priorities because it has no detrimental effect.

When we first started working together, they spending time testing on a lot of older devices. I didn't monitor that much initially, but now I do. I let them know what devices and what combinations to test on; otherwise, they'll test everything under the sun.

BACKGROUND

Please describe your organization.

My company is One Touch EMR. We sell a medical records software solution.

What is your position?

I am the founder and CEO.

OPPORTUNITY / CHALLENGE

What business challenge were you trying to address with QATestLab?

We needed software usability testing for software that we had built in-house. Our medical records solution had a lot of moving parts and we had to make sure it was reliable and scalable. We developed the software in-house and wanted an outside opinion on the stability of the final solution.

SOLUTION

Please describe the scope of their involvement in detail.

We have one person working on our QA [quality assurance] testing. It's been primarily usability and performance testing.

How did you come to work with QATestLab?

I did some online research. They were the only ones we approached for this project. The key drivers for my selection were cost and competence.

What is the status of this engagement?

We started working with them a couple years ago. Our relationship is long term, and I'm still using their services.

RESULTS & FEEDBACK

Could you share any statistics or metrics from this engagement?

I don't have anything tangible. We just use them for QA. Once they're finished, we release, and they seem to do a good job. We haven't had any issues with defective code or a significant amount of bugs.

How did QATestLab perform from a project management standpoint?

QATestLab has been very good and very easy to work with. I chose to use Assembla as a project management tool, and they were more than willing to use it with me. I appreciated that degree of flexibility.

What distinguishes QATestLab from other providers?

I'm not sure because I haven't used many others.

Is there anything QATestLab could have improved or done differently?

I can't think of any.

BACKGROUND

Please describe your organization.

Creative Technology Partners is a software solutions company specializing in responsive web development, mobile development and applications.

What is your position?

I am the senior producer or senior project manager.

OPPORTUNITY / CHALLENGE

What business challenge were you trying to address with QATestLab?

Since we specialize in responsive websites, and we were the first people on the market who started doing adaptive and responsive development. This created the challenge of finding people who would be able to verify the functionality of every single page – of every single feature of the website – in all possible resolutions with multiple browsers.

SOLUTION

Please describe the scope of their involvement in detail.

QATestLab worked with hundreds of devices verifying that the work we did was done right for those particular products.

We also had one of their specialists develop a product. We have also used the auto testing services from QATestLab, and that was great. Our customers were happy, and we were happy. We would request that they write some scripts for internal products or do automated tests.

Could you provide a sense of the size of this initiative in financial terms?

I would rather not disclose actual figures. It depends on the qualification of the QA [quality assurance] member that we work with.

What is the status of this engagement?

We first contracted them about five or six years ago [2009/2010], and we're still working with them. The most recent project, a website, will be finished in a week [August 2015]. Every month, we release something, or every two months at most.

RESULTS & FEEDBACK

Could you share any statistics or metrics from this engagement?

It takes less than a day for them to get onboard for a new project. I would provide all the information, and they usually start testing immediately. By the end of the same day, they would have around 10 or 20 bugs already entered into JIRA. You see them right away, and that's great.

It's better if there are less bugs entered, but the numbers are usually quite high because that's their responsive development. Their developers would first start with three or four basic device resolutions, and then QA would go over and specify problems for all other resolutions, interim sizes, and the browsers that they use.

Each QA specialist can work on two or three projects simultaneously. As soon as a developer completes fixing a bug, he informs the QA member. We check this all in JIRA so it's easier to understand. If the status for the task is changed, the QA will get to it in the next 10 to 20 minutes.

How did QATestLab perform from a project management standpoint?

I usually work directly with not more than three people from QATestLab, and there's always a manager that can help us with any organization questions or anything related. If I have a new project, or I'm not sure who to turn to, I would turn to the manager and describe the scope of project, the requirements that they have for it, and the manager would find the best people for this particular project.

We use JIRA internally. That's the tool that QA people work directly in. They post bugs there, with all the necessary information, and it's pretty convenient.

What we appreciate the most about QATestLab is that they're extremely responsive, and that's why we've never needed to hire any other company. As soon as I have a question, as soon as I contact them, I get an immediate response.

What distinguishes QATestLab from other providers?

These would be the three main things that I would point out: The responsiveness; the speed of onboarding and beginning of the testing itself; and the agile approach – the constant development, constant communication, and the quick back and forth between the team members. They have fantastic team members.

Is there anything QATestLab could have improved or done differently?

Not really. I once had a person replaced, but that was due to their internal need. I was warned in advance, and I didn't even notice it. The transition was all so smooth. I personally never had to request replacements due to some issues or problems.