DeviQA

BACKGROUND

Introduce your business and what you do there.

I’m a QA manager at a Swiss IT company that focuses on security solutions. We’ve developed a secure messenger app with video calling and file sharing features. Our main clients are governments, hospitals, and financial institutions. We’re a relatively small company, with about 30 people more or less, and I’m responsible for the entire process of testing and delivery.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with DeviQA?

There were too many things for us to test ourselves, and we couldn’t find enough good specialists in the region, so we outsourced some of the work to DeviQA. 

SOLUTION

What was the scope of their involvement?

DeviQA has helped us do several things. First of all, they worked on automation. We have a lot of test cases to automate—around 1,000 on Android and iOS each. We were doing this manually before hiring DeviQA, which amounted to a lot of time—around three weeks for three engineers. Their team has helped us automate everything, and they keep going, which means that we can basically run tests overnight.

They also assist with manual testing, as there are a lot of things to test before going to automation and to production. We’ve hired more testers from DeviQA in order to cover all product parts. Furthermore, they’re helping us lead a development project that we’ve outsourced somewhere else. One of DeviQA’s people is leading testing within that other company as well. 

Overall, they’re helping us a lot and I’m really happy we’ve hired DeviQA. I’d really love to hire more people from them, but we’re having some budget constraints. 

What is the team composition?

We’re working with three people at the moment: two automation testers and one manual tester. They work directly with me as a dedicated team, alongside the three testers on our side.  

How did you come to work with DeviQA?

I did a lot of research online and found DeviQA on Clutch. They had good feedback, so we started a small proof-of-concept project to see if they were as good as described. We liked them so much that we formally started working together, and we’ve been happy so far.

How much have you invested with them?

The cost so far has been between $300,000–$400,000 in total.

What is the status of this engagement?

We started working with DeviQA around two years ago. 

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

I’m 100% happy with them. We’ve worked with lots of other companies, but DeviQA is really responsible and has independent staff. Most of the time, they perform really well and I have no issues with them at all.

I’ve included them in our development environment, which has two-week milestones. I can see what they’re doing every day and whether they’ve achieved those milestones on a weekly basis. This is mainly how we measure their performance.

How did DeviQA perform from a project management standpoint?

If they have a problem, they will contact me directly and we’ll resolve it immediately. Everything is running smoothly and I can contact their principal whenever I have an issue, which he’ll likely know the solution to. DeviQA is pretty responsible, though, so I don’t normally need to call their boss.

We’re using several tools, including our own internal chat tool. It’s encrypted, which makes it more secure for exchanging important information. We’re also using Skype and Slack. I’m a Russian speaker as well, which has helped me work with them, but DeviQA has also worked with some of our English-speaking clients and the experience was positive. The communication part is easy-going, with no problems or misunderstandings.

What did you find most impressive about them?

They’re a really competent and solid team. We need to be careful when working with offshore providers, as some of them overpromise, but DeviQA’s people do their jobs and are professionals in what they do. I can leave them on their own and go on vacation, and trust that everything will be done and documented properly. I haven’t had any problems concerning their comprehension or ability to execute a task.

Are there any areas they could improve?

At this stage, they could only help us improve our own processes. DeviQA has helped us understand how we can improve internally. 

Do you have any advice for future clients of theirs?

It’s important to run a demo project and see if they can do what’s being asked of them. This is true for any provider, though. DeviQA will tell us when they lack the expertise to do something. 

BACKGROUND

Please describe your company and your position there.

I am the VP of business development, and I have a project management background. 

WiserBrand is a business process outsourcing company that helps clients complement their business with a team of dedicated professionals. Our company has about 60 employees, and we serve clients all over the world.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire DeviQA?

We needed a reliable, skilled partner to be responsible for the quality of all of our applications. They would build a structural testing approach, design automated test suite architecture, and develop automated tests. They would also conduct performance and manual testing.

What were your goals for this project?

We wanted our applications to be high quality.

SOLUTION

How did you select this vendor?

We contacted many different agencies and asked them to conduct a test task. DeviQA did it excellently.

Describe the project in detail.

We have mobile and web applications. We didn’t have a formal structural process. We had worked with other agencies before, but their processes were not as good. 

Once we started with DeviQA, our in-house team felt that their process differed much from what we had before. DeviQA’s services included automated testing.

What was the team composition?

The team consisted of four senior manual QA engineers, six senior automated testing engineers, and one project manager.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

The quality of our applications increased significantly. People started to recommend our applications thanks to DeviQA. We started getting perfect reviews in the Appstore and Google Play. Also, we needed to invest in automated testing to reduce manual work, and DeviQA did this perfectly and saved us money.

How effective was the workflow between your team and theirs?

Communication between our in-house team and the specialists from DeviQA was excellent. They are really fast. All of our team was happy with the workflow that they proposed.

What did you find most impressive about this company?

DeviQA is a high-class QA testing company. They are very experienced, skilled engineers. Their team is reliable and technical. Before DeviQA, I’d never met a partner I could trust with the quality of our apps.

Are there any areas for improvement?

No, they are perfect.

BACKGROUND

Please describe your company and your position there.

I’m the mobile applications manager at a mobile location data company.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire DeviQA?

We needed a partner to manage and do quality testing for our iOS and Android applications. They would also have to fix bugs and improve features reported by our users.

What were your goals for this project?

We wanted to maintain our apps’ full functionalities and ensure they complied with Twitter, Apple, and Google’s terms of use.

SOLUTION

How did you select this vendor?

I was not directly involved in the decision-making process. DeviQA was already working with us prior to me starting this role.

Describe the project in detail.

Devi QA works with three of our applications, two of which are for iOS and Android while the third is only for Android. They're responsible for quality testing release candidates for all of the applications and working alongside our developers to troubleshoot and resolve issues. The team also manages the App and Play Store reviews and assists in responding to user complaints.

What was the team composition?

We work with a team of QA testers, one of whom focuses on iOS while the other focuses on Android.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

All of our applications are functioning at the best level I’ve seen since starting my role. We continue to hear positive feedback from users about the apps’ stability and long-time usability. The apps have strong ratings in the App and Play stores thanks to DeviQA’s team.

How effective was the workflow between your team and theirs?

Despite the time difference, there hasn’t been any shortage of communication. Our developers and their QA team are in sync and able to schedule time easily to discuss and plan things. 

What did you find most impressive about this company?

Their work ethic and attention to detail are outstanding. The two resources on our team have great knowledge of our applications and processes and have significantly improved our workflow and productivity. Though one of them joined the team recently, they’ve stepped up quickly and displayed an ability to thoroughly integrate themselves.

Are there any areas for improvement?

No specific improvements are needed. Everything has been top notch and exceeded all expectations.

BACKGROUND

Please describe your company and your position there.

I’m the senior quality manager responsible for our flagship product, a video-based sales learning and readiness platform. I manage the quality of our web and mobile clients as well as the backend services.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire DeviQA?

Our company is growing out of its startup phase. We’re signing larger clients and need to add features quickly to accommodate those client needs while maintaining the base level of quality that our initial clients have come to expect. So, we hired DeviQA to assist with testing.

What were your goals for this project?

Automation is the key factor in ensuring quality while delivering product features quickly and consistently. Our goal is to build a reliable set of automation regression tests across our mobile and web applications.

SOLUTION

How did you select this vendor?

I inherited this team when I came on board with the company several months ago and have been pleased with their performance since the beginning.

Describe the project in detail.

The project included the development of a test framework based on open source tools and open standards. I supplied all guidelines, but the team was responsible for all of the coding and maintenance of the automated tests. They currently maintain the framework, including integration into other tools, writing new tests, monitoring test executions, analyzing test failures, and updating outdated tests.

What was the team composition?

The team consists of three full-time automation engineers. I provide guidance and set priorities on tasks, which are divided among the team based on who has the best experience for each task. This makes managing the team very easy for me, and I’m able to deliver a concise report of accomplishments to our executive management.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

The number of defects we find through the automation process before releasing to customers has been growing and we’re also able to spend more time writing comprehensive test plans and reducing the amount of manual testing required before each release.

How effective was the workflow between your team and theirs?

The workflow is very efficient and easy. We meet once a week for a status update and to discuss priorities. We communicate as needed throughout the week to discuss any issues and resolve problems blocking the team. There is a project manager available with whom I can discuss any problems with team members, but there haven’t been any. I’ve only needed to talk with him to add members to the team.

What did you find most impressive about this company?

Their talent pool is great. Each individual has a lot of experience and exceeds my expectations. They’re also able to support each other to stay engaged and to keep learning. Also, ramping up the team as needed is very easy.

Are there any areas for improvement?

I can't think of any.

BACKGROUND

Introduce your business and what you do there.

I’m the CTO of a marketing company.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with DeviQA?

We’re building innovative software, so we need trusted resources with the skills to test what we develop.

SOLUTION

What was the scope of their involvement?

DeviQA provides senior-level QA resources to test our software. To ensure they perform the most thorough tests possible, they’re involved with our entire development cycle. First, their team attends our development kickoff meetings to gain an understanding of our current project. Then, they start building applicable test scenarios for each product. They also take an active role in our regression and smoke testing efforts.

They’re available to test a variety of use cases whenever we’re deploying a new integration, a redesigned feature, or a bug fix. Their QA resources execute functional testing to make sure that all code changes are stable enough to function within a production environment. We use Jira and some internal automation tools to track each implementation.

What is the team composition?

They’ve assigned 2–3 QA people to our account.

How did you come to work with DeviQA?

They were referred to us by multiple colleagues. After an initial project with them, we realized that they’re a reliable firm. We’ve continued to work with them.

How much have you invested with them?

We’ve spent around $120,000–$150,000 thus far.

What is the status of this engagement?

We hired them in June 2017, and the engagement is ongoing. We’ll likely expand their contract with us to include automatic testing services.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

Each new software release has passed an intricate verification process. By looking at the bug reports listed in Jira, we can tell that they perform extensive testing.

How did DeviQA perform from a project management standpoint?

Their involvement with us consists of executing specific tasks. Therefore, they don’t require a lot of project management. However, we hold daily standup meetings with QA resources assigned to our projects. Those meetings consist of communicating outstanding issues and prioritizing bug fixes.

What did you find most impressive about them?

They’re able to implement a quick onboarding process. Since they’re expert-level QA testers, we don’t have to spend a lot of time teaching them our process. They can understand new software quickly, and then just go to work. It’s incredible.

Are there any areas they could improve?

I’d appreciate it if they provided more detailed progress reports. While they currently send frequent reports, more transparency would make things easier for us.

Do you have any advice for potential customers?

They’ve done an excellent job for us throughout the last two years, so I expect they’ll be able to help future clients in a similarly.

BACKGROUND

Introduce your business and what you do there.

We’re a business services company that provides consultancy and software platform services to clients. One of our core products is a software solution aimed at protecting users from malicious content. I’m the global development manager, in charge of projects across the global company space.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with DeviQA?

We didn’t have internal QA resources, specifically related to the software development lifecycle. We identified that as a significant risk, so we reached out to a number of companies to fill that position. DeviQA were the winners of the contract.

SOLUTION

What was the scope of their involvement?

They have a full-time engagement with us in the QA space of our project. We have two dedicated full-time resources handling testing on our software, which embodies two core sections: a web app and a mobile app, specifically for Android. They run a number of tests in the automated space—manual testing, unit testing, and some code review. They work remotely.

What is the team composition?

We project manage ourselves, so we have internal managers who interact with the two QAs from DeviQA. We also have direct communication with the owner of DeviQA and the senior partner, who we’ve met in person.

How did you come to work with DeviQA?

We initially engaged with them through the RentACoder.com platform. Our engagement grew to the point where we needed more resources than the platform could provide us, so we partnered with DeviQA on a direct engagement.

How much have you invested with them?

The total cost has been in excess of $170,000.

What is the status of this engagement?

We started working with them in September 2014 and the collaboration is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

We have very detailed success criteria; one of the core measurements is the number of issues raised as a result of new product releases. We measured before and after, throughout the product lifecycle, and specifically looked at the number of issues raised pre- and post- DeviQA. We saw a significant drop in all these. One metric showed an 82% reduction in the number of issues raised following the engagement with them.

How did DeviQA perform from a project management standpoint?

We run agile scrum, and they conform to that. They have a good understanding of the platform methodology, and of the system we use to administer that.

We’ve used a number of communication channels, the primary one being Atlassian Jira. We have daily scrum meetings performed through Zoom, the video conferencing platform, and interact directly with the developers using Slack.

What did you find most impressive about them?

Nothing was too much for them, which struck us the most about DeviQA. We can throw any challenge at them, and they jump at it.

Are there any areas they could improve?

Their English level is good, but I’d say respectfully that it could be improved a tiny bit, to make some communication easier. We don’t have any difficulties, but it would make it a bit easier for them to be a bit more conversant.

Do you have any advice for future clients of theirs?

Be sure to clearly define success metrics and ensure that you have the right communication channels.

BACKGROUND

Please describe your company and your position there.

I'm the VP of engineering of an IT services company and am in charge of all IT, development, QA, etc.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire DeviQA?

We needed QA for both our website and mobile apps.

What were your goals for this project?

We were mainly looking for automated test coverage for our website and mobile apps.

SOLUTION

How did you select this vendor?

We found them on UpWork.

Describe the project in detail.

Since we built our website and apps from scratch, we needed good test coverage to ensure we don't deploy bugs.

What was the team composition?

Initially, they assigned two developers and one QA specialist. This eventually expanded to three developers and two QA specialists.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

We have demonstrably fewer bugs making it through to the launched products.

How effective was the workflow between your team and theirs?

They are extremely good and require very little supervision. I would recommend them to anyone.

What did you find most impressive about this company?

They do QA exactly the way I want it. They also ask questions when needed, target whatever I ask them to target, document things well, and are generally extremely competent at their role.

Are there any areas for improvement?

No. They are as good as any QA company I've worked with.

BACKGROUND

Please describe your company and your position there.

I’m the CTO of an educational gaming company. I oversee all aspects of technology and directly manage the platform engineering team.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire DeviQA?

We were looking for continuous QA testing coverage including manual, black box, regression, and automated testing solutions.

What were your goals for this project?

Our company was growing quickly with a small staff, so we wanted to minimize software defects while continuing to move at a high velocity. Therefore, we needed QA expertise that could work seamlessly with our engineering team.

SOLUTION

How did you select this vendor?

We had two engineers who’d had great results with DeviQA at a previous company, so it was an easy choice.

Describe the project in detail.

A dedicated QA Engineer from DeviQA joined our daily standups and had full access to our Github, staging, and production environments. He learned our project requirements by interacting with product leads and engineers and would pick up tasks through our ticketing system. He would then deliver detailed bug reports, including steps to reproduce his fixes and video screen captures. Additionally, he maintained a suite of automated Selenium tests, which run daily, and reported any breaking tests on our staging server for remediation.

What was the team composition?

We worked with one full-time DeviQA resource.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

While we worked with DeviQA, our product’s user base grew from zero to over 200,000, and we added a vast number of innovative features. We couldn’t have grown this quickly without the confidence in QA that DeviQA provided.

How effective was the workflow between your team and theirs?

The DeviQA resource worked seamlessly as a core member of our own team. He had a company email address and was available continuously on Slack during overlapping business hours (about 3 hours per day), which was plenty of time to coordinate activities. He worked during our early morning hours, so we could start the day with fresh QA feedback from the previous day's engineering work.

What did you find most impressive about this company?

The level of expertise and the responsiveness of communication were both excellent.

Are there any areas for improvement?

To be honest, there aren’t any. DeviQA delivered everything we expected and more. We only decided to end the project when our company and budget allowed us to hire an in-house QA engineer. I couldn't have asked for a better experience working with an offshore company for staff augmentation.

BACKGROUND

Please describe your company and your position there.

I am the QA manager at Solebit.  Our company provides identification and prevention of zero-day malware and unknown threats.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire DeviQA?

DeviQA helped us build our SoleGATE™ cybersecurity platform. 

What were your goals for this project?

We needed to increase the quality of our product and develop automated tests to reduce manual testing work

SOLUTION

How did you select this vendor?

We found DeviQA through a recommendation and hired them after an interview process.

Describe the project in detail.

SoleGATE™ is a revolutionary cyber protection software platform that provides comprehensive coverage for advanced cyber attacks against organizations in real-time. Using evasion-proof, signature-less, patented technology, it instantly detects any malicious code and blocks it conclusively. No sandboxing is necessary.

DeviQA created complex and detailed automated scenarios for testing REST APIs using a Faraday library. They built a software development kit application that works on many different machines, including Azure, Google cloud, Docker, and both privileged and unprivileged LXC containers. They created complex methods for these integrations using SSH and SFTP connections. They added different scenarios for Microsoft Azure and Mac.

What was the team composition?

We started with one QA engineer from DeviQA.  After a while, we increased the size of their team by adding two more senior QA engineers.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

DeviQA did an excellent job, and I highly recommend them. They completed the entire scope of work we assigned and reduced the build’s duration from 15 to just three hours. Setting up automation jobs on TeamСity based on the scheduler delivers precise and straightforward reports.

Using parallel processes for file uploading, they resolved problems with scanning large amounts of data. The team created numerous automated tests for the application’s common functionality for settings related to SMPT services, sending emails, and changes in the database. 

How effective was the workflow between your team and theirs?

DeviQA synchronized with our team every morning and opened new tasks in Jira. At any time of the day, our teams could contact each other about questions, issues, or problems.

What did you find most impressive about this company?

DeviQA offers a very professional level of service. They understand the material and can easily learn about and adapt to new technologies

BACKGROUND

Please describe your company and your position there.

I'm the QA [quality assurance] manager for a midsize startup company involved in the fintech industry. We offer a unique, full-scale solution for paying suppliers. 

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire DeviQA?

Originally, we hired one person from DeviQA to test the most basic version of our system. As our company became more successful and grew, we increased DeviQA’s involvement in testing the product.

What were your goals for this project?

DeviQA conducts ongoing testing of the system. Our company works in sprints, and DeviQA integrates fully with our team while performing testing for the system. 

SOLUTION

How did you select this vendor?

They were selected by our CEO before I arrived at the company.

Describe the project in detail.

The project involves several different web platforms, which eventually integrate with each other to create a solution that allows companies to pay their suppliers. DeviQA handles the QA testing of our software.  

What was the team composition?

The team grew from one person to eight. Seven people act as manual testers, and one manages performance testing. 

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

DeviQA has been a major part of our company's success. They help us deliver a high-quality product, evidenced by relatively few production bugs and customer complaints.

How effective was the workflow between your team and theirs?

The workflow is fairly efficient. Our main development center is in Israel, and the DeviQA team is in Ukraine. Still, we’ve been able to communicate consistently via Slack, and our collaboration has improved over time.

What did you find most impressive about this company?

Their team members are committed and always try their best, sometimes staying later than required in order to finish testing items by the delivery deadline. I really appreciate their impressive level of dedication.

Are there any areas for improvement?

I'm very pleased overall. DeviQA’s testers should improve their written English skills, which are understandable but could be better.