DeviQA

BACKGROUND

Please describe your company and your position there.

I’m the mobile applications manager at a mobile location data company.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire DeviQA?

We needed a partner to manage and do quality testing for our iOS and Android applications. They would also have to fix bugs and improve features reported by our users.

What were your goals for this project?

We wanted to maintain our apps’ full functionalities and ensure they complied with Twitter, Apple, and Google’s terms of use.

SOLUTION

How did you select this vendor?

I was not directly involved in the decision-making process. DeviQA was already working with us prior to me starting this role.

Describe the project in detail.

Devi QA works with three of our applications, two of which are for iOS and Android while the third is only for Android. They're responsible for quality testing release candidates for all of the applications and working alongside our developers to troubleshoot and resolve issues. The team also manages the App and Play Store reviews and assists in responding to user complaints.

What was the team composition?

We work with a team of QA testers, one of whom focuses on iOS while the other focuses on Android.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

All of our applications are functioning at the best level I’ve seen since starting my role. We continue to hear positive feedback from users about the apps’ stability and long-time usability. The apps have strong ratings in the App and Play stores thanks to DeviQA’s team.

How effective was the workflow between your team and theirs?

Despite the time difference, there hasn’t been any shortage of communication. Our developers and their QA team are in sync and able to schedule time easily to discuss and plan things. 

What did you find most impressive about this company?

Their work ethic and attention to detail are outstanding. The two resources on our team have great knowledge of our applications and processes and have significantly improved our workflow and productivity. Though one of them joined the team recently, they’ve stepped up quickly and displayed an ability to thoroughly integrate themselves.

Are there any areas for improvement?

No specific improvements are needed. Everything has been top notch and exceeded all expectations.

BACKGROUND

Please describe your company and your position there.

I’m the senior quality manager responsible for our flagship product, a video-based sales learning and readiness platform. I manage the quality of our web and mobile clients as well as the backend services.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire DeviQA?

Our company is growing out of its startup phase. We’re signing larger clients and need to add features quickly to accommodate those client needs while maintaining the base level of quality that our initial clients have come to expect. So, we hired DeviQA to assist with testing.

What were your goals for this project?

Automation is the key factor in ensuring quality while delivering product features quickly and consistently. Our goal is to build a reliable set of automation regression tests across our mobile and web applications.

SOLUTION

How did you select this vendor?

I inherited this team when I came on board with the company several months ago and have been pleased with their performance since the beginning.

Describe the project in detail.

The project included the development of a test framework based on open source tools and open standards. I supplied all guidelines, but the team was responsible for all of the coding and maintenance of the automated tests. They currently maintain the framework, including integration into other tools, writing new tests, monitoring test executions, analyzing test failures, and updating outdated tests.

What was the team composition?

The team consists of three full-time automation engineers. I provide guidance and set priorities on tasks, which are divided among the team based on who has the best experience for each task. This makes managing the team very easy for me, and I’m able to deliver a concise report of accomplishments to our executive management.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

The number of defects we find through the automation process before releasing to customers has been growing and we’re also able to spend more time writing comprehensive test plans and reducing the amount of manual testing required before each release.

How effective was the workflow between your team and theirs?

The workflow is very efficient and easy. We meet once a week for a status update and to discuss priorities. We communicate as needed throughout the week to discuss any issues and resolve problems blocking the team. There is a project manager available with whom I can discuss any problems with team members, but there haven’t been any. I’ve only needed to talk with him to add members to the team.

What did you find most impressive about this company?

Their talent pool is great. Each individual has a lot of experience and exceeds my expectations. They’re also able to support each other to stay engaged and to keep learning. Also, ramping up the team as needed is very easy.

Are there any areas for improvement?

I can't think of any.

BACKGROUND

Introduce your business and what you do there.

I’m the CTO of a marketing company.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with DeviQA?

We’re building innovative software, so we need trusted resources with the skills to test what we develop.

SOLUTION

What was the scope of their involvement?

DeviQA provides senior-level QA resources to test our software. To ensure they perform the most thorough tests possible, they’re involved with our entire development cycle. First, their team attends our development kickoff meetings to gain an understanding of our current project. Then, they start building applicable test scenarios for each product. They also take an active role in our regression and smoke testing efforts.

They’re available to test a variety of use cases whenever we’re deploying a new integration, a redesigned feature, or a bug fix. Their QA resources execute functional testing to make sure that all code changes are stable enough to function within a production environment. We use Jira and some internal automation tools to track each implementation.

What is the team composition?

They’ve assigned 2–3 QA people to our account.

How did you come to work with DeviQA?

They were referred to us by multiple colleagues. After an initial project with them, we realized that they’re a reliable firm. We’ve continued to work with them.

How much have you invested with them?

We’ve spent around $120,000–$150,000 thus far.

What is the status of this engagement?

We hired them in June 2017, and the engagement is ongoing. We’ll likely expand their contract with us to include automatic testing services.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

Each new software release has passed an intricate verification process. By looking at the bug reports listed in Jira, we can tell that they perform extensive testing.

How did DeviQA perform from a project management standpoint?

Their involvement with us consists of executing specific tasks. Therefore, they don’t require a lot of project management. However, we hold daily standup meetings with QA resources assigned to our projects. Those meetings consist of communicating outstanding issues and prioritizing bug fixes.

What did you find most impressive about them?

They’re able to implement a quick onboarding process. Since they’re expert-level QA testers, we don’t have to spend a lot of time teaching them our process. They can understand new software quickly, and then just go to work. It’s incredible.

Are there any areas they could improve?

I’d appreciate it if they provided more detailed progress reports. While they currently send frequent reports, more transparency would make things easier for us.

Do you have any advice for potential customers?

They’ve done an excellent job for us throughout the last two years, so I expect they’ll be able to help future clients in a similarly.

BACKGROUND

Introduce your business and what you do there.

We’re a business services company that provides consultancy and software platform services to clients. One of our core products is a software solution aimed at protecting users from malicious content. I’m the global development manager, in charge of projects across the global company space.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with DeviQA?

We didn’t have internal QA resources, specifically related to the software development lifecycle. We identified that as a significant risk, so we reached out to a number of companies to fill that position. DeviQA were the winners of the contract.

SOLUTION

What was the scope of their involvement?

They have a full-time engagement with us in the QA space of our project. We have two dedicated full-time resources handling testing on our software, which embodies two core sections: a web app and a mobile app, specifically for Android. They run a number of tests in the automated space—manual testing, unit testing, and some code review. They work remotely.

What is the team composition?

We project manage ourselves, so we have internal managers who interact with the two QAs from DeviQA. We also have direct communication with the owner of DeviQA and the senior partner, who we’ve met in person.

How did you come to work with DeviQA?

We initially engaged with them through the RentACoder.com platform. Our engagement grew to the point where we needed more resources than the platform could provide us, so we partnered with DeviQA on a direct engagement.

How much have you invested with them?

The total cost has been in excess of $170,000.

What is the status of this engagement?

We started working with them in September 2014 and the collaboration is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

We have very detailed success criteria; one of the core measurements is the number of issues raised as a result of new product releases. We measured before and after, throughout the product lifecycle, and specifically looked at the number of issues raised pre- and post- DeviQA. We saw a significant drop in all these. One metric showed an 82% reduction in the number of issues raised following the engagement with them.

How did DeviQA perform from a project management standpoint?

We run agile scrum, and they conform to that. They have a good understanding of the platform methodology, and of the system we use to administer that.

We’ve used a number of communication channels, the primary one being Atlassian Jira. We have daily scrum meetings performed through Zoom, the video conferencing platform, and interact directly with the developers using Slack.

What did you find most impressive about them?

Nothing was too much for them, which struck us the most about DeviQA. We can throw any challenge at them, and they jump at it.

Are there any areas they could improve?

Their English level is good, but I’d say respectfully that it could be improved a tiny bit, to make some communication easier. We don’t have any difficulties, but it would make it a bit easier for them to be a bit more conversant.

Do you have any advice for future clients of theirs?

Be sure to clearly define success metrics and ensure that you have the right communication channels.

BACKGROUND

Please describe your company and your position there.

I'm the VP of engineering of an IT services company and am in charge of all IT, development, QA, etc.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire DeviQA?

We needed QA for both our website and mobile apps.

What were your goals for this project?

We were mainly looking for automated test coverage for our website and mobile apps.

SOLUTION

How did you select this vendor?

We found them on UpWork.

Describe the project in detail.

Since we built our website and apps from scratch, we needed good test coverage to ensure we don't deploy bugs.

What was the team composition?

Initially, they assigned two developers and one QA specialist. This eventually expanded to three developers and two QA specialists.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

We have demonstrably fewer bugs making it through to the launched products.

How effective was the workflow between your team and theirs?

They are extremely good and require very little supervision. I would recommend them to anyone.

What did you find most impressive about this company?

They do QA exactly the way I want it. They also ask questions when needed, target whatever I ask them to target, document things well, and are generally extremely competent at their role.

Are there any areas for improvement?

No. They are as good as any QA company I've worked with.

BACKGROUND

Please describe your company and your position there.

I’m the CTO of an educational gaming company. I oversee all aspects of technology and directly manage the platform engineering team.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire DeviQA?

We were looking for continuous QA testing coverage including manual, black box, regression, and automated testing solutions.

What were your goals for this project?

Our company was growing quickly with a small staff, so we wanted to minimize software defects while continuing to move at a high velocity. Therefore, we needed QA expertise that could work seamlessly with our engineering team.

SOLUTION

How did you select this vendor?

We had two engineers who’d had great results with DeviQA at a previous company, so it was an easy choice.

Describe the project in detail.

A dedicated QA Engineer from DeviQA joined our daily standups and had full access to our Github, staging, and production environments. He learned our project requirements by interacting with product leads and engineers and would pick up tasks through our ticketing system. He would then deliver detailed bug reports, including steps to reproduce his fixes and video screen captures. Additionally, he maintained a suite of automated Selenium tests, which run daily, and reported any breaking tests on our staging server for remediation.

What was the team composition?

We worked with one full-time DeviQA resource.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

While we worked with DeviQA, our product’s user base grew from zero to over 200,000, and we added a vast number of innovative features. We couldn’t have grown this quickly without the confidence in QA that DeviQA provided.

How effective was the workflow between your team and theirs?

The DeviQA resource worked seamlessly as a core member of our own team. He had a company email address and was available continuously on Slack during overlapping business hours (about 3 hours per day), which was plenty of time to coordinate activities. He worked during our early morning hours, so we could start the day with fresh QA feedback from the previous day's engineering work.

What did you find most impressive about this company?

The level of expertise and the responsiveness of communication were both excellent.

Are there any areas for improvement?

To be honest, there aren’t any. DeviQA delivered everything we expected and more. We only decided to end the project when our company and budget allowed us to hire an in-house QA engineer. I couldn't have asked for a better experience working with an offshore company for staff augmentation.

BACKGROUND

Please describe your company and your position there.

I am the QA manager at Solebit.  Our company provides identification and prevention of zero-day malware and unknown threats.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire DeviQA?

DeviQA helped us build our SoleGATE™ cybersecurity platform. 

What were your goals for this project?

We needed to increase the quality of our product and develop automated tests to reduce manual testing work

SOLUTION

How did you select this vendor?

We found DeviQA through a recommendation and hired them after an interview process.

Describe the project in detail.

SoleGATE™ is a revolutionary cyber protection software platform that provides comprehensive coverage for advanced cyber attacks against organizations in real-time. Using evasion-proof, signature-less, patented technology, it instantly detects any malicious code and blocks it conclusively. No sandboxing is necessary.

DeviQA created complex and detailed automated scenarios for testing REST APIs using a Faraday library. They built a software development kit application that works on many different machines, including Azure, Google cloud, Docker, and both privileged and unprivileged LXC containers. They created complex methods for these integrations using SSH and SFTP connections. They added different scenarios for Microsoft Azure and Mac.

What was the team composition?

We started with one QA engineer from DeviQA.  After a while, we increased the size of their team by adding two more senior QA engineers.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

DeviQA did an excellent job, and I highly recommend them. They completed the entire scope of work we assigned and reduced the build’s duration from 15 to just three hours. Setting up automation jobs on TeamСity based on the scheduler delivers precise and straightforward reports.

Using parallel processes for file uploading, they resolved problems with scanning large amounts of data. The team created numerous automated tests for the application’s common functionality for settings related to SMPT services, sending emails, and changes in the database. 

How effective was the workflow between your team and theirs?

DeviQA synchronized with our team every morning and opened new tasks in Jira. At any time of the day, our teams could contact each other about questions, issues, or problems.

What did you find most impressive about this company?

DeviQA offers a very professional level of service. They understand the material and can easily learn about and adapt to new technologies

BACKGROUND

Please describe your company and your position there.

I'm the QA [quality assurance] manager for a midsize startup company involved in the fintech industry. We offer a unique, full-scale solution for paying suppliers. 

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire DeviQA?

Originally, we hired one person from DeviQA to test the most basic version of our system. As our company became more successful and grew, we increased DeviQA’s involvement in testing the product.

What were your goals for this project?

DeviQA conducts ongoing testing of the system. Our company works in sprints, and DeviQA integrates fully with our team while performing testing for the system. 

SOLUTION

How did you select this vendor?

They were selected by our CEO before I arrived at the company.

Describe the project in detail.

The project involves several different web platforms, which eventually integrate with each other to create a solution that allows companies to pay their suppliers. DeviQA handles the QA testing of our software.  

What was the team composition?

The team grew from one person to eight. Seven people act as manual testers, and one manages performance testing. 

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

DeviQA has been a major part of our company's success. They help us deliver a high-quality product, evidenced by relatively few production bugs and customer complaints.

How effective was the workflow between your team and theirs?

The workflow is fairly efficient. Our main development center is in Israel, and the DeviQA team is in Ukraine. Still, we’ve been able to communicate consistently via Slack, and our collaboration has improved over time.

What did you find most impressive about this company?

Their team members are committed and always try their best, sometimes staying later than required in order to finish testing items by the delivery deadline. I really appreciate their impressive level of dedication.

Are there any areas for improvement?

I'm very pleased overall. DeviQA’s testers should improve their written English skills, which are understandable but could be better. 

BACKGROUND

Please describe your company and your position there.

The web application is a virtual assistant for your email, discovers opportunities in your inbox by proactively suggesting the next steps to take with your most important contacts.

First, we connect your email accounts to our system. Then, we analyze your conversation history for each relationship and automatically prompt you to re-engage with the people that are slipping off of your radar. This ensures that your most important relationships are in good standing and that you never let important people and business opportunities slip through the cracks.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire DeviQA?

We hired DeviQA to assist in both manual and automated QA.

What were your goals for this project?

We wanted to ensure our engineering team could move at a fast pace without sacrificing quality to our end users.

SOLUTION

How did you select this vendor?

They were referred to us by someone who had used them before.

Describe the project in detail.

The QA work included by-regression testing, as well as manual QA of new features.

What was the team composition?

The team has been composed of a project leader and various contractors over the years. We have increased and decreased the number of contractors working on the project as needed.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

DeviQA has done a great job of being flexible with working hours and continues to produce high-quality output. We consider our engagement with them a success for the stage of growth our company has achieved.

How effective was the workflow between your team and theirs?

It has been very effective.

What did you find most impressive about this company?

The tests are extremely thorough and catch edge cases we never even thought about. They do a great job.

BACKGROUND

Please describe your company and your position there.

I'm the CTO of a SaaS-based EHR platform, headquartered in Los Angeles. Our company is quickly growing and we serve a wide range of medical professionals in the US market.

OPPORTUNITY / CHALLENGE

For what reason(s) did your company hire DeviQA?

We are a fast growing team and we needed a quality assurance team with experience in setting up the right processes and the attention to detail it takes to deliver a high-quality software product.

What were your goals for this project?

Our company wanted to avoid customers' feeling they are working with an unfinished product, so we decided to invest in the proper processes and put in the effort to have a seamless feature release.

SOLUTION

How did you select this vendor?

We did some online research and got in touch with Oleg [Co-founder, DeviQA]. He was very knowledgeable about the QA processes and he staffed our team with great QA engineers.

Describe the project in detail.

After we started to work with DeviQA, their team soon dedicated some time to build and organize reproducible test cases, which raised our product to a higher quality level and simplified regression testing.

What was the team composition?

We had two QA engineers from DeviQA assigned to our project, they worked closely with 5 engineers on our side.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

We definitely improved the quality of our product and feature releases had been stable enough to avoid "fire-drills," which we had before investing in an experienced QA team.

How effective was the workflow between your team and theirs?

The communication with the DeviQA team was mostly handled via Slack and occasional video calls. I got a daily high-level update on the current QA progress and focus.

What did you find most impressive about this company?

Their most impressive attribute was the professionalism of the team and the availability of the co-founder to help out when needed. We really felt valued as a customer.

Are there any areas for improvement?

We have been very happy with their work. There are always occasional defects that are overlooked, but that is normal as we balance effort in QA and time to market. We ended up with a very organized QA process and a better-than-ever product quality.