a1qa

Introduce your business and what you do there.

I’m the CTO of a software company that provides back-office software for schools and school boards.

What challenge were you trying to address with a1qa?

We were looking for help building a test automation framework and automated test cases so we don’t have to do as much manual regression testing. Prior to this engagement, we had zero automation.

What was the scope of their involvement?

We went through a discovery phase where we learned about them and they learned about our specific challenges and what we wanted to achieve. They put together a demo of the things they’ve done and can do using our technology stack. After that, we had several other questions regarding how our teams would work together. Once that was done, we dealt with the commercial terms and what the business relationships would look like.

A1QA built for us the automation framework, reporting, integration with our build pipelines, and automated nightly runs.  In addition, we assign them automation scenarios which they then code and add to the suite. 

They also helped us migrate from Teamcity to Azure DevOps, and we also utilized them for performance testing and to build a PowerBI dashboard (using their parent organization - iTransition).

How did you come to work with a1qa?

We had referrals to three companies from private equity partners.  I interviewed all three and decided on A1QA based on a combination of technical skills and pricing.

How much have you invested with them?

We’ve spent over $100,000 since we started our engagement.

What is the status of this engagement?

We started working together in October 2018 and we continue to work together.

What evidence can you share that demonstrates the impact of the engagement? 

We measure our QA automation and how much time it saves us during our normal release processes. That has grown from zero automation to about 60% of our application flows being automated, saving us thousands of hours every quarter (we release every two weeks). In reality we wouldn't run all the manual regression testing so by having automation we have faster releases and higher quality.

How did a1qa perform from a project management standpoint?

They are very good at estimating work and communicating when things are slipping. Our teams chat daily.  Between the accuracy and communication they are easy to work with. They are good collaborators, and treat our work as a team effort as opposed to a vendor-client relationship.

What did you find most impressive about them?

They are technically very strong.

Are there any areas they could improve?

When we wanted to expand to performance testing and PowerBI development we had to work with their parent company - iTransition.  The parent company is great, but we now had to work with different project managers instead of having a single point of contact. Not a big deal, but it would be helpful if they could streamline that.

Please describe your company and your position there.

I am a Product Owner at a software company, based in the Netherlands.

For what projects/services did your company hire A1QA?

We introduced new components to our stack. The new functionality needed to be tested and part of GUI tests needed to be automated in a test framework. We did not had enough in-house capacity, so we needed to outsource these tasks.

What were your goals for this project?

We wanted to have enough test coverage to test the functionality of the newly developed software

Describe the project in detail.

The project is to create test cases, automate tests and customized a test framework. A1QA provided a test analyst, who was member of our scrum team and participated in all team-related events.

What was the team composition?

A test analyst and a test engineer were allocated to the team and participated in all team-related events.

Can you share any outcomes from the project that demonstrate progress or success?

within a few months hundreds of tests were created and a relative big portion was automated. This was according to expectation, since the specialists of A1QA could do this at a faster rate than the developers. Giving the developers more time to focus on actual development of new functionality.

How effective was the workflow between your team and theirs?

Very effective. Reporting was weekly and always on time. Although we had a geographical distance, but with Teams and Parabol we managed to minimize the distance and were considered full members of the team.

What did you find most impressive about this company?

Clear communication, clear goals, thinking along about solution directions, actively engaged.

Are there any areas for improvement?

no

Please describe your company and your position there.

I am a Senior Product Manager at smartclip Europe GmbH, responsible for managing the development of our ad server's/broadcaster's user interface. As a company, smartclip deploys proprietary, custom-developed technology to distribute video advertising across all platforms and devices, controlled by ad servers.

Whether online users consume video on the web, on mobile devices like smartphones and tablets, or on connected TV players – including smart TVs, game consoles, set-top boxes, and Blu-ray players, advertisers and their media agencies can reach the exact target audiences they seek for their brand message.

For what projects/services did your company hire A1QA?

While steadily developing new features for our platform on a grand scale, we have always been testing them on 2 separate test environments. In addition to writing the specifications for each feature and making sure the developers get all the feedback they need, it got very time and resource consuming to act as the QA team as well, which is what we hired a1qa for.

What were your goals for this project?

At the end of a two-month trial period, we wanted to see definite improvements in the frequency of our feature releases. This should be achieved by being able to simultaneously write new specs, manage the releases and test all successfully developed features, instead of doing all those steps in succession by ourselves.

How did you select this vendor?

Another department of our company had already hired a1qa in the past for acceptance and integration tests of our mobile SDK. They gave them an extraordinary review as well, which led us to our decision.

Describe the project in detail.

a1qa has helped us set up the QA processes and make them an integral part of the existing team’s workflow, which caters to quality control early in the process, at the requirements stage.

The team performs functional and UI testing of our application, including but not limited to checking the database, the files exported, user access rights, etc. Additionally, to foster quality control and test results documentation, a1qa guided us through the process of choosing a suitable test management system that fully meets our needs.

What was the team composition?

Initially, we were assigned a QA manager to support us and serve as the main point of contact for both testing our software and consulting me regarding our QA workflow. Once we had settled on how we wanted the workflow to look like, we hired two additional QA engineers to support the manager in the active testing of our UI, while he is responsible for defining the test scenarios based off the individual feature's requirements.

Can you share any outcomes from the project that demonstrate progress or success?

In the 10 months of our ongoing relationship, we were able to keep up the pace of our two-week release schedule with only skipping a beat once. Taking our requirements for an appropriate tool to put our test cases and results down, the QA manager provided us with a whole catalog of possibilities and only stopped his efforts to find the perfect one when we were 100% satisfied.

When integrating and training new workflows, the manager is always eager to take in our input and proactively trains his two engineers, who precisely follow all instructions we gave the manager. All the feedback and test results we receive from a1qa's manager and engineers are accurate and brim with love for even the smallest details.

How effective was the workflow between your team and theirs?

The QA manager that we were assigned initially was very communicative and provided me with at least daily updates. We used Slack for almost all communication. Overall, I was impressed by the timeliness of the team. Whenever the team identified a threat to our timelines, it was communicated immediately. When additional work was needed spontaneously, they jumped in without hesitation.

What did you find most impressive about this company?

All my questions and feedback were answered immediately. All employees of the company I was ever in contact with tried their very best to fulfill our goals at the highest standard.

Are there any areas for improvement?

While there is always headroom for efficiency in software development, I do not think there is anything that could be improved about our relationship to a1qa.

Please describe your company and your position there.

I'm the CEO of Hammerhead Technology, a technology holding company. Our companies operate in their own technology subdomains serving various regional markets.

For what projects/services did your company hire A1QA?

We’ve partnered with A1QA on software QA and cybersecurity projects for several industries, including banking and government.

What were your goals for this project?

We were looking for a service provider that would deliver substantial value to our end-customers.

How did you select this vendor?

We first met an affiliate/sister company of A1QA while doing discovery for a separate project. We paid A1QA a visit and things progressed into a partnership.

Describe the project in detail.

We can’t provide more detail because of confidentiality agreements with our end-customers.

What was the team composition?

Over the course of several projects, we’ve interacted with project managers, service delivery managers, business development managers, QA engineers, and cybersecurity engineers.

Can you share any outcomes from the project that demonstrate progress or success?

So far, we have a 100% completion rate when working with A1QA. We always deliver on time to our customers, whose feedback has been positive.

How effective was the workflow between your team and theirs?

We’ve put a smooth workflow in place after working together several times. Communication has never been a serious issue. We have frequent meetings and weekly progress reports for all our ongoing projects.

What did you find most impressive about this company?

Their technical staff is high quality compared to market standards. Additionally, their pure focus on software testing gives them an edge over shops with more diverse offerings.

Are there any areas for improvement?

The look and feel of their reports provided could be improved.

Introduce your business and what you do there.

I’m a QA manager for Sirin Labs, a company developing a smartphone with an integrated crypto wallet.

What challenge were you trying to address with A1QA?

We needed a partner to test our cold storage wallet, as well as the overall operation of the phone.

What was the scope of their involvement?

They took over the QA process after the completion of an internal QA scrum involving four of our testers. A1QA verified and validated the current version of the wallet and the phone itself, working through different versions. They did testing cycles in a waterfall manner.

What is the team composition?

We worked with eight manual testers from their team, two automation testers, and a system performance tester. They also had a QA manager who oversaw their process, and a product manager who helped with any peripheral issues. We also worked with their HR manager, who helped us identify the best people for the task.

How did you come to work with A1QA?

We chose them from a list of three finalists because of their demonstrated competence and their eagerness for the task. We thought they could provide the best solution.

How much have you invested with them?

I believe we’ve spent about $400,000–$500,000.

What is the status of this engagement?

We started working with them in June 2018, and the project concluded in January 2019. We hope to work with the same team when we contract for the next phase.

What evidence can you share that demonstrates the impact of the engagement?

They improved the quality of our product tremendously through their QA. They identified issues that our product team didn’t find. Also, their integration into our scrum process also helped our in-house team complete other tasks so that we could release our version for verification. We’d experienced a few timeline issues, and our version had fallen behind schedule. They integrated easily and didn’t slow down the existing work.

How did A1QA perform from a project management standpoint?

They became part of the team, talking to each developer in person, which allowed them to get the answers they needed and report bugs directly to the developers. It was though they were working locally with us, whether we were communicating through Skype meetings and chats, email, or Jira. They give their employee language lessons. At each step, they knew more and were better updated than I was.

Every week, they issued a progress report and a list of issues. We then solved issues through the following week and tried to find solutions for new challenges, of which we had a few. At the end of each testing run, they provided us with a full report of the validated version, with a quality grading.

What did you find most impressive about them?

They found good team members quickly and they were dedicated to our project. They didn’t treat our engagement as a task to be completed; rather, they helped us and they had a sense of responsibility. For example, we had an issue sending phone samples to Belarus. A1QA ensured the delivery of the phones, which is a complicated task due to regulation. They flew during the weekend to get the phones instead of waiting for deliveries. They also sent a team to Israel earlier in our engagement, before our SOP was signed. They wanted to understand our project so we could start on time, even though the engagement had been delayed by legal.

Are there any areas they could improve?

No, we're completely satisfied.

Do you have any advice for future clients of theirs?

Treat them as a partner and be as open as possible. They can solve a lot of the problems that clients face because they’re a mid-sized firm that’s been in the market for several years.

Introduce your business and what you do there.

I work for Faberlic as a project manager for our biggest project. We’re an international e-commerce platform with a multi-level marketing structure.

What challenge were you trying to address with A1QA?

We’re moving from a monolithic architecture to a service-based one, so we needed a testing team. We don’t have an internal testing team, so preferred to outsource the work.

What was the scope of their involvement?

We have new releases every three weeks. They perform functional, integration, and performance testing on our web platform, while providing us with all the testing documentation on an ongoing basis.

What is the team composition?

They’ve provided us with QA engineers. We’ve had 5–6 people in different roles, though we don’t have a project manager as it’s not needed. We have a manager in the testing team who acts as team leader.

How did you come to work with A1QA?

We went through a tender process. A1QA have a good reputation within the Russian market, and we’ve found that they have a really good internal structure. They share knowledge and train their staff, so we knew we wouldn’t have any problems; we knew we could give them tasks and they would deliver.

How much have you invested in them?

We’ve spent more than $50,000.

What is the status of this engagement?

We started working with them in August 2016 and have an ongoing relationship.

What evidence can you share that demonstrates the impact of the engagement?

We haven’t had any issues with the testing at all. It’s been cost effective for us to use A1QA as we don’t need to train people or hire people; in fact, we don’t need to do anything. We’re more confident with our platform due to their testing work.

How did A1QA perform from a project management standpoint?

They have performed excellently from a communication standpoint. They work remotely, and we communicate over Skype, as well as using Jira and Confluence to share documents and tasks.

What did you find most impressive about them?

As a company, they have excellent communication. They provide training programs and like to see their team and employees grow.

Are there any areas they could improve?

No, I don’t think I can give them any advice.

Do you have any advice for potential customers?

If it is an ongoing project, arrange an interview with the team that will to perform the testing.

Introduce your business and what you do there.

We’re a software development company. We also build our own products, because of new technologies we want to learn, and for their commercial potential.

I joined the company eight years ago, managing deployment for customers and making sure they’re happy with the deliverables. I’m currently the president of the company.

What challenge were you trying to address with A1QA?

We needed QA services for several projects. For example, we had a product capable of recording video conferences on most existing platforms and were adding editing capabilities for its users to edit and splice video and audio. We also added an integration with Watson Speech to Text, allowing users to transcribe their videos and do searches within that text.

What was the scope of their involvement?

It depends on the project; the way they work depends on the customer. For a customer who didn’t have any QA in place before, and who wanted to understand what they were missing, a tech lead would come in and do a survey of everything they had and come up with a plan. That plan would usually include writing test cases, selecting from a subset that made sense to automate. In the end, if it made sense, A1QA would propose penetration and load tests.

For some customers, they’ve done full security audits and testing: load testing, security penetration testing, and regression testing. In other cases, they provided manual testers, who wrote the test cases and tried to cover current issues. That is complementary to the unit testing written by the developers. With the customer, they decide which of the manual tests need automating and write those scripts. This allows customers to run both unit tests and automation tests, and it’s a good move toward CI/CD (continuous integration/continuous delivery).

What is the team composition?

The team fluctuates depending on the project. There’s usually a project manager who oversees all the activity and to whom things get escalated if there’s a need. There’s also a QA tech lead, who supervises the process.

How did you come to work with A1QA?

They’re well-known as a developer and they’re based in Minsk, where our main development team is also located.

How much have you invested with them?

The total cost since the beginning has been over $1 million.

What is the status of this engagement?

We started working with them around 2012. We’re still in touch regularly—they’re quite professional and helpful.

What evidence can you share that demonstrates the impact of the engagement?

The impact is great. The fact that we provide quality work is, in large part, due to their testing. Despite the movement to CI/CD, and every developer doing thorough tests, we’re not there yet. Without A1QA’s testing, we wouldn’t have our reputation for building quality products.

How did A1QA perform from a project management standpoint?

It all depends on the project, as different projects require different levels of documentation, but there’ve been no problems with their test cases. The personnel can change, but, if the test cases are still there, any new person can pick up on the project. Without those test cases, there would be no automation.

They’re very good in terms of communication. We’ve worked with customers and with them to perform the QA.

What did you find most impressive about them?

I think they’re very professional. They have a very thoughtful process of onboarding; an internal academy where all new people go through training on their processes and methodology. I think this makes for very good testers.

A1QA has good processes, and I think their management is professional. In terms of customer communication, I haven't had complaints from anyone about issues from them.

Are there any areas they could improve?

They should be hiring more people because there’s obviously demand for their services. I know that it’s hard to hire in Minsk.

Do you have any advice for future clients of theirs?

No.

Introduce your business and what you do there.

Cleeng is a platform that helps other people sell video content. I’m a QA specialist for the company.

What challenge were you trying to address with A1QA?

Historically, we’ve always had external companies provide testing services to us. Our contract with our last company was terminated, so we had to find someone new.

What was the scope of their involvement?

They provided automated and manual testing services to us. Right now, we’re working together on the security testing of our platform.

They’re able to run web and mobile testing, but we only need web testing on both mobile and desktop devices, not mobile app testing.

What is the team composition?

We have one team lead, one manual tester, one automation tester, and two security testers—five people in total.

I work with their manual tester on a daily basis, and we have a weekly catch up with their team lead in which we discuss what they’ve done, what plans they have for next week, and any problems or organizational issues.

How did you come to work with A1QA?

We sent an RFP to six companies, among which A1QA was one of the most professional. I had found them online, contacted them, and liked all their questions. They also had a lower price for their services.

How much have you invested with them?

We’ve spent around $18,000.

What is the status of this engagement?

We started working with them in May 2018, and the engagement is ongoing.

What evidence can you share that demonstrates the impact of the engagement?

Testing work is about quality, not about use cases or anything like that. I’m completely satisfied with the collaboration with them. They work professionally and provide great services.

Using them as a tester definitely makes us more confident about our software, especially in terms of security. It’s not a very good domain in our company, but we’re going to change it. They really help us.

How did A1QA perform from a project management standpoint?

I’m completely satisfied. If I have any questions, I can Skype the team leader or any other member I’m working with. I’ll usually have a response within a couple of minutes; if not, they’ll text us about it.

We don’t have any problems with the communication with them. They have a registered inbox in our Jira, for the security testing. They report to me and our CTO what work they’ve done on a weekly basis.

What did you find most impressive about them?

They’re very proactive, and they suggest good solutions that I don’t think of myself. They provide very good answers for different situations, especially for security testing. I’m not too familiar with this area, but I consider them professional and confident. They know what they’re doing.

Are there any areas they could improve?

It’s hard to say what they should improve.

Do you have any advice for future clients of theirs?

If the client isn’t satisfied with something, they just need to say it. A1QA is always open to dialogue and will change their approach if the client isn’t satisfied.

Introduce your business and what you do there.

I'm a delivery director for a development company that provides digital transformation strategy consulting services.

What challenge were you trying to address with A1QA?

We hired A1QA to fulfill QA staffing needs.

What was the scope of their involvement?

A1QA provides QA staffing for a range of projects, including three to four scrum teams. In addition to testing the features of our platforms, they also lead much of the QA test planning and work on some web redesign projects, integrating features as needed.

What is the team composition?

I work with a project lead, who coordinates resourcing and is available for any questions I have. There's also a QA lead on each of the few scrum teams we work with, who oversees coordination on testing, sending out reports, etc.

How did you come to work with A1QA?

We reached out to them, because they were well established in Eastern Europe, where we work, and their reputation preceded them.

How much have you invested with them?

We've invested between $1–$5 million thus far.

What is the status of this engagement?

We've been working with them since April 2014.

What evidence can you share that demonstrates the impact of the engagement?

A1QA has made my job much easier and our decade's-long engagement with our account much more stable overall. There are no weak links on their team—I can come to them with any issue without having to micromanage the process.

How did A1QA perform from a project management standpoint?

They communicate very well, especially considering they're offshore and that English is not their first language. Their QA staff are critical in guiding process improvements, working collaboratively with our team to drive the company in a positive direction.

What did you find most impressive about them?

I'm most impressed by their level of continuity; even when we increase staff from their team, they do it seamlessly, proactively shifting people in a way that's least disruptive to the team.

Please describe your organization.

Our company has been around for 22 years. We provide a pharmaceutical analytics solution to our clients. We have analytical suites that we provide our clients so they can monitor their performance compared to competitors or just monitor their products around the world. Where we're different is we actually manage this on an ongoing basis. When a client engages with us, it's not through a one-off product that we deliver and walk away from. The bespoke development of the product is only the first stage and then, after that, we manage the product for 10 to 20 years. Some clients have been with us from day one. We update it with new data and features fairly regularly. 

What is your position?

I'm the operations manager.

What business challenge were you trying to address with A1QA?

We already deal with their sister company, Itransition. Now, all of our developers are based full-time in Minsk. I have within my operational team a separate quality assurance team. One day, we just had so much work – we were really struggling to match resources with demand. One of my colleagues recommended that we have a quick word with the sister company. At the time of this original work, A1QA was still housed within Itransition, but it was an independent division. They said maybe they could help. We got one person involved and this was about four years ago. The relationship has expanded ever since. They've become an integral part of my operations team not just in QA but other disciplines now, too.

Please describe the scope of their involvement.

They're now fully integrated into our development process. To be fair to them, they actually drive it to a certain degree. We go to them for strategic advice. One thing that I really like about them is that they have their own QA academy. We have full-time people on our team from A1QA. We've been with them four years. We have members in the team from day one, and they've slowly grown and become more familiar with our processes and practices. They know our product like the back of their hands. They put together the QA scopes with us and help design our whole testing environment. They usually drive the scope themselves. We review it and agree and, usually, they're better than our original plans. They are incredibly professional. I give them full responsibility for selecting candidates that work on our QA. I have the telecoms and web QA boss run my team and, under her, there's a QA manager who oversees the daily operations. She's excellent. I give her full responsibility and trust her judgment. She keeps me fully informed. I have only changed one person in the past but to put them on a different project, not to remove them altogether. I'm kept fully informed of their experiences inside and outside of A1QA so their education or things that are relevant to Belarus.

How did you come to work with A1QA?

We were already working successfully with Itransition, and when we knew we needed additional QA resources, we approached them.

Could you provide a sense of the size of this initiative in financial terms?

It's hard to say because they're involved in both development QA and ongoing in-house QA. I would venture to say around $150,000 a year.

What is the status of this engagement?

This is still an ongoing engagement.

Could you share any statistics or metrics from this engagement?

Their role is looking at the development side of a product. When we get new projects for a client, they're usually involved from the beginning in requirements gathering and discovery phase, defining the QA process, then following that through. The way I would gauge the quality of their work would be going forward on the project. Once the project goes live, we would see whether or not we had any issues reported. I don't remember the last time we had an issue that got past them within the scope we've all agreed upon at the beginning of a project. Our sites are hugely complicated yet very reliable. Some pages have 20,000 to 30,000 possible combinations of links in them because it's all analytics – data driven. We define a tight overall spec, but within the actual QA they're doing I can say they just pick it up naturally. The last time I had an issue get past them was 11 months ago. They're extremely meticulous and methodical.

How did A1QA perform from a project management standpoint?

I get feedback from them and twice a year I go over and see them and do a review with each member of the team. That includes feedback from everyone involved with them from our company. I treat them exactly the same as one of our own staff. We have a whole series of questions. They always rate very highly considering they're based in Minsk. You would think there would be language barriers, but there just aren't. They're brilliant. It's the only outsourcing company I've ever had this much respect for. When I sit down with them we go over the last six months, is there any feedback? What's the communication been like between the teams? Have they been misunderstood? We're always trying to hone and improve our collaboration practices.

What distinguishes A1QA from other providers?

The one thing I'm impressed with is really the training they give their staff. Their QA academy is excellent. I know they have many people that come directly from a university. One of my team is at the university at the moment, but she works full-time on our project. Everybody we've had on our team has been first class. They're trained well, and it's always ongoing. I'm impressed with how that's managed. They're incredibly professional.

Is there anything A1QA could have improved or done differently?

One of our team managers was taking on too much work. He understands our product better than most of us here. We sat down and said we don't want to lose your service, but we need to relook at the structure of the engagement. They were brilliant and very accommodating. I put my thoughts forward and they jumped on it immediately. We had a slight change in the management structure of the team. It was professionally handled and, because of that, they do more services for us.

Do you have any advice to share with their future clients?

Because they were linked to the development side, it's always a bit of a risk. One part of the company is developing the product and another part is responsible for quality assurance. You always think that's a risk because you're putting faith in their honesty and transparency. I found early on that they are brutally honest about their sister company. They definitely act as if they're a separate company and, at this point, they are separate companies. The biggest thing was developing that trusting relationship. They have proved time and again that I could trust them. I have a lot of experience with outsourcing companies. They're good people to work with. They deliver what they say they're going to deliver. They deliver to schedule and to price.