a1qa

BACKGROUND

Please describe your company and your position there.

I am a Senior Product Manager at smartclip Europe GmbH, responsible for managing the development of our ad server's/broadcaster's user interface. As a company, smartclip deploys proprietary, custom-developed technology to distribute video advertising across all platforms and devices, controlled by ad servers.

Whether online users consume video on the web, on mobile devices like smartphones and tablets, or on connected TV players – including smart TVs, game consoles, set-top boxes, and Blu-ray players, advertisers and their media agencies can reach the exact target audiences they seek for their brand message.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire A1QA?

While steadily developing new features for our platform on a grand scale, we have always been testing them on 2 separate test environments. In addition to writing the specifications for each feature and making sure the developers get all the feedback they need, it got very time and resource consuming to act as the QA team as well, which is what we hired a1qa for.

What were your goals for this project?

At the end of a two-month trial period, we wanted to see definite improvements in the frequency of our feature releases. This should be achieved by being able to simultaneously write new specs, manage the releases and test all successfully developed features, instead of doing all those steps in succession by ourselves.

SOLUTION

How did you select this vendor?

Another department of our company had already hired a1qa in the past for acceptance and integration tests of our mobile SDK. They gave them an extraordinary review as well, which led us to our decision.

Describe the project in detail.

a1qa has helped us set up the QA processes and make them an integral part of the existing team’s workflow, which caters to quality control early in the process, at the requirements stage.

The team performs functional and UI testing of our application, including but not limited to checking the database, the files exported, user access rights, etc. Additionally, to foster quality control and test results documentation, a1qa guided us through the process of choosing a suitable test management system that fully meets our needs.

What was the team composition?

Initially, we were assigned a QA manager to support us and serve as the main point of contact for both testing our software and consulting me regarding our QA workflow. Once we had settled on how we wanted the workflow to look like, we hired two additional QA engineers to support the manager in the active testing of our UI, while he is responsible for defining the test scenarios based off the individual feature's requirements.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

In the 10 months of our ongoing relationship, we were able to keep up the pace of our two-week release schedule with only skipping a beat once. Taking our requirements for an appropriate tool to put our test cases and results down, the QA manager provided us with a whole catalog of possibilities and only stopped his efforts to find the perfect one when we were 100% satisfied.

When integrating and training new workflows, the manager is always eager to take in our input and proactively trains his two engineers, who precisely follow all instructions we gave the manager. All the feedback and test results we receive from a1qa's manager and engineers are accurate and brim with love for even the smallest details.

How effective was the workflow between your team and theirs?

The QA manager that we were assigned initially was very communicative and provided me with at least daily updates. We used Slack for almost all communication. Overall, I was impressed by the timeliness of the team. Whenever the team identified a threat to our timelines, it was communicated immediately. When additional work was needed spontaneously, they jumped in without hesitation.

What did you find most impressive about this company?

All my questions and feedback were answered immediately. All employees of the company I was ever in contact with tried their very best to fulfill our goals at the highest standard.

Are there any areas for improvement?

While there is always headroom for efficiency in software development, I do not think there is anything that could be improved about our relationship to a1qa.

BACKGROUND

Please describe your company and your position there.

I'm the CEO of Hammerhead Technology, a technology holding company. Our companies operate in their own technology subdomains serving various regional markets.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire A1QA?

We’ve partnered with A1QA on software QA and cybersecurity projects for several industries, including banking and government.

What were your goals for this project?

We were looking for a service provider that would deliver substantial value to our end-customers.

SOLUTION

How did you select this vendor?

We first met an affiliate/sister company of A1QA while doing discovery for a separate project. We paid A1QA a visit and things progressed into a partnership.

Describe the project in detail.

We can’t provide more detail because of confidentiality agreements with our end-customers.

What was the team composition?

Over the course of several projects, we’ve interacted with project managers, service delivery managers, business development managers, QA engineers, and cybersecurity engineers.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

So far, we have a 100% completion rate when working with A1QA. We always deliver on time to our customers, whose feedback has been positive.

How effective was the workflow between your team and theirs?

We’ve put a smooth workflow in place after working together several times. Communication has never been a serious issue. We have frequent meetings and weekly progress reports for all our ongoing projects.

What did you find most impressive about this company?

Their technical staff is high quality compared to market standards. Additionally, their pure focus on software testing gives them an edge over shops with more diverse offerings.

Are there any areas for improvement?

The look and feel of their reports provided could be improved.

BACKGROUND

Introduce your business and what you do there.

I’m a QA manager for Sirin Labs, a company developing a smartphone with an integrated crypto wallet.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with A1QA?

We needed a partner to test our cold storage wallet, as well as the overall operation of the phone.

SOLUTION

What was the scope of their involvement?

They took over the QA process after the completion of an internal QA scrum involving four of our testers. A1QA verified and validated the current version of the wallet and the phone itself, working through different versions. They did testing cycles in a waterfall manner.

What is the team composition?

We worked with eight manual testers from their team, two automation testers, and a system performance tester. They also had a QA manager who oversaw their process, and a product manager who helped with any peripheral issues. We also worked with their HR manager, who helped us identify the best people for the task.

How did you come to work with A1QA?

We chose them from a list of three finalists because of their demonstrated competence and their eagerness for the task. We thought they could provide the best solution.

How much have you invested with them?

I believe we’ve spent about $400,000–$500,000.

What is the status of this engagement?

We started working with them in June 2018, and the project concluded in January 2019. We hope to work with the same team when we contract for the next phase.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

They improved the quality of our product tremendously through their QA. They identified issues that our product team didn’t find. Also, their integration into our scrum process also helped our in-house team complete other tasks so that we could release our version for verification. We’d experienced a few timeline issues, and our version had fallen behind schedule. They integrated easily and didn’t slow down the existing work.

How did A1QA perform from a project management standpoint?

They became part of the team, talking to each developer in person, which allowed them to get the answers they needed and report bugs directly to the developers. It was though they were working locally with us, whether we were communicating through Skype meetings and chats, email, or Jira. They give their employee language lessons. At each step, they knew more and were better updated than I was.

Every week, they issued a progress report and a list of issues. We then solved issues through the following week and tried to find solutions for new challenges, of which we had a few. At the end of each testing run, they provided us with a full report of the validated version, with a quality grading.

What did you find most impressive about them?

They found good team members quickly and they were dedicated to our project. They didn’t treat our engagement as a task to be completed; rather, they helped us and they had a sense of responsibility. For example, we had an issue sending phone samples to Belarus. A1QA ensured the delivery of the phones, which is a complicated task due to regulation. They flew during the weekend to get the phones instead of waiting for deliveries. They also sent a team to Israel earlier in our engagement, before our SOP was signed. They wanted to understand our project so we could start on time, even though the engagement had been delayed by legal.

Are there any areas they could improve?

No, we're completely satisfied.

Do you have any advice for future clients of theirs?

Treat them as a partner and be as open as possible. They can solve a lot of the problems that clients face because they’re a mid-sized firm that’s been in the market for several years.

BACKGROUND

Introduce your business and what you do there.

I work for Faberlic as a project manager for our biggest project. We’re an international e-commerce platform with a multi-level marketing structure.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with A1QA?

We’re moving from a monolithic architecture to a service-based one, so we needed a testing team. We don’t have an internal testing team, so preferred to outsource the work.

SOLUTION

What was the scope of their involvement?

We have new releases every three weeks. They perform functional, integration, and performance testing on our web platform, while providing us with all the testing documentation on an ongoing basis.

What is the team composition?

They’ve provided us with QA engineers. We’ve had 5–6 people in different roles, though we don’t have a project manager as it’s not needed. We have a manager in the testing team who acts as team leader.

How did you come to work with A1QA?

We went through a tender process. A1QA have a good reputation within the Russian market, and we’ve found that they have a really good internal structure. They share knowledge and train their staff, so we knew we wouldn’t have any problems; we knew we could give them tasks and they would deliver.

How much have you invested in them?

We’ve spent more than $50,000.

What is the status of this engagement?

We started working with them in August 2016 and have an ongoing relationship.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

We haven’t had any issues with the testing at all. It’s been cost effective for us to use A1QA as we don’t need to train people or hire people; in fact, we don’t need to do anything. We’re more confident with our platform due to their testing work.

How did A1QA perform from a project management standpoint?

They have performed excellently from a communication standpoint. They work remotely, and we communicate over Skype, as well as using Jira and Confluence to share documents and tasks.

What did you find most impressive about them?

As a company, they have excellent communication. They provide training programs and like to see their team and employees grow.

Are there any areas they could improve?

No, I don’t think I can give them any advice.

Do you have any advice for potential customers?

If it is an ongoing project, arrange an interview with the team that will to perform the testing.

BACKGROUND

Introduce your business and what you do there.

Cleeng is a platform that helps other people sell video content. I’m a QA specialist for the company.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with A1QA?

Historically, we’ve always had external companies provide testing services to us. Our contract with our last company was terminated, so we had to find someone new.

SOLUTION

What was the scope of their involvement?

They provided automated and manual testing services to us. Right now, we’re working together on the security testing of our platform.

They’re able to run web and mobile testing, but we only need web testing on both mobile and desktop devices, not mobile app testing.

What is the team composition?

We have one team lead, one manual tester, one automation tester, and two security testers—five people in total.

I work with their manual tester on a daily basis, and we have a weekly catch up with their team lead in which we discuss what they’ve done, what plans they have for next week, and any problems or organizational issues.

How did you come to work with A1QA?

We sent an RFP to six companies, among which A1QA was one of the most professional. I had found them online, contacted them, and liked all their questions. They also had a lower price for their services.

How much have you invested with them?

We’ve spent around $18,000.

What is the status of this engagement?

We started working with them in May 2018, and the engagement is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

Testing work is about quality, not about use cases or anything like that. I’m completely satisfied with the collaboration with them. They work professionally and provide great services.

Using them as a tester definitely makes us more confident about our software, especially in terms of security. It’s not a very good domain in our company, but we’re going to change it. They really help us.

How did A1QA perform from a project management standpoint?

I’m completely satisfied. If I have any questions, I can Skype the team leader or any other member I’m working with. I’ll usually have a response within a couple of minutes; if not, they’ll text us about it.

We don’t have any problems with the communication with them. They have a registered inbox in our Jira, for the security testing. They report to me and our CTO what work they’ve done on a weekly basis.

What did you find most impressive about them?

They’re very proactive, and they suggest good solutions that I don’t think of myself. They provide very good answers for different situations, especially for security testing. I’m not too familiar with this area, but I consider them professional and confident. They know what they’re doing.

Are there any areas they could improve?

It’s hard to say what they should improve.

Do you have any advice for future clients of theirs?

If the client isn’t satisfied with something, they just need to say it. A1QA is always open to dialogue and will change their approach if the client isn’t satisfied.

BACKGROUND

Introduce your business and what you do there.

I'm a delivery director for a development company that provides digital transformation strategy consulting services.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with A1QA?

We hired A1QA to fulfill QA staffing needs.

SOLUTION

What was the scope of their involvement?

A1QA provides QA staffing for a range of projects, including three to four scrum teams. In addition to testing the features of our platforms, they also lead much of the QA test planning and work on some web redesign projects, integrating features as needed.

What is the team composition?

I work with a project lead, who coordinates resourcing and is available for any questions I have. There's also a QA lead on each of the few scrum teams we work with, who oversees coordination on testing, sending out reports, etc.

How did you come to work with A1QA?

We reached out to them, because they were well established in Eastern Europe, where we work, and their reputation preceded them.

How much have you invested with them?

We've invested between $1–$5 million thus far.

What is the status of this engagement?

We've been working with them since April 2014.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

A1QA has made my job much easier and our decade's-long engagement with our account much more stable overall. There are no weak links on their team—I can come to them with any issue without having to micromanage the process.

How did A1QA perform from a project management standpoint?

They communicate very well, especially considering they're offshore and that English is not their first language. Their QA staff are critical in guiding process improvements, working collaboratively with our team to drive the company in a positive direction.

What did you find most impressive about them?

I'm most impressed by their level of continuity; even when we increase staff from their team, they do it seamlessly, proactively shifting people in a way that's least disruptive to the team.

BACKGROUND

Please describe your organization.

Our company has been around for 22 years. We provide a pharmaceutical analytics solution to our clients. We have analytical suites that we provide our clients so they can monitor their performance compared to competitors or just monitor their products around the world. Where we're different is we actually manage this on an ongoing basis. When a client engages with us, it's not through a one-off product that we deliver and walk away from. The bespoke development of the product is only the first stage and then, after that, we manage the product for 10 to 20 years. Some clients have been with us from day one. We update it with new data and features fairly regularly. 

What is your position?

I'm the operations manager.

OPPORTUNITY / CHALLENGE

What business challenge were you trying to address with A1QA?

We already deal with their sister company, Itransition. Now, all of our developers are based full-time in Minsk. I have within my operational team a separate quality assurance team. One day, we just had so much work – we were really struggling to match resources with demand. One of my colleagues recommended that we have a quick word with the sister company. At the time of this original work, A1QA was still housed within Itransition, but it was an independent division. They said maybe they could help. We got one person involved and this was about four years ago. The relationship has expanded ever since. They've become an integral part of my operations team not just in QA but other disciplines now, too.

SOLUTION

Please describe the scope of their involvement.

They're now fully integrated into our development process. To be fair to them, they actually drive it to a certain degree. We go to them for strategic advice. One thing that I really like about them is that they have their own QA academy. We have full-time people on our team from A1QA. We've been with them four years. We have members in the team from day one, and they've slowly grown and become more familiar with our processes and practices. They know our product like the back of their hands. They put together the QA scopes with us and help design our whole testing environment. They usually drive the scope themselves. We review it and agree and, usually, they're better than our original plans. They are incredibly professional. I give them full responsibility for selecting candidates that work on our QA. I have the telecoms and web QA boss run my team and, under her, there's a QA manager who oversees the daily operations. She's excellent. I give her full responsibility and trust her judgment. She keeps me fully informed. I have only changed one person in the past but to put them on a different project, not to remove them altogether. I'm kept fully informed of their experiences inside and outside of A1QA so their education or things that are relevant to Belarus.

How did you come to work with A1QA?

We were already working successfully with Itransition, and when we knew we needed additional QA resources, we approached them.

Could you provide a sense of the size of this initiative in financial terms?

It's hard to say because they're involved in both development QA and ongoing in-house QA. I would venture to say around $150,000 a year.

What is the status of this engagement?

This is still an ongoing engagement.

RESULTS & FEEDBACK

Could you share any statistics or metrics from this engagement?

Their role is looking at the development side of a product. When we get new projects for a client, they're usually involved from the beginning in requirements gathering and discovery phase, defining the QA process, then following that through. The way I would gauge the quality of their work would be going forward on the project. Once the project goes live, we would see whether or not we had any issues reported. I don't remember the last time we had an issue that got past them within the scope we've all agreed upon at the beginning of a project. Our sites are hugely complicated yet very reliable. Some pages have 20,000 to 30,000 possible combinations of links in them because it's all analytics – data driven. We define a tight overall spec, but within the actual QA they're doing I can say they just pick it up naturally. The last time I had an issue get past them was 11 months ago. They're extremely meticulous and methodical.

How did A1QA perform from a project management standpoint?

I get feedback from them and twice a year I go over and see them and do a review with each member of the team. That includes feedback from everyone involved with them from our company. I treat them exactly the same as one of our own staff. We have a whole series of questions. They always rate very highly considering they're based in Minsk. You would think there would be language barriers, but there just aren't. They're brilliant. It's the only outsourcing company I've ever had this much respect for. When I sit down with them we go over the last six months, is there any feedback? What's the communication been like between the teams? Have they been misunderstood? We're always trying to hone and improve our collaboration practices.

What distinguishes A1QA from other providers?

The one thing I'm impressed with is really the training they give their staff. Their QA academy is excellent. I know they have many people that come directly from a university. One of my team is at the university at the moment, but she works full-time on our project. Everybody we've had on our team has been first class. They're trained well, and it's always ongoing. I'm impressed with how that's managed. They're incredibly professional.

Is there anything A1QA could have improved or done differently?

One of our team managers was taking on too much work. He understands our product better than most of us here. We sat down and said we don't want to lose your service, but we need to relook at the structure of the engagement. They were brilliant and very accommodating. I put my thoughts forward and they jumped on it immediately. We had a slight change in the management structure of the team. It was professionally handled and, because of that, they do more services for us.

Do you have any advice to share with their future clients?

Because they were linked to the development side, it's always a bit of a risk. One part of the company is developing the product and another part is responsible for quality assurance. You always think that's a risk because you're putting faith in their honesty and transparency. I found early on that they are brutally honest about their sister company. They definitely act as if they're a separate company and, at this point, they are separate companies. The biggest thing was developing that trusting relationship. They have proved time and again that I could trust them. I have a lot of experience with outsourcing companies. They're good people to work with. They deliver what they say they're going to deliver. They deliver to schedule and to price. 

BACKGROUND

Please describe your organization.

I work for a national economic research institute that conducts research and analysis on economic performance and makes strategic recommendations around ways to improve competitiveness in the global marketplace.

What is your position?

I am a deputy executive director.

OPPORTUNITY / CHALLENGE

What business challenge were you trying to address with A1QA?

We have what we call a case management system for one of our own customers, and they would perform the quality assurance for the finished product. We sometimes deal with quality assurance ourselves but, in this particular case, we were tight on time and resources, so we decided to find a partner with the skill set to do it for us.

SOLUTION

Please describe the scope of their involvement.

They integrated themselves directly into the development team that was working on the application. Depending on our needs, we had between five and ten testers included in our development team because we had more than one project going on with them. We wanted to be sure that we had folks that had prior experience testing a case management system.

As we explored different areas, we also moved toward mobile development as well. We needed folks that had experience testing mobile applications, and then folks that could also do automated testing on their part. They tested both on Android and iOS devices. We also expanded the scope of their involvement to include technical writing, which has worked out well so far.

How did you come to work with A1QA?

When we realized we had a need, our procurement rules were a little bit different than they are today, so we just went online and searched for several firms that could do quality assurance testing for us. A1QA came out of the search engine that we used back then. What really helped us is that, when trying to establish the scope of work and getting the quotes from other vendors took forever, A1QA was very responsive to get something back to us. They not only provided a reasonable quote, but they also gave us a sense of how they would approach working with us. They really made a good argument and were very responsive, which made it easy for us to make a decision and to go with them.

Could you provide a sense of the size of this initiative in financial terms?

The only number I'd be comfortable to say is that, on a yearly basis, we spent more than $200,000.

What is the status of this engagement?

We worked about 12 months on the application before it was prepared to go live. We are utilizing their resources more.

RESULTS & FEEDBACK

Could you share any statistics or metrics from this engagement?

That would be hard because what we provide right now is kind of a turnkey solution for our customers. Our customers are not really ever exposed to the QA [quality assurance] aspect of what we're doing, they're more like embedded as part of our group, and I don't have anything right now. That is something that we are considering as we're utilizing more of their resources so that we can make a good argument that we do have something that's worth spending money on. Right now, I do not have anything like that.

Having incorporated A1QA testers, they were more focused on developing value-added code than doing the rework of bugs that needed to be fixed. There's also a certain confidence that we have, that we have some good QA testers that are working with us. If the developers are missing something early on, everybody is feeling confident that there is a good chance it will be caught in the QA phase, which makes our product better and makes everybody feel better about what we're doing.

How did A1QA perform from a project management standpoint?

For project management, we are using the Atlassian Suite product that they have. That's one nice thing that A1QA has been able to do. They can plug-in into whatever we're using without us having to change any of our business processes.

What distinguishes A1QA from other providers?

One of the big strengths that we've seen with them is that we're working with them on a time and material basis. Whenever we had some projects that came last minute to us, and suddenly we had to scale up, A1QA been able to match and scale up their resources as well. We don't have to worry about whether the QA team can handle the workload if we find enough developers and the resources to do the work here. That has been something really nice. Some of the other vendors that we've worked with in the past required a substantial amount of notice before we could scale up our engagement with them. With A1QA, they've been able to match us on pretty much all the needs that we've ever had.

Is there anything A1QA could have improved or done differently?

I would say nothing that comes to mind right now. Being more competitive from the pricing point of view would be nice, but I would not say that they are the most expensive people out there.

BACKGROUND

Introduce your business and what you do there.

I’m the head of QA operations at Product Madness, a leading global mobile game studio specializing in top-grossing social casino games. We deliver entertaining and responsible gameplay experiences to millions of players worldwide.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with a1qa?

We needed support with our QA operations.

SOLUTION

What was the scope of their involvement?

a1qa provides functional QA for server and client operations, automation, and performance testing. The team also delivers test reports, automated test scripts, and performance analysis documents.

How did you come to work with a1qa?

a1qa was already working with our company when I joined.

What is the status of this engagement?

Ongoing engagement for the better part of a decade.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

Quality of our Products is the best testimony to the success of our partnership.

How did a1qa perform from a project management standpoint?

Their project managers and customer success specialists are always ready to help.

What did you find most impressive about them?

The entire organization is very passionate about QA and is always going the extra mile to ensure standards are met. The pace they can source candidates at has also been impressive.

Are there any areas they could improve?

Greater proactivity in identifying improvement opportunities would make an already strong partnership even stronger.

BACKGROUND

Please describe your company and position.

I am the QA Lead of Bally's Interactive

Describe what your company does in a single sentence.

A1QA provide great testers (Manual & Automation) to suit your companys needs.

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire a1qa to accomplish?

• Release on time
• Find high priority bugs

SOLUTION

How did you find a1qa?

Referral

Why did you select a1qa over others?

• Great culture fit
• Good value for cost
• Company values aligned

How many teammates from a1qa were assigned to this project?

2-5 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

• Feature acceptance testing
• Writing & maintaining test cases
• Sanity & Regression testing
• Automation framework for Web & native apps
• Defect reporting & retesting

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

• Project(s) delivered on time 
• High priority defects found & resolved before launch & regulator approval
• Great communication whether in virtual calls or messaging/email
• Worked with internal team without any issues

Describe their project management. Did they deliver items on time? How did they respond to your needs?

• Worked alongside internal teams to ensure we tested the priority each Day & Week of the project
• Issues raised communicated out so any blockers were prioritised appropriately
• Provided Holiday cover so we had enough resource at all times

What was your primary form of communication with a1qa?

• Virtual Meeting
• Email or Messaging App

What did you find most impressive or unique about this company?

• Their work ethic was great.
• Felt like & were part of the team rather than "outside" help
• Clear communication 

Are there any areas for improvement or something a1qa could have done differently?

N/A