Could you share any evidence that would demonstrate the productivity, quality of work, or the impact of the engagement?
We have moved away from an automated workflow. Whenever we made a new change in the code, it would be pushed automatically to our repositories, after which it would have unit testing performed, and be deployed. We experienced some issues with this workflow, so we are working together with QA Madness in order to test everything for errors and stability. We have implemented a new rollback tool which can undo any new features or bug fixes. We perform manual testing before each new release, and also test the rollback tool, ensuring that everything is okay in terms of previous and current versions.
The stability and security of our releases has increased a lot in the past two months, and we haven’t had any issues with new releases. We have one new release every week, which means that we are moving fast in terms of development, fixing bugs and releasing new features.
How did QA Madness perform from a project management standpoint?
We are working closely with the testing team itself through our Slack instance, which is a Skype-like application for communication. We have daily chats around the features to be tested, any reported bugs, and changes to the testing environment. For meetings, we use Zoom, which is similar to Google Hangouts. It allows us to share screens and have daily standups, and interview QA Madness’ resources.
As far as I know, our managers had conversations with members of QA Madness and assigned one person to work with us. After that, we had a couple of meetings with their manager and tester, and went through interviews and evaluations. The initial tester left the company, but a new resource was introduced to us before that happened, making the transition smooth. I don’t know if we were able to interview all the members of our testing team.
I haven’t looked at QA Madness’ reports. I believe that they are received by the company leader or someone else on our side.
What did you find most impressive about QA Madness?
This is the first company of its kind I’ve worked with, so I cannot compare QA Madness to anyone else. Their team is very organized, especially when it comes to using complicated applications like ours, with different accounts, users, and passwords. We didn’t have any issues with lost passwords on their side, which is a common occurrence for us.
QA Madness is also very detailed in terms of work. They perform different kinds of crazy things during testing, in order to break the software and get errors, like turning off the internet connection while performing a task. Over the past couple of years, QA Madness has helped us detect a number of strange user behaviors. In the end, we won’t know how good the software is until it’s released publicly, and users start doing crazy things with it. We appreciate QA Madness’ work, since this is the way to really test software. At some point, we will be introducing a new type of automatic and functional testing. We like to have this between different stages like testing, staging, production, private clouds, and so on. Our private cloud is similar to the production environment, but with more restrictions around security and data protection. Some of our customers need this kind of extra protection. We plan to introduce a new kind of automatic integration between these different stages, and already have a person from QA Madness’ side working with us, and learning the new technology, not only in order to test that everything is working fine, but to also convert the tests into real code that we can use. They will help us build the piece of software for performing this automatic integration and functional testing.
Are there any areas QA Madness could improve?
Since they are a growing company, improving knowledge around the software and technologies we’re using, specifically around HTML and security. We want to offer the best and most secure features to customers, so this is always a good offer. QA Madness could make suggestions related to security issues they identified in the latest version of Chrome, which could affect our platform, for example. It would be the perfect improvement to them.