Cro Metrics

BACKGROUND

Introduce your business and what you do there.

I work for Earnest, a student finance company offering student loan refinancing, student loans, and more valuable tools, such as scholarship applications.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with Cro Metrics?

We needed help to optimize our product.

SOLUTION

What was the scope of their involvement?

Cro Metrics has run an A/B test on our website to give us industry feedback. They operate around five teams to run experiments for us. Sometimes they make the design, and sometimes we make the design. Then, their engineering team implements the experiment and monitors it to see how it’s performing. If it’s a success, Cro Metrics helps us ship it live to 100%.

We’re currently working with them to optimize our landing pages and application. We use Google Optimize.

What is the team composition?

On average, we have 5–6 teammates, including a product manager, a few engineers, a project manager, a QA specialist, and a designer.

How did you come to work with Cro Metrics?

I don’t know; they were already in the company when I joined.

What is the status of this engagement?

We started working with them in January 2018, and the partnership is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

The annual cumulative impact is over 100 million in volume, although it isn’t in revenue. Overall, we keep working with Cro Metrics because their impact is very positive. For example, one test added the application to our landing page and increased conversion by 17%.

How did Cro Metrics perform from a project management standpoint?

They’re really good about meeting deadlines that they commit to and upfront about the cost. For communication, they’re in our Slack channel. The team also sends us designs on Figma for review, and we share Google Optimize for the experiments. Additionally, we use Iris as a dashboard of experimental metrics and do our experiment proposals in Google Docs.

What did you find most impressive about them?

They have a lot of experimentation experience because they’ve been doing it for so long. Moreover, Cro Metrics has clients that are similar to us, so they’re able to bring industry and domain-specific knowledge that gives us ideas for experiments that might work for us. In summary, Cro Metrics is an extension of our team, which is great for us because they can pick up important projects we don’t have time for.

Are there any areas they could improve?

No, there’s nothing to improve off the top of my head.

Do you have any advice for potential customers?

I suggest collaboratively coming up with solid hypotheses for the experiments. Cro Metrics is great at bringing in ideas, but the best experiments are the ones that have a clear vision and perspective to test.

BACKGROUND

Please describe your company and position.

I am the Managing Director, Content Marketing & Social Media of UNICEF USA

Describe what your company does in a single sentence.

UNICEF USA advances the global mission of UNICEF by rallying the American public to support the world’s most vulnerable children.

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire Cro Metrics to accomplish?

• Test and iterate our website as we redesign and move to a new platform
• Move past website redesign into a continuous improvement cycle

SOLUTION

How did you find Cro Metrics?

Referral

Why did you select Cro Metrics over others?

• Pricing fit our budget
• Great culture fit
• Good value for cost
• Company values aligned

How many teammates from Cro Metrics were assigned to this project?

6-10 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

Cro Metrics ran a website testing and iterating program to inform a website redesign. We have refreshed and rebuilt our website by updating the design, structure, cleaning up and making technical fixes to pages, and launching 20+ new features; the Cro Metrics testing program has been a foundational step in improving and maintaining our website to achieve our goals.

We largely accomplished this by establishing a testing and innovating program with Cro Metrics. With this program, we are continually improving the UX and leveraging the dynamic nature of the site by testing and iterating design improvements, led by Cro Metrics, ensuring that content is relevant and accessible so that the website can continue to drive impact for the organization. Objectives for this engagement have included: 

• Refresh the overall look and feel of universal site elements and primary site pages based on test learnings
• Ensure site performance post-replatforming with post-design testing; 
• Providing ad hoc testing strategy and engineering support as needed
• Improve online donation conversion rate and maximize monthly donations to increase long-term value
• Improve advocacy and engagement conversions to deepen ways to engage

Complete additional Optimizely integrations with other marketing stack technology. Cro Metrics has provided strategic and technical support to scale and operationalize UNICEF USA’s website refresh and rebuild. Cro Metrics has provided the resources necessary to maintain a successful optimization program for UNICEF USA, including: 

• Group Director
• Growth Strategist
• Program Manager
• UX Designer
• Engineers specializing in client-side test implementation
• Quality Assurance Engineers.

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

By investing in the continued upgrades of the website, we have been able to deliver effective conversion pathways for supporters, exceptional SEO ratings, improve target supporter experiences rather than UUSA’s internal needs and structure, and present a homepage that is modern, timely and news-driven and consistent with our brand while fulfilling our goal of providing up-to-date information on the impact of UNICEF programs and initiatives. 

In partnership with Cro Metrics, UNICEF USA has increased online donations through effective website experimentation. The site optimization program has measurably improved the usability of donation forms, increased recurring donations, and contextualized the organization’s mission in the donation funnel. 

In 2022-2023, Cro Metrics led the design and experimentation for overhauling UNICEFUSA.org, including designs, test builds, analysis, and strategic recommendations. Together, we: 

• Met goals for audience engagement identified based on stakeholder sessions and goal-tree mapping lead by Cro Metrics
• Measured success against baseline analytics developed for benchmarking 
• Executed approximately 12 pre-launch tests to inform final design
• Delivered experiment code gists, as needed, to enable UNICEF USA to efficiently productionize optimized experiences. 
• Developed and managed testing and page design roadmap

Describe their project management. Did they deliver items on time? How did they respond to your needs?

Cro Metrics delivered test designs and results on time and was always very responsive to our needs.

What was your primary form of communication with Cro Metrics?

• Virtual Meeting
• Email or Messaging App

What did you find most impressive or unique about this company?

The culture of CroMetrics is great; everyone is nice and fun to work with and seems to genuinely care about and get excited about the work. Everyone we have worked with at Cro Metrics has also been thorough, with great attention to detail as well as a knack for understanding and communicating the big picture.

Are there any areas for improvement or something Cro Metrics could have done differently?

I can't think of any areas of improvement. They have led a very successful testing program.

BACKGROUND

Introduce your business and what you do there.

I’m a director at a hotel chain corporation. I oversee our users’ digital shopping experience and contact centers.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with Cro Metrics?

After the COVID-19 pandemic, the hospitality industry was really affected, and we needed to rebound. We needed help optimizing our website and running A/B tests for our conversion strategy.

SOLUTION

What was the scope of their involvement?

Cro Metrics manages our site’s conversion rate optimization and A/B testing strategy, from the designing phase to the hypothesis testing and implementation, finishing up with the code’s development.

After that initial POC, we moved into a different phase where Cro Metrics took information from our analytics platform to work on the tests we wanted to run. Cro Metrics provides a recap after each test, whether successful or not. We still gain learnings from unsuccessful tests, which allows us to change our hypothesis for future tests. 

Cro Metrics changes specific things on our website and designs A/B tests to see which changes work best. We have different brands within our company, which allows us to test with different brands. For instance, a small change could be changing our call to action button, either the vocabulary or the shape and color. More extensive tests would be navigation redesigns or site redesign. We’ve conducted tests based on content, imagery, and also tests focused on functionality. 

Cro Metrics helps us not only to design these tests and create hypotheses based on analytics or the voice of the customer. They also help develop the tests, manage the testing software for us, and use the resulting information for our development pipeline or to change our site’s content. 

What is the team composition?

We work with an account manager and a strategic account partner. We also work with a development and strategic team, so we work with three Cro Metrics teammates.

How did you come to work with Cro Metrics?

We talked to different conversion rate optimization agencies, and Cro Metrics had a formal and solid approach to implementing testing. They also have a lot of knowledge in the industry and e-commerce background. One of our account representatives previously worked in the hospitality industry.

Moreover, when we went through the candidate’s RFP, the one from Cro Metrics was the one that looked more like a company we wanted to have a partnership with. We ran a proof of concept (POC) test where Cro Metrics ran tests they thought would provide good insights into our site’s performance, and that allowed us to see what they could bring to the table. When we looked at the ROI of those tests, we signed a long-term agreement with Cro Metrics. 

How much have you invested with them?

We have an hourly-based rate, so we only pay for the hours Cro Metrics spends working on our project. We’ve invested an average of $200,000 yearly. 

What is the status of this engagement?

We started working with them in December 2020, and our relationship is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement? 

Looking at the historic revenue based on the tests we’ve run, we’ve had an average ROI of one to 15, if not greater. The key metrics depend on the tests we’re running and their objectives. We’ve conducted business decision tests that weren’t necessarily looking to identify the best path forward and other tests that are more image-focused. 

During the bidding process, Cro Metrics supports testing and assists in helping a company establish a testing cadence. We’ve had a great partnership with Cro Metrics from the start, right out of the POC and the test pilot we ran. They stand by everything they've presented in their sales pitch. 

We weren't doing any testing prior to working with Cro Metrics, so this was a great way for us to optimize our site and improve its UX while improving key KPIs.

How did Cro Metrics perform from a project management standpoint?

I have no complaints. Cro Metrics is very responsive and organized. They have a robust dashboard platform where all our tests are stored and automatically pull in results. Cro Metrics updates our dashboard weekly, and we have a call every Monday to review which tests we have on the backlog or going live. In addition to that, we do quarterly touch bases which are broader sessions where we collaboratively share the progress of our tests. 

Moreover, Cro Metrics is upfront about our budget and how many hours they require to launch tests. 

What did you find most impressive about them?

Cro Metrics’ approach to testing is very professional, and their delivery is excellent. They take the time to understand our business and deliver the results we’re looking for. Cro Metrics has understood and helped us work towards our KPIs in a very collaboratively manner.  

Are there any areas they could improve?

I can’t think of anything for Cro Metrics to improve.

Do you have any advice for potential customers?

I recommend to get involved in your testing strategy and collaborate with Cro Metrics. I’ve been in the hospitality industry for 10 years and know the business very well. I didn’t know anything about A/B testing until this partnership, but I’ve learned as we progressed in our partnership with Cro Metrics.  We try different tests and run them collaboratively. 

To anyone interested in multivariate testing or A/B testing, I recommend understanding how website analytics work. Especially if you work in a unique industry, that will help drive different testing strategies to maximize results. Understanding the testing background and learning that piece, has helped us get to where we’re at now.

BACKGROUND

Please describe your company and your position there.

Bombas is a mission focused to DTC premium basics apparel brand. We sell Socks, T Shirts, Underwear and Slippers and donate one for every item purchased. At Bombas I lead our Site Experience team overseeing Product Management, UX Design and Site Merchandising; which are the main teams focused on optimizing the customers shopping journey and underlying tools and tech.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Cro Metrics?

We hired CRO Metrics to be our Center of Excellence for testing. Helping us establish the processes and proper approach to testing, understanding the testing tooling that exits and building out tests to address our priority hypothesis.

SOLUTION

How did you select this vendor and what were the deciding factors?

They had a proven track record of optimizing other larger brands and an expertise with the relevant tooling. Our previous vendor did a lot of different things, we liked that CRO was pretty focused on just....CRO (Conversion Rate Optimization).

Describe the project in detail and walk through the stages of the project.

This is an ongoing engagement and not a one off project. Several years ago we onboarded with CRO where they helped establish the process for creating and running tests. We have iterated on that process over time in effort to run more effective tests.

How many resources from the vendor's team worked with you, and what were their positions?

We have a primary strategist and project manager and range of engineers who work on building out projects, but we don't really work with them directly that is the project managers role.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

We have increased our testing program's velocity and overall ROI. This comes from improved process and education on how to identify and structure good hypothesis routed in clear customer pain points / problems.

How effective was the workflow between your team and theirs?

We have worked together quite well over the years which led to more impact and more test learnings.

What did you find most impressive or unique about this company?

Cro has such a good grasp on Optimizley (our testing tool) that we rarely need to speak with optimizley directly.

Are there any areas for improvement or something they could have done differently?

Helping identify helpful quantitative insights to go along with qualitative insights to help identify real problems / optimization opportunities.

BACKGROUND

Introduce your business and what you do there.

I’m the senior director of a capital solutions company.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with Cro Metrics?

I joined the company last year and we didn’t have a lot of data that showed how people visiting our website were converting to become leads which means, in particular, submitting forms. A lot of the testing we wanted to do was to identify different UX approaches and different tests for driving those conversions. Cro Metrics was originally hired to help with UX design and A/B testing on our platform.

SOLUTION

What was the scope of their involvement?

They worked with us in terms of our overall business goals, which for me, were related to KPI and conversion form submissions. Our A/B type tests were set up in Optimizely, and then we moved to Convert Experiences. They were primarily focused on conversion and journey testing.

We had things that we wanted to test. For example, we were getting submissions from visitors to our site who were mainly looking for support or looking to log in to our platform and were therefore not really marketing leads. We also did a test that changed the position of the login on the navigation and rearranged it to make guidance and support on the right part of the site so it was more prominent. 

They would do the design, we approved it, and then they would set up the tests in one of the platforms I mentioned and execute it. We had systems to record Optimizely documents that laid out exactly what the test was, what the intended results were or what we were looking for, and all the designs were stored there. We approved different stages before they moved to the build. They performed full turnkey solutions.

What is the team composition?

We had some changes on the account during the time that I was working with them which was about 18 months. I think there were probably about five different people we worked with. Two of them were either directors or higher.

How did you come to work with Cro Metrics?

It was an existing relationship when I came on board.

How much have you invested in them?

Knowing what we were paying on a monthly basis, it was in the region of over $200,000.

What is the status of this engagement?

It has been a long time based on what I have been told. I would say our engagement began in June 2015 and was terminated in October 2022.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

The reason we worked with them for so long was that they took care of everything. We implemented some of the ideas from the outcomes of the tests, and we built those into the website. I wouldn’t be able to give very specific quantitative metrics because the reason we implemented them was that they’d already shown themselves to be successful, or more successful than what we were doing before. I can’t give a business metric impact, but our conversion rates from visitors to form submissions increased.

How did Cro Metrics perform from a project management standpoint?

There was always very good communication, and they were generally on time. There weren’t rigid timescales or milestones, rather we used an outline expectation of how things would be delivered. Occasionally, we would have to chase them down to get things done, and if there was anything we wanted to do urgently, it made it a bit more difficult.

We mainly communicated through Google Meet and Slack. We normally met through virtual meetings. I never personally met any of them face-to-face, and I don’t think before me it was done that way either. 

What did you find most impressive about them?

They worked with our company for a long time and knew how we worked, so they fit in very much like an extension of the team. They were a good cultural fit, and they had a strong desire to make a difference. They were interested in delivering results, reliable, and good at what they did.

Are there any areas they could improve?

We terminated the engagement because of the cost. They wanted to increase costs substantially or do more work so that the business model changed. If that hadn’t been the case, we would still be working with them. I don’t think there is anything in particular that they could have done differently during the engagement.

Do you have any advice for potential customers?

Do your due diligence. Depending on what your business requirements are, there will normally be more than one service provider for you to choose from. You need to be clear about what is important to you and assess Cro Metrics against that. I think culture fit is important as well as technical capability and project management, but it’s the same as hiring any service provider because you have to do your due diligence. 

BACKGROUND

Please describe your company and your position there.

Calendly, a powerful yet simple automated scheduling platform, takes the work out of connecting with others so you can accomplish more. Millions of users benefit from an enjoyable scheduling experience, more time to spend on top priorities and flexibility to accommodate individual users and large teams alike. I am the Web Strategy Lead at Calendly, and have responsibility for our marketing website performance.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Cro Metrics?

With millions of monthly visitors, our marketing website plays a critical role for both our self-serve and sales-led pipelines. To assist us in growing signups and leads, we engaged Cro Metrics to help us with our experimentation strategy and conversion-rate optimization strategies.

SOLUTION

How did you select this vendor and what were the deciding factors?

We looked for a vendor that had worked with similarly-sized B2B SaaS companies, and that had inhouse resources to provide strategic support and executive with design and development work.

Describe the project in detail and walk through the stages of the project.

The project is ongoing, and Cro Metrics operates like an extension of our team. We strategize quarterly, meet weekly, and agree experiments for design, build and approval. Cro Metrics then works with us to analyze results and agree on next steps. We communicate daily, and Cro Metrics has an in depth understanding of our business.

How many resources from the vendor's team worked with you, and what were their positions?

We work with an Account Director, a Project Manager and a team of designers, developers, QA engineers and analysts.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

We have shipped hundreds of experiments, some of which have been successful and others not, but all of which have helped to improve the conversion rate of our site.

How effective was the workflow between your team and theirs?

Very effective! Cro Metrics feels like an extension of our team, and we are deeply integrated.

What did you find most impressive or unique about this company?

The amount of experience that Cro Metrics has is very impressive. We're able to learn best practices, and have the benefit of ideas and strategies that we wouldn't otherwise have access to if run internally.

Are there any areas for improvement or something they could have done differently?

Sometimes it can take some time for Cro Metrics to build, QA and launch experiments as we only pay for a proportion of a designer, developer and QA Engineer, as compared to having an internal resource.

BACKGROUND

Introduce your business and what you do there.

I’m the senior manager of product insights at a meal kit company. My team is responsible for generating insights from both our digital side and our actual physical products (i.e., the meals we produce).

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with Cro Metrics?

We needed help with A/B testing for insight generation.

SOLUTION

What was the scope of their involvement?

Cro Metrics helps us generate insights for the digital side of our business by running a lot of A/B tests on our website and mobile app. They’re in charge of building, launching, and analyzing these tests, using Optimizely to run them. Their efforts include testing strong copy we’re not sure of, testing assets our marketing team is developing, and coming up with ideas for new assets and copies to try.

What is the team composition?

We’ve worked with a lot of people from Cro Metrics. Currently, we mainly work with Isabel (Program Manager) and Zeinoun (Senior Growth Strategist), who recently joined the project. Zeinoun has been in charge of the test ideation and the analysis of results. Additionally, we’ve worked with one of their partners.

How did you come to work with Cro Metrics?

The partnership with Cro Metrics was already established before I joined my current role.

How much have you invested with them?

We pay around $50,000 per quarter, but the first quarter was a free trial. So far, we’ve spent around $400,000.

What is the status of this engagement?

We started working together around February 2020, although I joined the team in March 2020. Our engagement is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement? 

Cro Metrics’ involvement has allowed us to double our A/B testing speed so that we don’t have to do everything internally and we can focus on other priorities. One of the changes we’ve made as a result of the A/B tests conducted by Cro Metrics has been redesigning the user journey for when users go and close their accounts. This process’ design was outdated, so we refreshed it to ensure it was not confusing for users. We also added more information that could benefit the customer and hopefully make them reconsider closing their account.

How did Cro Metrics perform from a project management standpoint?

Cro Metrics’ project management is great, and they’re very proactive in their communication. We have weekly meetings where everyone comes always prepared with updates and thoughts — it is a very good communication approach. I also have a bimonthly meeting with their leadership to talk about the partnership and potential changes. We communicate mainly through Slack and use Google Docs and an in-house platform to manage the results of the tests.

What did you find most impressive about them?

We’ve been engaged with Cro Metrics for a while, so even when people have switched out, we’ve maintained a strongly cemented relationship. They really know our business, and we’ve learned together what works well and what doesn’t. Now, our conversations and tests are of a higher level due to the amount of work they’ve invested into learning about our business and our team members. At this point in time, they feel like part of our company.

Are there any areas they could improve?

No.

Do you have any advice for potential customers?

Have a dedicated person in your team to act as the main point of contact for them so that, if they have questions, they can easily refer to this person. Additionally, it has been very helpful for us to share with them our quarterly strategic priorities so that they can use them to prioritize testing ideas.

BACKGROUND

Please describe your company and your position there.

Blink Fitness is a value brand gym. I worked as a Product Manager for the website.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Cro Metrics, and what were your goals?

We hired CroMetrics to assist with A/B testing designs and features on our website and member portal. They helped us understand the impact of our changes on user behavior so that we could de-risk our decisions.

SOLUTION

How did you select this vendor and what were the deciding factors?

This decision was made before I started.

Describe the scope of work in detail, including the project steps, key deliverables, and technologies used.

I would write up tickets and include designs, review with cro metrics and define how we would measure the performance. After a round of QA we would schedule a release and launch features to pre-determined user segments. On our weekly call we would review performance and inform next steps.

How many people from the vendor's team worked with you, and what were their positions?

1 Account Manager, 1 Project Manager, and a team of engineers. (1-3?)

RESULTS & FEEDBACK

Can you share any measurable outcomes of the project or general feedback about the deliverables?

We saw a 14% decrease in online cancellations due to A/B testing with CroMetrics

Describe their project management style, including communication tools and timeliness.

We had a weekly meeting to go over projects in flight, met ad hoc for QA and new projects. We had a monthly checkin to review how we were tracking for our hours for the month. Lastly, we had a quarterly A/B testing shakeout to review key learnings with the broader company.

What did you find most impressive or unique about this company?

Their insight into the data they tracked was incredibly valuable. Data without knowing how to read it/tell a story, is useless. Cro Metrics made sure to make sense of what information we gathered.

Are there any areas for improvement or something they could have done differently?

No, they are great problem solvers and extremely collaborative. I loved working with them.

BACKGROUND

Introduce your business and what you do there.

We’re a WebOps platform that supports marketers and developers in building, launching, and iterating on their websites. I’m the director of brand and digital experience.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with Cro Metrics?

We hired Cro Metrics to help us facilitate a website experimentation program.

SOLUTION

What was the scope of their involvement?

They provide project management, strategic support for experiment ideas, and engineers to help us build experiments. They also have analysts to help us review and interpret the results of the experiments we’re running on our website.

Typically, we meet once a week to come up with experiment ideas on the website. For example, we might to swap out a new experience on our home page and see how it impacts our users.

We come up with a hypothesis, create mocks, and build those into the test. Then, we measure the current experience versus the variant. We run that until we reach statistical significance. Then, the project lead comes to us with the number of click throughs and conversions.

We decide which changes to implement, based on how successful the test was. From there, we modify the experience until it’s doing really well. From there, we’ll move onto another area of the site.

What is the team composition?

We work with Ann (VP of Business Development), Dave (VP of Operations), Steve (Senior Growth Strategist), Katie (Growth Strategist), and Angela (Program Manager).

How did you come to work with Cro Metrics?

I was introduced to them through one of our co-founders. He knew Ann and thought they would be a great partner for me with the experiments I’m running on the site.

How much have you invested with them?

We spend $185,000 per year.  

What is the status of this engagement?

We officially began our engagement in October 2018, and our work is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement? 

Working with them has been very successful, which is why I’ve been working with them for a few years now. We’re able to measure the impact that the experiments are having on our website very well and their impact on opportunity creation for our sales team.

It’s a very accountable program that we run that lets me easily measure ROI. We’ve been able to achieve increased lead generation and improved UX.

How did Cro Metrics perform from a project management standpoint?

They’re really good. The help me manage everything so I can focus on other issues. I come in to help with the ideation and approvals to make sure they have the assets they need. They’re really good about making sure that all of these experiments get built and are tracked properly. They also ensure everything runs on time and hold us accountable for deliverables on our end. We use Slack and a project management tool.

What did you find most impressive about them?

They’re an enjoyable team to partner with. They’re great thought partners and they’re great to bounce ideas off of. I trust them a lot.

Are there any areas they could improve?

We’ve had account changes where we’ve had our staff changes. I can’t really hold that against them, since that happens at any agency. I’ve been happy with their work.  

Do you have any advice for potential customers?

You get what you put into the relationship. The more information that you can share with Crometrics about your business and goals, the more they’ll be able to give you in terms of thoughtful experiment ideas. Holding back information will hurt the program.