Label Your Data

BACKGROUND

Please describe your company and your position there.

I am a Chief Scientific Officer of an AI startup leveraging tech for good. We assist Law Enforcement Agencies (LEAs) in fighting online crime in its many shapes and forms.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Label Your Data?

Our speciality lies in creating convenient AI-based investigative tools to help gain vital intelligence on online harassment, violence, fraud detection, and many more. For that we require huge news’ and messages’ datasets annotated and labelled properly.

SOLUTION

How did you select this firm and what were the deciding factors?

Multilingual, secure and knowledgeable — those were our criteria for selecting an annotation vendor. Label Your Data combines the three, that’s why we decided to go with them.

Describe the project in detail and walk through their service package.

The team assisted us with three projects since December 2020. The first one was connected to the upcoming elections where we needed them to gather a dataset with the related news and then label them as true, fake, or biased. Eventually, the annotators processed around 5000 news articles.

Our second inquiry had a limited timeframe: 1000 Arabic messages had to be annotated in just a few days. The team did it over the weekend, and we received the needed data on time. The third time, we asked them to annotate around 5000 news related to COVID-19 in Spanish, which was done well and in line with our requirements.

How many resources from the vendor's team worked with you, and what were their positions?

We remained in touch with the project manager the whole time. For each task, and depending on the language required, there were at least a couple of annotators involved.

RESULTS & FEEDBACK

Can you share any outcomes from the engagement that demonstrate progress or success?

With the assistance of Label Your Data, we successfully accomplished annotation goals for three of our projects.

How effective was the workflow between your team and theirs?

Our cooperation with Label Your Data started in December 2020. During all three projects, the annotation and labelling were done without a hitch, with constant feedback exchanges and always meeting the deadlines.

What did you find most impressive or unique about this company?

The swiftness they responded with to our urgent requests was impressive. Plus, their flexibility and a proactive approach.

Are there any areas for improvement or something they could have done differently?

Nothing much at this point.

BACKGROUND

Introduce your business and what you do there.

I’m the CTO of Raveler. We provide software for qualitative analysis of consumer reviews.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with Label Your Data?

We were trying to develop a new system based on some machine learning technology that required access to training data from human models.

SOLUTION

What was the scope of their involvement?

Label Your Data provides data annotation services to us. We specifically asked them to use a data annotation too called Doccano. Their tasks started small with only 200–300 data units. I also gave them some examples of data that I had annotated.

They then assigned 2–3 of their staff on the task. Once they completed that task, the team checked in to verify if the data quality aligned with my expectations. Label Your Data has repeated that same process about 7–8 times now, and they continue to provide data annotation.

What is the team composition?

My main point of contact is Karyna (CEO). I also work with a project manager who takes my requirements and turn them into tasks. Finally, there are 2–3 annotators who I haven’t directly spoken to.

How did you come to work with Label Your Data?

I searched for data annotation services on Google and found 2–3 agencies. I reached out to all of them, but I liked Label Your Data’s work the most. They can also work with any annotation tool, which was a plus.

How much have you invested with them?

We’ve spent between $3,000–$10,000. 

What is the status of this engagement?

The project started in May 2021 and is ongoing.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement? 

In terms of data quality, about 90%–95% of the annotations have been precisely what I wanted. Label Your Data can turn around any task of any volume within the space of five working days. For example, they once delivered 3,000 annotations in five days, which was quite impressive.

How did Label Your Data perform from a project management standpoint?

Their project management is perfect — we can contact them at any time, and they deliver within our deadlines. Throughout the process, they’ve been highly professional. We used Slack and Zoom to communicate.

What did you find most impressive about them?

The partnership has been going really well. The team is professional and provides excellent value for money. I feel like Label Your Data truly prioritizes our project — they’re willing to drop everything to fulfill our needs and keep us happy.

Label Your Data’s help has enabled us to be quite flexible about our technical direction. Whenever we have new ideas, they help us turn them into real-world results in less than a week.

Are there any areas they could improve?

No, there’s nothing they could do better.

Do you have any advice for potential customers?

Be specific about how you want them to annotate your data. The more specific you are, the quicker they can produce accurate work. 

BACKGROUND

Please describe your company and your position there.

We are a Data Science company specializing in data-oriented recruiting, consulting, and creating AI R&D centers from scratch. In addition, we conduct corporate trainings in data science, visualization and offer customizable courses to cover specific needs of our clients.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Label Your Data?

We needed a responsible partner to take on bounding boxes and semantic segmentation annotation for our project.

SOLUTION

How did you select this firm and what were the deciding factors?

The way it usually happens now — we found them online. The pilot went pretty well, and they acted on our feedback almost immediately. We liked what we saw, that’s why our communication continued.

Describe the project in detail and walk through their service package.

The task was to label videos and images captured by the security cameras on the work and building platforms. After the training, the annotation process began. The team had their own annotation tool as well as QA procedures. In the beginning, we had a couple of notes regarding the labeled data, which were considered right away. Then all went quick and well.

How many resources from the vendor's team worked with you, and what were their positions?

The team consisted of ~5 annotators, and an Account Manager, with whom all the communication was conducted. We agreed on using one of the messengers, which made it convenient for both parties. Feedback exchange was easy, and, more importantly, the team took all our needs and remarks seriously.

RESULTS & FEEDBACK

Can you share any outcomes from the engagement that demonstrate progress or success?

With Label Your Data taking over annotation, our team made great advancements in the software development for our client. Afterward, we came back to Label Your Data with a few more image annotation tasks.

How effective was the workflow between your team and theirs?

Label Your Data makes it feel like they're our own company’s department. They’ve shown readiness to do more than asked, they were interested in good results, and them being open to our suggestions turned it into a nice cooperation.

What did you find most impressive or unique about this company?

Their data annotation security standards are definitely worth mentioning. And the way they handle misunderstandings — quick and professional.

Are there any areas for improvement or something they could have done differently?

Nothing comes to mind, actually. They’ve been pretty responsive and flexible.

BACKGROUND

Please describe your company and your position there.

I am a CEO of a computer vision and traffic data platform. We work with the AI-powered LiDAR technology to provide real-time analytics with the aim of making traffic safer for every citizen

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Label Your Data?

Being a young company, we were looking for a reliable partner with whom we could start small and then grow together. In addition, LiDAR data requires a lot of persistency and skill, so we needed someone dedicated. Label Your Data appeared to be a perfect match

What were your goals for this project?

The main goal was quality LiDAR data annotation with 2D boxes for our platform. We were looking for all our annotation requirements to be met

SOLUTION

How did you select Label Your Data?

It all started with looking up companies providing annotation services. We reached out as their security standards and expertise appealed to us a lot. A flexible and involved long-term partner — that was the aim. Label Your Data keeps proving to be such

Describe the project in detail.

Our platform provides real-time traffic analytics using AI-powered sensors. The system helps define the speed and trajectory of every traffic participant. Annotation of sensor fusion data is more complicated than pictures and videos — it needs enormous attention to detail. Label Your Data covered that. They took every step of the integration with us.

What was the team composition?

A big team was not something we were planning on. At this point, it consists of 2 members, and we've been thinking about expanding it. Label Your Data was able to organize the workflow and communication in a way that we can track the progress at any moment

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

We can focus on enhancing the platform now. The annotation is taken care of, and no constant supervision over the team is needed. We receive the annotated data weekly and send our feedback

How effective was the workflow between your team and theirs?

The annotation was launched last year, in November. Label Your Data team found a tool that suits our needs for the LiDAR annotation, and even provided us with a free pilot. You can tell by their approach that they’ve got a clear understanding of our needs and goals

What did you find most impressive about this company?

They are flexible, dedicated, and their processes are well organized. We haven’t experienced any issues or miscommunication

Are there any areas for improvement?

Not that we noticed. Our collaboration has been pleasant and productive so far.

BACKGROUND

Please describe your company and your position there.

We are a tech company offering AI- and ML-based business solutions to help companies worldwide enter the new digital business era. I am an ML Engineer enhancing the infrastructure of our system.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Label Your Data?

The main problem we were working on was vehicle damage detection

What were your goals for this project?

The goal was to find a long-term partner flexible enough to do different annotation tasks quick, well, and whenever we needed it.

SOLUTION

How did you select Label Your Data?

We were looking into outsourcing annotation, an in-house team was not an option. As annotation types differed, we considered companies that were open to discussions and workflow modifications. The pilot with Label Your Data showed us that they were the right choice.

Describe the project in detail.

The main project was vehicle damage detection - i.e. finding and defining damages from a collection of photos of a damaged car. The photos might be close or long-shot, the damages can be on any of a number of car details, and there are several types of damages.

What was the team composition?

We spoke with a project manager from Label Your Data, who helped us flesh out our requirements. We know they had a team (I think of 6) data annotators, and a data engineer to aggregate/clean their labels.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

With the data annotated and tested, we were able to improve our system. In addition, during one of our cooperations they did an extra round of QA for the batch annotated, and together we found a hitch in the model that misidentified some objects.

How effective was the workflow between your team and theirs?

Very effective, to be honest. Their responsiveness and the way they switched between tasks made it a great experience.

What did you find most impressive about this company?

The coordinated and smooth performance of the Label Your Data team.

Are there any areas for improvement?

I really can't think of anything.

BACKGROUND

Please describe your company and your position there.

We are a mobile app development company with AI technology. I'm a senior researcher and development who is responsible for delivering high-quality machine learning models to support main features in mobile Apps.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Label Your Data?

Label You Data helped us label a large number of images with our customized labeling tool. They also collected and recorded videos for us to build specific datasets.

What were your goals for this project?

Build several datasets with high-quality annotations for training machine learning models.

SOLUTION

How did you select Label Your Data?

We communicated with several companies that provide similar services and worked on a pilot project with them to try out the services. The pilot project is a low-volume task but enough for us to make the judgment on which company is easier to work with. Finally, we like what Label Your Data provides most and selected them.

Describe the project in detail.

We have a Slack channel to communicate with them about our requirements. We sent them the data we have and provided our labeling tool, which is AI-assisted, and hopefully accelerate the process. They got the tool setup quickly without much help from us. Our requirements were a long list but they can digest quickly. In the beginning, we asked them to provide some initial labels for us to review and we keep enhancing the annotation quality through communication. The iteration is quick and effective. Their questions are short and precise. Once they fully understand what we want, we basically keep sending new data to them and they can get the tasks done within the expected time & budget we agreed on.

What was the team composition?

On our side it's just me talking to them with minimal bandwidth, providing guidelines/data/tools. I don't need to worry anything about how much resources are allocated in this project from Label Your Data. There's a main person I was talking to and she ensures everything is delivered under our requirements.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

We finally built a big dataset with several machine learning use cases within 2 months, which is very helpful on boosting the accuracy of our machine learning models, and eventually benefits our users.

How effective was the workflow between your team and theirs?

The communication is very smooth and efficient. We don't need to have long meetings to resolve anything. Most things I provide to them are just several slides with requirements. They can provide short and precise questions to make the communication clear, which saved time for both of us. Their responses to our messages were pretty quick (within hours, and sometimes instantly).

What did you find most impressive about this company?

They are humble, serious about their work, and they care about the feelings of their clients.

Are there any areas for improvement?

There's still space to improve the labeling quality. Some reviewing process on the results will help them eliminate some obviously bad labels. Having said that, their current quality is totally acceptable.

BACKGROUND

Please describe your company and your position there.

We’re a team of students from the University of Bayreuth (Germany) engaged in the development of a racing vehicle project for participation in the annual Formula Student festival. I work on the business and engineering sides of the project. It’s my job to see that the vehicle is commercially viable while it can also win in the race.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Label Your Data?

We needed an organization that could help us with the annotation work. We were looking for a company interested in sponsoring our project that also had the required level of knowledge and expertise in the annotation field. A young, ambitious start-up fit our goals perfectly.

What were your goals for this project?

This year, we will be participating in a driverless race. For this, we need to design an AI algorithm that requires annotated data to train correctly. We required help with the annotation task since we do not have the human resource capacity to do the job ourselves.

SOLUTION

How did you select Label Your Data?

We conducted our research based on our goals and core values. We were looking for a partner who combined expertise with curiosity and openness to new ideas. In the Label Your Data team, we’ve found everything we needed: a lot of interest in our project, a high level of professional expertise, open-mindedness, and ambition.

When we learned more about how they do business at Label Your Data, we were pleasantly surprised by the scope of their work and their vast experience. Their particular interest in driverless vehicles matched well with our goals. So we reached out to Label Your Data CEO, Karyna to offer participation in our project.

Describe the project in detail.

To train our algorithm, we need 20.000 images to be annotated using bounding boxes and key points labeling. Moreover, the project is ongoing as we add more images as we train the model. This allows us to control how our model trains and to fill in the blind spots on the go. Due to this, while we’re not tightly restricted in time, the timely delivery of the annotations is of importance to us.

What was the team composition?

We have a comparatively small project, so we don’t require a team of annotators. We need a single labeler to work continuously on the new images we provide, as well as supervision and QA to ensure that the quality of labeling is superb.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

For now, we’re still cooperating with Label Your Data, and we have only the good things to say about their labeling quality, security, and management. The team is very responsive and highly interested in our project. Whenever we need to contact them, they reply nearly instantly and are always friendly and eager to help. In addition, they offered to share our progress on their social media, which can be quite useful for us.

How effective was the workflow between your team and theirs?

The workflow is flawless. We can reach the annotator at any time we need. The same goes for the management. As our project goes on, it’s likely we will have an even more positive experience working with the Label Your Data team.

What did you find most impressive about this company?

We’re very impressed with their eagerness to help and support us. Besides, the communication is very pleasant and fast, and their security standards in data annotation can satisfy the pickiest of the clients.

Are there any areas for improvement?

Nothing comes to mind. We’d really like to work with them as a single team. We’d also be glad to see them during the racing events of 2021 :)

BACKGROUND

Please describe your company and your position there.

I am the CEO at a web development company. We deliver wholesome customer software solutions, from the backend to edge devices, using Microsoft technologies and modern front-end frameworks. My job is to oversee every project and make sure we meet the clients' demands.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Label Your Data?

We were building a state-of-the-art virtual assistant for our client and needed to ensure the proper training of the ML algorithm. We got the dataset from our client and were looking for a data annotation outsourcing company. Since it's not the focus of our business, we did not want to train a labeling specialist for our team.

What were your goals for this project?

High-quality and secure data annotation was an important part of our project, so we were looking for a partner we could trust. This was essential to ensure our team spends as little time and efforts on training as possible.

SOLUTION

How did you select Label Your Data?

There is a good choice of labeling companies on the Internet, but we had two factors to consider. First, we had a somewhat restricted budget, which ruled out the most expensive options. Second and probably most important, we needed a company that values security. This was crucial for us since we had to trust our client's data to a third party, so we paid the closest attention to the data protection protocols and standards of business practices of each candidate. For a final choice, we looked at the portfolio of the companies and went with the one that impressed us with their speedy delivery and high quality of services.

Describe the project in detail.

We required OCR with some text classification on the side. After reviewing our requirements, they offered us a pilot project so we could trace back any miscommunications. After that, we discussed and fixed the payment. After they started working, we had a few training sessions for one week so that the annotators working on our project could understand what classes and tags they needed to use. The integration was smooth: they were always there when we needed them, and they asked questions if any issue arose. They also have a great practice of a thorough QA that dramatically decreased the risk of mistakes. We paid in full when the project was finished.

What was the team composition?

We had a fixed team of ten annotators and a team manager who organized and overlooked their work. It was quite comfortable for us since we could reach out to the manager at any time to see what stage the project was on. We see as another benefit that the team of annotators was diverse, which allowed them to understand the requirements of the project better.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

It would have been expensive to hire and train an in-house data labeler for a single project, so we were able to fit within the budget by hiring an outsourcing annotation company. Besides, we saved the time and effort required to label a large dataset. And their competent QA allowed decreasing the error margin to a negligible 0.1%. We were able to concentrate on our primary task of designing the model. Also, due to the high quality of annotation, we could cut the time on algorithm training, without the need to go back to re-annotating the dataset. When the model was ready, we received positive feedback on the accuracy of its predictions from our client.

How effective was the workflow between your team and theirs?

Aside from the week-long training session, we met weekly for the duration of the project. They were reporting and seeking feedback proactively. When a hitch with the classification arose, they contacted us immediately, and we were able to resolve the issue within a day. For us, this meant focusing on the design of the model and its development. We knew that we had regular meetings to discuss progress and that the company would meet our needs.

What did you find most impressive about this company?

They work within very strict data protection protocols, from office security to personnel training. We were able to see how carefully the annotators treated every piece of data. Besides, they finished the project before the deadline, which was impressive given the volume of the dataset we assigned. They used this time to check with us whether we were happy with the results.

Are there any areas for improvement?

There's always room for improvement, however, I cannot think of anything to mention in this specific case. When we encountered small issues, they were always resolved quickly and professionally.

BACKGROUND

Introduce your business and what you do there.

I’m the CCO (Chief Communications Officer) of Sheis.ai. We're a startup, offering AI chat solutions for people who work in customer service roles.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with Label Your Data?

We needed our datasets labeled in order to help improve customer experience. Since we didn’t have these abilities in-house, we were looking for a vendor.

SOLUTION

What was the scope of their involvement?

We sent them our datasets, including directions and dialogues. In that process, we also instructed them on how we wanted each dataset labeled, indicators of different emotions, and more. They did the labeling.
Within our data, there's an algorithm that detects customer sentiments and identifies problems that need to be addressed. There are several datasets that indicate the mood or intent of the customer.

What is the team composition?

We worked with one person on development, initially. There were about 10 people overall who worked on our datasets.

How did you come to work with Label Your Data?

We sent a pilot project to potential vendors, in order to judge the quality of their work. They delivered the best quality, and we decided to work with them.

How much have you invested with them?

We spent $10,300.

What is the status of this engagement?

We worked together from June–July 2020.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement? 

They completed the work quickly. We ran some tests, checking the algorithm and its effectiveness. The data was labeled correctly, and the results were exactly what we wanted to see. We were pleased.

How did Label Your Data perform from a project management standpoint?

They were responsive and personable from the start. It was apparent by the relevant questions they asked that they were experienced in this type of work. They were available and tried to meet deadlines. We communicated via phone and email as needed. 

What did you find most impressive about them?

They possess qualities that we look for in a perfect partner. They do good quality work, understand our needs, and work using their own tools and systems. 

Are there any areas they could improve?

No, I can't think of anything. 

Do you have any advice for potential customers?

Clearly state what you want to receive and give detailed instructions at the beginning.

BACKGROUND

Please describe your company and position.

I am the CTO of SpexAI GmbH

Describe what your company does in a single sentence.

We simplify cultivation through spectral imaging.

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire Label Your Data to accomplish?

Get plant data annotated, with a high accuracy.

SOLUTION

How did you find Label Your Data?

Online Search

Why did you select Label Your Data over others?

• High ratings
• Pricing fit our budget
• Great culture fit
• Good value for cost
• Company values aligned

How many teammates from Label Your Data were assigned to this project?

6-10 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

Annotation of plant organs. After initial parts the annotation was really well done and quickly established.

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

• Actual data labels
• Reporting

Describe their project management. Did they deliver items on time? How did they respond to your needs?

All deliverables were delivered on time and with great feedback. Although the task is unusual and difficult, their annotation quality is incredible.

What was your primary form of communication with Label Your Data?

Virtual Meeting

What did you find most impressive or unique about this company?

Great quality and no upselling. Just solving the problems that I had at hand.

Are there any areas for improvement or something Label Your Data could have done differently?

Everything was done really well.