AI & API Development for Fleet Maintenance Company
Featured Review- AI Development API Development
- $200,000 to $999,999
- Dec. 2023 - Oct. 2024
- Quality
- 5.0
- Schedule
- 5.0
- Cost
- 5.0
- Willing to Refer
- 5.0
"Their ability to learn about our business and understand our goal was truly impressive."
- Automotive
- 1,001-5,000 Employees
- Online Review
- Verified
A fleet maintenance company hired NineTwoThree to automate their quality review process of review orders using AI. The team built an ML API to assess the details of each repair order.
NineTwoThree delivered a quality check API that helped the client reduce the time spent reviewing orders by the quality team. The team provided timely items, responded well to needs, took action against potential blockers, and shared constant updates on the progress and results of their work.
The client submitted this review online.
BACKGROUND
Please describe your company and position.
I am the Product Manger of Amerit Fleet Solutions
Describe what your company does in a single sentence.
Provide fleet maintenance services to large commercial fleets.
OPPORTUNITY / CHALLENGE
What specific goals or objectives did you hire NineTwoThree to accomplish?
- Determine if work completed by technician is correct.
- Create a mechanism to check the technicians work in real time.
- If the work is incorrect generate a note of what is incorrect.
SOLUTION
How did you find NineTwoThree?
Online Search
Why did you select NineTwoThree over others?
- High ratings
- Good value for cost
- Company values aligned
How many teammates from NineTwoThree were assigned to this project?
2-5 Employees
Describe the scope of work in detail. Please include a summary of key deliverables.
For the services we provide our clients we have implemented a repair order quality review. This review is manual and confirms that all information on a repair order is correct. This process is built to ensure that our customers are correctly invoiced.
We started with 923 to try to automate this process using AI. Most attributes of a repair order can be analyzed using if then logic but the technicians leave comments on the repair order that can also determine if the order is correct or not. Our original goal was to have 923 analyze the comments using AI to determine the correctness of the RO.
We followed this path and gathered data trying to complete this task before realizing our scope was too narrow. The repair order quality team does not only review comments, parts, labor, tasks to determine correctness they review the entire RO. We then decided to expand our scope to review the entire RO with the goal of determining the correctness of the RO.
In an ideal world we would have had millions of historical data points for the team to review. This was not the case due to the structure of our data that presented challenges when trying to see historical changes for the 923 team to analyze. We worked on building this historical data in real time for them to analyze but this provided a relatively small sample size.
Even with this small sample size 923 delivered on the goal of analyzing the correctness of the RO. They would not only determine the correctness of the RO but provide a score of how confident the model was in the results. They then went a step further and worked on pointing out the exact error in the repair order and what the failure reason was. We continued to go through testing delivering on our next goal of providing not only that the RO was correct or incorrect but also pointing out where and what the errors were.
By this point in our project we had determined that there would be too much risk in completely replacing the repair order quality review team. We shifted focus from replacing the repair order quality team to generating a message directly to the technician with the AI Models results. This solution, which was recommended by 923, eliminates this risk by displaying an error/quality check message to the tech that they could review or ignore. Instead of waiting until the last possible moment to check for correctness we are getting the root cause of the issue. The final deliverable was to create an API that could take the details of the Repair Order and deliver a score with error reasons for the repair order. This final piece was delivered by the 923 team.
The 923 team was fantastic to work with and taught me a lot about ML and AI. The team was patient while we worked through solutions to help them and always delivered on time. This was our first step into AI/ML and we have learned a lot to set us up in the future. Our biggest issue that 923 was able to overcome was data. We did not have the historical records to be able to supply for the model learning. Even with a small data set 923 was able to deliver great results. Rather than just taking our requirements and building something they took the time to learn our internal processes and make suggestions on how we could implement the solutions theyve developed.
RESULTS & FEEDBACK
What were the measurable outcomes from the project that demonstrate progress or success?
Determine the models accuracy - overall the model was around 75-80% accurate when comparing the models predictions to what the repair order quality team actually did.
Accuracy by cost - the model was able to determine the correct result with a high level on confidence on RO's with the a cost less than $500. Our biggest risk when replacing the repair order quality team is invoicing the customer the incorrect amount which results in either over charging the customer or under billing them and missing out on revenue. The 923 was able to demonstrate the accuracy in costs buckets and the possible dollar risk with each bucket. While we did not move forward with replacing the quality team this has set the framework to review orders at certain cost thresholds to reduce risk but still have a significant impact.
Model score - the model will generate a score for 100% of the RO's that are passed through. With this score we are able to determine if we should display the message to the end user or leave it alone. If the model is not confident in the results then we can choose not to display the result.
Failure reasons - Providing not only the score of the RO and if it was a failure but stating where the error occured and why it was generating. The 923 delivers this on every RO that is passed through to them. Comparing with notes left by the quality team we have achieved an almost exact match and caught errors they have missed. By producing these failure reasons this is a significant feature when generating the quality check message to technicians. Rather than simply stating something is wrong, double check your work (which they would likely ignore) we are able to generate a check that say X thing is wrong with this section of the order.
Quality check API - By developing this API we now have a mechanism to get results from the model and share them with the end user. In our case this will be when the technician completes a job we will pass the order details to the API to return the score and results. Implementing this mechanism will have a siginificant impact on the overall quality of the RO's because we are implementing a check at the source. In our current state when a technician completes an RO it can take the quality team 6-8+ hours to review the order. This check is now happening in seconds while the work is still fresh in the technicians mind. If the quality team finds an error they will flag it and send back to the technician who will correct, which can take an additional 4-6 hours. With this check happening at the time of completion we should see an overall decrease in time spent reviewing orders by the quality team, time to detect errors, and reduce time needed to complete and invoice the customer.
Describe their project management. Did they deliver items on time? How did they respond to your needs?
The 923 team always met timelines for deliverables and quickly responded to my teams needs. Because they took the time to get to know our business and learn our process they were not only able to adjust to changes in scope but were also able to recommend ways to implement solutions.
They set up our weekly touchpoints and were always available to schedule additional time as needed. They shared constant updates on the progress and results of their work. One of my favorite things about them was their ability to share the results of their work in a clear way that could be understood even if you are unfamiliar with AI/ML. This was my teams first time working with AI/ML and they were able to eliminate any confusion we had on the process/results which can be very complex.
The team always escalated potential blockers early on the process to give my team time to action. If we were delayed in actioning these blockers they continued to work and provided creative short term solutions while our plan was implemented. I greatly appreciate their commitment to the project and the way they were able to adjust on the fly when needed to deliver the best product. They are committed to providing the best experience to their customers and have always delivered on time.
What was your primary form of communication with NineTwoThree?
- Virtual Meeting
- Email or Messaging App
What did you find most impressive or unique about this company?
Their ability to learn about our business and understand our goal was truly impressive. Going from knowing nothing about a company and the way they work to delivering the results they did was impressive.
As I have mentioned this was our first time working with AI/ML so we really didnt know what was needed or what to expect. This was epsecially true in how our data was structured. What I found to be unique about 923 was their willingness to work with us while we built our historical data. It started with manual data queries and uploading to a shared filed before automating this process. Even during the early days of the project they were able to deliver fantastic results and learn more about our business. Because of the process we implemented with 923 we have set ourselves for future work in the AI space.
The final thing I was impressed with is their communication and ability to summarize results. We were constantly updated on their progress. They were able to provide great detail on how they acheived the results without causing confusion and answered any questions we had clearly. The AI/ML world an be overwhelming and complex but when talking through it with them I felt I had all the information necessary to make a decision and provide direction.
Are there any areas for improvement or something NineTwoThree could have done differently?
No the only problem we faced during our project was data. Mainly that we had not set up our data to track historical changes over time making it difficult to analyze. We have learned so much from this team and our currently working on projects to improve our data for future work with AI. Would love to work with this team again.
RATINGS
-
Quality
5.0Service & Deliverables
"Delivered great results and clearly defined expectations."
-
Schedule
5.0On time / deadlines
"Always delivered on time."
-
Cost
5.0Value / within estimates
"Achieved the results on time and within our budget. Everything clearly stated in the SOW including cost breakdown and deliverables."
-
Willing to Refer
5.0NPS
"They provide clear communications on the status of each deliverable and provide progress updates throughout the project.