UNIOUN

BACKGROUND

Please describe your company and position.

I am the CEO of Andala

Describe what your company does in a single sentence.

We build conversational customer agents

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire UNIOUN to accomplish?

• LLM Fine Tuning

SOLUTION

How did you find UNIOUN?

• Online Search
• Referral
• Clutch Site
• linkedin

Why did you select UNIOUN over others?

• Pricing fit our budget
• Good value for cost
• Referred to me
• Company values aligned

How many teammates from UNIOUN were assigned to this project?

2-5 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

fine-tuning a large language model (LLM) to create a conversational AI tailored for a specific use case customer support for industrial supplychain. The goal is to deliver a scalable, accurate, and context-aware AI with comprehensive documentation.

Deliverables:

Month 1: Requirements document, curated dataset (10,000+ samples), model selection report.
Months 2-4: Fine-tuned model, training scripts, 3 model checkpoints, evaluation report (BLEU, ROUGE, human scores).
Month 5: Test suite (1,000+ scenarios), validation report, bias analysis, user acceptance testing plan.
Month 6: Deployable model, deployment guide, user manual, final report, 2-hour knowledge transfer session.

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

Measurable Outcomes

Response Accuracy:

Expectation: ≥85% accuracy in addressing customer queries (e.g., order tracking, inventory, delivery schedules).
Outcome: The AI achieved 87% accuracy across 1,000+ test scenarios, verified through automated precision/recall metrics and human evaluation of 500 interactions, exceeding our target.

User Satisfaction:

Expectation: ≥4/5 user satisfaction score in user acceptance testing (UAT).
Outcome: UAT with 60 supply chain staff yielded an average score of 4.2/5, with users praising the AI’s clarity and relevance in handling procurement and logistics queries.

Response Latency:

Expectation: 95% of responses delivered in <2 seconds in a production-like environment.
Outcome: The AI delivered 96% of responses within 1.8 seconds during stress testing with 1,000 concurrent queries, meeting our performance requirements.

Query Resolution Rate:

Expectation: ≥80% of queries resolved without human escalation.
Outcome: The AI resolved 83% of 1,000+ logged queries independently, reducing reliance on human agents and aligning with our efficiency goals.

Context Retention:

Expectation: ≥90% accuracy in context-aware responses over 5+ turn dialogues.
Outcome: The AI maintained 91% context accuracy in 200 multi-turn dialogue samples, ensuring coherent interactions for complex supply chain inquiries.

Bias Mitigation:

Expectation: Biased responses reduced to <5% of interactions.
Outcome: Bias analysis of 500 interactions showed only 3% exhibited potential bias, with mitigation strategies implemented to ensure fair and neutral responses.

Operational Efficiency:

Expectation: 30% reduction in query handling time compared to human-only support (baseline: 5 minutes/query).
Outcome: The AI reduced average handling time to 3.2 minutes per query across 1,000 cases, achieving a 36% reduction, surpassing our target.

Team Unioun successfully delivered a conversational AI that met or exceeded all specified outcomes. The system has enhanced our customer support operations, reducing response times and human workload while maintaining high accuracy and user satisfaction. The final evaluation report and knowledge transfer session provided clear insights into the AI’s performance and maintenance, ensuring seamless integration into our operations.

Describe their project management. Did they deliver items on time? How did they respond to your needs?

Sprint Structure:

Six 4-Week Sprints: The project was divided into six sprints, each aligned with key phases (requirements, data prep, fine-tuning, testing, deployment).
Sprint Deliverables:Sprint 1: Requirements document, dataset (10,000+ samples), model selection.
Sprints 2-4: Fine-tuned model iterations, training scripts, checkpoints, evaluation metrics (87% accuracy achieved).
Sprint 5: Test suite (1,000+ scenarios), validation report (91% context retention), bias analysis (<3% bias).
Sprint 6: Deployable model, deployment guide, user manual, knowledge transfer session.

Project Management:

Methodology: Unioun used an Agile framework with bi-weekly sprint reviews and daily stand-ups, ensuring transparency.
Tools: Jira for task tracking, Slack for communication, and Git for version control.
Performance: Unioun met all milestones on time, with clear documentation and proactive issue resolution (e.g., data quality challenges resolved in Sprint 1).
Collaboration: Regular client check-ins and a final stakeholder presentation ensured alignment, achieving 4.2/5 user satisfaction and 36% query time reduction.

Unioun’s structured sprints and effective Agile management delivered a high-quality conversational AI, meeting our expectations for performance and operational efficiency.

What was your primary form of communication with UNIOUN?

• In-Person Meeting
• Virtual Meeting
• Email or Messaging App

What did you find most impressive or unique about this company?

The team and their understanding about the dynamic project structure and fast agile deployments really helped us get on track with our launches.

Are there any areas for improvement or something UNIOUN could have done differently?

The Unioun team should also start providing hosting and MLOps services so that it would be easier for me to expand and scale my project further , although the owner has promised to expand in the ML/AIOps domain for my specific project.