Dataeaze Systems

BACKGROUND

Please describe your company and your position there.

My company is an edtech platform, it is helping 3000+ schools across india to get digital upgradation. With around million+ students onboarded, there is a lot of data that needs to be managed at scale. I lead data engineering practice, our team works in building data automation and enable business with single central source of truth for data warehouse

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Dataeaze Systems, and what were your goals?

In our vision of building modern data platform, we hired dataeaze for their expertise in building modern data stack on AWS cloud. For building data engineering automation end to end

SOLUTION

How did you select this vendor and what were the deciding factors?

Dataeaze earlier experience building similar data platform, tech experts we talked to who were onboarded for building data platform These were the deciding factors

Describe the scope of work in detail, including the project steps, key deliverables, and technologies used.

Project involved building a data consolidation layer where data comes from various rdbms and nosql databases. There are distinct systems handling data about students, quizes, assignments, schools etc. This was to be onboarded as part one phase There was an earlier system in place which was required to be migrated to modern cloud data platform, that was one of phase Building a configurable spark based data migration framework was another stage of project

How many people from the vendor's team worked with you, and what were their positions?

2 to 4 resources worked at various stages. With positions:

• Principal data engineer
• Senior data engineer
• Data engineer

RESULTS & FEEDBACK

Can you share any measurable outcomes of the project or general feedback about the deliverables?

• Cloud based data lake and data pipelines are in place and are getting developed
• Framework is in place for smooth configuration driven data movement

Describe their project management style, including communication tools and timeliness.

We follow agile methodology & Jira to track

What did you find most impressive or unique about this company?

They have good talents in the resource pool and we were able to get talents as and when needed for short and long term assignments

Are there any areas for improvement or something they could have done differently?

.

BACKGROUND

Please describe your company and your position there.

I lead as Data Management and Engineering Head of a leading fibre optics manufacturing company with annual revenue in range of 800 million USD. I handle a team of Data Engineers and Cloud Engineers. We have created and manage a Data Lake on GCP since 2018/19

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Dataeaze Systems?

We have data platform built on Google Compute Platform. It involves complex data movement automation to consolidate data across various sources across organisation. Dataeaze has provided us team of experts who are implementing, maintaining and enhancing data engineering automation and data platform.

SOLUTION

How did you select this agency and what were the deciding factors?

We followed a Decision cycle which is an internal process to select the right vendor. We looked at their expertise in the area, projects and their performance in solving a particular business problem. Shortlisted vendors were given sample solutions to be developed and final selection was done based on their performance in all stages.

Describe the project in detail and walk through the stages of the project.

Project involves building centralised Data Lake on google compute storage. As well as do ELT data movement to BigQuery. Post that various analytics business needs are served from same. Stages involved are : Setting up base level infrastructure and enable it for first set of business use case. Second stage involves onboarding more data from other sources and enable analytics.

Describe the recruitment process in detail.

We discussed the resource requirements with the CEO of the company and worked with him and the leads through the POC phase. After the final decision were taken and contracts were signed the leads and additional Data Engineers were placed in charge of the projects.

RESULTS & FEEDBACK

Can you share any outcomes from the engagement that demonstrate progress or success?

For business teams and analytics teams now it becomes possible to give very quick turnaround time as data is timely refreshed in Data Warehouse. Multiple analytics use cases turnaround time has reduced drastically with ready available data in data warehouse

How effective was the workflow between your team and theirs?

We work on a very well-coordinated model with Dataeaze Systems. We run our projects on Scrum mode with Weekly delivery schedules. There are close interactions with all team members every day with active planning and execution of project deliverables.

What did you find most impressive or unique about this company?

They are able to provide quality resources at very reasonable costs. The resources themselves are able to work extremely well in an agile project delivery model. They also provide the necessary skills for various project challenges from infrastructure, ELT and analytics support.

Are there any areas for improvement or something they could have done differently?

I would like them to develop a very good training model for the resources to ensure continuous learning. This will help them to keep their skills constantly updated and ready for meeting new challenges. They have proven themselves in their technical abilities, now they can develop further in domain knowledge and consulting activities.

BACKGROUND

Please describe your company and your position there.

Company works in business of consumer finance, housing finance and corporate lending. I lead data Data and Analytics practice at this organisation as Senior Manager Data and Analytics.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Dataeaze Systems?

For building near real time data replication pipeline from source LMS, LOS system to data warehouse. Dataeaze is going to own overall support and maintenance of current data platform stack

SOLUTION

How did you select this vendor and what were the deciding factors?

Dataeaze is company having focussed services offering around data only. We reviewed dataeaze capabilities and experties. They performed a POC Demo and showcased capability for near real time data replication stack based on modern AWS based data stack.

Describe the project in detail and walk through the stages of the project.

Project involves 3 steps, - Immediate need : Near real time replication setup - Maintain existing data pipelines for batch and newly built realtime for longer duration - Build further roadmap for a modern data platform

How many resources from the vendor's team worked with you, and what were their positions?

5 Data Architect, Senior Data Engineer, Data Engineers

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

One of the primary data sources at our organization was getting replicated to the data warehouse once daily with an earlier data pipeline setup. With the first phase completion, it started getting replicated near real-time. This enabled dashboards and reports with a quick refresh interval

How effective was the workflow between your team and theirs?

It was well coordinated

What did you find most impressive or unique about this company?

We are impressed with their proactiveness and professionalism.

Are there any areas for improvement or something they could have done differently?

Should continue to improve and be proavtive and provide continuous visibility to client of the work progress

BACKGROUND

Introduce your business and what you do there.

I’m the founder of a marketing strategy company that works with tech companies to design their go-to-market campaigns.

OPPORTUNITY / CHALLENGE

What challenge were you trying to address with Dataeaze Systems?

We were looking at building a data crawler to extract information from various portals, which required an engine that would scrape websites, extract data from there, and then run AI libraries on top of that raw data to generate insight from that information.

SOLUTION

What was the scope of their involvement?

The range of services that they provided was around Python development on the data crawling activity and working on the database side to store all the data in the backend, which was created using MongoDB. They also used UML tools to extract information and intelligence from that data.

What is the team composition?

There was a project manager and architect involved because this was a very heavy research and development project. There was a team of one or two developers. Our main interaction was with the architect and some of the developers.

How did you come to work with Dataeaze Systems?

We reviewed quite a few sites. We were looking for a company that had strong data engineering capabilities and AI capabilities. We also wanted someone that could adapt to working with us as a small to medium-sized company as we are.

It was a very complex domain to work with, not necessarily the scope as it’s naturally large, but it certainly required strong technical expertise to be able to deliver it. Someone had recommended Dataeaze Systems to us, and we started working with them.

How much have you invested with them?

In total, we’ve invested around $40,000.

What is the status of this engagement?

We started this project in November 2019, and it was completed in June 2020. From the support side, it’s an ongoing engagement.

RESULTS & FEEDBACK

What evidence can you share that demonstrates the impact of the engagement?

This is a very niche area in which we’re operating, and there are a lot of unexplored things that we could do. We were also working on a very limited budget, but overall, I would rate them a four out of five in terms of the quality of work and deliverables. They also understood the tools we were working with.

We keep adding new sites and more AI in the library.

How did Dataeaze Systems perform from a project management standpoint?

In terms of communication, they were excellent. Because of the research-oriented nature of the work, we weren’t very clear on timelines.

What did you find most impressive about them?

What sets them apart is their strong focus on the data engineering side. They have certain solution accelerators that help companies like us rate development sites.

Are there any areas they could improve?

I think cost always, for companies like us, is an issue. It is something that, if it can be done at a lower cost, would always help an organization like us.

Do you have any advice for potential customers?

Invest the maximum time and effort during the initial architecture design for the project to become seamless.

Be clear on what business objectives you have and incorporate them much earlier in the overall development phase, rather than reaching out to them later. If someone isn’t 100% on the technical side, collaborating with them at an advisory level is something I recommend.

BACKGROUND

Please describe your company and your position there.

As the CTO my role was to oversee all of software development for Software Division. It was a role that managed teams spread across 2 locations.

We are an industry-leading integrator of digital networks engaged in design and integration of digital networks. We support global telecom companies, cloud companies, citizen networks and large enterprises to deliver solutions for their fixed and wireless networks for current and future needs.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Dataeaze Systems, and what were your goals?

Goal was to develop a first of its kind, Computer Vision system which is able to do anomaly detection on sensor data to identify these patterns in near real time and at a massive scale of ingesting millions of data points per second.

We understood the fiber optic side and the sensor side of the solution but were looking for a partner to help us with the data ingestion, data management and data analysis with AI-ML to help us build the complete product.

SOLUTION

How did you select Dataeaze Systems and what were the deciding factors?

We ran thru a selection process looked at 3 vendors etc. Dataeaze was selected based on the capabilities they demonstrated in the POC which they performed well.

Dataeaze was selected based on the vision and plan they had for the product development post POC Dataeaze was selected due to the knowledge of the team members who displayed and discussed the POC Dataeaze CEO and Lead people are knowledgeable. Dataeaze was flexible in their approach to partner with us.

Describe the scope of work in detail, including the project steps, key deliverables, and technologies used.

Protoype / POC Agile releases 0.1, 0.2, 0.3, 0.4, 0.5 and then 1.0 and 1.1 HDFS, Kafka, Java, Python etc.

How many people from the vendor's team worked with you, and what were their positions?

About 7 to 10 persons were engaged at various points in time ... with the roles as below

• Data Architect: Responsible for Designing Data pipeline
• Sr. Data Scientist: To Stitch the algorithm and perform model building and evaluation
• Machine Learning Engineer: To develop the model
• Data Engineer: Analyze the Data
• Support Engineer: Provide L1 & l2 support during development and Post go live

RESULTS & FEEDBACK

Can you share any measurable outcomes of the project or general feedback about the deliverables?

Product was delivered in reasonable amount of time . Pivots and changes were handled responsibly Commitment to make the solution work. Many unknowns in the solution were fixed by Dataeaze. Overall 8.5/10

Describe their project management style, including communication tools and timelines.

Agile / daily stand-up which was a good thing. Reporting on a weekly basis Hybrid project management approach. Are very transparent with things where they stand. 

What did you find most impressive or unique about this company?

Turnaround Time for experiments New ideas and implementing them Estimation techniques and transparency

Are there any areas for improvement or something they could have done differently?

Are very transparent with things where they stand. But then not proactive to share the information - but when asked are very transparent .. so from 3 rd week onward I started asking deep questions.

BACKGROUND

Please describe your company and your position there.

I am consulting CTO for Anand Rathi Wealth Services and we provide wealth solutions to clients in the Ultra HNI and HNI space. All our systems are hosted on AWS and portfolio analytics is a key offering to our clients.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Dataeaze Systems?

We hired the Dataeaze team to implement a big data processing pipeline using AWS EMR, Spark and mySQL. The tasks involved data ingestion from various operational data stores, aggregation and computation and creating data marts for analytics and BI.

What were your goals for this project?

Provide clients real time view of their portfolios with upto mark to market values, performance nos across various dimension - portfolio level, client level and family level. Similarly, the RM managing the client should be able to use the analytics to generate recommendations for portfolio action.

SOLUTION

How did you select Dataeaze Systems?

We selected Dataeaze based on the solution they had proposed in the pre-sales cycle. Moreever they had a strong track record of doing similar work in the past and the team came across as highly skilled and techno functional experts.

Describe the project in detail.

We wanted to implement real-time data ingestion from our operational databases to compute the performance of all the client and RM portfolios, mark to market and aggregate the AUM at various levels for analytics; AWS EMR was the data platform with underlying HDFS and Spark based processing and the final output was directed into mySQL databases for reporting, BI and Analytics.

What was the team composition?

The Dateaze team comprised of 1 solution architect, 2 data engineers and 1 BI developer.

RESULTS & FEEDBACK

Can you share any outcomes from the project that demonstrate progress or success?

The entire solution stack is running in production. we are running dynamic EMR instances at 1/10th the earlier infra cost, our processing speed has increased more than 1000% largely because of the ability to spin as many containers as needed to process large chunks of data in parallel and updated portfolios are available to customers before start of business day at 9am IST. Earlier portfolios would get updated late afternoon - which seriously compromised customer experience.

How effective was the workflow between your team and theirs?

All workflow was managed through JIRA boards and sprints. we run 2-3 week sprints and the Dataeaze team had the required tasks, they attend the daily standups and all the SCRUM ceremonies. all work communication happens over Slack and all interactions over Zoom. So overall, it was very structured and smooth.

What did you find most impressive about this company?

Its a small boutique data engineering company, highly professional and structured in their approach, their expertise in what they do and their commitment to scope. The quality of their work and timeliness has been very good.

Are there any areas for improvement?

They should scale up their team.

BACKGROUND

Please describe your company and position.

I am the CEO & Co-founder of Affirmo Technology

Describe what your company does in a single sentence.

Affirmo Technology provides solutions using AI, Data, Analytics and IoT for companies in Sustainability and Digital Transformation journeys.

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire Dataeaze Systems to accomplish?

• Data Management

SOLUTION

How did you find Dataeaze Systems?

Referral

Why did you select Dataeaze Systems over others?

• Great culture fit
• Good value for cost
• Company values aligned
• reliability

How many teammates from Dataeaze Systems were assigned to this project?

6-10 Employees

Describe the scope of work in detail. Please include a summary of key deliverables.

Ingest real-time IoT data for discovery of patterns to optimize operations and efficiency. Delivered:

• Data mart 
• Data organization 
• Dashboards and reporting 
• Real-time monitoring
• Data governance

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

Project was delivered meeting project parameters, on-time and on-budget.

Describe their project management. Did they deliver items on time? How did they respond to your needs?

They were experienced, reliable, professional, and trustworthy.

What was your primary form of communication with Dataeaze Systems?

Virtual Meeting

What did you find most impressive or unique about this company?

Has good technical skills and reliable people.

Are there any areas for improvement or something Dataeaze Systems could have done differently?

None.

BACKGROUND

Please describe your company and position.

I am the Director of Research of a software automation and automation platform.

Describe what your company does in a single sentence.

Uniphore develops software analytics and automation with conversational AI

OPPORTUNITY / CHALLENGE

What specific goals or objectives did you hire Dataeaze Systems to accomplish?

• Operationalising data processing functions during AI model building

SOLUTION

How did you find Dataeaze Systems?

Referral, Other

Why did you select Dataeaze Systems over others?

Referred to me

What was the size of Dataeaze Systems’s team?

2-5 Employees

What was your primary form of communication with Dataeaze Systems?

Virtual Meeting

Describe the scope of work in detail. Please include a summary of key deliverables.

The project involves developing data management activities including data versioning, cloud based storage of heterogenous datasets and data automation.

RESULTS & FEEDBACK

What were the measurable outcomes from the project that demonstrate progress or success?

• Successful deployment of data management platform
• Successful enablement of the platform towards adoption

Describe their project management. Did they deliver items on time? How did they respond to your needs?

The team achieved overall project goals in a timely manner with the periodic release of data management capabilities by following an mutually agreed plan Accommodating change in execution of plans and implementing key feature requests

What did you find most impressive or unique about this company?

High professional standards and commitment to achieving project goals.

Are there any areas for improvement or something Dataeaze Systems could have done differently?

Continue their focus on commitment to project goals and software quality.

BACKGROUND

Please describe your company and your position there.

Working for a multinational system integration based out in Japan. I am responsible for expanding our OpCos technical capabilities in the region in distributed computing, AI, etc.

OPPORTUNITY / CHALLENGE

For what projects/services did your company hire Dataeaze Systems, and what were your goals?

To create a data analytics and visualization system using AI and distributed computing tools. Seamless integration of AI technologies and distributed computing technologies while automating most of the tasks were objectives.

SOLUTION

How did you select Dataeaze Systems and what were the deciding factors?

Their past experiences and eagerness to try new things including the great people working at Dataeaze.

Describe the scope of work in detail, including the project steps, key deliverables, and technologies used.

Requirement gathering to Development. Ansible, Docker, Apache Spark, Apache Hadoop, Delta Lake, MLflow.

How many people from the vendor's team worked with you, and what were their positions?

5 members: 3 engineers and 2 senior executives.

RESULTS & FEEDBACK

Can you share any measurable outcomes of the project or general feedback about the deliverables?

We decided to continue to work with Dataeaze in other projects as their work is well above the standards.

Describe their project management style, including communication tools and timelines.

Used email based communication in the project. They communicated relevant matters in time allowing us to finish the project as planned.

What did you find most impressive or unique about this company?

We were impressed by their understanding of multiple technologies and coming up with multiple solutions for a problem.

Are there any areas for improvement or something they could have done differently?

Nothing particular.