Innovate · Build · Repeat
Datum Brain was founded in 2017 and is now a global IT service provider. Over the past years, we have developed a professional team of software developers, data scientists, and designers who excel in their domain.
We provide software development, design services, and business consultancy to enterprises operating locally and globally.
We have mastered the art of business alleviation by creating bespoke software solutions to achieve goals. From supporting business ideas to creation and implementation we acutely monitor every step by taking responsibility for the excellent end-user experience.
Doing IT right is what we do!
Recommended Providers
Focus
Portfolio
Parcelist, Polymer DLP, Examity Inc., PSI Services LLC, Functionize Inc., ReportBrain, Parsley
Nike’s Big Data
We worked on creating a recommendation engine for our client that grabbed the behavioral approach of the user in order to show and engage the user to use their platform to increase sales.
As we were working on this project, we encountered some techniques that were used the wrong way in order to ingest 'Big Data'. The problem was that it took over 22 hours to run over the cluster of 15 nodes in Azure. This consequently cost thousands of dollars to the client and which was obviously not suitable and wasn't a good practice for sure!
We steered our efforts towards minimizing the processing time. End result? 22 hours dropped to mere 6 minutes resulting in a huge cost saving! Another notable thing is, It wasn’t actually a client’s requirement, we had to do this by managing within our client-communicated time frame along with the tasks assigned by the client.
We used hashmaps and simple conventional dynamic programming algorithms in order to save cloud service bills.
It was learned that not every kind of data needs a solution for scalability, just thinking of an optimal code can save the day.
Technologies:
Microsoft Azure, Apache Spark, Apache Zeppelin, Scala
Scalable Web Crawler
The client hired us to develop a customized web crawler that queried data out of several web pages. Every website had different data representation methods but regardless of that, we managed to develop a Web Crawler that efficiently extracted the data for client’s readily usage.
It is a Scalable Web Crawler that scales up by adding a node horizontally and this makes it stand out from other web crawlers. Moreover, It runs on top of Apache Spark leveraging SpookyStuff, which is one of the fastest Big Data collection engines in history, with a speed record of querying 330404 dynamic pages per hour on 300 cores.
Technologies: Apache Spark, SpookyStuff, Scala, Zeppelin, Apache Nutch
Web Driver, A Human-like Crawler
We developed an intelligent Anti-Bot Crawler inspired by human behavior. This Human-Like Web Crawler is able to solve all sorts of tricky riddles and even custom questions.
One of its unique features is that it rotates the IP when a website blocks it due to security constraints. We programmed this Anti-Bot crawler in a way that it opts for a passive approach when the website detects that the crawling process is recurring.
Technologies: Scala, Apache NiFi, Spark, Zio-spark
Video Session Recording Service
A system was developed for our client to record the activities undertaken during test sessions for close monitoring and assessment. This service facilitates the proctor to inspect every statistical aspect of the test sessions being held to identify lagging areas. This keen invigilation helps the technical teams to resolve the issue quickly. Eventually, the final composition can be accessed later for auditing purposes.
Technologies: Go, Angular, JavaScript (for SDK), NATS, Redis, DynamoDB, TwilioS3
Reservations & Scheduling as a Service
Datum Brain collaborated with PSI as their software development team to develop a remote scrutinized system. We developed reservations and scheduling systems with a broader vision so they can be well utilized globally by businesses for their relevant use cases.
PSI uses this system for administrative purposes to manage its exam proctoring process. The core features of this system include,
-
Finding available locations
-
Selection of the slots (Scheduling)
-
Modification of the booked slots (Re-scheduling)
We also conducted the development of APIs to expose the core functionality of the system to be programmatically accessible by clients. This was done purposely so clients can integrate their own systems into this well-constructed software without varying interfaces and shifting focus.
Technologies: Go, AWS Lambda, Atlas
Polymer DLP—Enterprise App
With huge data floating over the wire, compliance becomes the top priority for organizations around the globe. Data security is a delicate matter to deal with when businesses need to share data within or outside the firm. A manual approach for reviewing data before sharing it with other entities is indeed agonizing. We tackled the problem for our client by developing this application to handle this data chaos.
Polymer DLP simplifies the tedious process by allowing organizations
-
To easily define what they classify as critical data
-
To define the reaction when something critical is found (warn/redact) in shared sources (text, Docx, CSV, pdf, images, etc) across SaaS apps (Slack, Github, Google Drive, Zendesk, etc)
-
To analyze and understand the sharing of sensitive data among the organization
Technologies:
Apache Spark, Scala, StanfordNLP, OpenNLP, Regex
Grouper App
With Ardent Growth, a B2B focused content marketing firm which takes pride in taking complete responsibility for the client’s revenue streams.
Datum Brain, as a technical partner, built a system where making content strategies for revenue generation became a breeze and it wasn’t guesswork anymore.
The system favors Total Addressable Market (TAM) gap analysis, to identify content gaps instead of competitor analysis. The reason for not choosing competitor analysis was that no difference would exist and we’ll be on the same surface as them. So, we chose TAM Gap Analysis to stay at the top and provide a real candidateship for our client in the market.
We then also devised and implemented an algorithm that smartly identifies content cannibalization and prevents it by providing data-driven direction. In this manner, our client’s content ranks higher resulting in increased, consistent, and reliable revenue streams.
Technologies: Python, Celery, SERP API
Mecku
The project included major assignment of carrying out ETLs while dealing with multiple sources of data. ETL is often considered to be a daunting task but we have simplified the overall process by developing an efficient ETL tool, Mecku.
This potent ETL tool is capable of carrying out the extraction of data from sources, applying transformations, and eventually loading them to relevant destinations. Data sources and data destinations can be of different natures like S3, Postgres, MySQL, and many other kinds. After the configuration of the data sources software activities can be performed easily.
We also made sure to develop an incredible user experience that included dragging and dropping nodes for user ease. This just not defined sources, transformation, and destinations but also helped to explain ETL flow through a connection.
Technologies:Apache Spark, Play Framework, Scala
Parcelist
We designed and integrated a whole system for our client ‘Parcelist’ to aid their delivery services by developing a cloud-based web application along with an interactive website.
The whole idea of safe and convenient package delivery was practically implemented by us which facilitated the end-user. The developed application is enriched with unconventional features and extraordinary design.
Team ‘Parcelist’ also wanted us to establish a cloud-based web app along with their website. We ensured that all the development steps undergo critical checks so the end product is enriched with quality features and our client receives maximum benefit.
Hence, a fully automated system for smart deliveries was developed and delivered to our client.
Technologies: Play Framework, Scala, Amazon ElasticBean
Reviews
the project
Scheduling System Development for Testing Solutions Company
"They clearly understood our issues and were able to quickly turnaround all of our requests."
the reviewer
the review
The client submitted this review online.
Please describe your company and your position there.
I'm the VP of Software Development at Testing Solutions Company
For what projects/services did your company hire Datum Brain, and what were your goals?
We needed a new scheduling service that utilized AWS serverless technology.
How did you select this vendor and what were the deciding factors?
We found them through Up Work. We were looking for Go developers that knew AWS lambdas and could trust to work independently.
Describe the scope of work in detail, including the project steps, key deliverables, and technologies used.
The designed and developed several new AWS services for us
- internal scheduling system
- a custom api for a large client that allowed tight integration with their system and a general use api for clients that have their own IT staffs. All 3 services are AWS server-less technology using Golang.
How many people from the vendor's team worked with you, and what were their positions?
I worked primarily with 2 people.
- Fahad the owner and
- Faison the lead developer.
Can you share any measurable outcomes of the project or general feedback about the deliverables?
We were able to launch all the projects on time or early. Within the first three weeks of each launch Fahad and group were available during US hours which helped with support. I am happy to say that we rarely ever have production support concerning their 3 services.
Describe their project management style, including communication tools and timeliness.
The project management was great. They regularly met deadlines, and we were always very responsive when changes inevitably occurred. We communicated through Teams and Emails.
What did you find most impressive or unique about this company?
We had direct access to Fahad and his lead developer. They clearly understood our issues and were able to quickly turnaround all of our requests.
Are there any areas for improvement or something they could have done differently?
Working with them was great any areas of improvement would be nitpicking.
the project
Custom Software Development for Marketing Consultancy Firm
“It’s nice to have a team that understands what we’re trying to accomplish.”
the reviewer
the review
A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.
Introduce your business and what you do there.
I’m the CEO of Ardent Growth, a marketing consultancy firm.
What challenge were you trying to address with Datum Brain?
We developed an algorithm and needed some engineers to help us with software architecture, production, and deployment to turn it into a custom web app.
What was the scope of their involvement?
Datum Brain handles our development needs. The technologies they use include Python, JavaScript, and various APIs. The frontend development is done through React, while the deployment is on Snowflake. It’s connected to BigQuery, Google Cloud, and other Google services.
The app has a prototype, but the Datum Brain team has done the heavy lifting for it. In terms of features, we’ve built the product for internal use. We do research on our end and feed the data into the product, which processes said data and finds matching information. Then, we analyze the output it gives us.
So far, Datum Brain continues to provide ongoing maintenance for the app through sprints whenever we need bug fixes, updates, or new features.
What is the team composition?
Two core people work with us, but the team comprises around six people whenever we do our sprint standups. My most direct contact is with their CEO, a Python developer, and a project manager.
How much have you invested with them?
We’ve invested between $120,000–$250,000.
What is the status of this engagement?
We started the partnership in January 2021, and it’s ongoing.
What evidence can you share that demonstrates the impact of the engagement?
Our company has reduced the time needed for our core processes from eight weeks to about a week.
How did Datum Brain perform from a project management standpoint?
Datum Brain meets our time and budget requirements. In terms of tools, we use Asana on our end, while they use Trello on theirs. I manage the product team in Asana, and they take the tasks to Trello for engineering. To communicate, we primarily use Slack, Zoom, and emails.
What did you find most impressive about them?
They’re great at thinking of use cases and scenarios I may not have considered whenever we’re looking into new features or mapping out ideas. This saves us a ton of time. It’s nice to have a team that understands what we’re trying to accomplish and provides feedback to us throughout the development lifecycle.
Are there any areas they could improve?
No, there aren’t any.
Do you have any advice for potential customers?
Have good documentation about your needs. Provide them with as many details as possible, and have regular standups. Make sure you know what you want whenever you engage Datum Brain.
the project
Custom Software Dev for E-Learning Company
"The team has delivered on time and exceeded expectations, delivering quality and functionality."
the reviewer
the review
The client submitted this review online.
Please describe your company and your position there.
My company provides proctoring services for the education, professional accreditation and certification marketplaces.
For what projects/services did your company hire Datum Brain, and what were your goals?
Our existing video platform was a bit old and not modernized to take full advantage of WebRTC capabilities for monitoring and responding to events to ensure a quality session.
How did you select Datum Brain and what were the deciding factors?
I had hired Datum Brain at a previous company and found their work to be superb. Their team engages like a true extension to my core teams, providing options and top-notch code with on time deliveries.
Describe the scope of work in detail, including the project steps, key deliverables, and technologies used.
The work entailed created a microservice to manage the life cycle of video recording rooms, from inception to managing deleting the assets when the SLAs warranted. The service monitors the health of the room, automatically adjusts recordings to the changing Internet conditions, and monitors the activities of people in the session
. A web SDK was also included for communicating with the backend, as well as a reporting/administrative internal website.
How many people from the Datum Brain team worked with you, and what were their positions?
There were 4-6 people at any given time. Fahad (owner/manager) lead the development team and there was mix of backend/front end developers.
Can you share any measurable outcomes of the project or general feedback about the deliverables?
The solution provided is currently being retrofitted into the existing platform. The real-time monitoring capabilities are a long awaited capability that we believe will reduce support calls by at least 50% and increase user satisfaction with their experience.
Describe their project management style, including communication tools and timeliness.
We communicated daily over Zoom to track progress and work through any open issues. The team has delivered on time and exceeded expectations, delivering quality and functionality -- it was very obvious that they invested in the project as truly their own, not just a job for a customer -- the results were excellent.
What did you find most impressive or unique about this company?
The team consistently rises to exceed expectations -- they listen to the requirements and come back with fantastic feedback for improvements and possibilities -- they truly engage as a partner in trying to produce the best possible solutions. Communications were excellent, never any negative surprises and consistent quality.
Are there any areas for improvement or something they could have done differently?
As with all development teams (internal as well as external), there can never be enough code comments and additional automated tests. For the most part, the work was high quality.
the project
Web Crawler Dev for SaaS Provider
"We’re very happy working with them and are going to hire them for another contract."
the reviewer
the review
A Clutch analyst personally interviewed this client over the phone. Below is an edited transcript.
Introduce your business and what you do there.
I’m the solution architect for ReportBrain, a SaaS provider.
What challenge were you trying to address with Datum Brain?
We needed to build a crawler for one of our client’s websites. We hired someone, and the crawler wasn’t scaling well. It took too much time to crawl so we needed another partner who could develop a distributed web crawler.
What was the scope of their involvement?
They developed an Apache Spark-based web crawler from scratch.
What is the team composition?
Fahad (Co-Founder, Datum Brain) worked on the project.
How did you come to work with Datum Brain?
Our founder found him on Upwork.
How much have you invested with them?
We spent $2,000 with them.
What is the status of this engagement?
We worked together from December 2019 until January 2020.
What evidence can you share that demonstrates the impact of the engagement?
The crawler is part of a much bigger data pipeline where we needed to crawl the website, get the data, process it, expose it, and send it to our customer so they can see what had changed in his data. Without the crawler, we couldn’t build inside this pipeline.
Datum Brain developed a wonderful distributed web crawler that scales how we wanted and has all the features we wanted. We’re very happy working with them and are going to hire them for another contract. The finished solution is reliable and functional.
How did Datum Brain perform from a project management standpoint?
Project management was good. Fahad kept me informed whether things were going well or not. If any issues came up that could affect delivery time, he quickly communicated it. They delivered on time and budget.
What did you find most impressive about them?
When we first hired them, I clearly remember Fahad telling me he has a plan A, plan B, plan C, and plan D, and that I didn’t have to worry. He applied Plan B and it worked well.
Are there any areas they could improve?
He could communicate more. Most of the time, he gave us enough information to see how things were going. Sometimes I had to go to him to ask for details about certain things. Overall, it wasn’t a big issue.
Do you have any advice for potential customers?
Trust them. They respond very well to trust. If a client wants to micromanage them and have them say exactly what line of code they have written and when, then I don’t think it will work out.
the project
Custom Web App Dev for Analytics Company
"They made it easier for me by being available during my work hours even though they are in a different time zone."
the reviewer
the review
The client submitted this review online.
Please describe your company and your position there.
I'm the Head of Product for a web analytics company that focuses on detecting online advertising fraud.
For what projects/services did your company hire Datum Brain, and what were your goals?
We had an existing application that needed new features built and we needed our documentation/github updated to reflect the current status of the application. The prior developer did a poor job managing documentation so it was hard for a new developer to pick up the project but Datum Brain did a great job.
How did you select this vendor and what were the deciding factors?
We posted a job on Upwork and Datum Brain had experience with large data sets and the language our application was built it. They were good communicators and had positive reviews already.
Describe the scope of work in detail, including the project steps, key deliverables, and technologies used.
- experts in golang -mysql / queries with very large data sets -writing / maintaining unit tests in Go -experience writing cli tools, REST apis -general linux/ubuntu administration -strong understanding of javascript to maintain/debug existing JS code
How many people from the vendor's team worked with you, and what were their positions?
2 back end developers and 2 front end developers
Can you share any measurable outcomes of the project or general feedback about the deliverables?
They have been able to complete all additional features requested in a time effective manor. They have improved our documentation/git and improved the overall efficiency of our product.
Describe their project management style, including communication tools and timeliness.
We have a weekly team call and manage tasks in trello and github issues. Good communication through slack
What did you find most impressive or unique about this company?
They have a well rounded team of good developers which gives them experience in a lot of different languages/products. They made it easier for me by being available during my work hours even though they are in a different time zone.
Are there any areas for improvement or something they could have done differently?
We have had some minor issues with communication of specific developers but the Datum Brain team lead quickly noticed this issue and fixed it.
Datum Brain's support enabled the client to launch their projects ahead of time. The project manager excellently managed the workflow, regularly met the deadlines, and responsively communicated the processes on Microsoft Teams and email. Overall, they were a one-stop-shop for our software needs.