Software Engineering, Big Data & Analytics
We are a technology outsourcing partner which helps tech driven companies to efficiently scale their engineering efforts quickly.
Our teams of agile practising problem solvers have specialised in building infrastructure to enable real time big data processing and enable our clients to extract value from vast amounts of information. Engineers at South Gate Tech deep dive into the business problems of our clients and are regularly contributing to the strategy and business outcomes of the products they build.
#142 – 757 WEST HASTINGS STREETVancouver, BC V6C 1A1Canada
We were contacted by BeAble – an American start-up company based in New Jersey to help them build their MVP.
Their main focus is on creating the Beable platform, including three major sections: rostering, taxonomy and literacy web-apps. The goal of this platform is to allow the enrollment of new students with literacy problems, which will then proceed to be assessed, and will ultimately be offered lessons in their predefined subjects, prepared specifically for their literacy level. An example of that would be a student in the 10th grade, who receives his material written for the 8th grade, so that he understands better and works his way up.
Technology – wise, the platform is built on top of micro-services architecture with a strict hierarchy of interfaces and helper libraries, as the main stack is NestJS, ReactJS (with hooks), MySQL and AWS.
Ramped up 3 augmented teams to cover key customer roadmap including:
- Custom BI and Analytics platform solution & set-up in AWS
- Custom BI and Analytics via Talend, Airflow, Looker
- Custom Real Time Data streaming via Python / Kafka / Spark / Airflow
Delivering key buisness outcomes and supporting the growth and scalability of tide.co.uk
Built and delivered a POC for Vodafone Germany to support their data warehous migration and optimization program.
Dedicated resources included a Senior Data Architect and Senior Data Engineers.
Solutioin consists of:
•Oracle GoldenGate, running in Docker containers on Exadata servers, servicing over 300 inbound and outbound connections via Kafka and applying real-time data validation.
•The architecture includes some of the latest AWS services.
Carried out performance and stress tests to proove the solution is stable and meets the KPIs.
Designed and built from scratch a production ready application for analysis, research and planning of TV advertisements based on peoplemetric data. Software is enabling advertising agencies, media companies and marketing departments to reach their target audiences by minimizing the cost and maximizing the coverage and plan rating points.
The product is computationally intensive and is built with high performance / high accuracy as its key non-functional requirements.
- Front end: React with Redux. Web Workers for local compute.
- Back end business logic: Serverless.com, AWS API Gateway, AWS Lambda, AWS Cognito, AWS S3
- Mathematical Optimization: Python with Numpy over Cython
- Database: AWS DynamoDB, Postgres
- Daily ETL process done over AWS S3, AWS Lambda, SQL stored procedures
The application is designed to be highly available, redundant and automatically scales up AWS resources to accommodate thecurrent usage optimising the hosting & running costs.
It is currently used by more than 300 agents.
• Growing product backlog
• Inability to find required talent internally
• Compounded by new Gambling Laws and Regulations requirements
In addition,any new starter had to:
• Meet demanding security requirements, including working with sensitive data (financial transactions, personal data, etc) and physical security
• Navigate a regulatory quagmire
Created a dedicated team of data engineers, complying with all customer security standards – from background employment checks and criminal records, through software and hardware policies, to office physical security – in total:
Team of seven talented engineers, a core team built in less than 4 weeks. We scaled with new members & tech stacks based on our customer ad-hoc needs
What we’ve achieved:
• Data streaming using Kafka, Talend, S3, Redshift. • Data latency & consistency checks with Talend
• Migration of regulatory framework from Oracle to AWS using Talend, AuroraDB
• Migration of datasets from Oracle to AWS using Talend, SQS, AuroraDB
• Automation of chargebacks ingestion and processing using Talend, S3, Redshift
Our customer PPB, has undertaken a program to speed up it’s go-to market time for new regulations and jurisdictions. Before that this meant building from scratch.
We begin by augmenting the customer teams with the following engineers:
- Java / backend - FrontEnd
- Scrum Masters and Product Owners The
The augmented teams delivered:
- 50 currencies support and configuration from CMS (including 3rd party game vendors)
- Real-time content translation tool
- Unified and configurable My Account experience