Supporting a high-tech startup unicorn
A high-tech startup unicorn in the VR/AR space, the company needed assistance with big data strategy, architecture design, development and implementation due to its innovative, unique product lines and stringent security requirements.
A previously proposed architectural design based on traditional off-the-shelf information technology tools was deemed insufficient. The proposed solution would not integrate, throttle-up scalability, nor provide significant flexibility with highly customized and bespoke systems.
Business and IT leadership stakeholders desired a strategy to empower the organization to understand business and product anomalies and proactively derive economic value via advanced analytics.
CHT was solely responsible for big data strategy, architecture design, and development as well as continuous integration and delivery pipeline automation. The implementation team was entirely composed of CHT resources consisting of a cloud oriented big data evangelist, data and enterprise architect, project manager, data scientist, business analysts, software development resources, dev ops specialists, and visualization experts.
Given client requirements, CHT’s strategy placed emphasis on grassroots technical tools implemented via software development best practices relative to application frameworks.
More specifically, CHT designed and deployed an AWS Cloud oriented “shared nothing” Massive Parallel Processing serverless architecture with Master Data Management and governance.
Overall, operational data was integrated relative to channel, geolocation, employees, financial, product hardware, person, price, product, recruiting, revenue, sales, security, device software, manufacturing supply chain, workflow management, and website clickstream to render a holistic enterprise wide 360-degree visualization with various scoring, internet sentiment analysis and segmentation analytics.
Technology & Architecture Benefits: CHT delivered event-driven advanced analytics with self-serve visualization capabilities which provide:
- Synchronous or Asynchronous Event based processing
- Big Data oriented Massive Parallel Processing (MPP)
- Shared Nothing Architecture
- Auto Scaling and Administration-Free Serverless Architecture
- Docker based Elastic Container Service via AWS Batch
- Application Framework oriented Development Process
- Glue based Integrated Development and Control Environment
- Python, Scala, GO, & SQL development tools
- Proactive SNS Alert Notification
- Data Lake direct data access (i.e., Parquet format, CSV, or JSON)
- Intrinsic Logging and Alerting Infrastructure
- Intrinsic System of Record Crawling and Data Cataloging
- On-the-Fly Integrated Data Science / Machine Learning for Scoring and Segmentation
- Key Management System for Encryption Key Rotation Strategy and Logging
Tools of the Trade: During the engagement, CHT’s resources relied on the following tools and applications:
- Restful APIs
- Database: Redshift, RDS
- Serverless Infrastructure: Apache Spark, Lambda
- Elastic Container Service: Docker, ECS, AWS Batch
- Security: KMS
- Audit & Logging: Cloud Watch, Cloud Trial
- Programming: Python, SQL, GO, Scala, AWS SDK, AWS CLI
- Dev Ops: Terraform
- Visualization: Tableau
- Streaming: Kinesis, Firehose
- MDM: Paxata, Alation
- Data Science: SK Learn, ML lib, R Studio
- Project Management: Jira