Lead Data Engineer/Architect - R01561308

Chicago, Illinois, United StatesFull-TimeLeadOther

You will be redirected to the company career page

Primary Skills

  • Athena, Step Functions, Spark - Pyspark, ETL Fundamentals, SQL (Basic + Advanced), Glue, Python, Lambda, Data Warehousing, EBS /EFS, AWS EC2, Lake Formation, Aurora, S3, Modern Data Platform Fundamentals, PLSQL, Data Modelling Fundamentals, Cloud front

Job requirements

  • About the Role
  • We are seeking an experienced Data Architect to design and build scalable, cloud-native data platforms supporting IoT, operational systems, and enterprise analytics.
  • This role requires a hands-on technical leader who can architect modern data pipelines, enable real-time and batch ingestion, and design performant serving layers for APIs and applications.
  • The ideal candidate blends deep AWS expertise, strong data modeling skills, and the ability to translate business requirements into scalable technical solutions.
  • Data Engineering & Architecture
  • Architect ingestion pipelines for streaming and event-based data (IoT telemetry)
  • Implement time-window logic and event-to-state transformations
  • Design, build, and optimize data pipelines using AWS Glue, Lambda, and Step Functions for ingestion, transformation, and curation.
  • Develop and maintain data lakes (S3 + Lake Formation) and data warehouses (Redshift) to support analytics and visualization.
  • Integrate Aurora (PostgreSQL/MySQL) as a serving layer for APIs and dashboards.
  • Design APIs or direct query interfaces that expose curated datasets for React-based dashboards and web applications.
  • Define and implement data models, partitioning strategies, and schema evolution best practices for performance and scalability.
  • Security, Governance & Compliance
  • Implement data security policies, encryption, and role-based access controls using AWS IAM, KMS, Lake Formation, and Secrets Manager.
  • Ensure compliance with organizational data governance and privacy standards.
  • Maintain data cataloging, lineage, and access tracking within AWS Glue Data Catalog and Lake Formation.
  • Implement CI/CD pipelines for data workflows in collaboration with DevOps teams.
  • 12+ years of experience in data engineering and AWS data architecture.
  • Strong expertise with AWS Glue, Redshift, S3, Lake Formation, Aurora (Postgres/MySQL).
  • Proven experience building APIs or exposing curated datasets for web applications (preferably React or similar front-end frameworks).
  • Strong programming skills in Python and SQL.
  • Experience with data modeling, ETL/ELT frameworks, and performance optimization.
  • Working knowledge of Terraform/CloudFormation and CI/CD practices.
  • Understanding of data security, IAM policies, encryption, and governance frameworks.
  • Experience with Athena for ad-hoc analytics.
  • Familiarity with API Gateway, AppSync, or GraphQL APIs for serving data to front-end applications.
  • Knowledge of event-driven pipelines (Kinesis, EventBridge, kafka) and data quality monitoring.

Job Summary

CompanyBrillio-2
LocationChicago, Illinois, United States
TypeFull-Time
LevelLead
DomainOther