Data Engineer

Role Summary
A senior data engineer position focused on designing, building, and optimizing large-scale, high-reliability data pipelines and lakehouse architectures. This role requires a highly experienced professional capable of leading core architectural decisions and implementing end-to-end data solutions that support analytics and business intelligence initiatives. The candidate will collaborate across teams to develop scalable, modular, and testable data systems, ensuring high performance and data quality.

Responsibilities

• Design, develop, and maintain ELT pipelines for data ingestion, transformation, modeling, and delivery across different layers (bronze, silver, gold).
• Implement incremental data loads, change-data-capture (CDC), merge/upsert, and idempotent workflows to ensure data reliability and repeatability.
• Define and apply data architectural patterns, including layered lakehouse architectures, domain datasets, and semantic models aligned with business goals.
• Engineer physical data structures, including partitioning strategies, partition key selection, clustering, micro-partitioning, and data compaction for optimized performance and cost efficiency.
• Develop curated datasets and data marts that facilitate analytics and self-service business intelligence solutions.
• Implement data quality frameworks, observability, and lineage tracking through validations, profiling, monitoring, and alerting mechanisms.
• Optimize data platform performance, particularly on cloud-based systems such as Snowflake, through query tuning, resource sizing, and task management.
• Design and manage lakehouse table formats (e.g., Apache Iceberg, Delta Lake) on object storage, including schema evolution and maintenance processes.
• Collaborate with data architects, analytics engineering teams, and stakeholders to translate business requirements into scalable data solutions.
• Mentor junior engineers, lead design reviews, and contribute to engineering standards, reusable frameworks, and automation efforts using CI/CD and infrastructure-as-code practices.

Qualifications

• 7 to 10+ years of experience in data engineering or related software engineering roles with a focus on data systems.
• Extensive SQL development skills, including advanced query optimization and troubleshooting.
• Strong proficiency in Python programming, software design principles, unit testing, version control, and refactoring.
• Experience building and orchestrating ELT/ETL pipelines with tools like Airflow, Astronomer, or similar, along with Git and CI/CD pipelines.
• Deep knowledge of core data engineering patterns such as data ingestion, transformation, dimensional modeling, slowly changing dimensions (SCDs), and data delivery.
• Proven expertise in physical database design, partitioning, clustering, and micro-partitioning on cloud data platforms.
• Experience with data quality frameworks, observability, monitoring, and operational SLAs.
• Familiarity with Lakehouse table formats (Apache Iceberg, Delta, Hudi) and columnar storage (Parquet) on cloud object storage (e.g., AWS S3).
• Effective communication skills to convey complex technical concepts to stakeholders at all levels.
• Must be legally authorized to work in the United States without employer sponsorship, now or in the future.

Publishing Pay Range: $94.00 – $95.00 Hourly
This position is based in office and requires employee to work on-site.