Lead Data Engineer

Description - External

Uniteds Digital Technology team designs, develops, and maintains massively scaling technology solutions brought to life with innovative architectures, data analytics, and digital solutions.

Our Values: At United Airlines, we believe that inclusion propels innovation and is the foundation of all that we do. Our Shared Purpose: "Connecting people. Uniting the world." drives us to be the best airline for our employees, customers, and everyone we serve, and we can only do that with a truly diverse and inclusive workforce. Our team spans the globe and is made up of diverse individuals all working together with cutting-edge technology to build the best airline in the history of aviation.

Job overview and responsibilities:

The Lead Data Engineer will design, implement, and manage end-to-end data pipeline architectures, ensuring data quality and accuracy in the data warehouse. They will collaborate with business stakeholders to understand and translate requirements into technical solutions while mentoring and guiding junior team members. Additionally, they will oversee data validation and data cleansing processes to maintain high data integrity and reliability in the reporting warehouse.

Responsibilities:

• Design, implement, and oversee the end-to-end data pipeline architecture, ensuring robust data ingestion, processing, and integration that supports business objectives and maintains operational efficiency.
• Utilize expertise in data warehousing technologies to optimize and manage data storage, retrieval, and organization, ensuring that the data warehouse supports accurate and efficient reporting and analytics.
• Develop and enforce data quality frameworks and validation procedures to ensure the accuracy, completeness, and integrity of data within the data warehouse and throughout the data lifecycle.
• Collaborate with business stakeholders to accurately translate business requirements into technical solutions, ensuring that data engineering efforts align with organizational goals and deliver actionable insights
• Provide leadership and mentorship to junior data engineers, guiding their professional development, fostering a collaborative team environment, and ensuring high performance and growth within the team.
• Apply advanced programming skills in languages such as Python or SQL for querying and managing data, contributing to efficient and effective data processing and analysis.

This position is offered on local terms and conditions. Expatriate assignments and sponsorship for employment visas, even on a time-limited visa status, will not be awarded. This position is for United Airlines Business Services Pvt. Ltd - a wholly owned subsidiary of United Airlines Inc.

Required:

· Bachelors degree in computer sciences or computer engineering

· 7 years of relevant experience in ETL Tools and Technologies: Talend, Informatica, SSIS, AWS Glue, Google Cloud Dataflow. Data Transformation: Data cleansing, enrichment, aggregation, scripting (Python, SQL)

· Data Integration: Data pipeline orchestration, scheduling tools (Apache Airflow, Luigi).

· AWS Services: S3, Lambda, Glue, IAM.

· Amazon Redshift: Cluster setup, management, optimization, query writing.

· Data Migration: Data migration strategies and tools for Redshift.

· Apache Spark: Spark SQL, DataFrames, RDDs.

· Integration and APIs: Databricks APIs, integration with AWS services.

· Data Modeling: Schema design, data warehousing

· SQL: Advanced querying and manipulation.

· Programming: Python.

· Cloud Architecture: Scalability, security, cost management.

Preferred:

• Masters Degree