Senior Data Engineer (AWS, Big Data, ETL, Netezza, Spark, Kafka, Data Warehousing)
About the role
Senior Data Engineer | AWS Cloud | Big Data | ETL | Netezza | Spark | Kafka | Data Platform Modernization
Job Summary
The ideal candidate will have strong expertise in designing, developing, and supporting scalable data pipelines and distributed systems, along with hands-on experience in Big Data ecosystem tools, AWS services, and real-time data processing.
This role involves working on data platform modernization, cloud migrations, Data Warehousing, and ETL in a fast-paced enterprise environment.
Required Technical Skills
Cloud Technologies
• Strong experience in AWS Cloud services: • EMR • EC2 • S3 • VPC • RDS • Redshift • AWS Glue • IAM • CloudWatch • CloudFormation
• Experience with: • Airflow (or AWS Managed Workflows)
Databases
• Experience working with: • Netezza (Mandatory) • SQL-based systems: • SQL Server • PostgreSQL
• Data warehouses: • Teradata • Redshift • Netezza
ETL Tools
• Hands-on experience with: • SSIS • Pentaho or similar ETL tools
Programming & Scripting
• Strong proficiency in: • SQL • Shell scripting
• Good to have: • Python
Operating Systems
• Strong experience in Unix/Linux environments.
Key Qualifications
• 10+ years of experience in Data Engineering / Big Data / Platform Engineering.
Key Responsibilities
• Design, develop, maintain, and support scalable data pipelines using Big Data and AWS technologies.
• Lead and support data platform migration initiatives (On-Prem to AWS Cloud), ideally with Netezza background.
• Develop and manage ETL/ELT processes using tools like SSIS, Pentaho, or similar.
• Implement and manage AWS services such as: • EMR • S3 • EC2 • Redshift • Glue • Airflow
• Build and optimize data workflows and orchestration pipelines using Airflow.
• Work with real-time streaming technologies such as: • Kafka • Spark Streaming
• Perform data ingestion, transformation, and validation from multiple data sources.
• Optimize SQL queries for performance and scalability.
• Monitor system performance, troubleshoot issues, and ensure system reliability.
• Collaborate with cross-functional teams including: • Developers • Architects • Infrastructure teams.
About Astra-North Infoteck Inc. ~ Conquering today’s challenges, achieving tomorrow’s vision!
Astra - North Infoteck Inc.
Agile - Systems - Technologies - Resources - Applications
Apply for Jobs@ https://careers.astra-north.com/jobs/
We are a global information technology services, management consulting and outsourcing company headquartered in Canada. Astra North helps clients navigate the ever-changing complex information technology landscape with its portfolio of IT consulting services, managed services and staffing and talent management solutions.
The core challenge facing many businesses in their IT functions is the increasing shortage of talent across the globe and its resultant impact on productivity. To overcome this challenge Astra-North provides its clients with a single source Integrated Recruitment Process Outsourcing - iRPO Service. Astra-North works as a specialist strategic partner for its clients by providing the best talent pool of IT Professionals on contract hire, permanent hire or managed services basis through its comprehensive and integrated recruitment, staff augmentation, workforce solutions, recruitment process outsourcing and managed services programs which are customized for individual client needs. Astra North Staffing Services include managing programs that are in compliance with matters focussing on careers, communications, legal and regulatory issues, technology and outsourcing, budgeting and metrics, IT staffing management best practices and global staffing management issues.
Our technical service extends the entire software development lifecycle from Consulting, Business Process Analysis, Requirement Mapping, Analysis, Design, Development, Testing to Implementation and Support. Astra-North Infoteck has expertise in building, maintaining and reengineering IT solutions to meet its client’s evolving needs.We help clients become agile to meet and exceed their goals.
Astra-North ~ Conquering Today's Challenges, Achieving Tomorrow's Vision!
Similar Jobs
Senior Data Engineer (AWS, Big Data, ETL, Netezza, Spark, Kafka, Data Warehousing)
About the role
Senior Data Engineer | AWS Cloud | Big Data | ETL | Netezza | Spark | Kafka | Data Platform Modernization
Job Summary
The ideal candidate will have strong expertise in designing, developing, and supporting scalable data pipelines and distributed systems, along with hands-on experience in Big Data ecosystem tools, AWS services, and real-time data processing.
This role involves working on data platform modernization, cloud migrations, Data Warehousing, and ETL in a fast-paced enterprise environment.
Required Technical Skills
Cloud Technologies
• Strong experience in AWS Cloud services: • EMR • EC2 • S3 • VPC • RDS • Redshift • AWS Glue • IAM • CloudWatch • CloudFormation
• Experience with: • Airflow (or AWS Managed Workflows)
Databases
• Experience working with: • Netezza (Mandatory) • SQL-based systems: • SQL Server • PostgreSQL
• Data warehouses: • Teradata • Redshift • Netezza
ETL Tools
• Hands-on experience with: • SSIS • Pentaho or similar ETL tools
Programming & Scripting
• Strong proficiency in: • SQL • Shell scripting
• Good to have: • Python
Operating Systems
• Strong experience in Unix/Linux environments.
Key Qualifications
• 10+ years of experience in Data Engineering / Big Data / Platform Engineering.
Key Responsibilities
• Design, develop, maintain, and support scalable data pipelines using Big Data and AWS technologies.
• Lead and support data platform migration initiatives (On-Prem to AWS Cloud), ideally with Netezza background.
• Develop and manage ETL/ELT processes using tools like SSIS, Pentaho, or similar.
• Implement and manage AWS services such as: • EMR • S3 • EC2 • Redshift • Glue • Airflow
• Build and optimize data workflows and orchestration pipelines using Airflow.
• Work with real-time streaming technologies such as: • Kafka • Spark Streaming
• Perform data ingestion, transformation, and validation from multiple data sources.
• Optimize SQL queries for performance and scalability.
• Monitor system performance, troubleshoot issues, and ensure system reliability.
• Collaborate with cross-functional teams including: • Developers • Architects • Infrastructure teams.
About Astra-North Infoteck Inc. ~ Conquering today’s challenges, achieving tomorrow’s vision!
Astra - North Infoteck Inc.
Agile - Systems - Technologies - Resources - Applications
Apply for Jobs@ https://careers.astra-north.com/jobs/
We are a global information technology services, management consulting and outsourcing company headquartered in Canada. Astra North helps clients navigate the ever-changing complex information technology landscape with its portfolio of IT consulting services, managed services and staffing and talent management solutions.
The core challenge facing many businesses in their IT functions is the increasing shortage of talent across the globe and its resultant impact on productivity. To overcome this challenge Astra-North provides its clients with a single source Integrated Recruitment Process Outsourcing - iRPO Service. Astra-North works as a specialist strategic partner for its clients by providing the best talent pool of IT Professionals on contract hire, permanent hire or managed services basis through its comprehensive and integrated recruitment, staff augmentation, workforce solutions, recruitment process outsourcing and managed services programs which are customized for individual client needs. Astra North Staffing Services include managing programs that are in compliance with matters focussing on careers, communications, legal and regulatory issues, technology and outsourcing, budgeting and metrics, IT staffing management best practices and global staffing management issues.
Our technical service extends the entire software development lifecycle from Consulting, Business Process Analysis, Requirement Mapping, Analysis, Design, Development, Testing to Implementation and Support. Astra-North Infoteck has expertise in building, maintaining and reengineering IT solutions to meet its client’s evolving needs.We help clients become agile to meet and exceed their goals.
Astra-North ~ Conquering Today's Challenges, Achieving Tomorrow's Vision!