MLOps Engineer (Focusing AWS platform) - RDT Pharma Medicines

Data Engineer is responsible for designing, developing, and maintaining scalable and robust AWS based data pipelines and infrastructure. This role involves independently building ETL/ELT processes, optimizing data storage solutions (such as data warehouses , data lakes and Lake House ), ensuring data quality and reliability, and monitoring data systems. You will collaborate closely with data product owners, data scientists and data and business analysts to meet their specific data requirements, utilizing strong programming skills in Python, PySpark, expert SQL knowledge, and proficiency in big data technologies like Spark. Turn raw commercial data (claims, sales, specialty pharmacy, payer, CRM) into reliable, compliant, analytics-ready data products that power sales force effectiveness, market access, brand analytics and forecasting. Works independently, owns projects end to end, and partners with data product owners, data scientists and business analysts.

Job Responsibilities

• Scope: Leads the design, build, and maintenance of scalable data pipelines. Works independently to execute specific data engineering projects from start to finish.
• Accountability/Problem Solving: Solves complex data ingestion and processing problems. Optimizes data flows and actively enhances overall system performance.
• Stakeholder Management: Engages directly with broader business units, business analysts and data scientists to thoroughly understand and meet their data needs. Communicates effectively with both technical and non-technical stakeholders.
• Impact/Strategy: Handles big-sized data engineering projects. Implements robust strategies that significantly improve the organization's data infrastructure.
• Complexity : Manages complex, large-scale data systems. Integrates multiple diverse data sources and ensures efficient, reliable data flow across platforms

Qualifications

• 6-10 years in data engineering with proven independent delivery of large-scale pipelines.
• Demonstrated experience handling big-sized data engineering projects and managing complex, large-scale data systems.
• Proven track record of independently building ETL/ELT processes and optimizing data storage solutions such as data warehouses and data lakes.
• Experience ensuring data quality, reliability, and monitoring data systems

Technical Skills

• Proficiency in programming languages such as SQL, Python, PySpark
• Proficiency in AWS Cloud based Data Engineering skills - S3, Lambda, AWS Glue, Athena, Airflow, DAGs, RedShift, RDS, AWS DataSync, AWS Step Functions, QuickSight, dbt
• Strong, expert knowledge of cloud platforms and Big Data tools, specifically Spark.
• Strong data modeling (dimensional, data vault, lakehouse), partitioning and file formats.

Data quality, monitoring and incident troubleshooting experience

• Experience building data products, designing a semantic layer, and handling unstructured data
• Terraform or CloudFormation, Git-based CI/CD, containerization.
• Advanced capability to solve complex data ingestion and processing problems, and to optimize data flows for enhanced system performance.

Preferred Qualifications

• Exceptional communication and collaboration skills, with the proven ability to engage effectively with both technical audiences (like data scientists) and non-technical business stakeholders
• HCP/HCO master data (Veeva Network, OneKey), patient-level longitudinal data, de-identification. …