IT Product Engineer Business Intelligence
The posted compensation range of $37.05 - $53.72 /hour is a reasonable estimate that extends from the lowest to the highest pay CommonSpirit in good faith believes it might pay for this particular job, based on the circumstances at the time of posting. CommonSpirit may ultimately pay more or less than the posted range as permitted by law.
CommonSpirit Health was formed by the alignment of Catholic Health Initiatives (CHI) and Dignity Health. With more than 700 care sites across the U.S. from clinics and hospitals to home-based care and virtual care services CommonSpirit is accessible to nearly one out of every four U.S. residents. Our world needs compassion like never before. Our communities need caring and our families need protection. With our combined resources CommonSpirit is committed to building healthy communities advocating for those who are poor and vulnerable and innovating how and where healing can happen both inside our hospitals and out in the community.
Responsibilities
This is a remote position.
About the Enterprise Business Intelligence Team
The Enterprise Data & Analytics team is responsible for building and maintaining the robust and scalable data infrastructure that empowers CommonSpirit Health to deliver exceptional patient care, improve operational efficiency, and drive data-driven decision-making. We are a team of passionate data professionals dedicated to delivering high-quality, secure, and reliable data products. We are committed to leveraging cutting-edge technologies, particularly Google Cloud Platform (GCP), to build a world-class, automated data foundation. We adhere to strict HIPAA compliance standards.
Role Description
The Product Engineer Role is a critical contributor to the design, development, optimization,automation, and maintenance of our core data platform and analytical solutions, with a strong focus on leveraging GCP services. The Product Engineer will work closely with data analysts, clinicians, and other engineers to understand their data needs and translate them into robust, scalable, performant, and automated data pipelines and infrastructure. The Product Engineer will be responsible for ensuring the reliability, performance, security, data quality, and HIPAA compliance of our data infrastructure. This role is heavily focused on data engineering and infrastructure development, with a strong emphasis on workflow orchestration and automation, hands-on development of ETL processes using Python and SQL, and leveraging Dataplex for data profiling and quality management.
Responsibilities:
- Data Pipeline Design, Development, and Automation
- Design, develop, test, deploy, and automate highly scalable and reliable data pipelines and ETL/ELT processes using GCP services such as Dataflow, Dataproc, BigQuery, Cloud Functions, Cloud Composer, and Cloud Storage.
- Implement data ingestion, transformation, and loading strategies for various data sources, including clinical systems, operational databases, and external data feeds.
- Optimize data pipelines for performance, scalability, cost-effectiveness, and automated execution.
- Implement robust error handling and monitoring mechanisms to ensure data quality and pipeline stability.
- Design, develop, and maintain complex DAGs (Directed Acyclic Graphs) in Apache Airflow (or Cloud Composer) to orchestrate data pipelines and workflows.
- Implement automated testing and deployment strategies for Airflow DAGs.
- Develop and maintain ETL processes using Python and SQL to transform and load data into BigQuery and other data stores.
- Implement data quality checks and validation rules within data pipelines to ensure data accuracy and completeness.
Data Governance and Quality Management:
- Utilize Google Cloud Dataplex to profile data assets, discover data quality issues, and implement data quality rules.
- Develop and maintain data quality dashboards and reports to monitor data quality metrics.
- Collaborate with data owners and stewards to resolve data quality issues and improve data governance practices.
- Implement and maintain data governance policies and procedures.
Data Modeling and Schema Design:
- Design and implement data models and schemas in BigQuery optimized for analytical workloads and reporting needs.
- Ensure data consistency and integrity across different data sources.
- Collaborate with data analysts to understand their data requirements and design data models that meet their needs.
Collaboration and Communication:Collaborate with data analysts, business leaders, healthcare operators and other engineers to understand their data requirements and translate them into technical solutions.
- Communicate technical concepts clearly and effectively to both technical and non-technical audiences, including stakeholders with varying levels of understanding of healthcare data and regulations.
- Participate in code reviews and contribute to team knowledge sharing.
- Contribute to the development of data engineering best practices and standards, with a strong emphasis on HIPAA compliance, data privacy, automation, and data quality.
Innovation and Improvement:
- Stay up-to-date with the latest trends and technologies in data engineering, GCP, Airflow, and automation techniques, Dataplex, and healthcare data standards and regulations.
- Identify opportunities to improve the efficiency, scalability, reliability, automation, and data quality of our data platform while maintaining HIPAA compliance.
- Propose and implement innovative solutions to complex data engineering challenges in the healthcare domain.
Qualifications
Required:
- Bachelor's degree in Computer Science, Engineering, or a related field or relevant experience.
- 2-3+ years of experience in data engineering or a related role, with a strong focus on building, maintaining, and automating data pipelines and infrastructure, and ensuring data quality.
- Strong proficiency in SQL and experience with relational databases (e.g., SQLSERVER, MySQL).
- Hands-on experience with Google Cloud Platform (GCP) services such as BigQuery, Dataflow, Dataproc, Cloud Functions, Cloud Composer, and Cloud Storage.
- Experience with data modeling and schema design.
- Expert proficiency in Python and SQL for developing ETL processes.
- Extensive experience with data pipeline development and ETL/ELT processes.
- Experience with version control systems (e.g., Git).
- Strong problem-solving and analytical skills, specifically related to data engineering challenges, automation, and data quality.
- Excellent communication and collaboration skills.
- Solid understanding of HIPAA regulations and data privacy principles.
- Experience working with Protected Health Information (PHI).
- Proven experience designing, developing, and deploying complex DAGs in Apache
- Airflow (or Cloud Composer).
- Experience with automating data pipelines and infrastructure using scripting and orchestration tools.
- Experience with data profiling and data quality assessment techniques.
- Experience with data governance principles and practices.
- Strong attention to detail.
- Ability to work independently and as part of a team.
- Ability to manage multiple projects simultaneously.
- Strong organizational skills and time management skills.
- Understanding of HIPAA regulations and data privacy policies.
Preferred:
- Experience with streaming data processing technologies such as Apache Kafka or Apache Beam.
- GCP certifications (e.g., Google Cloud Certified Professional Data Engineer).
- Experience with healthcare data standards (e.g., HL7, FHIR, ICD-10, CPT).
- Experience with data de-identification and anonymization techniques.
- Experience with implementing and maintaining security controls in a healthcare environment.
- Experience with data warehousing and data lake architectures.
- Experience with performance tuning and optimization of data pipelines and infrastructure.
- Experience with CI/CD pipelines for data engineering workflows.
- Cloud Dataplex for data profiling, data quality monitoring, and data governance.
- Experience with building data quality dashboards using tools like Looker or Qlik.
#LI-Remote
#LI-CSH
Unless directed by a Collective Bargaining Agreement, applications for this position will be considered on a rolling basis. CommonSpirit Health cannot anticipate the date by which a successful candidate may be identified.
Apply
Depending on the position offered, CommonSpirit Health offers a generous benefit package, including but not limited to medical, prescription drug, dental, vision plans, life insurance, paid time off (full-time benefit eligible employees may receive a minimum of 14 paid time off days, including holidays annually), tuition reimbursement, retirement plan benefit(s) including, but not limited to, 401(k), 403(b), and other defined benefits offerings, as may be amended from time to time. For more information, please visit https://www.commonspirit.careers/benefits.
No featured jobs
No recently viewed jobs
You have no saved jobs
Equal Opportunity
CommonSpirit Health™ is an Equal Opportunity/Affirmative Action employer committed to a diverse and inclusive workforce. All qualified applicants will be considered for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, marital status, parental status, ancestry, veteran status, genetic information, or any other characteristic protected by law. For more information about your EEO rights as an applicant, please click here [PDF].
CommonSpirit Health™ will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their pay or the pay of another employee or applicant. However, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with the contractor’s legal duty to furnish information. 41 CFR 60-1.35(c). External hires must pass a post-offer, pre-employment background check/drug screen. Qualified applicants with an arrest and/or conviction will be considered for employment in a manner consistent with federal and state laws, as well as applicable local ordinances, ban the box laws, including but not limited to the San Francisco and Los Angeles Fair Chance Ordinances. If you need a reasonable accommodation for any part of the employment process, please contact us by telephone at (415) 438-5575 and let us know the nature of your request. We will only respond to messages left that involve a request for a reasonable accommodation in the application process. We will accommodate the needs of any qualified candidate who requests a reasonable accommodation under the Americans with Disabilities Act (ADA). CommonSpirit Health™ participates in E-Verify.