Data Engineer

Core Responsibilities:

Legacy Data Platform Support

• Maintain and enhance SSIS packages for data extraction, transformation, and loading
• Support SQL Server data warehouse (staging, ODS, reporting layers)
• Troubleshoot data issues, job failures, and performance bottlenecks
• Optimize SQL queries, stored procedures, and indexing strategies
• Ensure reliability of scheduled jobs via SQL Server Agent

Cloud Data Engineering (Azure + Databricks)

• Design and develop data pipelines using Azure Data Factory (ADF)
• Ingest and organize data into Azure Data Lake (Bronze/Silver/Gold layers)
• Build scalable data transformations using Databricks (Spark SQL, PySpark)
• Create curated, analytics-ready datasets for Power BI
• Implement Delta Lake and support data governance (e.g., Unity Catalog)

Migration & Modernization

• Analyze and document existing SSIS/SQL pipelines
• Translate legacy ETL processes into modern ELT patterns
• Support phased migration strategy (coexistence of legacy and modern platforms)
• Reduce technical debt and improve pipeline maintainability
• Establish standards for data modeling, naming, and architecture

Data Modeling & Business Value Creation

• Design dimensional models (fact and dimension tables) aligned to business processes
• Integrate and standardize data across multiple ERP systems
• Translate business requirements into scalable data solutions
• Partner with stakeholders to identify high-impact use cases for data and analytics
• Deliver datasets that enable reporting, forecasting, and operational insights

Data Quality & Governance

• Implement data validation, reconciliation, and monitoring processes
• Ensure data accuracy and consistency across systems during migration
• Define and enforce data quality standards and controls
• Support data lineage, documentation, and transparency initiatives

Collaboration & Stakeholder Engagement

• Work closely with business stakeholders, analysts, and BI developers
• Support Power BI semantic models and reporting solutions
• Communicate technical solutions in business terms
• Act as a bridge between IT/data teams and business functions

Qualifications:

Required Qualifications

• 4-8+ years of experience in data engineering or data warehousing
• Strong SQL skills (T-SQL and/or Spark SQL)
• Hands-on experience with SSIS and SQL Server
• Experience with Azure Data Factory (ADF) or similar tools
• Experience with Databricks (Spark, Delta Lake, or similar platforms)
• Solid understanding of data warehousing concepts (star schema, fact/dimension modeling)
• Experience integrating data from multiple source systems (ERP experience preferred)
• Proven ability to translate business requirements into technical solutions

Preferred Qualifications

• Experience migrating legacy ETL systems (SSIS) to cloud-based architectures
• Proficiency in Python or PySpark
• Familiarity with Medallion architecture (Bronze/Silver/Gold)
• Experience with Power BI data modeling and performance optimization
• Knowledge of data governance tools (e.g., Unity Catalog)
• Experience with Git and CI/CD pipelines
• Exposure to dbt or similar frameworks

Technical Skills

• SQL Server (T-SQL), SSIS
• Azure Data Factory (ADF)
• Azure Data Lake Storage (ADLS)
• Databricks (Spark SQL, PySpark, Delta Lake)
• Data modeling (Kimball methodology preferred)
• Performance tuning and query optimization
• Version control (Git)