Technology Partners is currently seeking a talented Principal Data Engineer (Snowflake & Azure Lakehouse | Technical Lead).
Do you have experience leading large-scale Snowflake data architectures, building production-grade pipelines, and driving client-facing data engineering engagements?
Let us help you make your next big career move a reality!
What You Will Be Doing:
You will serve as the hands-on technical lead for a high-stakes, enterprise data modernization program focused on transforming large-scale claims data ingestion into a modern Snowflake + Azure lakehouse architecture. This is a delivery-critical role where you own the technical outcome, lead architecture decisions, build core pipelines, and act as the primary engineering voice with senior client stakeholders. You will operate in a fast-paced, high-visibility environment, balancing execution, leadership, and client engagement.
Key Responsibilities:
- Lead the design and implementation of end-to-end data architecture across Snowflake and Azure environments
- Build and deliver multiple data architecture patterns (Snowflake-native, hybrid Azure + Snowflake, event-driven ingestion, and notebook-based workflows)
- Implement Medallion Architecture (Raw ? Silver ? Gold) with full lineage, governance, and promotion rules aligned with business definitions
- Design and implement real-time ingestion pipelines using Snowpipe, Event Grid, and secure file ingestion (e.g., PGP-encrypted sources)
- Develop scalable pipelines for high-volume, multi-source data ingestion, transformation, and validation using Snowpark (Python/Scala)
- Build and optimize dbt Core models including transformations, testing, and data quality enforcement
- Configure and manage Azure data services (ADLS Gen2, Azure Data Factory, Blob Storage, Private Endpoints, Managed Identity)
- Develop and maintain operational dashboards for monitoring pipeline health, SLA adherence, and data quality (targeting 99.5% uptime)
- Lead client-facing technical discussions, demos, and working sessions with executive stakeholders
- Translate complex technical decisions into clear business outcomes and data value narratives
- Scope work, estimate delivery timelines, and manage technical risks and change requests
- Lead and mentor supporting engineering resources while ensuring high-quality deliverables
- Drive AI/ML integration initiatives including Snowflake Cortex, Streamlit, and Azure AI services
- Ensure all architecture, pipelines, and transformations are well-documented and production-ready
Required Skills & Experience:
- 5+ years of experience in data engineering, with at least 2+ years in a technical lead or principal role
- Deep, hands-on expertise in:
- Snowflake architecture and optimization
- Snowpark (Python/Scala) for pipeline development
- Advanced SQL (complex transformations, performance tuning, schema design)
- dbt Core (modeling, testing, incremental processing, source management)
- Strong experience with Azure cloud ecosystem, including:
- ADLS Gen2, Azure Data Factory, Azure Functions
- Event Grid, Blob Storage, Private Endpoints, Managed Identity
- Proven experience building event-driven and streaming data pipelines (e.g., Snowpipe)
- Experience implementing Medallion/Lakehouse architecture patterns in production environments
- Experience with data governance tools (e.g., Microsoft Purview or equivalent)
- Proven ability to operate in client-facing consulting environments, managing stakeholders and delivering under pressure
- Strong leadership, communication, and problem-solving skills
- Ability to balance speed of delivery with system reliability and scalability
Desired Skills & Experience:
- Experience with AI/ML integration on data platforms (Snowflake Cortex, Streamlit, Azure AI Foundry, or similar)
- Exposure to PGP/GPG encryption workflows in automated pipelines
- Familiarity with Iceberg table formats or modern table storage frameworks
- Experience building lightweight data applications or UI layers (e.g., Streamlit)
- Domain knowledge in insurance, claims, or TPA data ecosystems
- Experience scaling data platforms across multiple business domains
- Advanced degree in Computer Science, Data Engineering, or related quantitative field
Pay: $87.50 – $125.00 /hr.
We are interested in every qualified candidate who is eligible to work in the United States. However, we are not able to provide sponsorship at this time or accept candidates who would require a corp-to-corp agreement.
If this position sounds like you, WE SHOULD TALK!
Your better future is ready, and we want to put the right tools in your hands to get you there. Let's go!
Keywords: Principal Data Engineer, Snowflake, Snowpark, Azure, ADLS Gen2, Data Lakehouse, Medallion Architecture, dbt Core, SQL, Python
Looking for more opportunities with Technology Partners? Check out technologypartners.net/jobs!
All offers of employment at Technology Partners are contingent upon clear results of a thorough background check and drug screening that meet corresponding laws and regulations at the city, state and federal level.
Pay ranges are influenced by candidate qualifications, experience, and role specifics, with the actual rate determined considering skills, market conditions, and are subject to change by the employer; pay negotiations follow all state and federal legal guidelines.
