Senior Databricks Architect

Job Title: Sr. Databricks Architect

Location: Remote

Company: Zachary Piper Solutions

Clearance Required: Public Trust

Full Job Description:

Zachary Piper Solutions is looking for an Senior Databricks Architect to join a growing team in support of a government agency. This position will be fully remote.

Responsibilities/Overview:

• Design and implement event-driven architecture to support scalable, resilient, and dynamic GenAI applications.

• Refactor and rehost existing applications onto cloud platforms, optimizing for performance and scalability.

• Collaborate on the development of Platform Architecture, ensuring seamless integration and efficient operation of cloud services.

• Develop and optimize Databricks Solution Architectures to handle large-scale data processing and analytics workloads.

• Engineer robust Bedrock Solution Architectures to support infrastructure provisioning and management at scale.

• Define and execute Deployment Architectures that allow for automated and reliable deployments of applications and services.

• Establish and enforce Security Architectures to protect data and maintain compliance with industry and regulatory standards.

• Create Use Case Solution Architectures, transforming business requirements into technical solutions.

• Proactively identify and address technical issues and challenges to improve the efficiency and reliability of cloud-based systems.

• Work with cross-functional teams to facilitate the smooth deployment of cloud services across various operational environments.

• Document architectural decisions and create best practice guidelines for cloud development and deployment.

• Stay abreast of emerging cloud technologies and recommend adaptations to keep ahead of industry trends.

• Utilize Agile and DevSecOps methodologies and practices.

• Coach and provide direction to junior engineers in the Databricks and data engineering areas.

• Lead efforts to integrate various data sources, ensuring seamless data flow and transformation. Develop strategies for data ingestion, cleansing, normalization, and integration.

• Leverage experience with MLFlow and general CI/CD principles for automating model deployment.

• Collaborate with customers and critical resources in an SME program to come up with solutions that align with long-term roadmaps and strategy.

• Execute Databricks upgrades, particularly in the federal space.

Required Qualifications:

• Must be a US Citizen with the ability to obtain a Public Trust clearance

• Bachelors degree and seven (7) years or more of related experience or Masters and five (5) or more years of related experience. 

• Proven experience in cloud engineering and architecture, with a focus on event-driven solutions.

• In-depth knowledge of public cloud platforms (e.g., AWS, Azure, Google Cloud, Oracle).

• Proficiency with Databricks and understanding of big data ecosystems.

• Experience with Infrastructure as Code (IaC) using tools like Terraform, AWS CloudFormation, etc.

• Strong understanding of deployment pipelines and automated testing frameworks.

• Demonstrated experience in implementing security practices in cloud architectures.

• Clear understanding of microservices architecture and containerization technologies (e.g., Docker, Kubernetes).

• Excellent verbal and written communication skills, with the ability to document and explain complex architectures to non-technical stakeholders.

• Databricks Certification and any other AI/ML or data engineering certifications (AWS, Azure, GCP).

Desired:

• Previous experience working in a regulated environment, particularly within government agencies like the IRS.

• Certifications related to AWS or cloud architecture.

• Experience with AI/ML platforms and understanding of their integration with cloud environments.

• Prior experience with event-driven architectures and related technologies (e.g., Apache Kafka, RabbitMQ).

• Security Clearance desired

Compensation and Benefits:

·        $130,000 - $155,000+ Dependent on experience

·        Comprehensive Benefits: Medical, Dental, Vision, 401K, PTO, Sick Leave as required by law, and Holidays.

This job opens for applications on 7/16/25. Applications for this job will be accepted for at least 30 days from the posting date.