Guidehouse
AI and Data Defense and Security
DatabricksEngineer
Neural analysis suggests this role is
optimal for Mid candidates.
“Databricks Engineer at Guidehouse. Skills: Databricks, PySpark, SQL, Delta Lake, ETL pipelines, batch and stream processing, CI/CD practices. Design, build, and maintain scalable ETL pipelines using Databricks, PySpark, SQL, and Delta Lake. Proficiency in batch and stream processing, autoloader, and compute optimization”
Industry & Context.
Troubleshoot and resolve Databricks job failures, cluster performance issues, data quality gaps, and pipeline bottlenecks
Ability to Obtain Public Trust, Ability to OBTAIN and MAINTAIN a Federal or DHS Public Trust
What They're Looking For.
Must Have
Bachelors Degree, 5+ years of relevant experience, Ability to OBTAIN and MAINTAIN a Federal or DHS Public Trust
Nice to Have
Active DHS Public Trust, Law Enforcement experience, Scrum Master Certification, Experience managing projects in similar size and scope, Experience managing IT projects using either Agile Scrum or Kanban methodologies, Experience managing projects leveraging commercial cloud infrastructure, Databricks Data Engineer Associate/Professional Certification, Experience with streaming frameworks (e. g. , Kafka, Event Hub), Experience with CI/CD tools (Azure DevOps, Jenkins, GitHub Actions), Experience with Unity Catalog, Terraform, or related IaC tools
What You'll Do.
and maintain scalable ETL pipelines using Databricks
Proficiency in batch and stream processing
and compute optimization
Troubleshoot and resolve Databricks job failures
cluster performance issues
and pipeline bottlenecks
Implement CI/CD practices for notebooks
Ensure all engineering work aligns with data governance
and compliance standards (Unity Catalog
data quality validation
How You'll Work.
Team & Collaboration
Collaborate with cross‑functional teams including data scientists, solution architects, and client stakeholders
Communication Scope
Communicate technical concepts and delivery impacts to non‑technical audiences
Process & Methodology
Lead Agile Project Management (stand‑ups, sprint planning, retrospectives) as support backlog refinement, Experience managing projects in similar size and scope, Experience managing IT projects using either Agile Scrum or Kanban methodologies
Full Job Description
**_Job Family_ :** Data Engineering & Architecture Consulting ** _Travel Required_ :** None ** _Clearance Required_ :** Ability to Obtain Public Trust ** _What You Will Do_ :** Guidehouse is seeking a Databricks Engineer to join our AI and Data Defense and Security team to support client projects involving large‑scale data pipelines, troubleshooting production workloads, and leading Agile delivery teams in support of organizational and mission goals within the Department of Homeland Security (DHS). This role requires hands‑on technical expertise, strong communication skills, and the ability to own workstreams independently. Responsibilities include, but are not limited to: * Design, build, and maintain scalable ETL pipelines using Databricks, PySpark, SQL, and Delta Lake. * Proficiency in batch and stream processing, autoloader, and compute optimization. * Troubleshoot and resolve Databricks job failures, cluster performance issues, data quality gaps, and pipeline bottlenecks. * Implement CI/CD practices for notebooks, workflows, and data jobs. * Lead Agile Project Management (stand‑ups, sprint planning, retrospectives) as needed; support backlog refinement. * Collaborate with cross‑functional teams including data scientists, solution architects, and client stakeholders. * Communicate technical concepts and delivery impacts to non‑technical audiences. * Ensure all engineering work aligns with data governance, security, and compliance standards (Unity Catalog, Delta constraints, Data Lineage). * Perform code reviews, data quality validation, and documentation. **_What You Will Need_ :** * Ability to OBTAIN and MAINTAIN a Federal or DHS Public Trust * Bachelors Degree * 5+ years of relevant experience ** _What Would Be Nice To Have_ :** * Active DHS Public Trust * Law Enforcement experience * Scrum Master Certification * Experience managing projects in similar size and scope. * Experience managing IT projects using either Agile Scrum or Kanban methodologies. * Experie
Applying for this Databricks Engineer role?
Most applicants get filtered before a human reads their resume. See if yours makes the cut.
How to Apply on Workday
- Workday has a multi-step form — save your progress after every section.
- "Apply With LinkedIn" can fail or lose data; manual entry is more reliable.
- Watch for the "Submit for Review" final step — hitting "Save" alone does not submit.
- Job requisition numbers are useful when following up with HR by email.
ANONYMOUS · UNFILTERED
What do employees actually say about Guidehouse?
Real rants from real employees. Read before you apply.