Senior HPC Systems Administrator @ Siemens Healthineers

Job Requirements

The AI Supercomputing team at Siemens Healthineers is looking for a Senior HPC Systems Administrator to support the day-to-day operation of Sherlock, our enterprise AI supercomputing platform.

Sherlock powers AI research and development across medical imaging, diagnostics, and foundation model initiatives. The environment includes NVIDIA GPU clusters, high-speed InfiniBand networking, petabyte-scale storage, and Slurm workload management.

This is a hands-on operations role focused on maintaining system reliability, resolving production issues, and ensuring a high-quality user experience. You will work closely with the engineering team responsible for infrastructure architecture and future platform enhancements.

Key Responsibilities

HPC Operations

• Administer and maintain a large-scale Linux-based HPC environment.
• Monitor system health, troubleshoot issues, and ensure high availability.
• Perform OS maintenance, patching, upgrades, and configuration management.
• Conduct capacity planning, preventive maintenance, and system health checks.
• Maintain operational documentation and configuration records.

Slurm & GPU Administration

• Manage Slurm scheduling, resource allocation, and job accounting.
• Troubleshoot job failures, performance issues, and scheduling problems.
• Monitor and support NVIDIA GPU infrastructure, including hardware, drivers, and CUDA-related services.
• Support researchers and engineers with workload execution issues.

Storage & Network Support

• Monitor and troubleshoot storage, networking, and connectivity issues.
• Coordinate incident resolution with internal teams and external vendors.
• Support parallel file systems, backup, recovery, and data integrity processes.
• Help investigate network performance and infrastructure bottlenecks.

Monitoring & Automation

• Improve monitoring, alerting, and operational health checks.
• Automate routine administration tasks using Bash, Python, Ansible, or similar tools.
• Perform root-cause analysis and implement long-term fixes for recurring issues.
• Maintain incident response and service recovery procedures.
• Produce operational and reliability reports.

Incident Management & User Support

• Respond to and resolve production incidents across Linux, HPC, storage, and network systems.
• Communicate system status and incident impacts to stakeholders.
• Support users with cluster access, development environments, and software environments.
• Escalate complex infrastructure issues when needed.

Collaboration

• Partner with the AI Supercomputing Engineering team to ensure reliable platform operations.
• Support the transition of new infrastructure into production.
• Provide operational feedback on reliability, supportability, and maintainability.
• Assist with testing and implementation activities while maintaining focus on production operations.

Work Experience

• Bachelor’s degree in Computer Science, Engineering, IT, or equivalent experience.
• 5+ years of Linux system administration experience, ideally in HPC, research computing, or large-scale compute environments.
• Strong Linux troubleshooting skills, including networking, storage, system services, and kernel diagnostics.
• Experience with Slurm or a similar workload manager.
• Experience managing Linux server clusters.
• Familiarity with NVIDIA GPU hardware, drivers, and monitoring.
• Knowledge of distributed storage systems and high-performance networking (TCP/IP, InfiniBand, RDMA).
• Scripting and automation experience (Bash, Python, or similar).
• Proven ability to resolve complex production issues, perform root-cause analysis, and maintain reliable operations.
• Ability to work independently, manage priorities, and own production system stability.

Preferred Qualifications

• Experience with HPC management platforms such as NVIDIA Base Command Manager, Bright Cluster Manager, or Open OnDemand.
• Experience with distributed file systems (e.g., Ceph, Lustre, BeeGFS, VAST Data, IBM Spectrum Scale).
• Experience with NVIDIA DGX/HGX and multi-GPU environments.
• Knowledge of InfiniBand administration and troubleshooting.
• Familiarity with Slurm resource scheduling, accounting, and performance analysis.
• Experience with monitoring tools such as Prometheus, Grafana, NVIDIA DCGM.
• Knowledge of Linux access management, SSH, and enterprise authentication.
• Experience supporting containers (Apptainer/Singularity) and Python/Conda environments.
• Familiarity with remote development tools and operational processes such as incident management and change control.