Document Outline

• User's Guide
• Table of Contents
• About This Document
  • Intended Audience
  • New and Changed Information in This Edition
  • Typographic Conventions
  • HP XC and Related HP Products Information
  • Related Information
  • Manpages
  • HP Encourages Your Comments
• 1 Overview of the User Environment
  • 1.1 System Architecture
    • 1.1.1 HP XC System Software
    • 1.1.2 Operating System
    • 1.1.3 Node Platforms
    • 1.1.4 Node Specialization
    • 1.1.5 Storage and I/O
    • 1.1.6 File System
    • 1.1.7 System Interconnect Network
    • 1.1.8 Network Address Translation (NAT)
  • 1.2 Determining System Configuration Information
  • 1.3 User Environment
    • 1.3.1 LVS
    • 1.3.2 Modules
    • 1.3.3 Commands
  • 1.4 Application Development Environment
    • 1.4.1 Parallel Applications
    • 1.4.2 Serial Applications
  • 1.5 Run-Time Environment
    • 1.5.1 SLURM
    • 1.5.2 Load Sharing Facility (LSF-HPC)
    • 1.5.3 Standard LSF
    • 1.5.4 How LSF-HPC and SLURM Interact
    • 1.5.5 HP-MPI
  • 1.6 Components, Tools, Compilers, Libraries, and Debuggers
• 2 Using the System
  • 2.1 Logging In to the System
    • 2.1.1 LVS Login Routing
    • 2.1.2 Using the Secure Shell to Log In
  • 2.2 Overview of Launching and Managing Jobs
    • 2.2.1 Introduction
    • 2.2.2 Getting Information About Queues
    • 2.2.3 Getting Information About Resources
    • 2.2.4 Getting Information About System Partitions
    • 2.2.5 Launching Jobs
    • 2.2.6 Getting Information About Your Jobs
    • 2.2.7 Stopping and Suspending Jobs
    • 2.2.8 Resuming Suspended Jobs
  • 2.3 Performing Other Common User Tasks
    • 2.3.1 Determining the LSF Cluster Name and the LSF Execution Host
  • 2.4 Getting System Help and Information
• 3 Configuring Your Environment with Modulefiles
  • 3.1 Overview of Modules
  • 3.2 Supplied Modulefiles
  • 3.3 Modulefiles Automatically Loaded on the System
  • 3.4 Viewing Available Modulefiles
  • 3.5 Viewing Loaded Modulefiles
  • 3.6 Loading a Modulefile
    • 3.6.1 Loading a Modulefile for the Current Session
    • 3.6.2 Automatically Loading a Modulefile at Login
  • 3.7 Unloading a Modulefile
  • 3.8 Viewing Modulefile-Specific Help
  • 3.9 Modulefile Conflicts
  • 3.10 Creating a Modulefile
• 4 Developing Applications
  • 4.1 Application Development Environment Overview
  • 4.2 Compilers
    • 4.2.1 MPI Compiler
  • 4.3 Examining Nodes and Partitions Before Running Jobs
  • 4.4 Interrupting a Job
  • 4.5 Setting Debugging Options
  • 4.6 Developing Serial Applications
    • 4.6.1 Serial Application Build Environment
    • 4.6.2 Building Serial Applications
      • 4.6.2.1 Compiling and Linking Serial Applications
  • 4.7 Developing Parallel Applications
    • 4.7.1 Parallel Application Build Environment
      • 4.7.1.1 Modulefiles
      • 4.7.1.2 HP-MPI
      • 4.7.1.3 OpenMP
      • 4.7.1.4 Pthreads
      • 4.7.1.5 Quadrics SHMEM
      • 4.7.1.6 MPI Library
      • 4.7.1.7 Intel Fortran and C/C++Compilers
      • 4.7.1.8 PGI Fortran and C/C++ Compilers
      • 4.7.1.9 GNU C and C++ Compilers
      • 4.7.1.10 Pathscale Compilers
      • 4.7.1.11 GNU Parallel Make
      • 4.7.1.12 MKL Library
      • 4.7.1.13 ACML Library
      • 4.7.1.14 Other Libraries
    • 4.7.2 Building Parallel Applications
      • 4.7.2.1 Compiling and Linking Non-MPI Applications
      • 4.7.2.2 Compiling and Linking HP-MPI Applications
      • 4.7.2.3 Examples of Compiling and Linking HP-MPI Applications
  • 4.8 Developing Libraries
    • 4.8.1 Designing Libraries for the CP4000 Platform
• 5 Submitting Jobs
  • 5.1 Overview of Job Submission
  • 5.2 Submitting a Serial Job Using LSF-HPC
    • 5.2.1 Submitting a Serial Job with the LSF bsub Command
    • 5.2.2 Submitting a Serial Job Through SLURM Only
  • 5.3 Submitting a Parallel Job
    • 5.3.1 Submitting a Non-MPI Parallel Job
    • 5.3.2 Submitting a Parallel Job That Uses the HP-MPI Message Passing Interface
    • 5.3.3 Submitting a Parallel Job Using the SLURM External Scheduler
  • 5.4 Submitting a Batch Job or Job Script
  • 5.5 Submitting Multiple MPI Jobs Across the Same Set of Nodes
    • 5.5.1 Using a Script to Submit Multiple Jobs
    • 5.5.2 Using a Makefile to Submit Multiple Jobs
  • 5.6 Submitting a Job from a Host Other Than an HP XC Host
  • 5.7 Running Preexecution Programs
• 6 Debugging Applications
  • 6.1 Debugging Serial Applications
  • 6.2 Debugging Parallel Applications
    • 6.2.1 Debugging with TotalView
      • 6.2.1.1 SSH and TotalView
      • 6.2.1.2 Setting Up TotalView
      • 6.2.1.3 Using TotalView with SLURM
      • 6.2.1.4 Using TotalView with LSF-HPC
      • 6.2.1.5 Setting TotalView Preferences
      • 6.2.1.6 Debugging an Application
      • 6.2.1.7 Debugging Running Applications
      • 6.2.1.8 Exiting TotalView
• 7 Monitoring Node Activity
  • 7.1 Installing the Node Activity Monitoring Software
  • 7.2 Using the xcxclus Utility to Monitor Nodes
  • 7.3 Plotting the Data from the xcxclus Datafiles
  • 7.4 Using the xcxperf Utility to Display Node Performance
  • 7.5 Plotting the Node Performance Data
  • 7.6 Running Performance Health Tests
• 8 Tuning Applications
  • 8.1 Using the Intel Trace Collector and Intel Trace Analyzer
    • 8.1.1 Building a Program — Intel Trace Collector and HP-MPI
    • 8.1.2 Running a Program – Intel Trace Collector and HP-MPI
  • 8.2 The Intel Trace Collector and Analyzer with HP-MPI on HP XC
    • 8.2.1 Installation Kit
    • 8.2.2 HP-MPI and the Intel Trace Collector
  • 8.3 Visualizing Data – Intel Trace Analyzer and HP-MPI
• 9 Using SLURM
  • 9.1 Introduction to SLURM
  • 9.2 SLURM Utilities
  • 9.3 Launching Jobs with the srun Command
    • 9.3.1 The srun Roles and Modes
      • 9.3.1.1 The srun Roles
      • 9.3.1.2 The srun Modes
    • 9.3.2 Using the srun Command with HP-MPI
    • 9.3.3 Using the srun Command with LSF-HPC
  • 9.4 Monitoring Jobs with the squeue Command
  • 9.5 Terminating Jobs with the scancel Command
  • 9.6 Getting System Information with the sinfo Command
  • 9.7 Job Accounting
  • 9.8 Fault Tolerance
  • 9.9 Security
• 10 Using LSF-HPC
  • 10.1 Information for LSF-HPC
  • 10.2 Overview of LSF-HPC Integrated with SLURM
  • 10.3 Differences Between LSF-HPC and LSF-HPC Integrated with SLURM
  • 10.4 Job Terminology
  • 10.5 Using LSF-HPC Integrated with SLURM in the HP XC Environment
    • 10.5.1 Useful Commands
    • 10.5.2 Job Startup and Job Control
    • 10.5.3 Preemption
  • 10.6 Submitting Jobs
  • 10.7 LSF-SLURM External Scheduler
  • 10.8 How LSF-HPC and SLURM Launch and Manage a Job
  • 10.9 Determining the LSF Execution Host
  • 10.10 Determining Available System Resources
    • 10.10.1 Examining System Core Status
    • 10.10.2 Getting Information About the LSF Execution Host Node
    • 10.10.3 Getting Host Load Information
    • 10.10.4 Examining System Queues
    • 10.10.5 Getting Information About the lsf Partition
  • 10.11 Getting Information About Jobs
    • 10.11.1 Getting Job Allocation Information
    • 10.11.2 Examining the Status of a Job
    • 10.11.3 Viewing the Historical Information for a Job
  • 10.12 Translating SLURM and LSF-HPC JOBIDs
  • 10.13 Working Interactively Within an Allocation
  • 10.14 LSF-HPC Equivalents of SLURM srun Options
• 11 Advanced Topics
  • 11.1 Enabling Remote Execution with OpenSSH
  • 11.2 Running an X Terminal Session from a Remote Node
  • 11.3 Using the GNU Parallel Make Capability
    • 11.3.1 Example Procedure 1
    • 11.3.2 Example Procedure 2
    • 11.3.3 Example Procedure 3
  • 11.4 Local Disks on Compute Nodes
  • 11.5 I/O Performance Considerations
    • 11.5.1 Shared File View
    • 11.5.2 Private File View
  • 11.6 Communication Between Nodes
  • 11.7 Using MPICH on the HP XC System
    • 11.7.1 Using MPICH with SLURM Allocation
    • 11.7.2 Using MPICH with LSF Allocation
• A Examples
  • A.1 Building and Running a Serial Application
  • A.2 Launching a Serial Interactive Shell Through LSF-HPC
  • A.3 Running LSF-HPC Jobs with a SLURM Allocation Request
    • A.3.1 Example 1. Two Cores on Any Two Nodes
    • A.3.2 Example 2. Four Cores on Two Specific Nodes
  • A.4 Launching a Parallel Interactive Shell Through LSF-HPC
  • A.5 Submitting a Simple Job Script with LSF-HPC
  • A.6 Submitting an Interactive Job with LSF-HPC
  • A.7 Submitting an HP-MPI Job with LSF-HPC
  • A.8 Using a Resource Requirements String in an LSF-HPC Command
• Glossary
• Index