Document Outline

• HP StorageWorks Scalable File Share System User Guide
  • Contents
  • About this guide
    • Audience
    • Assumptions
    • New and changed features
    • Structure of this guide
    • HP SFS documentation
    • Documentation conventions
    • Naming conventions
    • For more information
    • Providing feedback
  • Safety considerations
    • Disconnect power cord before performing maintenance
  • 1: Overview
    • 1.1 Product overview
      • 1.1.1 Overview of the Lustre file system
      • 1.1.2 Overview of HP SFS
      • 1.1.3 HP SFS configurations
        • 1.1.3.1 Object Storage Servers
        • 1.1.3.2 Storage
          • 1.1.3.2.1 EVA4000 storage arrays
          • 1.1.3.2.2 SFS20 storage arrays
        • 1.1.3.3 Network connections
          • 1.1.3.3.1 Site network and interconnect
          • 1.1.3.3.2 Management network
    • 1.2 Supported operating system software
    • 1.3 Supported hardware
  • 2: The sfsmgr utility
    • 2.1 Overview
    • 2.2 Starting the SFS CLI
    • 2.3 Running sfsmgr commands
      • 2.3.1 Running sfsmgr commands at the command line
    • 2.4 Troubleshooting the sfsmgr command
  • 3: Operating the system
    • 3.1 Booting the system
    • 3.2 Shutting down the system
    • 3.3 Booting a server
    • 3.4 Booting multiple servers
    • 3.5 Shutting down an Object Storage Server or the MDS server
    • 3.6 Shutting down the administration server
    • 3.7 Stopping a file system
    • 3.8 Starting a file system
      • 3.8.1 Responding to formatting messages from the start filesystem command
    • 3.9 Unconfiguring storage arrays
      • 3.9.1 Unconfiguring EVA4000 arrays attached to the administration and MDS servers
      • 3.9.2 Unconfiguring EVA4000 arrays attached to Object Storage Servers
      • 3.9.3 Unconfiguring SFS20 arrays attached to the administration and MDS servers
      • 3.9.4 Unconfiguring SFS20 arrays attached to Object Storage Servers
    • 3.10 Deleting SFS20 array information from the database
    • 3.11 Managing licenses
      • 3.11.1 Understanding license file concepts
      • 3.11.2 Viewing license information
      • 3.11.3 Adding or removing licenses
      • 3.11.4 Licensing additional storage capacity to the system
      • 3.11.5 Upgrading from an SFSMDSOST license
      • 3.11.6 Deleting all licenses
    • 3.12 Managing remote access
      • 3.12.1 Creating authorizations
      • 3.12.2 Deleting authorizations
      • 3.12.3 Viewing authorizations
      • 3.12.4 Setting up ssh access to the system alias
    • 3.13 Locating servers and SFS20 arrays
  • 4: Viewing system information
    • 4.1 Viewing server information
      • 4.1.1 Server information after system parameter is changed
    • 4.2 Viewing file system information
    • 4.3 Viewing OST service information
    • 4.4 Viewing LUN information
    • 4.5 Viewing array information
    • 4.6 Viewing event logs
      • 4.6.1 Using the show log command
        • 4.6.1.1 Event filters
        • 4.6.1.2 Event severity
        • 4.6.1.3 Event facility
        • 4.6.1.4 Examples of event filters
      • 4.6.2 Capturing the system event log in a file
    • 4.7 Viewing performance statistics
      • 4.7.1 Overview of the collectl utility
      • 4.7.2 Overview of the information gathered by the collectl utility
      • 4.7.3 Using the Web server to view information gathered by the collectl utility
        • 4.7.3.1 Viewing overall throughput to OST devices from a server
        • 4.7.3.2 Viewing throughput to each OST device from a server
        • 4.7.3.3 Viewing throughput to OST devices from network devices on a server
        • 4.7.3.4 Viewing RPC transfer sizes or disk I/O transaction sizes
      • 4.7.4 Viewing information in the /proc file system
  • 5: Creating and modifying file systems
    • 5.1 Creating a file system - EVA4000 storage
      • 5.1.1 Step 1: Identifying unassigned LUNs - EVA4000 storage
      • 5.1.2 Step 2: Matching array numbers to physical arrays - EVA4000 storage
      • 5.1.3 Step 3: Setting roles, preferred controllers, and disk group information for LUNs - EVA4000 storage
      • 5.1.4 Step 4: Identifying file system information - EVA4000 storage
        • 5.1.4.1 OST Service ordering - EVA4000 storage
      • 5.1.5 Step 5: Using the create filesystem command - EVA4000 storage
        • 5.1.5.1 Using the create filesystem command in interactive mode - EVA4000 storage
        • 5.1.5.2 Using the create filesystem command in scripted mode - EVA4000 storage
      • 5.1.6 Step 6: Initializing the EVA4000 LUNs for file-system access
      • 5.1.7 Step 7: Backing up the system database
    • 5.2 Creating a file System - SFS20 storage
      • 5.2.1 Step 1: Assigning roles to LUNs - SFS20 storage (optional)
      • 5.2.2 Step 2: Identifying file system information - SFS20 storage
        • 5.2.2.1 OST service ordering - SFS20 storage
        • 5.2.2.2 Creating mirrored OST LUNs on SFS20 arrays
      • 5.2.3 Step 3: Using the create filesystem command - SFS20 storage
        • 5.2.3.1 Using the create filesystem command in interactive mode - SFS20 storage
        • 5.2.3.2 Using the create filesystem command in scripted mode - SFS20 storage
      • 5.2.4 Step 4: Backing up the system database
    • 5.3 LUN Roles, used by, and preferred server changes
    • 5.4 Operating a file system
    • 5.5 Recovering from failure of the create filesystem command
      • 5.5.1 If the create filesystem command times out while configuring file system services
    • 5.6 Modifying file system attributes
      • 5.6.1 Adding OST services to a file system
        • 5.6.1.1 Usable space on new OST services
      • 5.6.2 Changing the preferred server for OST and MDS services
      • 5.6.3 Modifying service mount options
      • 5.6.4 Modifying file system interconnects
      • 5.6.5 Changing other file system attributes
      • 5.6.6 Rewriting LDAP configuration data
    • 5.7 Managing quotas
      • 5.7.1 Understanding quota tuning
      • 5.7.2 Configuring quotas on a file system
        • 5.7.2.1 Step 1: Verifying that the quota option is enabled on the HP SFS system
        • 5.7.2.2 Step 2: Enabling quota functionality on a file system
        • 5.7.2.3 Step 3: Preparing the client nodes to use quotas on the file system
        • 5.7.2.4 Step 4: Configuring user and group information on a file system
        • 5.7.2.5 Step 5: Activating quota functionality on a file system
        • 5.7.2.6 Step 6: Specifying quota limits for users and/or groups
      • 5.7.3 Checking quota limits
      • 5.7.4 Disabling quota functionality on one or all file systems
    • 5.8 Managing supplementary groups resolution
    • 5.9 Creating a system nickname
    • 5.10 Deleting file systems
    • 5.11 Managing space on OST services
      • 5.11.1 Managing out-of-space alerts
      • 5.11.2 Identifying files on OST services
      • 5.11.3 Deactivating OST services
  • 6: Verifying, diagnosing, and maintaining the system
    • 6.1 Verifying the system
      • 6.1.1 Verifying the system configuration and diagnosing errors using the syscheck command
      • 6.1.2 Verifying the operation of the power management
      • 6.1.3 Verifying EVA4000 storage failover configuration
      • 6.1.4 Confirming the status of SFS20 hardware using the hpacucli utility
        • 6.1.4.1 Identifying SFS20 array serial numbers
        • 6.1.4.2 Reviewing SFS20 array information
        • 6.1.4.3 Identifying Smart Array 6404 HBA controller serial numbers
        • 6.1.4.4 Reviewing Smart Array 6404 HBA controller information
      • 6.1.5 Verifying LUN performance
        • 6.1.5.1 Verifying the performance of all unmirrored OST LUNs
        • 6.1.5.2 Verifying the performance of LUNs on a single server
      • 6.1.6 Verifying the management network
      • 6.1.7 Verifying the operation of supplementary groups
        • 6.1.7.1 Verifying the operation of the ssh connection
        • 6.1.7.2 Verifying access for supplementary groups
      • 6.1.8 Interconnect diagnostics
        • 6.1.8.1 Testing Gigabit Ethernet interconnect performance
        • 6.1.8.2 Examining the Myrinet interconnect
          • 6.1.8.2.1 Examining the Myrinet adapter and interconnect link
          • 6.1.8.2.2 Testing Myrinet interconnect performance using the net_test.bash command
          • 6.1.8.2.3 Testing Myrinet interconnect performance using the gm_allsize command
        • 6.1.8.3 Examining the Quadrics adapter and interconnect link
          • 6.1.8.3.1 Testing for presence of Quadrics adapter
          • 6.1.8.3.2 Running the qselantest diagnostics
          • 6.1.8.3.3 Example output from a qselantest test
          • 6.1.8.3.4 Testing Quadrics interconnect performance
        • 6.1.8.4 Examining the Voltaire InfiniBand interconnect HCA adapter and interconnect link
          • 6.1.8.4.1 Testing for presence of Voltaire HCA adapter
          • 6.1.8.4.2 Testing Voltaire InfiniBand interconnect performance
      • 6.1.9 Client performance diagnostics
        • 6.1.9.1 Sequential OST service tests
        • 6.1.9.2 Parallel OST service tests
        • 6.1.9.3 Overall file system performance tests
    • 6.2 Managing email alerts
      • 6.2.1 Configuring default email alert addresses
      • 6.2.2 Configuring email alerts
      • 6.2.3 Constructing email alert filters
      • 6.2.4 Default email alerts
      • 6.2.5 Modifying email alerts
      • 6.2.6 Creating email alerts
      • 6.2.7 Viewing email alerts
      • 6.2.8 Disabling and enabling email alerts
      • 6.2.9 Deleting email alerts
      • 6.2.10 Verifying email alert operation
    • 6.3 Backing up and restoring system data
      • 6.3.1 Backing up the system database
      • 6.3.2 Saving database backup files
      • 6.3.3 Restoring a system database backup
      • 6.3.4 Deleting a system database backup
    • 6.4 Backing up and restoring file system data
      • 6.4.1 Using HP OpenView Storage Data Protector to back up file systems
    • 6.5 Removing log files
    • 6.6 Viewing and clearing the Integrated Management Log
      • 6.6.1 Overview
      • 6.6.2 Viewing the IML
      • 6.6.3 Clearing the IML
  • 7: Changing the system parameters
    • 7.1 Changing system parameters
    • 7.2 Using the configure system command
    • 7.3 Using the configure server command
    • 7.4 Adding, changing, or removing the Alias IP on a Gigabit Ethernet network
    • 7.5 Changing the IP address of an individual server
    • 7.6 Changing the attributes of bonded Gigabit Ethernet networks
    • 7.7 Partitioning an InfiniBand interconnect
    • 7.8 Removing InfiniBand partitioning
    • 7.9 Changing the iLO user name and password
  • 8: Replacing, adding, and removing hardware, and upgrading firmware
    • 8.1 Replacing hardware components
      • 8.1.1 Replacing the administration or MDS server, or the motherboard component in the administration or MDS server
      • 8.1.2 Replacing an Object Storage Server or the motherboard component in an Object Storage Server
      • 8.1.3 Replacing a single disk on a server
      • 8.1.4 Replacing both internal disks on the administration or the MDS server
      • 8.1.5 Replacing an internal disk on an Object Storage Server
      • 8.1.6 Replacing a Storage Host Bus Adapter on an EVA4000 array (HP Part Number FCA2214 DC)
      • 8.1.7 Replacing a Fibre Channel cable (from server to Fibre Channel switch)
      • 8.1.8 Replacing a Fibre Channel switch
      • 8.1.9 Replacing an EVA4000 controller
      • 8.1.10 Replacing a disk in an SFS20 array
      • 8.1.11 Replacing an SFS20 controller module
      • 8.1.12 Replacing an SFS20 battery pack and cache
      • 8.1.13 Replacing an SFS20 fan assembly, power supply unit, or mounting bracket covers
      • 8.1.14 Replacing SFS20 non-hot-pluggable components
      • 8.1.15 Replacing Smart Array 6404 adapter
      • 8.1.16 Replacing a Quadrics network adapter
      • 8.1.17 Relocating a Quadrics cable to a different port on a Quadrics switch
      • 8.1.18 Replacing a Quadrics cable
      • 8.1.19 Replacing a Myrinet network adapter
      • 8.1.20 Relocating a Myrinet cable to a different port on a Myrinet network
      • 8.1.21 Replacing a Myrinet cable
      • 8.1.22 Replacing a dual-port Gigabit Ethernet adapter (for dual or bonded Gigabit Ethernet)
      • 8.1.23 Relocating a Gigabit Ethernet cable to a different port on a Gigabit Ethernet network
      • 8.1.24 Replacing a Gigabit Ethernet cable
      • 8.1.25 Replacing a Voltaire HCA adapter
      • 8.1.26 Replacing an InfiniBand interconnect cable (from server to InfiniBand switch)
      • 8.1.27 Replacing a Voltaire InfiniBand switch
      • 8.1.28 Relocating an InfiniBand cable to a different port on the InfiniBand switch
      • 8.1.29 Replacing a Power Distribution Unit (PDU) on a rack
      • 8.1.30 Replacing a Power Distribution Module (AC power strip) on a rack
      • 8.1.31 Replacing a Power Distribution Unit (PDU) on an HP ProCurve Switch 2650 or HP ProCurve Switch 2626
    • 8.2 Upgrading firmware
      • 8.2.1 Upgrading firmware on a server
        • 8.2.1.1 Upgrading online using the OnlineROM Flash Component executable
        • 8.2.1.2 Upgrading offline using a USB pen drive
        • 8.2.1.3 Upgrading offline using a floppy disk drive - G3 servers only
      • 8.2.2 Upgrading firmware on Smart Array 6404 adapters and SFS20 arrays
      • 8.2.3 Upgrading firmware on an InfiniBand adapter
        • 8.2.3.1 Upgrading the firmware on a Voltaire PCI-X HCA adapter
    • 8.3 Adding and removing components
      • 8.3.1 Adding Object Storage Servers
      • 8.3.2 Removing Object Storage Servers
      • 8.3.3 Adding SFS20 arrays
      • 8.3.4 Removing SFS20 arrays
      • 8.3.5 Adding a dual or a bonded Gigabit Ethernet interconnect
      • 8.3.6 Removing a dual or a bonded Gigabit Ethernet interconnect
  • 9: Troubleshooting
    • 9.1 Server fails to boot during installation
    • 9.2 Server fails to boot
    • 9.3 Server stops responding
    • 9.4 Server fails to mount administration LUN - EXT3-fs error
    • 9.5 Benign message displayed by the configure system command
    • 9.6 The configure server command fails because static routes are incorrectly configured
    • 9.7 Server with Quadrics interconnect may fail to boot
    • 9.8 The scan mac command fails
    • 9.9 Setting Lustre debug level
    • 9.10 System database has become corrupted
    • 9.11 Service LUN has become corrupted
    • 9.12 Replacing a service LUN with a spare service LUN
    • 9.13 Booting a server in single-user mode
    • 9.14 The SFS CLI reports that the database is not ready
    • 9.15 The configure array command fails
      • 9.15.1 SFS20 array cabling problems
      • 9.15.2 Preferred server for the SFS20 array is down
    • 9.16 Command remains in the Allocated state
    • 9.17 The configure server command remains in the Unallocated state
    • 9.18 Emergency clustat events occur during configure server command
    • 9.19 The show log command fails with corrupted record
    • 9.20 SFS CLI command fails with error: Could not insert duplicate record
    • 9.21 Press "F1" key to continue prompt
    • 9.22 Gigabit Ethernet -113 error code
    • 9.23 Troubleshooting the Quadrics interconnect
      • 9.23.1 Start, stop, and loading problems
      • 9.23.2 Nodeset and Node ID information
      • 9.23.3 Checking active Lustre communications over the Quadrics interconnect
      • 9.23.4 Gathering debugging information
    • 9.24 Troubleshooting the Myrinet interconnect
      • 9.24.1 Start, stop, and check status
      • 9.24.2 Myrinet interconnect stress test
      • 9.24.3 Nodeset and Node ID information
      • 9.24.4 Disconnected link in Myrinet 2XP interconnect
    • 9.25 Troubleshooting the Voltaire InfiniBand interconnect
      • 9.25.1 Start, stop, and loading problems
        • 9.25.1.1 Determining whether Voltaire InfiniBand interconnect is loaded
        • 9.25.1.2 Starting, stopping, and auto-starting the Voltaire InfiniBand interconnect
          • 9.25.1.2.1 Using the ib-setup utility
          • 9.25.1.2.2 From the command line
        • 9.25.1.3 Server hangs when Voltaire InfiniBand interconnect is started
      • 9.25.2 Voltaire HCA adapter is not recognized
      • 9.25.3 Voltaire HCA adapter is not activated
      • 9.25.4 Connection and data transfer problems
      • 9.25.5 AD_TAVOR : vvi_mlx_poll_for_completion messages
    • 9.26 Troubleshooting file systems
      • 9.26.1 Problems creating a file system
      • 9.26.2 Identifying servers serving OST services
      • 9.26.3 The start filesystem command may fail twice
      • 9.26.4 Troubleshooting the stop filesystem command
      • 9.26.5 Using the MPI Lustre repair utility to repair file systems
        • 9.26.5.1 Using the repair-lfsck script and the generated file system-specific shell scripts
          • 9.26.5.1.1 Running a generated file system repair script
        • 9.26.5.2 Repairing or verifying individual MDS or OST services
      • 9.26.6 MDS or OST services stay in the recovering state
      • 9.26.7 MDS and OST service recovery process
      • 9.26.8 Rebalancing file system services
      • 9.26.9 Troubleshooting supplementary groups access
    • 9.27 Troubleshooting file system performance
      • 9.27.1 Performance troubleshooting
      • 9.27.2 Verifying file striping
        • 9.27.2.1 Recreating files
      • 9.27.3 Checking for unbalanced distribution of OST services
      • 9.27.4 Checking for unbalanced controllers in EVA4000 arrays
      • 9.27.5 Examining the system logs for errors
      • 9.27.6 Examining EVA4000 storage subsystems for errors
      • 9.27.7 Examining SFS20 storage subsystems for errors
      • 9.27.8 Examining the interconnect switch for errors
      • 9.27.9 Verifying performance statistics on Fibre Channel switches
      • 9.27.10 Troubleshooting slow commit messages
    • 9.28 Troubleshooting EVA4000 array connectivity
    • 9.29 Troubleshooting LUN presentation
    • 9.30 Accessing consoles
    • 9.31 Accessing the iLO component
      • 9.31.1 Configuring the iLO component
        • 9.31.1.1 Using the remote console
        • 9.31.1.2 Using a Web browser
      • 9.31.2 Troubleshooting iLO access
    • 9.32 Troubleshooting licenses
    • 9.33 Troubleshooting failed SFS20 arrays
      • 9.33.1 Identifying failed SFS20 arrays
      • 9.33.2 Recovering from a temporary SFS20 array failure
      • 9.33.3 Recovering degraded MDS or OST services
    • 9.34 Handling Disk Errors on SFS20 storage
      • 9.34.1 Disks showing the removed/failed state
      • 9.34.2 Disks showing the predict fail state
      • 9.34.3 Disks showing the logging errors state
    • 9.35 Recovering degraded MDS services on systems using EVA4000 storage
    • 9.36 System log files
    • 9.37 Administration service restarts every one minute (attempting to start the evlogd daemon)
    • 9.38 The MDS service fails with an ASSERTION(ino ==inode->i_ino) message
    • 9.39 The MDS service repeatedly crashes with an LBUG error
    • 9.40 Rebuilding logical drives after disk failures
    • 9.41 Determining if the Network ID of a server on a Quadrics or Myrinet interconnect has been changed
    • 9.42 Troubleshooting client mount failures
    • 9.43 Recovering after using the ifdown command
    • 9.44 Password needed for ssh access
    • 9.45 The syscheck command reports that the Insight Manager Agents are not running
  • A: CLI commands
    • A.1 General information
    • A.2 accept license command
    • A.3 activate ost command
    • A.4 boot server command
    • A.5 configure commands
      • A.5.1 configure array
      • A.5.2 configure server
      • A.5.3 configure system
    • A.6 create commands
      • A.6.1 create alert
      • A.6.2 create attribute
      • A.6.3 create authorization
      • A.6.4 create database_backup
      • A.6.5 create filesystem
    • A.7 deactivate ost command
    • A.8 delete commands
      • A.8.1 delete alert
      • A.8.2 delete array
      • A.8.3 delete attribute
      • A.8.4 delete authorization
      • A.8.5 delete database_backup
      • A.8.6 delete filesystem
    • A.9 disable commands
      • A.9.1 disable alert
      • A.9.2 disable server
    • A.10 enable commands
      • A.10.1 enable alert
      • A.10.2 enable server
    • A.11 help command
    • A.12 kill command command
    • A.13 modify commands
      • A.13.1 modify alert
      • A.13.2 modify filesystem
    • A.14 monitor command command
    • A.15 pause filesystem command
    • A.16 quit command
    • A.17 restore database_backup command
    • A.18 scan mac command
    • A.19 set commands
      • A.19.1 set array
      • A.19.2 set attribute
      • A.19.3 set lun
      • A.19.4 set nics
      • A.19.5 set password
      • A.19.6 set server
    • A.20 show commands
      • A.20.1 show alert
      • A.20.2 show array
      • A.20.3 show attribute
      • A.20.4 show authorization
      • A.20.5 show commands
      • A.20.6 show database_backups
      • A.20.7 show filesystem
      • A.20.8 show license
      • A.20.9 show log
      • A.20.10 show lun
      • A.20.11 show network
      • A.20.12 show ost
      • A.20.13 show server
      • A.20.14 show support
      • A.20.15 show system
      • A.20.16 show version
    • A.21 shutdown server command
    • A.22 start filesystem command
    • A.23 stop filesystem command
    • A.24 syscheck command
    • A.25 unconfigure array command
    • A.26 ! command
  • B: Performance figures
    • B.1 I/O performance
    • B.2 Network performance
    • B.3 SFS20 array configuration
    • B.4 SFS20 RAID5 and ADG performance
    • B.5 Default file system stripe count
    • B.6 Bandwidth variation-number of OST services and number of client nodes
    • B.7 Single client node bandwidth
    • B.8 Gigabit Ethernet bandwidth
    • B.9 Meta-data operations from a single client node
  • C: File system configuration examples
    • C.1 EVA4000 storage examples
    • C.2 SFS20 storage examples
  • D: RAID rebuild timing information
    • D.1 RAID rebuild information
  • E: HP SFS specifications
    • E.1 Supported number of Object Storage Servers
    • E.2 Supported number of OST services
    • E.3 Supported number of file systems
    • E.4 File and file system limits
    • E.5 Maximum file system default stripe size is 1024MB
    • E.6 File system default stripe size and client page size
    • E.7 File stripe size and client page size
  • Glossary
  • Index