Executar a ferramenta de verificação de integridade e pré-atualização do Hyperflex

Introdução

Este documento descreve o processo para executar a ferramenta Hypercheck Health e Pre-upgrade.

Pré-requisitos

Requisitos

A Cisco recomenda ter conhecimento deste tópico:

• Hiperflex

Componentes Utilizados

As informações neste documento são baseadas na ferramenta Hypercheck Health e Pre-upgrade.

As informações neste documento foram criadas a partir de dispositivos em um ambiente de laboratório específico. Todos os dispositivos utilizados neste documento foram iniciados com uma configuração (padrão) inicial. Se a rede estiver ativa, certifique-se de que você entenda o impacto potencial de qualquer comando.

Informações de Apoio

Esta ferramenta é um utilitário que executa autoverificações proativas em sistemas Hyperflex para garantir sua estabilidade e resiliência. Ele ajuda a automatizar uma lista de verificações de integridade e pré-atualização nos sistemas Hyperflex para economizar tempo durante as operações de atualização e manutenção do Hyperflex.

Observação: sempre faça o download da versão mais recente da ferramenta antes de usá-la. Como a ferramenta é aprimorada com frequência, o uso de uma versão anterior pode resultar na ausência de verificações importantes.

Sistemas HX suportados

• Versões Hyperflex - 1.8, 2.0, 2.1, 2.5, 2.6, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5
• Cluster Padrão Hyperflex
• Cluster Estendido Hyperflex
• Cluster de borda Hyperflex (2 nós, 3 nós e 4 nós)
• Compatível somente com cluster Hyperflex no VMWare ESXi

Observação: para obter informações sobre como executar o Hypercheck no cluster HyperV Hyperflex, consulte Hypercheck: Hyperflex Health & Pre-Upgrade Check Tool - HyperV.

Quando usar

Os horários efetivos para usar a ferramenta de verificação de pré-atualização e integridade do Hyperflex são:

• Antes das atualizações do Hyperflex
• Verificação de integridade do Hyperflex antes e depois das janelas de manutenção
• Para identificar unidades/discos com falha
• Quando você trabalha com o Cisco TAC
• Verificação de integridade proativa a qualquer momento

How to Use

HX Versão 4.5 e Mais Recente

Etapa 1. Inicie uma conexão SSH com a SCVM (Storage Controller VM, máquina virtual do controlador de armazenamento) usando o CMIP (Cluster Management IP, IP de gerenciamento de cluster), que é seu IP de conexão HX.

Etapa 2. Execute o comando hypercheck .

admin:~$ hypercheck

Etapa 3. Insira a senha do administrador do SCVM quando solicitado e insira a senha raiz do ESXi.

admin:~$ hypercheck

                  HX Health Check 4.5.0

Please enter below info of HX-Cluster:
Enter the HX-Cluster Root Password:
Enter the HX-Cluster Admin Password:
Enter the ESX Root Password:

HX Versão 4.0 e Anterior

Etapa 1. Faça o download Hyperflex-Hypercheck.zip da conta Cisco github devnet. Obtenha a cópia mais recente, que tem as últimas melhorias e atualizações.

Observação: somente usuários registrados da Cisco podem acessar ferramentas, arquivos e informações internas da Cisco.

Observação: use somente o script baixado da conta Cisco github devnet.

Etapa 2. Carregue-o no SCVM com o CMIP.

Use seu método preferido - scp/sftp/ftp/tftp - Para copiar o Hyperflex-Hypercheck.zip para o /tmp diretório.

Para MAC:

Executar SCP a partir do CLI (confirme se o Hyperflex-Hypercheck.zip está na mesma pasta de onde você executa SCP).

# scp Hyperflex-Hypercheck.zip root@<scvm-eth0:mgmtip>:/tmp/.

Use isso para identificar o IP de gerenciamento de cluster em seu ambiente HX - Manual do Hyperflex.

Para Windows:

Você pode usarWINSCP o para transferir os arquivos como mostrado aqui:

Etapa 3. Extraia o conteúdo de Hyperflex-Hypercheck.zip.

Digite cd /tmp para mudar para o diretório/tmp.



root@SpringpathController7PVQWP6WV1:~# cd /tmp/


Digite unzip Hyperflex-Hypercheck.zip para extrair os arquivos.



root@SpringpathController7PVQWP6WV1:/tmp# unzip Hyperflex-Hypercheck.zip 
Archive:  Hyperflex-Hypercheck.zip
b61c59f7962b72902692ce70548ba3d760efdf06
   creating: Hyperflex-Hypercheck/
  inflating: Hyperflex-Hypercheck/HXTool.py 
  inflating: Hyperflex-Hypercheck/LICENSE.txt 
  inflating: Hyperflex-Hypercheck/ReadMe.txt 
  inflating: Hyperflex-Hypercheck/TestInfo.txt 
  inflating: Hyperflex-Hypercheck/prettytable.py 
  inflating: Hyperflex-Hypercheck/progressbar.py 
root@SpringpathController7PVQWP6WV1:/tmp# 


Etapa 4. Execute o script Python HXTool.

Digite cd Hyperflex-Hypercheck para navegar para o diretórioHyperflex-Hypercheck.



root@SpringpathControllerABCDE01234:/tmp# cd Hyperflex-Hypercheck


Digite python HXTool.py  para executar o script.



root@SpringpathControllerABCDE01234:/tmp/Hyperflex-Hypercheck# python HXTool.py


Etapa 5. Insira a senha raiz do SCVM quando solicitado.



Enter this information of HX-Cluster:
Enter the HX-Cluster Root Password:
Enter the ESX Root Password:

Observação: para interromper a execução do script, use a tecla (CTRL+Z) e ela será interrompida imediatamente.

Etapa 6. A ferramenta Hyperflex-Hypercheck inicia suas verificações. Leva cerca de 3 a 10 minutos para a execução ser concluída, o que depende do número de nós convergentes no cluster.

Passo 7. Obter um relatório das saídas do script. Você pode recebê-lo como mostrado:

O Hypercheck Report tar arquivo é salvo em /var/log/springpath  e /tmp/Hyperflex-Hypercheck. Assim, você pode baixar o pacote tar de em /var/log/springpath ou /tmp/Hyperflex-Hypercheck. Como alternativa, você pode apenas gerar e carregar um pacote de suporte a storfs que também contenha o tar do relatório de hiperverificação.

Exemplo de arquivo tar de relatório - HX_Report_2020_08_30_10_43_50.tar é copiado para o caminho: /var/log/springpath.

Digite ls -l | grep HX_Report para examinar os arquivos criados pela ferramenta Hyperflex-Hypercheck.



Under /var/log/springpath,

root@SpringpathControllerABCDE01234:/var/log/springpath# ls -l | grep HX_Report
-rw-r--r-- 1 root root 380K Sep 23 15:41 HX_Report_2020_08_30_10_43_50.tar
root@SpringpathControllerABCDE01234:/var/log/springpath#

Under /tmp/Hyperflex-Hypercheck,

root@SpringpathControllerABCDE01234:/tmp/Hyperflex-Hypercheck# ls
HX_Report_2020_08_30_10_43_50.tar prettytable.py HX_Report_2020_08_30_10_43_50 TestInfo.txt progressbar.py
HXTool.py prettytable.pyc ReadMe.txt progressbar.pyc LICENSE.txt
root@SpringpathControllerABCDE01234:/tmp/Hyperflex-Hypercheck# 


Arquivos e logs no pacote de log do Hypercheck:



root@SpringpathControllerABCDE01234:/tmp/Hyperflex-Hypercheck# ls HX_Report_2020_08_30_10_43_50/
HX_Tool_2020-08-30_10-43-50.log 
HX_Tool_Main_Report_2020-08-30_10-54-34.txt 
HX_Tool_Summary.json


Etapa 8. Exportar o HX_YYYY_MM_DD_HH_MM_SS.tar e compartilhá-lo com o TAC.

Use o método de sua preferência para exportar os logs do Hypercheck com o uso de scp/sftp/ftp/tftp do SCVM ou você pode simplesmente baixar o pacote storfs-support que contém o pacote tar HX_Report.

Entender Saídas/Verificações Executadas

Verificações Executadas por Hiperverificação

Essas verificações são realizadas pela ferramenta Hyperflex-Hypercheck:



Hyperflex Checks: (Below checks are performed on all the storage controller VMs)
  Cluster services check - Verifies the status of storfs, stMgr and stNodeMgr services
    Enospc state check - Checks if the cluster space usage is above the warning threshold or no   
    Zookeeper check - Checks whether the Zookeeper is running or no
    Exhibitor check - Verfies the status of the Exhibitor service which manages the ZK
 System Disks Usage - Checks if /sda1, var/stv and /var/zookeeper is less than 80% 
    HDD health check - Reports if you have any blacklisted disk in your cluster      
    DNS check - Checks whether DNS is configured and reachable                
    vCenter reachability check - Checks whether the vCenter is reachable on the required ports
    Timestamp check - Checks if all the controller VMs have the exact same time           
    NTP sync check - Checks whether NTP is reachable from the storage controller VMs and synced
   Check package & versions - Checks for packages and versions on Storage Controller VMs
 Check Iptables count - Checks for Iptables count on and ensure it is same on all Storage Controller VMs.              
    Extra pnodes check - Looks for any extra/duplicate pnode entries in the cluster
    Out of memory check - Checks through the log files if the cluster had any oom event
    Supported vSphere versions - Shows all the vSphere Versions supported with your current HXDP version
 Permissions for /tmp - Checks if the /tmp permissions are set correctly
 Check Cluster Policy - Checks the Configured Cluster Policy
 Check springpath_keystore.jceks file - Check if All the SCVM have same keystore file
 SED Capable - Checks if the cluster is SED Capable
 SED Enabled - Checks if Encryption is enabled in the Cluster
 USB-0 Check - If Encryption is enabled, Checks that USB0 interface is present on all the SCVMs
 SED 5100/5200 Drive Check - If we have Micron SED 5100 drives and version is below 3.5.2b, we wont be able to replace or add new 5200 drives
 Disk Lock Check - If Encryption is enabled, Checks for any Locked drives
 Network Checks - Checks the connectivity in Storage network
 Check ZK-Cleanup-Script - Checks to identify ZKTxnCleanUp Script
 Replication Checks - If replication is enabled, we check the local and remote network connectivity (HX 4.5 Only)
 Stretched Cluster Checks - Checks the latency between the sites and the witness VM (HX 4.5 Only)

ESXi Checks: (Below checks are performed on each ESXI node)
    HX User Account check - Verifies if the HXUser is created on all the esxi hosts and has admin rights
    vMotion enabled check - Checks if the vMotion network is confirgued
 Check for ESXI Failback timer - Check for ESXi Failback timer on ESXi host
    Check connectivity between vmk1 and eth1 - Checks the connectivity between the Mgmt and Storage network
    No extra controller vm folders check - Checks for duplicate Controller SCVM Folders
 VMware Tools location check  - Checks for Non default VMware Tools location
 vfat Disk Usage check - Checks for vfat Disk Usage
 Check /tmp usage - Checking for /tmp usage
 Compute Node Checks - All the ESXI checks are also performed on Compute nodes (HX 4.5 Only)


Exemplo de saída de hiperverificação de um cluster estendido de 4 nós



Enter this information of HX-Cluster: Enter the HX-Cluster Root Password: Enter the ESX Root Password: Cluster Name: HX-10-Stretched Site-100 Site-97 Cluster Type: STRETCH_CLUSTER SSH connection established to HX Node: 192.168.53.135 SSH connection established to HX Node: 192.168.53.136 SSH connection established to HX Node: 192.168.53.137 SSH connection established to HX Node: 192.168.53.138 HX Cluster Nodes: +-------+--------------+--------------------------------+ | Nodes | IP Address | HostName | +-------+--------------+--------------------------------+ | 1 | 14.39.53.134 | SpringpathControllerOHCWUK9X3N | +-------+--------------+--------------------------------+ | 2 | 14.39.53.135 | SpringpathController37MHMEIBCY | +-------+--------------+--------------------------------+ | 3 | 14.39.53.136 | SpringpathControllerDWRWWIBFLF | +-------+--------------+--------------------------------+ | 4 | 14.39.53.137 | SpringpathControllerWB4UNXDKX3 | +-------+--------------+--------------------------------+ SSH connection established to ESX Host: 14.39.53.133 SSH connection established to ESX Host: 14.39.53.130 SSH connection established to ESX Host: 14.39.53.132 SSH connection established to ESX Host: 14.39.53.131 HX Controller: 192.168.53.135 Cluster services check [########################] COMPLETED ZooKeeper & Exhibitor check [########################] COMPLETED HDD health check [########################] COMPLETED Pre-Upgrade Check [########################] COMPLETED Network check [########################] COMPLETED HX Controller: 192.168.53.136 Cluster services check [########################] COMPLETED ZooKeeper & Exhibitor check [########################] COMPLETED HDD health check [########################] COMPLETED Pre-Upgrade Check [########################] COMPLETED Network check [########################] COMPLETED HX Controller: 192.168.53.137 Cluster services check [########################] COMPLETED ZooKeeper & Exhibitor check [########################] COMPLETED HDD health check [########################] COMPLETED Pre-Upgrade Check [########################] COMPLETED Network check [########################] COMPLETED HX Controller: 192.168.53.138 Cluster services check [########################] COMPLETED ZooKeeper & Exhibitor check [########################] COMPLETED HDD health check [########################] COMPLETED Pre-Upgrade Check [########################] COMPLETED Network check [########################] COMPLETED HX Controller: 192.168.53.135 Test Summary: +--------------------------------------+-----------+----------------------------------------------------------------+ | Name | Result | Comments | +--------------------------------------+-----------+----------------------------------------------------------------+ | Cluster services check | PASS | Checks storfs, stMgr, sstNodeMgr service running on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Enospc state check | PASS | Checks if the cluster storage utilization is above threshold. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Zookeeper check | PASS | Checks if Zookeeper service is running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Exhibitor check | PASS | Checks if Exhibitor in running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | System Disks Usage | PASS | Checks if /sda1, var/stv and /var/zookeeper is less than 80%. | +--------------------------------------+-----------+----------------------------------------------------------------+ | HDD Health check | PASS | Checks if any drive is in blacklisted state. | +--------------------------------------+-----------+----------------------------------------------------------------+ | DNS check | PASS | Checks if configured DNS is reachable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | vCenter reachability check | PASS | Checks if vCenter is network reachable using PING. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Timestamp check | PASS | Checks if the timestamp is same across all Nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | NTP sync check | PASS | Checks if the NTP is synced with NTP server. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check package & versions | PASS | Checks for count and version of HX packages on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Iptables count | PASS | Checks if the IP Table count matches on all nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Extra pnodes check | PASS | Checks for any stale Node entry. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Memory usage check | PASS | Checks for available memory more than 2GB. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Incidence of OOM in the log file | PASS | Checks for any previous incidence of Out Of Memory Condition. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Supported vSphere versions | 6.0.0-U3 | Prints the supported ESXi versions. | | | 6.5.0-U1 | | | | 6.5.0-U2 | | | | 6.5.0-U3 | | | | 6.7.0-UGA | | | | 6.7.0-U1 | | | | 6.7.0-U2 | | | | 6.7.0-U3 | | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check permissions for /tmp | PASS | Checks if the /tmp permissions are set correctly. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Cluster Policy | Lenient | Checks the Configured Cluster Policy | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check springpath_keystore.jceks file | PASS | All the SCVM have same keystore file. | +--------------------------------------+-----------+----------------------------------------------------------------+ | SED Capable | NO | Checks if the cluster is SED Capable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Witness Reachability | PASS | Checks Witness VM IP address is reachabile. | +--------------------------------------+-----------+----------------------------------------------------------------+
| Check ZK-Cleanup-Script | PASS | Checks to identify ZKTxnCleanUp Script. | 
+--------------------------------------+-----------+----------------------------------------------------------------+ HX Controller: 192.168.53.136 Test Summary: +--------------------------------------+-----------+----------------------------------------------------------------+ | Name | Result | Comments | +--------------------------------------+-----------+----------------------------------------------------------------+ | Cluster services check | PASS | Checks storfs, stMgr, sstNodeMgr service running on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Enospc state check | PASS | Checks if the cluster storage utilization is above threshold. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Zookeeper check | PASS | Checks if Zookeeper service is running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Exhibitor check | PASS | Checks if Exhibitor in running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | System Disks Usage | PASS | Checks if /sda1, var/stv and /var/zookeeper is less than 80%. | +--------------------------------------+-----------+----------------------------------------------------------------+ | HDD Health check | PASS | Checks if any drive is in blacklisted state. | +--------------------------------------+-----------+----------------------------------------------------------------+ | DNS check | PASS | Checks if configured DNS is reachable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | vCenter reachability check | PASS | Checks if vCenter is network reachable using PING. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Timestamp check | PASS | Checks if the timestamp is same across all Nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | NTP sync check | PASS | Checks if the NTP is synced with NTP server. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check package & versions | PASS | Checks for count and version of HX packages on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Iptables count | PASS | Checks if the IP Table count matches on all nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Extra pnodes check | PASS | Checks for any stale Node entry. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Memory usage check | PASS | Checks for available memory more than 2GB. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Incidence of OOM in the log file | PASS | Checks for any previous incidence of Out Of Memory Condition. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Supported vSphere versions | 6.0.0-U3 | Prints the supported ESXi versions. | | | 6.5.0-U1 | | | | 6.5.0-U2 | | | | 6.5.0-U3 | | | | 6.7.0-UGA | | | | 6.7.0-U1 | | | | 6.7.0-U2 | | | | 6.7.0-U3 | | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check permissions for /tmp | PASS | Checks if the /tmp permissions are set correctly. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Cluster Policy | Lenient | Checks the Configured Cluster Policy | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check springpath_keystore.jceks file | PASS | All the SCVM have same keystore file. | +--------------------------------------+-----------+----------------------------------------------------------------+ | SED Capable | NO | Checks if the cluster is SED Capable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Witness Reachability | PASS | Checks Witness VM IP address is reachabile. | +--------------------------------------+-----------+----------------------------------------------------------------+
| Check ZK-Cleanup-Script | PASS | Checks to identify ZKTxnCleanUp Script. | 
+--------------------------------------+-----------+----------------------------------------------------------------+ HX Controller: 192.168.53.137 Test Summary: +--------------------------------------+-----------+----------------------------------------------------------------+ | Name | Result | Comments | +--------------------------------------+-----------+----------------------------------------------------------------+ | Cluster services check | PASS | Checks storfs, stMgr, sstNodeMgr service running on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Enospc state check | PASS | Checks if the cluster storage utilization is above threshold. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Zookeeper check | PASS | Checks if Zookeeper service is running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Exhibitor check | PASS | Checks if Exhibitor in running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | System Disks Usage | PASS | Checks if /sda1, var/stv and /var/zookeeper is less than 80%. | +--------------------------------------+-----------+----------------------------------------------------------------+ | HDD Health check | PASS | Checks if any drive is in blacklisted state. | +--------------------------------------+-----------+----------------------------------------------------------------+ | DNS check | PASS | Checks if configured DNS is reachable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | vCenter reachability check | PASS | Checks if vCenter is network reachable using PING. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Timestamp check | PASS | Checks if the timestamp is same across all Nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | NTP sync check | PASS | Checks if the NTP is synced with NTP server. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check package & versions | PASS | Checks for count and version of HX packages on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Iptables count | PASS | Checks if the IP Table count matches on all nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Extra pnodes check | PASS | Checks for any stale Node entry. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Memory usage check | PASS | Checks for available memory more than 2GB. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Incidence of OOM in the log file | PASS | Checks for any previous incidence of Out Of Memory Condition. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Supported vSphere versions | 6.0.0-U3 | Prints the supported ESXi versions. | | | 6.5.0-U1 | | | | 6.5.0-U2 | | | | 6.5.0-U3 | | | | 6.7.0-UGA | | | | 6.7.0-U1 | | | | 6.7.0-U2 | | | | 6.7.0-U3 | | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check permissions for /tmp | PASS | Checks if the /tmp permissions are set correctly. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Cluster Policy | Lenient | Checks the Configured Cluster Policy | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check springpath_keystore.jceks file | PASS | All the SCVM have same keystore file. | +--------------------------------------+-----------+----------------------------------------------------------------+ | SED Capable | NO | Checks if the cluster is SED Capable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Witness Reachability | PASS | Checks Witness VM IP address is reachabile. | +--------------------------------------+-----------+----------------------------------------------------------------+
| Check ZK-Cleanup-Script | PASS | Checks to identify ZKTxnCleanUp Script. | 
+--------------------------------------+-----------+----------------------------------------------------------------+ HX Controller: 192.168.53.138 Test Summary: +--------------------------------------+-----------+----------------------------------------------------------------+ | Name | Result | Comments | +--------------------------------------+-----------+----------------------------------------------------------------+ | Cluster services check | PASS | Checks storfs, stMgr, sstNodeMgr service running on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Enospc state check | PASS | Checks if the cluster storage utilization is above threshold. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Zookeeper check | PASS | Checks if Zookeeper service is running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Exhibitor check | PASS | Checks if Exhibitor in running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | System Disks Usage | PASS | Checks if /sda1, var/stv and /var/zookeeper is less than 80%. | +--------------------------------------+-----------+----------------------------------------------------------------+ | HDD Health check | PASS | Checks if any drive is in blacklisted state. | +--------------------------------------+-----------+----------------------------------------------------------------+ | DNS check | PASS | Checks if configured DNS is reachable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | vCenter reachability check | PASS | Checks if vCenter is network reachable using PING. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Timestamp check | PASS | Checks if the timestamp is same across all Nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | NTP sync check | PASS | Checks if the NTP is synced with NTP server. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check package & versions | PASS | Checks for count and version of HX packages on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Iptables count | PASS | Checks if the IP Table count matches on all nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Extra pnodes check | PASS | Checks for any stale Node entry. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Memory usage check | PASS | Checks for available memory more than 2GB. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Incidence of OOM in the log file | PASS | Checks for any previous incidence of Out Of Memory Condition. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Supported vSphere versions | 6.0.0-U3 | Prints the supported ESXi versions. | | | 6.5.0-U1 | | | | 6.5.0-U2 | | | | 6.5.0-U3 | | | | 6.7.0-UGA | | | | 6.7.0-U1 | | | | 6.7.0-U2 | | | | 6.7.0-U3 | | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check permissions for /tmp | PASS | Checks if the /tmp permissions are set correctly. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Cluster Policy | Lenient | Checks the Configured Cluster Policy | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check springpath_keystore.jceks file | PASS | All the SCVM have same keystore file. | +--------------------------------------+-----------+----------------------------------------------------------------+ | SED Capable | NO | Checks if the cluster is SED Capable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Witness Reachability | FAIL | Checks Witness VM IP address is reachabile. | +--------------------------------------+-----------+----------------------------------------------------------------+
| Check ZK-Cleanup-Script | PASS | Checks to identify ZKTxnCleanUp Script. | 
+--------------------------------------+-----------+----------------------------------------------------------------+ | Check Disk for SMART Failure. | PASS | Checks disks for SMART Failure. |
+--------------------------------------+-----------+----------------------------------------------------------------+ ################################################################################ Network check: ################################################################################ ESX vmk0: 14.39.53.130, 14.39.53.131, 14.39.53.132, 14.39.53.133 ESX vmk1: 192.168.53.130, 192.168.53.131, 192.168.53.132, 192.168.53.133 SCVM eth0: 14.39.53.134, 14.39.53.135, 14.39.53.136, 14.39.53.137 SCVM eth1: 192.168.53.135, 192.168.53.136, 192.168.53.137, 192.168.53.138 ESX Host: 14.39.53.130 +--------------------------------------+--------+-------------------------------------------------+ | Name | Result | Comments | +--------------------------------------+--------+-------------------------------------------------+ | HX User Account check | PASS | Checks if HXUSER is present. | +--------------------------------------+--------+-------------------------------------------------+ | vMotion enabled check | PASS | Checks if vMotion is enabled on the host. | +--------------------------------------+--------+-------------------------------------------------+ | Check for ESXI Failback timer | PASS | Checks for ESXi FAILBACK timer set to 30000ms. | +--------------------------------------+--------+-------------------------------------------------+ | Check vmk1 ping to eth1 | PASS | Checks Network between ESXi vmk1 and SCVM eth1. | +--------------------------------------+--------+-------------------------------------------------+ | No extra controller vm folders check | PASS | Checks for duplicate Controller SCVM Folders. | +--------------------------------------+--------+-------------------------------------------------+ | VMware Tools location check | PASS | Checks for Non default VMware Tools location. | +--------------------------------------+--------+-------------------------------------------------+ | vfat Disk Usage check | PASS | Checks for vfat Disk Usage. | +--------------------------------------+--------+-------------------------------------------------+ | Check /tmp usage | PASS | Checking for /tmp usage. | +--------------------------------------+--------+-------------------------------------------------+ ESX Host: 14.39.53.131 +--------------------------------------+--------+-------------------------------------------------+ | Name | Result | Comments | +--------------------------------------+--------+-------------------------------------------------+ | HX User Account check | PASS | Checks if HXUSER is present. | +--------------------------------------+--------+-------------------------------------------------+ | vMotion enabled check | PASS | Checks if vMotion is enabled on the host. | +--------------------------------------+--------+-------------------------------------------------+ | Check for ESXI Failback timer | PASS | Checks for ESXi FAILBACK timer set to 30000ms. | +--------------------------------------+--------+-------------------------------------------------+ | Check vmk1 ping to eth1 | PASS | Checks Network between ESXi vmk1 and SCVM eth1. | +--------------------------------------+--------+-------------------------------------------------+ | No extra controller vm folders check | PASS | Checks for duplicate Controller SCVM Folders. | +--------------------------------------+--------+-------------------------------------------------+ | VMware Tools location check | PASS | Checks for Non default VMware Tools location. | +--------------------------------------+--------+-------------------------------------------------+ | vfat Disk Usage check | PASS | Checks for vfat Disk Usage. | +--------------------------------------+--------+-------------------------------------------------+ | Check /tmp usage | PASS | Checking for /tmp usage. | +--------------------------------------+--------+-------------------------------------------------+ ESX Host: 14.39.53.132 +--------------------------------------+--------+-------------------------------------------------+ | Name | Result | Comments | +--------------------------------------+--------+-------------------------------------------------+ | HX User Account check | PASS | Checks if HXUSER is present. | +--------------------------------------+--------+-------------------------------------------------+ | vMotion enabled check | PASS | Checks if vMotion is enabled on the host. | +--------------------------------------+--------+-------------------------------------------------+ | Check for ESXI Failback timer | PASS | Checks for ESXi FAILBACK timer set to 30000ms. | +--------------------------------------+--------+-------------------------------------------------+ | Check vmk1 ping to eth1 | PASS | Checks Network between ESXi vmk1 and SCVM eth1. | +--------------------------------------+--------+-------------------------------------------------+ | No extra controller vm folders check | PASS | Checks for duplicate Controller SCVM Folders. | +--------------------------------------+--------+-------------------------------------------------+ | VMware Tools location check | PASS | Checks for Non default VMware Tools location. | +--------------------------------------+--------+-------------------------------------------------+ | vfat Disk Usage check | PASS | Checks for vfat Disk Usage. | +--------------------------------------+--------+-------------------------------------------------+ | Check /tmp usage | PASS | Checking for /tmp usage. | +--------------------------------------+--------+-------------------------------------------------+ Main Report File: HX_Tool_Main_Report_2020-08-26_09-54-59.txt Report tar file: HX_Report_2020_08_26_09_43_18.tar Report file copied to path: /var/log/springpath Release Notes: https://www.cisco.com/c/en/us/support/hyperconverged-systems/hyperflex-hx-data-platform-software/products-release-notes-list.html Upgrade Guides: https://www.cisco.com/c/en/us/support/hyperconverged-systems/hyperflex-hx-data-platform-software/products-installation-guides-list.html Note: 1) If upgrading to HX 4.0(2a), please review the following link and perform workaround – https://tinyurl.com/wc7j5qp 2) Please check the status of Compute nodes manually, script only verifies the config on the converged nodes. 3) Hypercheck doesnot perform FAILOVER TEST, so please ensure that the upstream is configured for network connectivity for JUMBO or NORMAL MTU size as needed.
4) If you have performed any activity on your vcenter(like upgrade, certificate replacement,etc.), it is recommended to reregister your cluster to the vcenter before performing upgrades. Refer to the Hyperflex CLI guide for relevant commands. 


Analisar saída da ferramenta

Próximas etapas

• A ferramenta automatiza o processo de execução de comandos manuais em sistemas Hyperflex.
• Se a ferramenta executar OK e der um PASS em todos os testes, o sistema HX será adequado para todas as verificações que o script executou.
• Em situações em que a ferramenta FALHA em algumas verificações ou não é executada com êxito, você pode usar os comandos CLI (listados) para executar as mesmas verificações no Sistema Hyperflex feitas manualmente pelo script.
• A ferramenta não verifica se há avisos antigos/novos/abertos/resolvidos e, portanto, é altamente recomendável revisar as Notas de versão e os Guias de atualização do Hyperflex antes de qualquer atividade de atualização ou manutenção.

Observação: não abra um caso de TAC porque o script não pôde ser executado. Execute os comandos manualmente, identifique o problema e abra um SR para o problema identificado.

Comandos CLI

No Hyperflex SCVM:



SSH to All Hyperflex SCVMs-

# service_status.sh

# sysmtool --ns cluster --cmd enospcinfo

# echo srvr | nc 0 2181

# pidof exhibitor

# stcli disk list --ip <Corresponding ESXi Mgmt IP Address> |grep -B 2 -A 8 blacklisted

# stcli services dns show (and ping the IPs listed)

# ping <vCenter IP Address>

# date ; compare the time on all SCVMs. They should ideally be identical

# stcli services ntp show

# stcli cleaner info

# ntpq -p -4 

# dpkg -l | grep -i springpath | grep -v storfs-support*

# sysmtool --ns disk --cmd list | grep -i blacklisted

# iptables -L -n | wc -l 

# stcli cluster info

# df -h ; check that /var/stv should be less than 80%

# zgrep -i “out of memory” /var/log/springpath/debug-storfs.*

# ping -I eth0 <eth0> of all SCVMs

# ping -I eth1 <eth1> of all SCVMs

# "ls -ld /tmp" check for 775 and 777

# stcli cluster info | grep -i 'clusterAccessPolicy:' | head -1

# md5sum /etc/springpath/secure/springpath_keystore.jceks

# cat /etc/springpath/sed_capability.conf

# cat /etc/springpath/sed.conf

# cat /var/log/springpath/diskslotmap-v2.txt

# stcli cluster info | grep dataZkIp (ping dataZkIp for latency)


Em sistemas ESXi:



SSH to all ESXi hosts

# esxcli system account list

# esxcli network firewall ruleset list | grep -i vMotion

# esxcli software vib list| egrep -i 'scvm|stHyper|stfs'

# chkconfig --list | grep -E 'ntpd|hostd|vpxa|stHypervisorSvc|scvmclient|hxctlvm' 

# esxcfg-vmknic -l ; confirm that vMotion VMK2 is created

# vmkping -I vmk1 <eth1> of all SCVMs

# cd /vmfs/volumes/Springpath-XXXXXXXXXX ; Ensure that it has only one Folder that has the Storage Controller VM

# df -h | grep vfat ; Ensure dir has free space

Informações Relacionadas

• Suporte técnico e downloads da Cisco