Troubleshoot Automation Scripts

This chapter provides information about troubleshooting the automation scripts.

Collect Debug Logs

Table 1. Feature History Table

Feature Name

Release Information

Description

Debug Automation Scripts

Release 7.5.1

Use this feature to collect logs that contain debug information for ltraces and tech-support data. These logs aid in troubleshooting whenever the scripts are not working as expected.

This feature introduces the show tech-support script command.

To automatically run show commands that display the debugging information specific to automation scripts, use the show tech-support script command in EXEC mode.

  • show version

  • show platform

  • show logging

  • show running-config

  • show install active

  • show processes blocked location all

  • show processes script_watcher_main location all

  • show processes script_agent_main location all

  • show processes checksum_verifier_main location all

  • show memory summary location all

  • show tech cfgmgr

  • show tech eem

  • show tech appmgr

  • show script status detail

  • show script execution detail

In addition, the debug command collects the following data:

  • All the script management log files in /var/log directory

  • List all the files under /pkg/lib/python3/ directroy

  • Collects data about top processes consuming high CPU resources

  • List all the processes initiated by the script manager

  • Collect information about /harddisk:/mirror/script-mgmt directory

Run the debug command to collect information about the automation scripts (in zip format):
Router#show tech-support script
Mon Nov 15 23:28:46.849 UTC
++ Show tech start time: 2021-Nov-15.232847.UTC ++
Mon Nov 15 23:28:47 UTC 2021 Waiting for gathering to complete
.............................
Mon Nov 15 23:30:19 UTC 2021 Compressing show tech output
Show tech output available at 
0/RP0/CPU0 : /harddisk:/showtech/showtech-script-2021-Nov-15.232847.UTC.tgz
++ Show tech end time: 2021-Nov-15.233019.UTC ++
View the collected debug zip files:
Router#dir harddisk:/showtech
Mon Nov 15 00:32:17.218 UTC

Directory of harddisk:/showtech
262146 -rw-rw-rw-. 1 1101085 Nov 15 23:24 showtech-script-2021-Nov-15.232322.UTC.tgz
262147 -rw-rw-rw-. 1 1143339 Nov 15 23:30 showtech-script-2021-Nov-15.232847.UTC.tgz

70553000 kbytes total (66887640 kbytes free)
Untar the collected zip file to view the list of debug log files:
Router#run
Mon Nov 15 00:32:29.724 UTC
[node0_RP0_CPU0:~]$cd /harddisk\:/showtech/
[node0_RP0_CPU0:/harddisk:/showtech]$ls -ltr
total 2196
-rw-rw-rw-. 1 root iosxr 1101085 Nov 15 23:24 showtech-script-2021-Nov-15.232322.UTC.tgz
-rw-rw-rw-. 1 root iosxr 1143339 Nov 15 23:30 showtech-script-2021-Nov-15.232847.UTC.tgz

[node0_RP0_CPU0:/harddisk:/showtech]$gunzip showtech-script-2021-Nov-15.232847.UTC.tgz
[node0_RP0_CPU0:/harddisk:/showtech]$ls -l
total 2612
-rw-rw-rw-. 1 root iosxr 1101085 Nov 15 23:24 showtech-script-2021-Nov-15.232322.UTC.tgz
-rw-rw-rw-. 1 root iosxr 1572864 Nov 15 23:30 showtech-script-2021-Nov-15.232847.UTC.tar

[node0_RP0_CPU0:/harddisk:/showtech]$tar -xvf showtech-script-2021-Nov-15.232847
showtech-script-2021-Nov-15.232847.UTC/
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-ps-grep-python-output
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script_action_log
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script-mgmt/
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script-mgmt/config/
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script-mgmt/logs/
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script-mgmt/logs/exec_sample_script.py_exec_1625009314/
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script-mgmt/logs/exec_sample_script.py_exec_1625009314/stderr.log
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script-mgmt/logs/exec_sample_script.py_exec_1625009314/stdout.log
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script-mgmt/exec/
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script-mgmt/exec/exec_sample_script.py
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script-mgmt/process/
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script-mgmt/eem/
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script-mgmt/.script-mgmt/
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script-mgmt/.script-mgmt/request_queue.json
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script-mgmt/.script-mgmt/script_db.json
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-ps-grep-python-output
showtech-script-2021-Nov-15.232847.UTC/cfg-node0_RP0_CPU0.tar
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script_watcher_log
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-top-output-2
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0.tech.gz
showtech-script-2021-Nov-15.232847.UTC/system.tech.gz
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-top-output-2
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script_agent_log
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-script-mgmt/
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-script-mgmt/config/
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-script-mgmt/logs/
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-script-mgmt/logs/exec_sample_script.py_exec_1625009314/
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-script-mgmt/logs/exec_sample_script.py_exec_1625009314/stderr.log
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-script-mgmt/logs/exec_sample_script.py_exec_1625009314/stdout.log
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-script-mgmt/exec/
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-script-mgmt/exec/exec_sample_script.py
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-script-mgmt/process/
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-script-mgmt/eem/
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-script-mgmt/.script-mgmt/
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-script-mgmt/.script-mgmt/request_queue.json
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-script-mgmt/.script-mgmt/script_db.json
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0.tech.gz
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-top-output-1
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script_control_log
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script_manager_log
showtech-script-2021-Nov-15.232847.UTC/node0_RP1_CPU0-top-output-1
showtech-script-2021-Nov-15.232847.UTC/node0_RP0_CPU0-script_oper_log