no way to compare when less than two revisions
Differences
This shows you the differences between two versions of the page.
— | monitors:6540 [2010/09/08 09:01] (current) – created - external edit 127.0.0.1 | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== 6540.ksh ====== | ||
+ | ^ Author | [[ everett.vernon@gmail.com | Vernon Everett ]] | | ||
+ | ^ Compatibility | Xymon 4.2 (Will work on others, but needs SPLITNCV for some graphs) | | ||
+ | ^ Requirements | Solaris CAMS client - SUNWstkcam | | ||
+ | ^ Download | None | | ||
+ | ^ Last Update | 2010-09-08 | | ||
+ | |||
+ | ===== Description ===== | ||
+ | We needed some visibility into what's happening on the 6540 without giving too many people access to CAMS. Slowly we added more functionality, | ||
+ | |||
+ | The script queries the CAMS application, | ||
+ | |||
+ | If somebody wants to test this and give feedback, I would appreciate it. | ||
+ | ===== Installation ===== | ||
+ | === Client side === | ||
+ | Download 6540.ksh and put it into ~hobbit/ | ||
+ | You will need to make sure you have the CAMS CLI installed on the box, and that hobbit has sudo rights to run the appropriate commands without password. | ||
+ | Add this to sudoers | ||
+ | hobbit | ||
+ | |||
+ | Add the following code to ~/ | ||
+ | [6540] | ||
+ | ENVFILE $HOBBITCLIENTHOME/ | ||
+ | CMD $HOBBITCLIENTHOME/ | ||
+ | LOGFILE $HOBBITCLIENTHOME/ | ||
+ | INTERVAL 5m | ||
+ | |||
+ | |||
+ | === Server side === | ||
+ | Add this to TEST2RRD= in hobbitserver.cfg | ||
+ | cache=ncv, | ||
+ | Add this to GRAPHS= in hobbitserver.cfg | ||
+ | cntrlr, | ||
+ | # The ::7 indicates the number of lines on a each graph. Edit to taste. | ||
+ | Add this to hobbitserver.cfg | ||
+ | NCV_cntrlr=" | ||
+ | NCV_details=" | ||
+ | SPLITNCV_cache=" | ||
+ | SPLITNCV_iops=" | ||
+ | SPLITNCV_reads=" | ||
+ | SPLITNCV_writes=" | ||
+ | |||
+ | Add this to hobbitgraph.cfg | ||
+ | [cache] | ||
+ | FNPATTERN cache, | ||
+ | TITLE Cache Hits Percentage | ||
+ | YAXIS % Hits | ||
+ | DEF: | ||
+ | LINE1: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | | ||
+ | [iops] | ||
+ | FNPATTERN iops, | ||
+ | TITLE Total IOs Per Second | ||
+ | YAXIS IOPS | ||
+ | DEF: | ||
+ | LINE1: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | | ||
+ | [reads] | ||
+ | FNPATTERN reads, | ||
+ | TITLE Total READS Per Second | ||
+ | YAXIS Reads/s | ||
+ | DEF: | ||
+ | LINE1: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | | ||
+ | [writes] | ||
+ | FNPATTERN writes, | ||
+ | TITLE Total WRITES Per Second | ||
+ | YAXIS Writes/s | ||
+ | DEF: | ||
+ | LINE1: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | | ||
+ | [cntrlr] | ||
+ | TITLE Controller IOPs | ||
+ | YAXIS IOPS | ||
+ | DEF: | ||
+ | DEF: | ||
+ | LINE2: | ||
+ | LINE2: | ||
+ | -l 0 | ||
+ | COMMENT:\n | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | | ||
+ | [details] | ||
+ | TITLE PERFORMANCE SUMMAY | ||
+ | YAXIS Percent | ||
+ | DEF: | ||
+ | DEF: | ||
+ | DEF: | ||
+ | AREA: | ||
+ | STACK: | ||
+ | LINE2: | ||
+ | -l 0 | ||
+ | -u 100 | ||
+ | COMMENT:\n | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | GPRINT: | ||
+ | |||
+ | ===== Source ===== | ||
+ | ==== 6540.ksh ==== | ||
+ | |||
+ | <hidden onHidden=" | ||
+ | < | ||
+ | # | ||
+ | SSCS="/ | ||
+ | ARRAYS=$($SSCS list array | cut -d" " -f2) | ||
+ | export COLUMNS=255 | ||
+ | TMPFILE=$BBTMP/ | ||
+ | SITE=$($SSCS list site) | ||
+ | EVENTS=$($SSCS list event) | ||
+ | for ARRAY in $ARRAYS | ||
+ | do | ||
+ | # | ||
+ | # | ||
+ | # | ||
+ | |||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | echo > $TMPFILE | ||
+ | echo " | ||
+ | echo " | ||
+ | echo >> $TMPFILE | ||
+ | echo "FRU SUMMARY" | ||
+ | $SSCS list -d $ARRAY | ||
+ | echo >> $TMPFILE | ||
+ | $SSCS list -a $ARRAY registeredarray >> $TMPFILE | ||
+ | echo >> $TMPFILE | ||
+ | echo " | ||
+ | echo "< | ||
+ | echo " | ||
+ | echo " | ||
+ | echo " | ||
+ | echo "</ | ||
+ | |||
+ | $BB $BBDISP " | ||
+ | |||
+ | | ||
+ | | ||
+ | $BB $BBDISP " | ||
+ | |||
+ | |||
+ | | ||
+ | | ||
+ | | ||
+ | if [ -n " | ||
+ | then | ||
+ | # If there disks left after removing optimal disks, then it must non-green. | ||
+ | COLOUR=" | ||
+ | # Now let's find out how non-green | ||
+ | DISKSUMMARY=$(echo " | ||
+ | # Take out green and yellow, if we still have something, it must be red. | ||
+ | [ -n " | ||
+ | fi | ||
+ | echo " | ||
+ | # We know most of these states. Bypassed is special. It is detailed under bug id 6371462 | ||
+ | # Search for it on sunsolve or here http:// | ||
+ | # Particularly useless workaround. | ||
+ | # "Power off the arrays and set the link rate switch back to 2 Gb/s." WTF! | ||
+ | $BB $BBDISP " | ||
+ | rm $TMPFILE.disk | ||
+ | |||
+ | | ||
+ | | ||
+ | echo " | ||
+ | [ $? -eq 0 ] && COLOUR=yellow | ||
+ | $BB $BBDISP " | ||
+ | sed "s/not at baseline.$/ | ||
+ | | ||
+ | | ||
+ | do | ||
+ | echo "< | ||
+ | done) | ||
+ | $BB $BBDISP " | ||
+ | echo "< | ||
+ | </ | ||
+ | |||
+ | | ||
+ | | ||
+ | do | ||
+ | echo "< | ||
+ | done) | ||
+ | $BB $BBDISP " | ||
+ | echo "< | ||
+ | </ | ||
+ | |||
+ | | ||
+ | | ||
+ | | ||
+ | [ -z $LINKSUP ] && COLOUR=green | ||
+ | $BB $BBDISP " | ||
+ | |||
+ | | ||
+ | | ||
+ | $BB $BBDISP " | ||
+ | |||
+ | | ||
+ | | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo " | ||
+ | s/ | ||
+ | s/Remote volume: /< | ||
+ | s/ | ||
+ | s/ | ||
+ | echo "</ | ||
+ | $BB $BBDISP " | ||
+ | rm $TMPFILE | ||
+ | |||
+ | | ||
+ | | ||
+ | [ -z $SNAPSHOTS ] && SNAPSHOTS=" | ||
+ | $BB $BBDISP " | ||
+ | |||
+ | | ||
+ | | ||
+ | > | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo " | ||
+ | | ||
+ | | ||
+ | | ||
+ | echo "</ | ||
+ | $BB $BBDISP " | ||
+ | rm $TMPFILE | ||
+ | |||
+ | | ||
+ | | ||
+ | > | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo " | ||
+ | s/Type: /< | ||
+ | s/Pool: /< | ||
+ | s/Profile: /< | ||
+ | s/ | ||
+ | echo "</ | ||
+ | $BB $BBDISP " | ||
+ | rm $TMPFILE | ||
+ | |||
+ | | ||
+ | > | ||
+ | | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | # | ||
+ | #echo " | ||
+ | # | ||
+ | $SSCS list -d $ARRAY -t all fru | egrep -v -- " | ||
+ | | while read NAME B C D E F G | ||
+ | do | ||
+ | if [ " | ||
+ | then | ||
+ | | ||
+ | else | ||
+ | if [ " | ||
+ | then | ||
+ | E="$E & | ||
+ | [ " | ||
+ | else | ||
+ | E="$E & | ||
+ | COLOUR=red | ||
+ | fi | ||
+ | fi | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | done | ||
+ | echo "</ | ||
+ | $BB $BBDISP " | ||
+ | |||
+ | | ||
+ | | ||
+ | | ||
+ | [ -z " | ||
+ | #$BB $BBDISP " | ||
+ | | ||
+ | | ||
+ | | sed "s/[ \t][ \t]*/ | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "</ | ||
+ | for i in 2 3 4 5 6 7 8 9 10 11 12 | ||
+ | do | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "< | ||
+ | echo "</ | ||
+ | done | ||
+ | echo "</ | ||
+ | echo "< | ||
+ | echo ctrlA: | ||
+ | echo ctrlB: | ||
+ | echo "</ | ||
+ | $BB $BBDISP " | ||
+ | |||
+ | # | ||
+ | # $SSCS list -a $ARRAY --type controller_stats performance | ||
+ | # $SSCS list -a $ARRAY --type array_stats performance | ||
+ | # $SSCS list -a $ARRAY --type volume_stats performance | ||
+ | # 1 NAME | ||
+ | # 2 TOT IOPS | ||
+ | # 3 READ % | ||
+ | # 4 WRITE% | ||
+ | # 5 TOT DATA | ||
+ | # 6 AVG R SIZE | ||
+ | # 7 READS/s | ||
+ | # 8 PEAK R/s | ||
+ | # 9 AVG W SIZE | ||
+ | # 10 WRITES/s | ||
+ | # 11 PEAK W/s | ||
+ | # 12 CACHE% | ||
+ | |||
+ | # Cache Hits | ||
+ | | ||
+ | | ||
+ | $BB $BBDISP "data $ARRAY.cache $COLOUR $(date) $(echo;echo " | ||
+ | # IOPS | ||
+ | | ||
+ | $BB $BBDISP "data $ARRAY.iops $COLOUR $(date) $(echo;echo " | ||
+ | # READS | ||
+ | | ||
+ | $BB $BBDISP "data $ARRAY.reads $COLOUR $(date) $(echo;echo " | ||
+ | # WRITES | ||
+ | | ||
+ | $BB $BBDISP "data $ARRAY.writes $COLOUR $(date) $(echo;echo " | ||
+ | done | ||
+ | |||
+ | |||
+ | </ | ||
+ | </ | ||
+ | |||
+ | ===== Known Bugs and Issues ===== | ||
+ | Sometimes, the FRU listing | ||
+ | $SSCS list -d $ARRAY -t all fru | ||
+ | wraps columns, causing the output to be unparsable, and making the resulting table look like absolute garbage. | ||
+ | |||
+ | ===== To Do ===== | ||
+ | * iSCSI support. We don't use it. Somebody want to add iSCSI support? | ||
+ | |||
+ | * SAS support. See iSCSI above. | ||
+ | |||
+ | * Get Sun/Oracle to fix a CAMS bug that sometimes causes controller B to show 0 for all values, even though there are plenty LUNs using it as their preferred controller. This shows up on the CAMS GUI too. | ||
+ | * **Update :** 2010-07-21 Sun/Oracle BugID 6967353 created. | ||
+ | |||
+ | * Figure out why sometimes the FRU listing wraps columns. See Bugs above. | ||
+ | |||
+ | ===== Credits ===== | ||
+ | |||
+ | Can't really blame anybody except myself for this one. | ||
+ | |||
+ | ===== Changelog ===== | ||
+ | |||
+ | * **2010-07-12** | ||
+ | * Initial release | ||
+ | * **2010-09-08** | ||
+ | * Updated to cater for the curious disk state " |