Linux Certif

Toute la documentation sur la certification Linux LPI

collectl

NAME

collectl collectl - Collects data that describes the current system status.

SYNOPSIS

Record Mode - read data from live system and write to file or display on terminal

collectl [-f file] [options]

Playback Mode - read data from one or more raw data files and display on terminal

collectl -p file1 [file2 ...] [options]

OPTIONS

Record Mode

In this mode data is taken from a live system and either displayed on the terminal or written to one or more files or a socket.

--align

If the HiRes modules is present, collectl sample monitoring will be aligned such that a sample will always be taken at the top of a minute (this does NOT mean the first sample will occur then) so that all instances of collectl running on any systems which have their clocks synchronized will all take samples at the same time. Furthermore, if one is doing process monitoring, those samples will also be taken at the top of the minute and so can delay the start of sampling up to 2 full process monitoring intervals.

--all

Collect summary data for ALL subsystems except slabs, since slab monitoring requires a different monitoring interval. This also means you won't get any detail data which also includes processes and environmementals. You can use this switch anywhere -s can be used but not both together. If the system supports lustre and/or interconnect monitoring those statistics will be provided but the warnings produced when they are not available you try to select them with -s will not be displayed.

-A, --address address[:port] | server[:port]

In the first form, one species an address or hostname and optional port. All data is then written to that socket prefaced with the current host name at the named address and port until the socket is closed, at which time collectl will exit.
In the second form one enters the text "server" and optional port. In this form, collectl runs as a server, waiting for a connection and once established writes data on that socket. The key difference here is if the client exists collectl keeps running and will again look for a new connection, allowing it to survive client restarts or crashes. If collectl receives the text "exit" from the client, it will shut down.
The default port is set at 2655 but can be changed - see collectl.conf.
In both forms, one can additionally request local data logging by specifying a combination of -P and -f. See man collectl-logging for more details.

-c, --count Samples

The number of samples to record. This is one way of 3 ways of describing how long collectl should run (see -r and -R ). Note that these 3 switches are mutually exclusive.

-C, --config filename

Name/location of the collectl configuration file. If not specified, collectl searches for collectl.conf first in /etc (the default), then in the same directory the collectl executable is in, and finally the current working directory.

-D, --daemon

Run collectl as a daemon, primarily used when starting as a service. One caveat about this mode is you can only run one copy.

--export file[,options]

This requests that collectl does not print anything on the terminal (or send it to a socket) using the standard brief/verbose/plot formats. Instead it executes a perl "require" on the named file, using an extension of ph if not specified. It first looks in the current directory and if not there the directory the executable is in. It then calls the function "file"Init(options) towards the beginning of collectl and again as simply "file"(@options) to generate the exported formatted output. See the document sections on Exporting Custom Output and Logging for more details.

-f, --filename Filename

This is the name of a file to write the output to. See the description of File Naming for further details.

-F, --flush seconds

Flush output buffers after this number of seconds. This is equivalent to issuing kill -s USR1 at the same frequency (but a lot easier!). If 0, a flush will occur every data collection interval.

--grep pattern

The main purpose of this switch is for those users who have discovered there is some data in the raw files that never appears in any display and have taken to displaying it themselves with grep. Unfortunately this method does not include timestamps and so makes it difficult to interpret the results. Even if you include the timestamp from the file it is in UTC and so needs to be translated to be of any real value. This switch does just that and then some.
Specifically, it allows you to playback a file and instead of processing it normally it simply searches for any entries that match the perl pattern and reports those lines prefaced with time stamps. You can optionally change the time format with the usual -o options and can even select the timeframe with --from and --thru.

-G, --group

Treat process data as an entirely separate group of data, writing it into its own raw file, named "rawp". These separate process files can be played back and processed just like any other collectl raw files and in fact one can even play back both at the same time if that is what is desired. The only real purpose of this switch is that on some systems with many processes, it is possible to generate huge raw files (some have been observerd to be >250MB!) and while collectl will happily play back/process these files it can take a long time. By using the -G switch one still gets a hugh rawp file, but the raw file is a much more manageable size and as a result much faster to process.

--home

Always start the display for the current interval at the top of the screen also known as the home position (non-plot format only). This generates a real-time, continously refreshing display when the data fits on a single screen.

-i, --interval interval[:interval2[:interval3]]

This is the sampling interval in seconds. The default is 10 seconds when run as a daemon and 1 second otherwise. The process subsystem and slabs (-sY and -sZ) are sampled at the lower rate of interval2. Environmentals (-sE), which only apply to a subset of hardware, are sampled at interval3. Both interval2 and interval3, if specified, must be an even multiple of interval1. The daemon default is -i10:60:300 and all other modes are -i1:60:300. To sample only processes once every 10 seconds use -i:10.

-r, --rolllogs time[[,days][,minutes]]

When selected, collectl runs indefinately (or at least until the system reboots). The maximum number of raw and/or plot files that will be retained (older ones are automatically deleted) is controlled by the days field, the default is 7 days. The increment field which is also optional (but is position dependent) specifies the duration of an individual collection file in minutes the default of which is 1440 or 1 day.

--quiet

Whenever collectl wants to tell the user something, it assigns a category to it such as Informational, Warning, Error or Fatal. When run with -m, all messages are displayed for the user and if logging data to a file with -f, these messages are also sent to a log file which is in the data collection directory and has an extenion of "log". However, if -m is not specified Informational messages (such as collectl starting or stopping) are not reported on the terminal but the other 3 are. Sometimes the warnings can be annoying and one can suppress these with --quiet though they will still be written to the message log in -f. You cannot suppress Error or Fatal errors.

--rawtoo

Only available in conjunction with -P, this switch causes the creation/logging of raw data in addition to plottable data. While this may seem excessive, keep in mind that unlike plottable data, raw data can be played back with different switches potentially providing more details. The overhead to write out this additional data is minimal, the only real cost being that of extra disk space.

-R, --runtime duration

Specify the duration of data collection where the duration is a number followed by one of wdhms, indicating how many weeks, days, hours, minutes or seconds the collection is to be taken for.

--sep separator

Specify the plot format separator - default is a space. If this is a numeric field it is interpretted as the decimal value of the associated ASCII character code. Otherwise it is interpretted as the character itself. In other words, "--sep :" sets the separator character to a colon and "--sep 9" sets it to a horizontal tab. "--sep 58" would also set it to a colon.

-S, --ssh

This is typically used when starting collectl on another system via ssh or rsh. It causes collectl to "watch" for its parent (who started it locally) to exit at which point it will exit as well. The reason for this switch is that when the remote command that started collectl exists, collectl's parent will exit as well but NOT collectl, unless -S is specified.

Playback Mode

In this mode, data is read from one or more data files that were generated in Record Mode

-f, --filename Filename

If specified, this is the name of a file or directory to write the output to (rather than the terminal). See the description for details on the format of this field. This requires the -P flag as well.

--from time range

Play back data starting with this time, which may optionally include the ending time as well, which is of the format of [date:]time[-[date:]time]. The leading 0 of the hour is optional and if the seconds field is not specified is assumed to be 0. If no dates specified the time(s) apply to each file specified by -P. Otherwise the time(s) only apply to the first/last dates and any files between those dates will have all their data reported.

--passwd filename

When reporting usernames associated with a UID, use this file for the mapping. This is particularly important on systems running NIS where this are no user names in /etc/passwd.

--pname name

By default, collectl uses the file /var/run/collectl.pid to indicate the pid of the running instance of collectl and prevent multiple copies from being run. If you DO want to run a second copy, this switch will cause collectl to change its process name to collectl-name and use that name as the associated pid file as well.

--offsettime seconds

This field originally was used before collectl reported the timezone in the file headers and allowed one to compensate. Since then it is rarely needed except in two possible cases, one in which data on two systems is to be compared and they weren't synchonized with ntp. This allows all the times to be reported as shifted by some number of seconds. The other case (and this is very rare) is when a clock had changed in the middle of a sample and will not be converted correctly. When this happens one may have to play back the samples in pieces and manually set the time offset.

-p, --playback Filename

Read data from the specified playback file(s), noting that one can use wildcards in the filename if quoted (if playing back multiple files to the terminal you probably want to include -m to see the filenames as they are processed). The filename must either end in raw or raw.gz. As an added feature, since people sometimes automate the running of this option and don't want to hard code a date, you can specify the string YESTERDAY or TODAY and they will be replaced in the filename string by the appropriate date.

--procanalyze

When specified and there is process data in the raw file, a summary file will be generated with one entry unique process containing such things as the total cpu consumed for both user and system, min/max utilization of various memory types, total page faults and several others.

--slabanalyze

When specified and there is slab data in the raw file, a summary file will be generated with one entry unique slab containing data on physical memory usage by that slab.

--thru time

Time thru which to play back a raw file. See --from for more

Common Switches - both record and playback modes

-d, --debug debug

Control the level of debugging information, not typically used. For details see the source code.

-h, --help, -x, --helpext, -X, --helpall

Display standard, extended help message (which doesn't include the optional displays such as --showoptions, --showsubsys, --showsubopts, --showtopopts) or everything.

--hr, --headerrepeat num

Sets the number of intervals to display data for before repeating the header. A value -1 will prevent any headers from being displayed and a value of 0 will cause only a single header to be displayed and never repeated.

--iosize

In brief mode, include iosize with disk, infiniband and network data.

-l, --limits limit

Override one or more default exception limits. If more than one limit they must be separated by hyphens. Current values are:
SVC:value

Report partition activity with Service times >= 30 msec

IOS:value

Report device activity with 10 or more reads or writes per second

LusKBS:value

Report client or OSS activity greater than limit. Only applies to Client Summary or OSS Detail reporting. [default=100000]

LusReints:value

Report MDS activity with Reint greater than limit. Only applies to MDS Summary reporting. [default=1000]

AND

Both the IOS and SCV limits must be reached before a device is reported. This is the default value and is only included for completeness.

OR

Report device activity if either IOS or SVC thresholds are reached.

-L, --lustsvcs [c|m|o][:seconds]

This switch limits which servics lustre checks for and the frequency of those checks. For more information see the man page collectl-lustre.

-m, --messages

Write status to a monthly log file in the same directory as the output file (requires -f to be specified as well). The name of the file will be collectl-yyyymm.log and will track various messages that may get generated during every run of collectl.

-N, --nice

Set priority to a nicer one of 10.

-o, --options Options

These apply to the way output is displayed OR written to a plot file. They do not effect the way data is selected for recording. Most of these switches work in both record as well as playback mode. If you're not sure, just try it.
1

Data in plotting format should use 1 decimal point of precision as appropriate.

2

Data in plotting format should use 2 decimal points of precision as appropriate.

a

Always append data to an existing plot file. By default if a plot file exists, the playback file will be skipped as a way of assuring it is associated with a single recorded file. This switch overrides that mechanism allowing muliple recorded files to be processed and written to a single plot file.

A

When playing back one or more files to the terminal in brief mode, append the Average and Totals.

c

Always open newly named plot fies in create mode, overwriting any old ones that may already exists. If one processes multiple files for the same day in append mode multiple times, the same data will be appended to the same file mulitple times. This assures a new file is created at the start of the processing.

d

For use with terminal output and brief mode. Preceed each line with a date/time stamp, the date being in mm/dd format. This option can also be applied to plot formatit which will cause the date portion to also be displayed in this format as opposed to D format.

D

For use with terminal output and brief mode. Preceed each line with a date/time stamp, the date being in yyyymmdd format.

g

For use with terminal output and brief mode. When displaying values of 1G or greater there is limited precision for 1 digit values. This options provides a way to display additional digits for more granularity by substituting a "g" for the decimal point rather than the trailing "G".

G

For use with terminal output and brief mode. This is similar to "g" but preserves the trailing "G" by sacrificing a digit of granularity.

m

Whenever times are reported in plot format, in the normal terminal reporting format at the bginning of each interval or when when one of the time reporting options (d, D, T or U is selected), append the milliseconds to the time.

n

Where appropriate, data such as disk KBs or transfers are normalized to units per second by taking the change in a counter and dividing by the number of seconds in that interval. Normalization can be disabled via this option, the result being the reported values are not divided by the duration of the interval.

T

For use with terminal output and brief mode, preceeds each line with a time stamp.

u

Create plot files with unique names by include the starting time of a colletion in the name. This forces multiple collections taken the same day to be written to multiple files.

U or --utc

In plot format only, report timestamps in Coordinated Universal time which is more commonly know as UTC.

x

Report only exception records for selected subsystems. Exception reporting also requires --verbose. Currently this only applies to disk detail and Lustre server information so one must select at least -s D, l or L for this to apply. If writing to a detail file, this data will go into a separate file with the extension X appended to the regular detail file name.

X

Report both exceptions as well as all details for selected subsystems, for -s D, l or L only.

z

If the compression library has been installed, all output files will be compressed by default. This switch tells collectl not to compress any plottable files. If collectl tries to compress but cannot because the library hasn't been installed, it will generate a warning which can be suppressed with this switch.

-P, --plot

Generate output in plot format. This format is space separated data which consists of a header (prefaced with a # for easy identification by an analysis program as well as identifying it as a comment for programs, such as gnuplot, which honor that convention). When written to disk, which is the typical way this option is used, summary data elements are written to the tab file and the detail elements written to one or more files, one per detail subsystem. If -f is not specified, all output is sent to the terminal. Output is always one line per sampling interval.

-s, --subsys subsystem

This field controls which subsystem data is to be collected or played back for. The rules for displaying results vary depending on the type of data to be displayed. If you write data for CPUs and DISKs to a raw file and play it back with -sc, you will only see CPU data. If you play it back with -scm you will still only see CPU data since memory data was not collected. However, when used with -P, collectl will always honor the subsystems specified with this switch so in the previous example you will see CPU data plus memory data of all 0s. To see the current set of default subsystems, which are a subset of this full list, use -h.
You can also use + or - to add or subtract subsystems to/from the default values. For example, "-s-cdn+N"< will remove cpu, disk and network monitoring from the defaults while adding network detail.
The default is "cdn", which stands for CPU, Disk and Network data.
SUMMARY SUBSYSTEMS

b - buddy info (memory fragmentation)
c - CPU
d - Disk
f - NFS V3 Data
i - Inode and File System
j - Interrupts
l - Lustre
m - Memory
n - Networks
s - Sockets
t - TCP
x - Interconnect
y - Slabs (system object caches)
DETAIL SUBSYSTEMS
This is the set of detail data from which in most cases the corresponding summary data is derived. There are currently 2 types that do not have corresponding summary data and those are "Environmental" and "Process". So, if one has 3 disks and chooses -sd, one will only see a single total taken across all 3 disks. If one chooses -sD, individual disk totals will be reported but no totals. Choosing -sdD will get you both.

C - CPU
D - Disk
E - Environmental data (fan, power, temp), via ipmitool
F - NFS Data
J - Interrupts
L - Lustre OST detail OR client Filesystem detail
N - Networks
T - 65 TCP counters only available in plot format
X - Interconnect
Y - Slabs (system object caches)
Z - Processes

--showheader

In collectl mode this command will cause the header that is normally written to a data file to be displayed on the terminal and collectl then exists. This can be a handy way to get a brief overview of the system configuration.

--showoptions

This command shows only the portion of the help text that desribes the -o and --options switches to save the time of wading through the entire help screen.

--showsubopts

List all the subsystem specifice options

--showtopopts

Show all the different values for the --top type field, which specify the field(s) by to sort the data

--showrootslabs

This command only works on systems using the new slab allocator and will list the root name (these are those entries in /sys/slab which are not soft links) along with all its alias names. If a name doesn't have an alias, it will not appear in this report.

--showslabaliases

This command only works on systems using the new slab allocator. Like --showrootslabs, it will name a slab and all its aliases but rather than show the root slab name it will show one of the aliases to provide a more meaningful name. If there are any slabs that only have a single (or no) alias they will not be included in this report.

--showsubopts

Similar to --showoptions, this command summaries just the paramaters associated with -O and --subopts.

--showsubsys

Yet another way to summare a portion of the help text, this command only shows valid subsystems.

--top [type][,num]

Include the top "num" consumers by resource for this interval. The default number is the height of the window if it can be determined otherwise 24, and the default resource is the total cpu time which is taken as the sum of SysT and UsrT. See --showtopopts for a list of other types of data you can sort on.
This switch can also be used with -s in which case a portion of the window is reserved at the top to fill in the subsystem data, which is currently in verbose mode though a brief format is contemplated for some time in the future.
In interactive mode and if not specified, the process monitoring interval will be set to that for other subsystems. The screen will be cleared for each interval resulting in a display similar to the "top" utility. In playback more the screen will NOT be cleared. You cannot use this switch in "record" mode.

--umask mask

Sets collectl's umask to control output file permissions. Only root can set the umask. See "man umask" for details.

-v

Show version and whether or not Compression and/or HiResTime modules have been installed and exit.

-V

Show default parmeter and control settings, all of which can be changed in /etc/collectl.conf

--verbose

Display output in verbose mode. This often displays more data than in the default mode. When displaying detail data, verbose mode is forced. Furthermore, if summary data for a single subsystem is to be displayed in verbose mode, the headers are only repeated occasionally whereas if multiple subsystems are involved each needs their own header.

-w

Disply data in wide mode. When displaying data on the terminal, some data is formatted followed by a K, M or G as appropriate. Selecting this switch will cause the full field to be displayed. Note that there is no attempt to align data with the column headings in this mode.

SUBSYSTEM OPTIONS

The following options are subsystem specific and typically filter data for collection and/or display as well as affect the output format:

--dskfilt perl-regx[,perl-regx...]

This ONLY applies to disk detail output and not data collection. Only data for disk names that match the pattern(s) will be displayed. If you don't know perl, a partial string will usually work too.

--envopts Environmental Options

The default is to display ALL data but the following will cause a subset to be displayed

c - display current (power) data
f - display fan data
t - display temperature data
C - convert temperature to Celcius if in Farenheit
F - convert temperature to Farenheit if in Celcius
M - display each type of data on separate line
9 - any number, will tell ipmitool to read on this device number

--envfilt regx

If specified, this regx is evaluated against each line of data returned by ipmitool and only those that match are retained. All other data is lost.

--envremap

If specified as a comma separated list of perl regular substitution expressions without the =~s portion, each expression is applied to each environmental field name, thereby allowing one to rename the column headers. This can be most useful when running on heterogeneuos systems and you want consistent column names.

--lustopts Lustre Options

B - For clients and servers, show buffer stats
D - For MDSs and OSTs AND running earlier versions of HPSFS, collect disk block iostats
M - For clients, collect metadata
O - For OSTs, show detail level stats
R - For client, collect readahead stats

--netfilt perl-regx[,perl-regx...]

This ONLY applies to network detail output and not data collection. Only data for network interface names that match the pattern(s) will be displayed. If you don't know perl, a partial string will usually work too.

--nfsfilt NFS Filters

Specify one or more comma separated filters as a C/S followed by an nfs version number and only those will have data reported on. For example, C2 says to report data on V2 Clients. As a data collection performance optimization, if one or more client filters are specified, data will actually be collected for all clients as is also done for servers.

--nfsopts NFS Options

z - only display detail lines which have data

--procfilt Process Filters

These filters restrict which processes are selected for collection/display and replaces -Z which is now deprecated. The format of a filter is a one charter type followed by a match string. Multiple filters may be specified if separated by commas.

c - substring of the command being executed
C - any command that starts with the specified string
f - full path of the command, including arguments
p - pid
P - parent pid
u - any process ownerd by this user's UID or in the range specifide by uxxx-yyy
U - any process owned by this username

--procopts options

These options control the way data is displayed and can also improved data collection performance

c - include CPU time of children who have exited (same as ps -S)
f - use cumulative totals for page faults in process data instead of rates
i - show process I/O counters in display instead of default format
m - show breakdown of memory utilization instead of default format
p - never look for new pids or threads during data collection
r - show root command name only (no directory) for narrower display
t - include ALL process threads (increases collection overhead)
w - widen display by including whole argument string, with optional max width
z - exclude any processes with 0 in sort field (in --top mode)

--slabfilt Slab Filters

One can specify a list of slab names separated by commas and only those slabs whose names start with those strings will be listed or summaried.

--slabopts Slab Options

s - exclude any slabs with an allocation of 0

S - only show those slabs whose allocations changed since last display

DESCRIPTION

The collectl utility is a system monitoring tool that records or displays specific operating system data for one or more sets of subsystems. Any set of the subsystems, such as CPU, Disks, Memory or Sockets can be included in or excluded from data collection. Data can either be displayed back to the terminal, or stored in either a compressed or uncompressed data file. The data files themselves can either be in raw format (essentially a direct copy from the associated /proc structures) or in a space separated plottable format such that it can be easily plotted using tools such as gnuplot or excel. Data files can be read and manipulated from the command line, or through use of command scripts.

Upon startup, collectl.conf is read, which sets a number of default parameters and switch values. Collectl searches for this file first in /etc, then in the directory the collectl execuable lives in (typically /usr/sbin) and finally the current directory. These locations can be overriden with the -C switch. Unless you're doing something really special, this file need never be touched, the only exception perhaps being when choosing to run collectl as a service and you wish to change it's default behavior which is set by the DaemonCommand entry.

RESTRICTIONS/PROBLEMS

Thread reporting currently only works with 2.6 kernels.

The pagesize has been hardcoded for perl 5.6 systems to 4096 for IA32 and 16384 for all others. If you are running 5.6 on a system with a different pagesize you will see incorrect SLAB allocation sizes and will need to scale the numbers you're seeing accordingly.

I have recently discovered there is a bug in /proc in that an extra line is occasionally read with the end of the previous buffer! When this occurs a message is written (if -m enabled) and always written to the terminal. Since this happens with a higher frequency with process data I silently ignore those as the output can get pretty noisey. If for any reason this is a problem, be sure to let me know.

Since collectl has no control over the frequency at which data gets written to /proc, one can get anomolous statistics as collectl is only reporting a snapshot of what is being recorded. For more information see http://collectl.sourceforge.net/TheMath.html.

At least one network card occasionally generates erroneous network stats and to try to keep the data rational, collectl tries to detect this and when it does generates a message that bogus data has been detected.

FILES, EXAMPLES AND MORE INFORMATION

http://collectl.sourceforge.net OR /opt/hp/collectl/docs

ACKNOWLEDGEMENTS

I would like to thank Rob Urban for his creation of the Tru64 Unix collect tool, which collectl is based on.

AUTHOR

This program was written by Mark Seger (mjseger@gmail.com).
Copyright 2003-2010 Hewlett-Packard Development Company, LP
collectl may be copied only under the terms of either the Artistic License or the GNU General Public License, which may be found in the source kit