PW

Section: Pipe Watch (1)
Updated: 26 April 2022
Index Return to Main Contents

 

NAME

pw - Pipe Watch: monitor recent lines of output from pipe

 

SYNOPSIS

command | pw [-i interval] [-l interval] [-n number-of-lines] [-dEB]

 

DESCRIPTION

pw stands for Pipe Watch, a utility that continuously reads lines of text from a pipe or pipe-like source, passes them through a FIFO buffer, and maintains a display based on the occasional sampling the contents of the FIFO buffer, with useful features such as triggering and filtering.

Upon successful startup, pw simultaneously monitors its standard input for the arrival of new data, as well as the TTY for interactive commands. Lines from standard input are placed into a FIFO buffer. While the FIFO buffer is not yet full, lines are displayed immediately. After the FIFO buffer fills up with the specified number of lines (controlled by the -n option) then pw transitions into the free-running mode in which, old lines are removed from the tail of the FIFO as new ones are added to the head, and the display is no longer refreshed for every new line arriving into the FIFO.

In free-running mode, the display is automatically refreshed with the latest FIFO data only if:

1.
suspended mode is not in effect; and
2.
trigger mode is not in effect; and
3.
pw isn't executing in the background (see the Ctrl-Z command).

When the above conditions do not hold, and thus the display is being automatically refreshed, it is refreshed at these times:

1.
when there is some keyboard activity from the terminal; or
2.
when the interval period has expired without new input having been seen; or else
3.
whenever the long period elapses.

In other words, while the input is rapidly producing data, and there is no keyboard input, the display is updated infrequently, only according to the long interval.

If pw is executing in the foreground, and if display updates aren't suspended, and if trigger mode is effect, then the display updates automatically only when a trigger is activated by a pattern match on the FIFO contents. Trigger mode is activated using the > and ? commands below.

When standard input indicates end-of-data. pw terminates, unless the -d (do not quit) option has been specified, in which case it pw stays in interactive mode. The the end-of-data status is indicated by the string EOF being displayed in the status line after the data.

 

DISPLAY

Lines displayed by pw are trimmed not to exceed the number of terminal columns. When a line is trimmed, it is terminated by the < character to indicate that there are more characters. The display may be scrolled interactively to read the long lines.

The ASCII DEL character (127) is displayed as ^? and ASCII control characters are displayed as ^@, ^A, ... ^Z, ... ^_. All other characters are sent to the display as-is. No attempt is made to account for the width of East Asian Unicode characters, and such.

When the display is scrolled horizontally, the > character appears at the start of each line to indicate this state. Thus, if neither end of a long line is currently visible, then its visible portion appears delimited by >...<.

 

COMMANDS

While reading data from standard input, pw monitors the TTY for input, providing the following commands.

The commands may be prefixed by a numeric argument, which is ignored by commands to which it is not applicable. The numeric argument is specified by entering digits, which are not echoed. The last 3 digits of the input are retained, so that the argument has an effective range from 0 to 999. Leading zeros are interpreted as the 0 command.

The commands are:

q, Ctrl-C
Quit the program.

l, Left Arrow
Scroll the display to the left.

h, Right Arrow
Scroll the display to the right.

0, Home
Reset scroll to first column.

Space
Suspend the acquisition of new snapshots. In suspended mode, input continues to pass through the FIFO, but new snapshots of data aren't taken. The display updates only due to horizontal scrolling, history browsing, resize, or Ctrl-Z suspend and resume. The status indicator SUSPENDED is displayed below the data. If the the status is already EOF then suspend mode cannot be entered.

Enter
Cancel suspended mode, resuming display refresh. The display is refreshed immediately and the SUSPENDED status indicator disappears.

:
Enter colon command mode. In colon command mode, the status line clears and a colon prompt appears there. An extended command can be entered. Pressing Enter in colon mode dispatches the command. Simple editing is available: backspace, Ctrl-W word erase and Ctrl-U line erase. If backspace is used in an empty colon line, colon mode terminates without executing a command. Ctrl-C and Esc also terminate colon mode without executing a command. Colon commands are documented in the COLON COMMANDS section below.

[pos]/[!]pattern, [pos]?[!]pattern
Set a trigger if a non-empty pattern is specified, or else cancel a trigger if an empty pattern is specified. The / command specifies a head-referenced trigger mode, whereas ? specifies tail-referenced trigger mode.

Under trigger mode, the display is suspended, similarly to suspend mode. It is only refreshed when data arrives into the FIFO such that the state of the FIFO then matches certain regular expression patterns.

A head-referenced trigger is relative to most recently received data which appears at the bottom of the display. A tail-referenced trigger is on least-recently received data, relative to the top of the display. If one or more triggers are set, then trigger mode is in effect. If all triggers are removed, trigger mode is canceled.

Tail-referenced and head-referenced triggers are mutually exclusive. If a tail-referenced trigger command is successfully executed, then all head-referenced triggers are removed. Vice versa, a tail-referenced trigger command cancels all head-referenced triggers.

If the numeric prefix pos is omitted, or specified as zero, then it defaults to 1. The prefix specifies the relative position of the pattern. A head-referenced trigger's position is relative to the most recent data in the FIFO, and therefore is effectively a reverse line number relative to the bottom of the display. For example /foo or, equivalently, 1/foo triggers when the bottom line of the display contains a match for the expression foo. Whereas 3/bar triggers when the third line from the bottom matches bar. To cancel the 3/bar trigger without canceling the 1/foo trigger, the command 3/ can be used: 3/ with an empty pattern. Inversely, the tail-referenced pattern positions are from the top of the display. Thus ?foo or 1?foo trigger when the top line contains a match for foo.

If the value of pos specifies a line beyond the display range, or a value greater than 100, the command is ignored.

Trigger patterns saved in a history which may be navigated for recall using the up and down arrow keys or Ctrl-P and Ctrl-N. While the trigger pattern is being edited, the current set of triggers remains in effect. Editing a trigger pattern may be canceled with ESC, in which case it is not entered into the history and and the trigger command isn't executed. The history stores only the patterns, not the pos prefix or the ? or / command. Erroneous patterns, and the patterns of ignored commands are still entered into the history; empty patterns are not.

The pattern of a newly issued command interpreted as either an extended regular expression (ERE) or (BRE) depending on the current setting. This is true even if it is recalled from a history entry which had been created under a different mode.

If the pattern is preceded by the character ! then it is logically inverted. The trigger will activate on lines which do not match the pattern. To write an uninverted pattern which begins with !, precede the ! with a backslash. This is not a regex character escape sequence, but part of the trigger command syntax. It must not be used anywhere in a pattern, other than immediately after the / or ? command character.

In trigger mode, the status string TRIG/ or TRIG? is shown followed by the list of triggers in parentheses. Patterns which have a position other than 1 are preceded by the position shown in square brackets.

[count]a, [count]d
Advance or delay the currently active triggers by count lines, defaulting to 1. Advancing means that all of the triggers are assigned to more recently received lines closer to the head of the FIFO. on earlier data. Delaying is the opposite: the triggers are assigned to less recently received items, closer to the tail of the FIFO. The advance/delay commands will not move any trigger to a position which corresponds to a line that is not displayed.

k, Up Arrow
Switch the display to earlier snapshot. Each time pw updates the display with a snapshot of data, the existing snapshot is entered into a history window, where it becomes snapshot 1. The previous snapshot 1 becomes 2, and so on. 19 snapshots are retained. The k command switches to an older snapshot: from current to 1, or from 1 to 2, and so on. When snapshot 1 to 19 is being displayed, the status line shows HISTnum where num is the snapshot number.

Tip: the snapshot history is particularly useful when it is gathered by triggering. See the / and ? commands above. Triggered snapshots can reveal recurring patterns in the input. The history can be used to identify differences among triggered snapshots which appear and disappear too rapidly to be read in real time. One of the possible uses of that information is to refine the trigger patterns in order to capture snapshots that are more similar to each other. In the absence of triggering, historic snapshots may be completely unrelated.

Tip: it is usually useful to pause the acquisition of snapshots when browsing the history: see the Space command above. Snapshot acquisition is be observed while browsing the history, since every historic snapshot moves to a higher number when a snapshot is taken, while the displayed number stays the same, and the display updates accordingly.

l, Down Arrow
Go back to earlier snapshot in history. If snapshot 1 is currently being displayed, then pw switches to the current snapshot and the HIST indicator in the status line disappears.

[count]+
Increases the display size by count lines, defaulting to 1. This doesn't come into effect immediately; the newly opened position must be filled by a line of data from standard input. There is no way to reduce the size dynamically. Resizing is limited to the terminal height, if the height is known.

Resizing clears the snapshot history. The command is disabled when viewing snapshots from the history; it must be executed while viewing the current snapshot.

#
Toggle the display of line numbers. In tail-referenced trigger mode, the lines are numbered from bottom to top. In all other situations, top to bottom.

Ctrl-Z
Suspends pw, and indeed the entire surrounding process group, to the background to gain shell access. This requires a job control shell. When pw is foregrounded again, it will refresh the display in a new text area below the cursor, avoiding overwriting anything in the terminal. Thus suspending and then immediately foregrounding provides a way to save a snapshot of the window into the terminal sessions history.

It is possible to execute pw in the job control background. This may happen in two ways: either the pw job is backgrounded from the beginning using the shell's & operator, or else it is suspended with Ctrl-Z and then requested to execute in the background using the shell's bg command. When pw executes in the background, it continues reading from the pipe and discard input, but doesn't update the display. This useful behavior allows pw to be used for monitoring multiple programs which continuously produce output, all from the same job control session. Redirecting the output of such programs to pw and putting them into the background effectively muffles their output, while allowing them to execute indefinitely. Because pw is reading their output, they never block on a pipe write. At any time any such a backgrounded job can be brought into the foreground to visualize its most recent output in the pw display.

 

COLON COMMANDS

First, some general remarks. Display refresh doesn't pause during the editing of a colon command. If that is required, the suspend command must be used. The space between the command and the argument may be omitted. After a command is executed, a result message appears in its place. This message persists over the poll interval period. The longer the poll interval (-i option), the longer the result message persists.

Colon commands are saved in a history which may be navigated for recall using the up and down arrow keys or Ctrl-P and Ctrl-N.

:w filename
Write the currently displayed lines into the specified file.

:a filename
Append the currently displayed lines into the specified file.

:! filename
Pipe the currently displayed lines into the specified command.

:g pattern, :v pattern
Push a grep pattern onto the grep stack. The :g command pushes a plain pattern; the :v command pushes a logically inverted pattern. The grep stack holds up to 64 patterns. If the pattern is successfully pushed onto the stack, it restricts which lines are admitted into the FIFO and available for display. The plain pattern admits only the lines which match pattern; the inverted pattern admits only lines which do not match pattern. Lines which are already in the FIFO at the time a new pattern is introduced are not affected. The status line shows GREP (pattern[, ...]) to indicate that grep mode is in effect, and gives the list of patterns, separated by commas. The inverted patterns are indicated by a leading ! character.

:r[!]
Remove and forget the most recent grep pattern (:gor:v) from the grep stack. If the stack depth decrements from 1, the grep mode is disabled and the GREP status line disappears. If the optional ! modifier is included in the command, then it pops the entire stack, forgetting all the patterns and disabling grep mode.

:i real
Set the poll interval to the number of seconds specified by real. See the description of the -i command line option for the argument format and semantics.

:l real
Set the long interval to the number of seconds specified by real. See the description of the -i command option for the argument format and the -l option for the semantics.

:E, :B
Enable extended regular expressions (EREs) or basic regular expressions (BREs), respectively. If no -E option is given to the program, then BRE mode is in effect on start up. This setting does not affect the meaning of patterns that are currently in effect in the grep stack or in the trigger. The setting affects new patterns that will subsequently be presented to the program. Note that the currently active patterns shown in the status line are not accompanied by any indication of whether they were compiled as ERE or BRE.

Any other command string results in a brief error message.

 

OPTIONS

-i real
Set the poll interval to number of seconds specified by real. This is is a floating-point constant such as 3 or 1.5. Exponential notation such as 5E-1 (equivalent to 0.5) is permitted. This interval determines the input poll timeout of pw's input processing loop. If no data arrives from the primary input, or from the TTY for this amount of time, a timeout occurs, and the display is refreshed if the FIFO has changed since the last refresh. The default poll interval is 1s.

-l real
Set the long update interval to number of seconds specified by real. The format is the same as -i, and the default value is 10s. Every time the long update interval passes, the display is updated, if the FIFO has changed since the last update. This happens even when input is arriving too rapidly to permit the poll timeout to take place. The purpose of the long interval is to ensure that there are updates even when the data source continuously and rapidly produces data, and there is no TTY input activity. The long interval should be at least several times longer than the short interval. The granularity of the timing of the long interval updates depends on the poll interval; in the absence of TTY input, pw will not perform any display updates more often that the poll interval, even if the long interval is made smaller than the poll interval.

-n integer
Set the number of lines N to the specified decimal integer value. The default value is 15. This value must be positive, and is clipped to the number of display lines available in the terminal, less one.

-d
Disable auto-quit: when no more input is available, instead of updating the display one last time and terminating, pw will updating the status to EOF and staying in the interactive mode. This is useful when the last portion of the input is of interest, and contains long lines that require horizontal scrolling.

-E
Enable extended regular expressions (EREs). By default, regular expressions processed by pw are treated as basic regular expressions (BREs). The mode can be switched at run time using the :E and :B commands.

-B
Disable extended regular expressions (EREs), switching to basic (BREs). Since that is the default, this option has no effect unless it appears after a -E option.

 

ENVIRONMENT

pw ignores the TERM environment variable, requiring an ANSI terminal.

pw requires the following conditions to hold in regard to its execution environment, otherwise it terminates with an error diagnostic:

1.
It must be possible to open the /dev/tty device,
2.
Standard input is either not a TTY device, or else not the same TTY device as /dev/tty.

 

TERMINATION STATUS

If pw reaches EOF on standard input without encountering a read error, then its termination status will be successful. This is regardless of whether it automatically quits, or whether it stays in interactive mode and then quits due to a quit command. If the data ends prematurely due to a read error, or if the program is asked to quit before all of the data has been read, then unsuccessful termination status will be indicated.

Incorrect usage, such as nonexistent options or bad arguments to options, result in a diagnostic on standard error and an unsuccessful termination.

Unexpected conditions like out of memory result in abnormal termination (abort).

 

BUGS

This was written over the course of a couple of hours, and tested only interactively.

Many of the issues which follow are easy; patches welcome.

The display format, such as the handling of control characters, is hard-coded.

The program uses hard-coded ANSI sequences, so it doesn't support interesting old terminals. On the other hand, it carries no dependency on any terminal abstraction library/data.

There is no support for unwrapping long lines, which would be useful for copy and paste. However, the features like :w to save the displayed data somewhat compensate for this.

During the :! command's execution, the TTY settings are not restored.

The timeout isn't applied to the arrival of data into the FIFO, but on the arrival of data into the program. Therefore, the grep stack feature doesn't necessarily do what you want in terms of the timeout-based refresh behavior. Consider a situation in which ten lines are arriving per second at a steady rate. The default 1s timeout therefore never goes off. The long refresh interval is what is updating the display, every 10s by default. Now suppose you put in a :gpattern command which only passes every 20th line. Effectively, it now looks as if a line appears every 2s, and so the 1s timeout should be going off to refresh the display. Unfortunately, that's not how it works. The program is still receiving ten lines per second from standard input, and that's where the timeout is applied. Until that is fixed, the way to get more timely refresh behavior under heavy filtering is to play with the long interval.

If the terminal window is resized to a narrower size, many terminals wrap any lines that would get truncated by the operation and this makes a mess of the display. While pw recovers the display area, wrapped lines get pushed above it and remain. This issue is likely unfixable, other than by implementing a text-editor-like full screen mode which has no scroll back above the display, and which recovers the prior content when exiting.

 

AUTHOR

Kaz Kylheku <kaz@kylheku.com>

 

COPYRIGHT

Copyright 2022, BSD2 License.


 

Index

NAME
SYNOPSIS
DESCRIPTION
DISPLAY
COMMANDS
COLON COMMANDS
OPTIONS
ENVIRONMENT
TERMINATION STATUS
BUGS
AUTHOR
COPYRIGHT

This document was created by