History Files

The History File

Since V2.30, Magma saves the lines the user types to a history file, so lines from earlier sessions can be recalled in later ones. This behaviour is on by default. Before V2.30 the history buffer, which the line editor (Section The Magma Line Editor) scrolls through and the ‘%’ commands of Section History Commands act on, was lost each time Magma exited. Everything in this section is new in V2.30.

How it works.

  • At start-up, the last lines of the file are loaded into the history buffer, ready for the up arrow, the incremental search, and %h.

  • Each line is appended to the file as soon as it is entered, before it is run. Nothing is left to do at exit, so a killed session loses nothing.

  • The file is not created until the first real command is entered.

  • The file is a rolling window: it keeps the last 1000 lines the user has typed, whatever sessions they came from, and once it is full the oldest lines drop off to make room. Nothing else is ever removed from it. Use the option history-lines (Section The Options File, below) to change the default 1000 to another value; 0 means no limit.

  • The file is trimmed at start-up, not at exit, and only once it exceeds twice history-lines, so it is not rewritten on almost every start. Only one session trims at a time, locking history.lock (see below); a line typed in another session at the instant a trim finishes can be lost.

  • Recall is also capped by the history buffer size, set by SetHistorySize (default 2000). If history-lines is larger, Magma says so at start-up, and whenever SetHistorySize makes the buffer smaller than it. The extra lines stay in the file but are not recalled.

  • Sessions running at the same time interleave their lines in the file, in the order they were typed, as shells do. To keep each session’s lines separately, see Section Saved Sessions and the Browser.

Recalling lines.

  • The up arrow and the incremental search reach every line in the history buffer (normally the whole history file), from this session and earlier ones.

  • Given no range, the ‘%’ commands act only on the lines entered in this session; a line number reaches any line in the history buffer, and the word all takes in the whole buffer, which normally holds the whole history file.

What is not recorded.

  • Only interactive sessions, with input from a terminal, use the history file. A run such as magma < file records nothing, reads no options file, creates no directory, and ignores MAGMA_CONFIG_DIR (Section Environment Variables).

  • Input supplied to read and readi is not saved.

  • Blank lines and a bare quit; are not saved, unless the option remove-quit is turned off (Section The Options File).

  • Only the text of each line is saved, not the random number generator state, so the seed-setting forms %s and %S (Section History Commands) are not meaningful for lines from earlier sessions.

The ~/.magma directory. Here are details of how the history is stored, for when the user wants to change the defaults (nothing needs to be understood or edited for the default behaviour).

  • The first interactive session creates the directory ~/.magma:

    ~/.magma/
    options            the user's settings
    state/
        history        the history file
        history.lock   used while trimming the file

    When Magma creates the directory, the file options (Section The Options File) starts as a copy of the default file (libs/auxiliary/options in the standard Magma distribution), with every line a comment initially, and can be edited by the user. After the initialization, Magma itself writes only under state.

  • To use another directory, set the environment variable MAGMA_CONFIG_DIR (Section Environment Variables):

    setenv MAGMA_CONFIG_DIR ~/.magma-history          # csh/tcsh
export MAGMA_CONFIG_DIR=~/.magma-history          # sh/bash

    It takes precedence over ~/.magma, and is created if it does not exist.

  • To keep no history at all, put history no in the options file, or set the environment variable MAGMA_NO_HISTORY to anything. The variable also stops Magma creating any directory.

  • If ~/.magma exists but is not a directory, Magma keeps no history and says so at the start of every session.

Saved Sessions and the Browser

Magma can also keep each session’s lines in a file of its own (in the sub-directory saved of state), so that whole earlier sessions can be browsed, searched, and re-run with the browser described below. This archive of earlier sessions is off by default. To turn it on, do one of the following:

  • Simply open the browser on an empty line (press <Ctrl>-R in vi mode, <Esc> r in emacs mode, or enter %b in either mode) and answer y when Magma asks No saved sessions. Keep them from now on? (y/n). The lines already typed in the current session are kept as well, and the browser opens on them. Magma asks at most once a session, whatever the answer, and not at all if the options file sets saved-sessions explicitly.

  • Set saved-sessions yes in the options file (Section The Options File).

  • Create the directory: mkdir -p ~/.magma/state/saved.

Turning the archive off later does not delete it; Magma just stops adding to it.

The archive. The archive sits beside the history file under state:

~/.magma/
    options
    state/
        history
        history.lock
        saved/
            history20140317-101535
            history20140317-142208
            ...

  • Each session has its own file, named by the date and time the session began, with a numeric suffix if needed to make the name unique.

  • Each command line is appended to the session’s file as well as to the history file.

  • Like the history file, the session’s file is not created until the first real command is entered, so a session in which nothing is typed leaves no file behind.

Opening the browser. The browser shows the archive, starting with the current session. It is available only in interactive sessions; with no archive kept, opening it offers to start one (see above).

  • In vi mode, <Ctrl>-R on an empty line opens the browser, or, with no archive kept, asks whether to start one, opening the browser if the answer is yes. The incremental search is on / in command mode instead (see Section The Magma Line Editor).

  • In emacs mode, <Ctrl>-R is the incremental history search (see Section The Magma Line Editor), so the browser is <Esc> r, or a second <Ctrl>-R at an empty search prompt, on an empty line.

  • The keys may be changed with the options browser-key-emacs and browser-key-vi (Section The Options File). The %b command opens the browser whatever they are set to.

The display. Each session is shown as a short, fixed-size preview: a banner giving the session’s date and time, its first few command lines, an ellipsis, and its last few lines.

  • Each line is truncated to the screen width, so the display never wraps.

  • Three lines are shown above the ellipsis and three below, unless the options browser-lines-top and browser-lines-bottom (Section The Options File) say otherwise. A session short enough to fit is shown whole, without an ellipsis.

  • Tabs in the saved commands are expanded to the next tab stop, eight columns apart unless the option browser-tab-width says otherwise, so indentation is preserved in the preview.

Keys. The browser recognises the following keys:

k, up-arrow      move to an older session
j, down-arrow    move to a newer session
g                jump to the newest session
G                jump to the oldest session
Return, Enter, r re-enter and run the commands of the shown session
Space            load the commands into the history buffer, without running them
0-9, -           a count prefix for the next key (see below)
&                filter to the sessions whose commands match a pattern
/                search for the next matching session, toward older ones
?                search for the next matching session, toward newer ones
n                repeat the last search in the same direction
N                repeat the last search in the other direction
i                toggle case-insensitive matching
E                edit the shown session file in place, then redisplay it
e                edit a copy of the shown session and run the result
s                save the shown session (or a line range) to a named file
c                browse a different directory (empty returns to saved/)
d                delete the shown session file
q, Esc           leave the browser

Loading or running an older session’s commands adds them to the current history, so they may then be recalled and edited like any others.

Finding a session. There are two ways of finding a session by what it contains, and both take the same kind of pattern. A session matches if any of its command lines does.

  • & prompts for a pattern and restricts the browser to the matching sessions, leaving the others out of the list until an empty pattern clears the filter. The pattern in force is shown in the banner.

  • / searches instead: the list is left as it is, and the browser moves to the next matching session, toward older ones; ? does the same toward newer ones. n repeats the search in the same direction and N in the other, as does an empty pattern at the / or ? prompt.

  • A count before any of these keys asks for the \(N\)-th match rather than the first.

  • The pattern is a regular expression, matched against each command line, unless the option browser-search-literal (Section The Options File) says it is to be taken literally. <Tab> at the prompt switches between the two for that pattern, and the prompt shows which is in force: &re or &lit for the filter, /re or /lit for a search. A literal pattern is the one to use for text full of brackets, as Magma code is.

  • i toggles case-insensitive matching for both kinds of pattern; a [ci] tag in the banner shows when it is on. A case-insensitive regular expression is matched by folding both the pattern and the line to lower case, so a range such as [A-Z] becomes [a-z] and then matches any letter.

Counts. Typing digits (optionally preceded by -) before a key builds a count \(N\), shown in the banner, that modifies the next key:

  • k/j (and the arrow keys) move by \(N\) sessions.

  • g/G jump to the \(N\)-th session: \(1\) is the newest, and a negative count counts from the oldest, so \(-1\) is the oldest.

  • /, ?, n, and N go to the \(N\)-th match.

  • r/Return, Space, e, and s act on only the last \(N\) commands of the session, or the first \(|N|\) if \(N\) is negative, instead of all of them.

  • d deletes \(N\) files: the shown one and the next \(N-1\) older ones, or newer ones for a negative count.

Editing and deleting.

  • E opens the shown session file in the user’s editor; any changes are saved to the file permanently, and the browser redisplays it. The editor is the option browser-editor, else the EDITOR environment variable (as for the %e command, Section History Commands), else /bin/ed.

  • e edits a temporary copy instead. If the user makes no real change, or leaves it blank, the edit is abandoned; otherwise the new contents are entered and run as if the user had accepted them.

  • d asks for confirmation before deleting a session file, unless the option browser-confirm-delete is set to no (Section The Options File).

Saving and browsing other directories.

  • s saves the shown session (or, with a count, just its last \(N\) or first \(|N|\) command lines) to a file the user names, so that a session worth keeping can be moved out of the archive. Naming the shown session itself is refused: the copy would empty the file it was about to read.

  • c browses such a directory: it prompts for one, and shows the files there in place of the saved archive. An empty answer returns to saved.

  • The files in another directory need not be Magma’s own saved sessions, and their names are arbitrary, so they are listed by modification time, newest first, and each is shown in the banner by that time followed by its name (see the option browser-name-max, Section The Options File). In the saved archive only the date is shown, so the name also shows that another directory is being browsed.

  • All the other keys behave as usual there, except that d always asks for confirmation: those are the user’s own files, not Magma’s.

  • When s or c prompts for a name on the bottom row, <Tab> completes it from the files (for c, the directories) that match what the user has typed, filling in as far as the possibilities agree and appending a / when the name is a directory. The prompt is also a small line editor with the usual keys, as in the main line editor. These are the left and right arrows, <Ctrl>-A/<Ctrl>-E to the start/end, <Ctrl>-B/<Ctrl>-F by a character, <Ctrl>-D to delete, <Ctrl>-K to the end, <Ctrl>-W the previous word, and <Ctrl>-U the whole line. <Esc> abandons the prompt and returns to the browser unchanged.

Appearance.

  • By default the browser draws the session inside a coloured box (banner, contents, and key menu). The colour is blue unless the option browser-color (Section The Options File) says otherwise.

  • The box can be turned off, for a plain, uncoloured display, with the option browser-plain (Section The Options File), or, for a single session and without editing the options file, by setting the environment variable MAGMA_HIST_PLAIN to any value (the variable can only make the display plainer, never fancier). It is also turned off automatically when the screen is very narrow.

  • The setting browser-plain dumb goes further: the browser then emits no terminal escape sequences at all, neither colour nor the cursor movement it normally repaints itself with. Since it cannot move the cursor back up, each keystroke prints a fresh block below the last one and the screen scrolls, like a printed listing; everything else about the browser is unchanged. This is the setting for a terminal that does not understand the usual sequences, and it is chosen automatically when TERM is unset, empty, or dumb. Setting MAGMA_HIST_PLAIN to dumb asks for it for a single session.

Limiting the archive. The saved sub-directory grows without bound by default, one file per session for ever.

  • The option saved-max-files (Section The Options File) keeps only the newest \(N\) sessions, and the option saved-max-days only those from the last \(N\) days. Either or both may be given; both default to zero, meaning no limit. These options delete files, so nothing is discarded unless the user asks for it.

  • The limits are applied at start-up, and the current session’s file is created afterwards, so the directory holds one more file than saved-max-files says until the next start trims it back. The point is to bound the growth, not to hit an exact count.

  • Only files named as Magma itself names them are ever considered, so anything else the user keeps in that directory is left alone.

  • The archive is the real record of the user’s work; the history file is only the last history-lines lines of it. The & filter and the / and ? searches read every file in the archive, so a very large archive makes them slower, which is the other reason to set a limit.

The Options File

The file options sets options for everything described above: the history file, the saved sessions, and the browser.

  • It is at the top of the history directory: ~/.magma/options, unless MAGMA_CONFIG_DIR says otherwise.

  • It can be edited by the user. When Magma makes ~/.magma it puts the default options file there, with every line a comment showing the built-in value (except saved-sessions and browser-editor, whose defaults depend on the directory and on EDITOR), so that the settings are in front of the user and not only in this section. Uncomment whatever is to be changed.

  • After that Magma never writes to it, not even to recreate it: if it is deleted, every option goes back to its built-in default.

  • A directory named by MAGMA_CONFIG_DIR gets no such file: copy it from libs/auxiliary/options in the Magma distribution, or write one from scratch.

Format. Each line names one option and gives its value:

browser-lines-top       10
browser-lines-bottom    4
browser-color           green
browser-confirm-delete  no

# blank lines and lines beginning with '#' are ignored
browser-date-format     %Y-%m-%d %H:%M

  • White space separates the name from the value. The value runs to the end of the line, so it may itself contain spaces, as browser-date-format generally will.

  • Names are matched without regard to case. If an option is named more than once, the last line wins.

  • Naming an option that does not exist, or giving a value that cannot be understood, draws a warning at start-up and otherwise has no effect. A value that is understood but out of range is quietly brought into range.

  • A boolean (yes/no below) may be written as yes/no, true/false, on/off, or 1/0.

History file options.

  • history (yes/no, default yes): the off switch. history no stops the history being kept at all.

  • history-lines (\(N\), default 1000): how many lines the global history file keeps: the last \(N\) the user has typed, whatever sessions they came from, which is what is read into the history buffer when Magma starts. Zero means no limit. Keep it below the size of the history buffer itself (SetHistorySize, 2000 lines by default), or the lines read at start-up fill the buffer before the user has typed anything, and each line the user then types pushes out one that was read.

  • history-print-lines (\(N\), default 100): how many lines a printing command shows when it is given all with no count of its own: %h, %PA, %p all, and the seed forms. Zero means no limit. It exists so that a full history is not emptied onto the screen by accident; the command says when it has shortened its output, and a count, or 0, still asks for everything. It does not affect %e or %E.

  • remove-quit (yes/no, default yes): whether a line consisting of just quit;, the command that ends the session, is dropped from the saved file. By default it is, since it merely repeats in every session and is no help on recall. Turn it off if the user types code in which a whole physical line is legitimately quit; (for instance inside a multi-line string literal) and wants the saved session to reproduce it exactly.

Saved session options.

  • saved-sessions (yes/no, default yes if state/saved exists, otherwise no): keep the per-session archive (Section Saved Sessions and the Browser). Answering y in the browser writes nothing to this file: creating state/saved is itself what keeps the archive on in later sessions, just as an existing archive goes on being kept.

  • saved-max-files (\(N\), default 0): how many session files to keep; 0 keeps all.

  • saved-max-days (\(N\), default 0): how many days of sessions to keep; 0 keeps all.

Browser display options.

  • browser-lines-top and browser-lines-bottom (\(N\) or \(N\)%, default 3 each): the lines shown above and below the ellipsis.

    • A count is either a plain number of lines or a percentage of the height of the terminal, written N%, so browser-lines-top 25% asks for a quarter of the screen.

    • A percentage is worked out afresh each time the display is drawn, so it follows the terminal if the window is resized while the browser is open.

    • At least one line is always shown above and below, and if the two counts together would not leave the whole block room on the screen, they are reduced until they do.

  • browser-color, also spelt browser-colour (default blue): the box colour. One of blue, cyan, green, red, magenta, yellow, white, or none (which draws the box without colour).

  • browser-plain (yes/no or dumb, default no): a plain, unboxed display; dumb makes it free of escape sequences as well.

  • browser-tab-width (\(N\), 1 to 32, default 8): the tab stop width.

  • browser-date-format (a format, default %a %b %e %H:%M:%S %Y): the format of the date in the banner. It is passed to the C strftime function, and if it yields nothing, or is too long to fit, the file name is shown instead. The default omits %Z (the time zone), which is the same for every session the user is likely to browse and only takes up banner width; add it back if wanted.

  • browser-name-max (\(N\), default 32): applies only when browsing a directory other than saved (the c key), where the file name is shown in the banner as well as the date. It is the greatest number of characters of that name to show, or \(0\) for all of it (at the risk of pushing the (n/m) position off the edge of the box).

Browser key and matching options.

  • browser-key-emacs (a key, default esc-r): opens the browser in emacs mode.

  • browser-key-vi (a key, default ^R): opens the browser in vi mode.

  • search-key (a key, default ^R): starts the incremental search in emacs mode. It must be a control key: an esc-x value is rejected with a warning.

  • search-key-vi (a key, default /): starts the incremental search in vi mode. Besides the forms below, it may be / or ?, which then works only in command mode; in insert mode it is typed as usual. If it is set to the same key as browser-key-vi, a second press at an empty search prompt, on an empty line, opens the browser, as in emacs mode. An esc-x value is rejected with a warning.

  • The four key options (browser-key-emacs, browser-key-vi, search-key, and search-key-vi) name a key in any of these forms, or none (or off) to leave the action unbound; the letter in a control or meta key is not case-sensitive.

    • ^X, C-x, ctrl-x, or control-x: a control character.

    • esc-x, M-x, meta-x, or alt-x: the two-character sequence <Esc> then x.

    • A meta key (esc-x form) for browser-key-vi has no effect, since in vi mode <Esc> means “enter command mode” and so cannot begin a binding.

    • Keys that cannot reach Magma at all are rejected with a warning rather than bound to nothing: ^@, which never arrives, and ^S, ^Q, and ^Z, which the terminal driver takes for flow control and suspension. So are the keys the editor cannot do without: ^H (<Backspace>), ^I (<Tab>), ^J and ^M (<Return>), and ^[ (<Esc>).

    • ^O and ^T are accepted but work only on Linux, being the “discard” and “status” characters elsewhere, which is why neither is the default.

  • search-ignore-case (yes/no, default no): whether the incremental search ignores case, in either mode.

  • browser-filter-ignore-case (yes/no, default no): whether matching in the browser starts case-insensitive. It sets only the state the browser starts in; the i key still toggles case-insensitivity from there. It applies to regular expressions as well as to literal patterns.

  • browser-search-literal (yes/no, default no): whether the &, /, and ? prompts start out expecting a literal string rather than a regular expression. <Tab> at the prompt switches between the two whatever it says.

  • browser-confirm-delete (yes/no, default yes): whether d asks before deleting.

  • browser-editor (a command, default unset): the command the browser’s E and e keys run. It may include arguments; the file name is appended to it. If it is not set, the EDITOR environment variable is used, and failing that /bin/ed, so this is worth setting only if the user wants those keys to use a different editor from %e and everything else.