:toc: macro
:toc-title: Table of contents
:source-language: lisp
ifndef::env-github[:icons: font]
ifdef::env-github[]
:warning-caption: :warning:
:caution-caption: :fire:
:important-caption: :exclamation:
:note-caption: :paperclip:
:tip-caption: :bulb:
endif::[]
:uri-stable: http://stable.melpa.org/#/logview
:uri-unstable: http://melpa.org/#/logview


= Logview mode

image:https://img.shields.io/badge/license-GPL_3-green.svg[License: GPL 3, link=http://www.gnu.org/licenses/gpl-3.0.txt]
image:https://img.shields.io/github/release/doublep/logview.svg[Latest release, link=https://github.com/doublep/logview/releases]
image:http://stable.melpa.org/packages/logview-badge.svg[MELPA Stable, link=http://stable.melpa.org/#/logview]
image:https://github.com/doublep/logview/workflows/CI/badge.svg[CI, link=https://github.com/doublep/logview/actions?query=workflow%3ACI]

Logview major mode for Emacs provides syntax highlighting, filtering
and other features for various log files.  The main target are files
similar to ones generated by Log4j, Logback and other Java logging
libraries, but there is really nothing Java-specific in the mode and
it should work just fine with any log that follows similar structure,
probably after some configuration.

The mode is meant to be operated in read-only buffer, so most of the
command bindings lack modifiers.

Out-of-the-box the mode should be able to parse standard SLF4J (Log4j,
Logback) files, Apache error logs, PHP Monolog logs, certain UNIX
files in `/var/log` and output of Jupyter Lab.  Nearly all typical
timestamp formats, including localized ones, should be picked up
automatically.

toc::[]


== Installation

Logview is available from MELPA (both {uri-stable}[stable] and
{uri-unstable}[unstable].  Assuming your `package-archives` lists
MELPA, just type:

    M-x package-install RET logview RET

to install it.

Installing Logview from source is not difficult either.  First, clone
the source code:

    $ cd SOME-PATH
    $ git clone https://github.com/doublep/logview.git

Now, from Emacs execute:

    M-x package-install-file RET SOME-PATH/logview

Alternatively to the second step, add this to your `.emacs` file:

    (add-to-list 'load-path "SOME-PATH/logview")
    (require 'logview)

In this case you should probably byte-compile the sources:

    $ eldev compile


== Submodes

Since there is no standard log file format, Logview mode has to try
and guess how the log file it operates on is formatted.  It does so by
trying to parse the first few lines of the file against various
submodes it has defined.

If it succeeds in guessing, you will see major mode specifed as
“Logview/...” in the modeline, where the second part is the submode
name.

In case it fails, you will see it complain in the echo area.  The
buffer will also not be highlighted or switched to read-only mode, so
you will be able to edit it.

=== What to do if Logview mode fails to guess format

This depends on _why_ it fails.  If the mode doesn’t know about this
log format at all, customize the relevant options.  `C-c C-s` will
show you the three variables that are important for submode guessing.
You will need to customize at least one of those, or maybe all three.
All the variables are well-documented in customization interface.

However, it is also possible that Logview fails because it looks only
at the first lines of the buffer.  As of 0.14 this can be up to 500
lines (see option `logview-guess-lines`), but in the unlikely case
real log entries in your files start even later, you can customize
this value.  However, there is an optimization in Logview: if it
discovers several lines that do look like log entry start, yet cannot
be understood by the mode, guessing is aborted.  This is done because
otherwise guessing could take very long time and still be
unsuccessful.  This optimization is triggered upon seeing
`logview-max-promising-lines` such lines (default value is just 3).
You can always customize this setting if needed, but remember, that
this can lead to very long guessing times.

Finally, you can always force Logview to switch to appropriate submode
using `C-c C-c` shortcut.  Remember that you still need to have it
registered first.  In case you don’t want to manually select submode
each time in such cases, you can set up some automatic switching from
a hook using Elisp function `logview-choose-submode`.

If you think your log format is standard enough, you can open an issue
and request format addition to the list of mode built-ins.


[#filters-explained]
== Filters

In addition to level (INFO, ERROR, etc.) filtering, Logview
supports filtering by entry’s logger name, thread and message.

These filters are regular expression and come in two kinds: _include_
and _exclude_ filters.  If at least one include filter is set, only
those entries where relevant part matches at least one of regular
expressions are shown, all others are filtered out.  If any exclude
filter is set, all entries where relevant part matches its regular
expression are filtered out (regardless of any other filters) and
hidden.

Easiest way to add filters is by using `a` / `A` commands (add
include/exclude name filter correspondingly), `t` / `T` (thread
filters), and `m` / `M` (message filters).  You can reset all filters
of given type: `r a` for name, `r t` for thread and `r m` for message
filters, `R` for all filters at once.

However, oftentimes you need to adjust existing filters, e.g. to fix a
typo or simply change one, while keeping others the same.  For this
purpose use `f` command.  It pops up a separate buffer with all
currently active filters (both level and text), which you can edit as
you like, using any standard Emacs features.

Lines beginning with ‘#’ character are comments.  They and empty lines
are ignored.  Lines for level filters must start with prefix ‘lv ’
(note the single trailing space!) for normal filtering or ‘LV ’ for
“always show” pseudo-filter, and contain the textual level
representation afterwards, just as you would see it in the logfile.
For example:

    lv DEBUG
    LV ERROR

means “show all entries except trace-level, and additionally show all
errors with no regard to text filters.”

Lines for text filters are similar.  They must start with ‘a+ ’
(again, remember the trailing space) for name inclusion filters, ‘a- ’
for name exclusion, and similarly ‘t+ ’, ‘t- ’, ‘m+ ’, ‘m- ’ for
thread and message filters.  Additionally, multiline message filters
must use ‘.. ’ prefix (two dots and a space) for continuation lines.
For example:

    a+ database
    m+ ^insert.+
    .. values

means “show entries with word ‘database’ in the logger name and which
message has a line beginning with ‘insert’ and the line after that
beginning with ‘values’.”

The buffer mode has some syntax highlighting support, so you should
see if anything goes wrong.  The easiest way to figure it out is to
add a few filters using commands described earlier and then open this
buffer with `f` and see how they are represented.

Changes in the buffer popped up with `f` (or <<thread-narrowing,Y>>)
are previewed in the main buffer on-the-fly.  If you don’t like this,
customize variable `logview-preview-filter-changes`, toggle this in
the main buffer using `o p` or in the filter buffer with `C-c C-p`.

=== Filter regexp details

Regular expressions can be matched against entry parts either
case-sensitively or case-insensitively, depending on standard Emacs
variable `case-fold-search`.

Filters are matched against relevant entry parts as strings, not
against the whole buffer.  Therefore, you can use `^` and `$` special
characters for the expected meaning.  For example, adding `^org` as
name exclusion filter will hide all entries where logger name begins
with string ‘org’.

Unlike name and thread filters, message filters can span multiple
lines.  To enter linefeed in message buffer (after `m` or `M`) use
`C-q C-j`.  When editing a multiline filter with `f`, prefix all
continuation lines with ‘.. ’.

Commands `a`, `A`, `t` and `T` default to the name (or thread) of the
current entry.  You can also use `C-p` (`<up>`) to browse history of
previously entered values and `C-n` (`<down>`) for a few default
values.

[#thread-narrowing]
=== Thread-narrowing filters

In addition to “normal” or “main” filters, Logview supports a separate
set of _thread-narrowing filters_, naturally only in those submodes
that have threads.  Thread-narrowing filters are independent from
normal filters and are combined with an ‘and’ operation when filtering
out entries.

These filters can be quickly changed using command `y` that toggles
between “narrowing” to the current entry’s thread and clearing
thread-narrowing filters altogether.  They can also be changed by
command `c c` (see <<sections-explained,the topic on sections>>).
Finally, just as command `f` allows you to edit normal filters,
command `Y` can be used to edit thread-narrowing filters.  However,
only ‘t+ ’ and ‘t- ’ filters are understood here.

Thread-narrowing filters are not affected by filter resetting commands
with the exception of `r e`, that resets “everything possible”.
Instead, they are treated as part of buffer narrowing.  In particular,
command `w` (“widen”) resets thread-narrowing filters in addition to
standard Emacs buffer narrowing.  In the same vein, thread-narrowing
filters are not considered part of <<views-explained,views>>.

This is largely the justification for their existence: to decouple
quick changes to displayed thread(s) from the main filters.


[#views-explained]
== Views

_Views_ are named sets of filters that you can activate quickly.  They
are especially useful if you use name or message filters a lot, and
often find yourself typing in the same filters over and over again.

The easiest way to define a view is by first adding all the filters
you need.  This way you can see in the buffer if the filtering result
matches what you expect.  After you are satisfied, type `V s` and a
name for the new view.  Notice that the mode line now displays name of
the view in square brackets after the submode name, e.g.:

    Logview/SLF4J [useful-view-1]

Now type `R` to reset all the filters.  All previously hidden entries
will be shown again and the view name disappear from the mode line.
However, to restore the filters now you don’t have to re-create them
one-by-one.  Simply type `v` and whatever name you used when saving
your first view.  You can also use text completion to pick among all
the defined views.

To make choosing views even easier, you can optionally assign quick
access indices to views.  For this, activate a view normally (or have
it just saved), type `V i` and enter a number, say 3.  After this, the
view can be quickly activated again by typing `M-3` or `3 v`.

Remember that further filtering doesn’t affect view definition.  If
you want to change a view, save filters as a view with the same name
again, and confirm that you do want to replace the previous
definition.  Alternative way is to edit views using `V e`.  This pops
up a separate buffer just like `f` command does, but instead of
filters you will edit all defined views for the current submode at
once.  This way you can change existing definitions, delete unneeded
or add more.  Commands like `V s` or `V d` (delete a view by name) can
be seen as just a convenience.

Views come in two kinds: globally accessible and bound to a specific
submode.  This distinction is important if you use logs of different
kinds.  Most often you need submode-specific views, because text
filters usually can’t be meaningfully applied without changes to
different programs.  When you use `v` command, only the views for the
current submode plus any global views are available for selection.

In addition to applying view filters, it is also possible to move
between entries in a view without activating it.  For this, define a
view and then set it as as a _navigation view_ with `V n` command.
After this, use commands `M-n` and `M-p` to quickly navigate forward
and backward.  Remember that these commands skip all hidden entries,
whether because of your main view (or filters) or manual entry hiding.

Finally, you can highlight all entries in a view, or, more precisely,
those that are visible currently.  This can be done with `V h`
command.  Cancel this by highlighting entries from a different view or
removing highlighting altogether with `V u`.

To summarize:

* You can have any number of named views.  Their definitions are
  stored permanently across Emacs session and are available from all
  Logview buffers.
* At any time you can switch to a view, i.e. replace current filters
  with those stored in the view’s definition.  Changing filters itself
  doesn’t alter any view definitions.
* You can appoint one view as a <<sections-explained,section>> view.
  It will be used for highlighting section headers and all
  <<sections,section commands>>.
* You can choose one navigation view, independently from the currently
  applied view.  Navigation view is used by commands `M-n` and `M-p`.
* You can highlight entries of a view, again, independently from
  current, section or navigation view.


[#sections-explained]
== Sections

Logview can split your log files into _sections_ to simplify
navigating and comprehending what would otherwise be an endless flow
of entries.  For this, you need to create a <<views-explained,view>>
that matches entries that you define as _section headers_.  For
example, if a log is generated by some kind of a server, each section
could span one request to the server and the section header view
should match only the “intro” entries of request processing.  An
example view definition could look somewhat like this:

    view Server X sections
    submode Server X
    lv INFO
    a+ ^my\.server\.Class$
    m+ ^serving request to

Since section views are supposed to be used often, it is recommended
to include “excessive” filters (e.g. the level and name filters in the
example above) to make them faster.

You can now activate the created section view with `V c` command or
any of the section commands (`c \...`).

Section headers will be highlighted with inverted colors and bold
text, allowing you to easily spot boundaries between different
requests.  Perhaps even more importantly, various section commands,
e.g. `c a` or `c n` let you navigate the log in terms of sections, and
command `c c` lets you instantly narrow (as in Emacs buffer narrowing
combined with <<thread-narrowing,thread narrowing>>) to the current
section.

Sections in Logview can be either _thread-bound_ or not.  By default,
if the log has a concept of threads, sections are thread-bound.  You
can toggle this using command `c t`; additionally, there are several
commands like e.g. `c N` that temporarily treat sections as
non-thread-bound, even if they normally are.

When sections are thread-bound, they can have overlap each other:
entries in different threads always belong to different sections.
This actually reflects how threaded programs (that create logs with
different threads) work, so shouldn’t be seen as unexpected.

Sections work particularly nicely in combination with timestamp
difference commands.  Using command `z c` you can order Logview to
replace timestamps for all non-header entries with the difference to
the timestamp in the corresponding section’s header.  This is useful
when investigating logs for sources of performance problems.

Another use for sections is long-distance navigation in a huge log
file.  You can use command `c h` to temporary “narrow” to only section
headers.  Typically, the headers should be informative enough,
e.g. contain accessed resource name (but this depends on how _you_
define the section view), that you can quickly find the section you
are interested in at the moment.  This functionality is also available
from within <<isearch,Isearch>> as `M-h`.


[#isearch]
== Special Isearch support ==

Logview adds a few extensions to Isearch.  When performing an
incremental search, usually started with `C-s`, you can switch between
showing entries normally and only section headers with `M-h` (see
<<sections-explained,the explanation of log section>>).  Similarly,
you can temporarily switch between searching all visible text (the
default) and only visible message using `M-m` (outside Isearch this is
available with `o m`).  When you activate any of these options from
within Isearch, the change is temporary and is cancelled once you are
done with the search.  This is for consistency with how e.g. `M-c` in
Isearch operates: its change in case-folding also affects only the
current search and is not remembered outside it.

In addition, when searching only in messages, Logview will dim all
other text, but only when incremental search is in progress.  This is
done regardless of how the option is activated.


== Commands

Nearly all commands have some use for prefix argument.  It can be
usually just guessed, but you can always check individual command
documentation within Emacs.

When buffer is switched to read-write mode, Logview automatically
deactivates all its commands so as to not interfere with editing.
Once you switch the buffer back to read-only mode, commands will be
active again.

=== Movement

* All standard Emacs commands
* Go to the beginning of entry’s message: `TAB`
* Go to next / previous entry: `n` / `p`
* Go to next / previous “as important” entry: `N` / `P`
* Go to next / previous entry in the navigation view: `M-n` / `M-p`
* Go to the next / previous entry with large timestamp gap after the
  previous: `z n` / `z p`
* Same as above, but only considering entries in the same thread: `z
  N` / `z P`
* Go to first / last entry: `<` / `>`

“As important” means entries with the same or higher level.  For
example, if the current entry is a warning, “as important” include
errors and warnings.

Many <<sections,section commands>> also just move the point.

=== Narrowing and widening

* Narrow from / up to the current entry: `[` / `]`
* Widen (and cancel thread-narrowing filters): `w`
* Widen upwards / downwards only: `{` / `}`
* Toggle narrowing to the current entry’s thread: `y`
* Edit thread-narrowing filters: `Y` (pops up a separate buffer)

Command `y` toggles between narrowing to the current entry’s thread
and completely cancelling all thread-narrowing filters.

See also some <<sections,section commands>>.

=== Filtering by entry level

* Show only errors: `l 1` or `l e`
* Show errors and warnings: `l 2` or `l w`
* Show errors, warnings and information: `l 3` or `l i`
* Show all levels except trace: `l 4` or `l d`
* Show entries of all levels: `l 5` or `l t`
* Show entries “as important” as the current one: `+` or `l +`

=== Always showing entries of certain levels

It is possible to always display entries of certain levels, regardless
of any additional text filters.

* Always show errors: `L 1` or `L e`
* Always show errors and warnings: `L 2` or `L w`
* Always show errors, warnings and information: `L 3` or `L i`
* Always show all levels except trace: `L 4` or `L d`
* Disable “always show” feature: `L L` or `L 0`

=== Filtering by entry’s logger name, thread or message

See <<filters-explained,more detailed description above>>.

* Edit current name, thread and message filters: `f` (pops up a separate buffer)
* Add name include / exclude filter: `a` / `A`
* Add thread include / exclude filter: `t` / `T`
* Add message include / exclude filter: `m` / `M`

<<narrowing-and-widening,Thread narrowing commands>> can also be seen
as filtering.

=== Resetting filters

* Reset level filter: `r l`
* Reset name filters: `r a`
* Reset thread filters: `r t`
* Reset message filters: `r m`
* Reset all filters: `R`
* Reset all filters, widen and show all explicitly hidden entries: `r e`

=== Views

See <<views-explained,more detailed description above>>.

* Switch to a view: `v`
* Choose a <<sections-explained,section>> [header] view: `V c`
* Choose navigation view (for `M-n` and `M-p`): `V n`
* Select a view to highlight its entries: `V h`
* Remove view highlighting: `V u`
* Save the current filters as a view for this submode: `V s`
* Save the current filters as a global view: `V S`
* Edit submode views: `V e` (pops up a separate buffer)
* Edit all views: `V E` (pops up a separate buffer)
* Assign a quick access index to the current view: `V i`
* Delete a view by name: `V d`

You can also switch to views using their quick access index:
`M-0`..`M-9` or e.g. `1 4 v` (for quick access index 14).  Prefix
argument works also for `V n` and `V h`.

=== Sections

See <<sections-explained,more detailed description above>>.

* Go to the current section’s beginning/end: `c a` / `c e`
* Go to the next / previous section: `c n` / `c p`
* Go to the next / previous section in any thread: `c N` / `c P`
* Go to first / last section: `c ,` / `c .`
* Go to first / last section in any thread: `c <` / `c >`
* Narrow to the current section: `c c`
* As above, but don’t touch thread narrowing filters: `c C`
* “Narrow” to section headers, i.e. don’t show any other entries: `c h`
* Toggle whether sections are bound to threads: `c t`

=== Explicitly hide or show individual entries

* Hide one entry: `h`
* Hide entries in the region: `H`
* Show some explicitly hidden entries: `s`
* Show explicitly hidden entries in the region: `S`
* Show all manually hidden entries in the buffer: `r h`

In Transient Mark mode `h` and `s` operate on region when mark is
active.

=== Entry timestamp commands

* Replace timestamps with their difference to that of the current
  entry: `z a`.
* Same as above, but only within the same thread: `z t`
* Same as above, but within _every_ section relative to its header: `z c`
* Go to the entry difference to which timestamp is shown: `z z`
* Don’t show timestamp differences: `z A`
* Don’t show timestamp differences for this thread: `z T`
* Don’t show timestamp differences in every section: `z C`

Timestamp differences are displayed in seconds.  See also
<<sections-explained,description of sections above>>.

=== Explicitly hide or show details of individual entries

The mode terms all message lines after the first “details”.
Oftentimes these contain exception stacktrace, but most logging
libraries let you write anything here.

* Toggle details of the current entry: `d`
* Toggle details of all entries in the region: `D`
* Toggle details in the whole buffer: `e`
* Show all manually hidden entry details in the buffer: `r h`

In Transient Mark mode `d` operates on region when mark is active.

=== Change options for the current buffer

These options can be customized globally and additionally temporarily
changed in each individual buffer.

* Change gap length for `z n` and similar commands: `o g` or `z g`
* Toggle Auto-Revert mode: `o r`
* Toggle Auto-Revert Tail mode: `o t`
* Toggle “copy only visible text”: `o v`
* Toggle “search only in messages”: `o m`
* Toggle “preview filtering results”: `o p`
* Toggle “show ellipses”: `o e`

=== Miscellaneous

* Pulse (briefly highlight) the current log entry: `SPC`
* Manually choose appropriate submode and timestamp format: `o s` or `C-c C-c`
* Customize options that affect submode selection: `o S` or `C-c C-s`
* Bury buffer: `q`
* Refresh the buffer (appending, if possible) preserving active filters: `g`
* Prepare the buffer for new contents: `G`
* Append log file tail to the buffer: `x`
* Revert the buffer preserving active filters: `X`
* Universal prefix commands are bound without modifiers: `u`, `-`, `0`..`9`

Preparing for new contents (`G`) is useful in situations where you
want to abstract from existing entries.  Next time you issue command
`g` after it, the buffer will contain all entries added between `G`
and `g`.  `G` is roughly equivalent to the following sequence of
commands: `g`, `w`, `>`, `[`.

=== Extensions for Isearch mode ===

* “Narrow” to section headers, i.e. don’t show any other entries: `M-h`
  (normally available as `c h`)
* Toggle “search only in messages”: `M-m` (normally available as `o m`)


[#locked-narrowing]
== Locked narrowing in Emacs 29

Emacs 29 (in development) has introduced locked narrowing as a way to
improve performance.  At the same time, it broke compatibility by
making function `widen` not always do what packages would expect from
it.

It is extremely difficult to adapt Logview to `widen` not widening as
it has always does.  Or likely even impossible without completely
redesigning the mode, sacrificing lazy filtering in the process.

Locked narrowing restrictions can be lifted, but for this you need to
know the “tag” used to install them.  There is no way to find the tag
(unless you have locked with it yourself and thus just know), but at
least Emacs itself uses only a few hardcoded tags.  Logview tries to
unlock all of those.  However, if restrictions are still locked after
that, Logview will fail before causing more (and unpredictable)
problems: e.g. previously it could even cause a full freeze in Emacs
(where even `C-g` does nothing) from its fontification code.

Normally, such errors should not happen now, but if they do then
either Emacs invented yet another place where it has to
+++<del>+++break everything+++</del>+++ lock restrictions, or this has
made it into some minor mode you use (they made function
`narrowing-lock` public for everyone to try and break other code).

I have tried arguing with Emacs developers about this, but that is
pointless, as I have found on this and several other occasions.
