Synopsis:
This thread covers what TextFinder does, in no particular language. This page is the
whole utility at a glance.
-
Take a root path and a regular expression, walk the tree below the root, evaluate
the expression against each line of each selected file, write every matching file
to stdout.
-
Nine switches control the search and every one has a default, so any partial
command line resolves to a full option set.
Everything stdout carries is one of three things.
- A block, one per matching file - a path line, then a detail line per matching line.
- An announcement, for a file or directory that produced no block. Four of them, in two kinds, and the kind decides whether
/h can hide it.
- One run summary line counting the files and directories the run reached.
Three exit codes are fixed rather than left to the implementation, so two
implementations can be compared by exit code as well as by output.
- Neither an error announcement nor an empty result changes the code - finding nothing is a successful search.
- Section 6 sketches the four-component pipeline the remaining Behavior pages are organized by.
1. The Utility in One Page
TextFinder takes a root path and a regular expression, walks the tree below the root,
evaluates the expression against each line of each file it selects, and writes every
matching file to stdout. That is the whole of it, per Spec_TextFinder.md §1.
Nine switches control the search, every one of them with a default, so any partial command
line resolves to a full option set (§3.1). The
Command Line page
covers them.
One command line names no work at all. Given the bare executable name, TextFinder writes
the resolved option listing, exits 0, and traverses nothing, so a user who types the
program's name learns the settings a real invocation would start from instead of waiting
on a search of the current tree (§3.1).
Spec_TextFinder.md §2 scopes all of this: the document specifies behavior
common to every implementation, in no particular language, and each language's own
documents refine it without redefining it. Four implementations are planned - C++, Rust,
C#, and Python - and the first three are complete.
2. The Shape of a Block
Matches reach stdout as blocks, one block per matching file. A block opens with a line
naming the file and continues with a detail line for each matching line, when
/n or /L asks for one (§3.4).
<path>
<lineNumber> - <matchedLine>
<lineNumber> - <matchedLine>
The first line is <path> alone, with nothing before or after it. Each
detail line is indented exactly two spaces and carries the fields /n and
/L select, joined by the three-character separator -
when both are present.
The two-level form exists to write a path once and once only. A file's path appears on its
block's first line whatever the number of its matches, and nothing below repeats it. No
announcement names a file that produced a block, so within a single root no path is ever
written twice.
With /n and /L both at their defaults a block has no detail
lines. Its path line then says everything the block can say, so one block is emitted per
matching file and evaluation of that file stops at its first match. The
Matching page
covers the four field combinations and the case that needs no file content at all.
Blocks are emitted as they occur. A block's path line is written the moment that file's
first match is found, and each detail line as its own line is evaluated. Nothing is
buffered to the end of a file, a directory, or the run, so blocks appear in the
depth-first traversal order of §3.2.
3. The Four Announcements
Besides blocks, TextFinder announces the files and directories that produced no block,
through the same destination. An announcement is a fixed form followed by a path, emitted
as TextFinder deals with the entry it names. There are four, in two kinds, and the kind
decides whether /h can hide it.
| Announcement |
Kind |
Emitted when |
searched <path> |
file |
a file was admitted, read, searched, and matched nothing |
skipped <path> |
file |
a file was rejected by the NUL or UTF-8 test of §3.3 |
too large <path> |
error |
a file exceeded the size limit of §3.3 |
cannot open <path> |
error |
a file, directory, or root path could not be opened, is a symbolic link named as a root, is neither a regular file nor a directory, or carries a name the implementation cannot render as text |
File announcements report a file that was examined and yielded no match. They appear only
when /h is false; under the default true such a file
contributes nothing. Error announcements report work TextFinder was asked to do and could
not, and appear whatever /h says.
The division was not there at first. An earlier §3.4 contradicted itself in
consecutive sentences, saying that under the default no file announcement appears and then
that too large and cannot open are emitted whatever
/h says - and both are file announcements under the old naming. Splitting the
two kinds into separate tables retired the contradiction, and the reason the split falls
where it does is that silencing a 10 MB skip under the default would hide requested work
that did not happen.
Every file TextFinder examines therefore contributes at most one of two things, never both:
a block if it matched, a file announcement if it did not. Under /h false every
examined file appears exactly once.
The last clause of the cannot open row covers a case only a filesystem
produces: a name the implementation's string type cannot carry, which means bytes that are
not valid UTF-8 on POSIX and unpaired surrogate code units on Windows. Such a file is not
searched, and its announcement names it with U+FFFD REPLACEMENT CHARACTER substituted for
each unit that will not render, since an announcement naming nothing would leave the reader
no way to find the file. §3.4 states the two costs it accepts for that: the path
reported is not the path on disk, and which names reach the case depends on the
implementation's string type, so §6's match set agrees only over trees whose names
every implementation can carry.
Rendering a name may not end the run. Where an implementation's conversion reports failure
by throwing, or by anything else that would propagate out of the walk, it is contained
where it arises: the entry draws its announcement and traversal continues with the next one.
One unrenderable name costs one file, never the remainder of the search, and the run still
exits 0.
Announcements route through the implementation's output component, not to stderr, and
nothing announced affects the exit code. Usage diagnostics are the only text that goes to
stderr, and §5.2 binds their shape while leaving their wording to each implementation.
4. The Run Summary
Blocks and announcements are not the whole of what a traversing run writes. After the last
root path is finished, TextFinder closes its output with one line counting what the run
reached (§3.6):
accessed <fileCount> files, <dirCount> directories
That line is the last thing stdout carries. Its form is fixed here rather than left to the
implementation, for the reason the block form and the exit codes are: two implementations
free to word it differently could not be compared line-for-line. Both numbers are written in
decimal with no grouping separator, and neither noun is inflected - a run that reached one
file writes 1 files. Inflecting would put a plural rule in four implementations
to disagree over, and §3.6 accepts the cost of not inflecting instead: one line that
reads badly in a case a real search rarely reaches.
A file counts when TextFinder examines it - when it passed the /p filter and
TextFinder went on to take its size from the filesystem. A directory counts when TextFinder
reads its entries or tries to. The counts are therefore of work attempted rather than of
announcements written: a file admitted and then reported too large is counted,
while an entry refused before selection is not - one the /p list excluded, a
symbolic link, anything beneath a pruned directory, a name that will not render, and an
entry that is neither a regular file nor a directory. The
Traversal page
covers those gates; each of them sits ahead of the count.
/h does not suppress the line, and no switch does. It is neither a block nor an
announcement - it names no path and reports no entry - so nothing in §3.4 governs it,
and /h, which hides only files that matched nothing, leaves it in place.
Only a run that begins traversal writes it. The bare command line of §3.1 traverses
nothing and writes the option listing alone, /H prints help and traverses
nothing, and a usage diagnostic and a startup failure both precede traversal. A run that
does traverse writes the summary whatever it found, so a search that matched nothing still
reports what it reached, and a run whose one root is a symbolic link writes
accessed 0 files, 0 directories beneath that root's cannot open.
§3.6 also fixes one thing the rest of the document leaves to each language: which
component holds the two counts. It is the traversal component in every implementation, and
the entry binary asks it for the line once the last root is finished, since only the entry
binary knows which root was the last. The
Contracts page
covers why that one division is fixed where the others are not.
5. The Three Exit Codes
§3.4 fixes the three values rather than leaving them to the implementation, so that
two implementations can be compared by exit code as well as by output. No other value is
returned.
| Code |
Returned when |
| 0 |
The invocation succeeded. Whether matches were found, and whether any file or root path drew an error announcement, does not change this. /H returns 0, as does the bare command line of §3.1 |
| 1 |
The command line was invalid. TextFinder wrote the usage diagnostic §5.2 fixes for the violation, to stderr, and traversed nothing. An invalid /r writes the §5.3 listing to stdout ahead of that diagnostic; every other violation leaves stdout empty |
| 2 |
TextFinder could not start, for a reason that is not about what the user typed - a failure to initialize its output component, for instance. No usage diagnostic is written, and traversal does not begin |
Code 2 separates a failure of the program from a failure of the command line, so a test can
tell the two apart. §5.2 fixes the text a code-1 failure writes and leaves the text of
a code-2 failure to the implementation, which is why cannot initialize output
appears in the C++ Entry specification rather than in the project specification.
Two things deliberately do not reach the exit code. An error announcement does not: a root
path that cannot be opened is reported and the run still exits 0. Neither does an empty
result: finding nothing is a successful search.
6. The Pipeline the Components Form
The project specification names no components; that is each language's own structure
document. Every implementation so far divides the work the same way, and the division is
worth stating here because the remaining Behavior pages are organized by it.
Cmdline ---> Dirnav ---> Output
^ ^ ^
| | |
+----- Entry (binary) ---+
- Cmdline - converts the argument vector into a resolved option set,
every switch carrying its supplied value or its default. Opens no stream and touches no
filesystem.
- Dirnav - walks, selects, reads, matches, and formats. The only
component that touches file contents, and it writes to no stream. It also holds the two
run counts of §3.6, being the only component that sees the entries.
- Output - the sink. Receives fully formed strings and writes each as
one line to stdout, absorbing any write failure.
- Entry - the binary. Owns the skip list, wires the three together,
drives traversal across the root paths, asks Dirnav for the run summary once the last
one is finished, and owns the exit code.
Emission is a pipeline rather than a report: a line leaves Dirnav for Output the moment it
is produced, which is what §3.4's "as they occur" requires and what makes the output
of a long search start immediately.
The remaining four pages of this thread follow the boundaries above. The
Command Line page
covers §4 and §5; the
Traversal page
covers §3.2 and §3.3; the
Matching page
covers §3.3, §3.4, and §3.6; and the
Contracts page
covers §3.5, §3.6, §6, §6.1, and §7. For one language's realization of the
same boundaries, cross with the header row above.