Synopsis:
This page covers how TextFinder reaches a file - the walk, and the gates an entry
passes before it is searched.
-
Depth-first, in the order the platform's directory-reading facility yields. No
sorting, no grouping of files ahead of directories.
-
An 11-name skip list prunes whole subtrees by directory basename. Pruning draws no
announcement - it is work TextFinder chose not to do, not work it could not do.
-
Symbolic links are never opened. One met during traversal is passed over silently;
one named as a root is announced.
One rule runs under several of these: an entry TextFinder finds on its own it may pass
over in silence, and an entry the user typed it owes an explanation for.
- That is why the skip list is not consulted for a root path, and why a root that is a link is announced.
/p matches the file's last dot-suffix, so .gitignore has extension gitignore - which is why the C++ spec rejects path::extension() by name.
- Three admission tests in a fixed order: size from the filesystem, then no NUL byte, then valid UTF-8.
Two retreats are on the record, and Section 6 settles which gates the run counts.
- Sorted sibling order gave way to filesystem order, and three places changed together so the document would not claim what it no longer delivered.
- Read-then-test gave way to test-then-read, which had been reading a 200 MB file in order to refuse it.
1. Depth-First in Filesystem Order
Traversal is depth-first: on reaching a directory that is not pruned, TextFinder descends
into it and completes its subtree before returning to the next entry of the parent. Within
a directory, entries are visited in the order the platform's directory-reading facility
presents them, files and directories interleaved rather than grouped, so a directory is
descended at the point it is reached in that order (§3.2).
Two requirements fix that order, and §6's comparison of one implementation against
another rests on both.
- An implementation enumerates a directory through the facility its platform provides -
readdir on POSIX, FindFirstFileW and
FindNextFileW on Windows, or whatever its own standard library wraps
around them - and does not substitute a facility that imposes an order of its own.
- It visits entries in the order that facility yields. It does not sort them, and it
does not group files ahead of directories or the reverse. Collecting a directory's
entries before visiting any of them is permitted, since it changes no order;
reordering them is not.
The visit order is therefore the filesystem's own, and it is the same order in every
implementation on one machine against one tree. What that leaves guaranteed is narrower
than sorted order would give, and §6 states the limit rather than claiming more: the
total emission order is reproducible only across runs over the same tree on the same
platform and filesystem. Elsewhere the match set still agrees and the order of matches from
different directory entries may not.
An earlier draft required ascending basename order, compared bytewise. The retreat to
filesystem order was a deliberate trade, and the record states what each side buys.
Filesystem order streams a level instead of buffering it and reads more cleanly, which
counts for a teaching codebase. Sorted order buys cross-platform reproducibility, since
NTFS returns uppercased-name index order while ext4 with dir_index returns
hash order, and buys stability over time, since ext4 hash order shifts as entries are added
and removed. Three places had to change together so the document would not claim what it no
longer delivered: §3.2, §6, and the C++ traversal rule.
Recursion is controlled by /s and is on by default. With /s false
TextFinder searches the files directly within the root path and descends into none of its
subdirectories. Multiple /P occurrences accumulate, and their subtrees are
traversed in the order given.
2. The Skip List
TextFinder maintains a list of directory names never entered during traversal. When a
directory whose name appears in the list is met, its entire subtree is pruned. The default
list holds 11 names, directories that typically carry version-control metadata or
intermediate build output:
archive, .git, .svn, .hg, build, out, target, bin, obj, __pycache__, node_modules
A skip-list entry is compared against the directory's basename - the final component of
its path - and matches when the two strings are equal. Matching is case-sensitive on POSIX
and case-insensitive on Windows, following each platform's filesystem conventions. The list
governs directory names only; file-name filtering by extension is /p.
A pruned directory is not a failure and draws no announcement. That distinction matters:
cannot open reports work TextFinder could not do, and pruning is work it chose
not to do.
The list governs directories met during traversal, never a root path the user supplied on
/P. A root whose basename appears in the list is traversed like any other
root, for the same reason a root that is a symbolic link is announced rather than passed
over: the user named it explicitly, and returning nothing without a word would leave that
unexplained. Pruning resumes immediately below the root, so a build directory
found beneath a root named build is pruned as usual.
That carve-out was removed once and restored. An intermediate version consulted the list
for every directory including roots, so /P build was pruned before any entry
was read; the current text exempts the root again. The C++ unit suite tests both halves -
a root whose name is in the skip list is traversed, and the same directory reached during
traversal is still pruned.
Nothing on the command line and no configuration file extends the list. Extension happens
at build time, through the one function §3.5 defines; the
Contracts page
covers it.
3. Symbolic Links and Root Paths
Symbolic links are never opened. One met during traversal is passed over silently, since
TextFinder makes no attempt to open it and so has no failure to report. A root path that is
a symbolic link is announced with cannot open, because the user named it
explicitly and it will not be searched.
The asymmetry is the point rather than an oversight, and it follows the same rule the skip
list follows: an entry TextFinder finds on its own it may pass over in silence, and an
entry the user typed it owes an explanation for.
A root path resolves three ways.
| Root resolves to |
What happens |
| a directory |
Traversed, skip list not consulted for the root itself |
| a regular file |
Searched as that single file, filtered by /p like any other file |
| anything else |
Not searched, and reported with cannot open per §3.4 |
No root-path outcome affects the exit code. A run whose only root could not be opened
announces that and exits 0, which is the §3.4 rule that an error announcement does not
change the code.
One more entry never gets as far as being selected. A name the implementation's string type
cannot carry - bytes that are not valid UTF-8 on POSIX, unpaired surrogate code units on
Windows - is announced with cannot open and skipped, and §3.4 places that
test over every such entry rather than over the selected ones, so it fires whatever
/p holds. It also runs after the symbolic-link test, so a link with such a name
stays silent like any other link. The
Overview page
covers the announcement and the two costs §3.4 accepts for it.
The path a block reports begins with the root path currently being traversed - the
/P occurrence whose subtree holds the file, not the first /P
given - and continues with the entry names descended through to reach it. The root's own
text is part of the path, so a root of src yields
src/foo.cpp rather than foo.cpp, and two roots holding files of
the same name yield blocks that differ. A root of . is the one exception: it
contributes no leading ./. Every path is rendered with /
separators on every platform.
4. Extension Rules
/p takes a quoted, comma-separated list of bare extensions. Four rules govern
it, and the first is the one an implementer is likeliest to get wrong by reaching for a
standard-library call.
- An extension is the file's last dot-suffix, a leading dot on the name
notwithstanding.
.gitignore has extension
gitignore, so a dot-file is searched like any other file and is excluded
only by /p or by the skip list.
- Each item is normalized. Trimmed of surrounding whitespace, then
stripped of one leading dot if present, so
cpp and .cpp are
equivalent. Empty items are discarded, so "cpp,,rs" and
"cpp, rs" name the same two extensions.
- Comparison follows the platform, case-sensitive on POSIX and
case-insensitive on Windows, as skip-list entries do.
- The empty list and the non-empty list differ on extensionless files.
An empty list searches every file, files with no extension included. A non-empty list
excludes them.
The dot-file rule is why the C++ specification rejects
std::filesystem::path::extension() by name: that function returns an empty
string for .gitignore, where §5 gives it the extension
gitignore. The warning is kept in the specification deliberately, because the
wrong call is the plausible one.
5. The Three Admission Tests
A selected file is tested before it is searched, and the order of the tests is part of the
specification rather than an implementation detail.
- Size. Taken from the filesystem, it must not exceed 10 MB
(10,485,760 bytes). A file above the limit is never read, and draws the error
announcement
too large.
- No NUL byte. This is the binary-file test of §7. A file within
the size limit is read in full - so that one failing a later test is skipped entirely
rather than searched in part - and rejected if its content holds a NUL byte.
- Valid UTF-8. Truncated sequences, overlong encodings, encoded
surrogates, and scalar values above U+10FFFF all fail.
A file that fails test 2 or test 3 draws skipped, a file announcement, so
/h can hide it. A file that fails test 1 draws too large, an
error announcement, which /h cannot hide.
The NUL test is there because UTF-8 validity alone does not serve: a UTF-16 file of ASCII
text is valid UTF-8, so validation would admit it and the search would run fruitlessly over
text the specification says not to search. The reason is stated in §3.3 rather than
left implicit, and it survived the trimming passes for that reason.
Test 1 reads metadata and tests 2 and 3 read bytes, and the ordering follows from that. An
earlier draft said TextFinder reads the file in full and then admits it if it passes three
tests, the first being size, which would have read a 200 MB file in order to refuse it. The
current text is test-then-read: the filesystem-reported size is checked first, and the
read-in-full justification attaches only to the NUL and UTF-8 tests.
A leading UTF-8 BOM is consumed and does not belong to the first line. One case skips
tests 2 and 3 entirely, and the
Matching page
covers it: with the default expression and neither /n nor /L, no
file content is needed, so no file is opened and the size test runs alone.
6. Which of These Gates Counts
§3.6 closes every traversing run with a line counting the files and directories it
reached, and the gates this page describes are what decide the two numbers. The rule is one
line each. A file counts when it passed the /p filter of Section 4 and
TextFinder went on to take its size for test 1. A directory counts when TextFinder reads
its entries or tries to.
Put that against the sections above and the counts fall out without a second rule.
- A file refused by
/p is not counted. TextFinder learned its name and
nothing more.
- A symbolic link met during traversal is not counted, being passed over silently
without an attempt to open it (Section 3).
- Nothing beneath a directory the skip list pruned is counted, traversal never having
reached it - and the pruned directory is not counted either, pruning meaning it was
never entered (Section 2).
- A root path the skip list would have pruned is counted, since Section 2
exempts roots from the list: the user named it explicitly.
- Under
/s false the root counts and no subdirectory does.
- A file that passed
/p and then failed test 1 is counted, the count
standing ahead of the test. So is one whose size could not be read at all, which draws
cannot open.
- An entry refused before selection is not counted even though it draws
cannot open: a name that will not render, which Section 3 announces
whatever /p holds, and an entry that is neither a regular file nor a
directory.
The last two items are the ones worth reading twice, because together they say the counts
are of work attempted rather than of announcements written. A run can print more
cannot open lines than its file count admits to, and that is correct: the two
entries in the second item never reached the tests this page is about.
The no-content case counts the same way it selects. It opens no file, yet every file it
selects is counted, including a zero-length one that produced neither a block nor an
announcement - the size test still ran on metadata, which is the whole of what the count
requires.